Utterance Level Feature Aggregation with Deep Metric Learning for Speech Emotion Recognition release_aaq3ttoexjd2lk2tng3ngd43ia

by Bogdan Mocanu, Ruxandra Tapu, Titus Zaharia

Published in Sensors by MDPI AG.

2021   Volume 21, Issue 12, p4233

Abstract

Emotion is a form of high-level paralinguistic information that is intrinsically conveyed by human speech. Automatic speech emotion recognition is an essential challenge for various applications; including mental disease diagnosis; audio surveillance; human behavior understanding; e-learning and human–machine/robot interaction. In this paper, we introduce a novel speech emotion recognition method, based on the Squeeze and Excitation ResNet (SE-ResNet) model and fed with spectrogram inputs. In order to overcome the limitations of the state-of-the-art techniques, which fail in providing a robust feature representation at the utterance level, the CNN architecture is extended with a trainable discriminative GhostVLAD clustering layer that aggregates the audio features into compact, single-utterance vector representation. In addition, an end-to-end neural embedding approach is introduced, based on an emotionally constrained triplet loss function. The loss function integrates the relations between the various emotional patterns and thus improves the latent space data representation. The proposed methodology achieves 83.35% and 64.92% global accuracy rates on the RAVDESS and CREMA-D publicly available datasets, respectively. When compared with the results provided by human observers, the gains in global accuracy scores are superior to 24%. Finally, the objective comparative evaluation with state-of-the-art techniques demonstrates accuracy gains of more than 3%.
In application/xml+jats format

Archived Files and Locations

application/pdf  6.0 MB
file_isi2yo6ninhc7k5afi3l4fv2di
res.mdpi.com (publisher)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article-journal
Stage   published
Date   2021-06-20
Language   en ?
Container Metadata
Open Access Publication
In DOAJ
In ISSN ROAD
In Keepers Registry
ISSN-L:  1424-8220
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: f0f6f780-eae1-4afe-b084-71ad19d13f08
API URL: JSON