Speaker-independent speech emotion recognition by fusion of functional and accompanying paralanguage features

MAO Qi-rong; ZHAO Xiao-lei; HUANG Zheng-wei; ZHAN Yong-zhao

doi:10.1631/jzus.CIDE1310

Your Location：

Home >

Browse articles >

Speaker-independent speech emotion recognition by fusion of functional and accompanying paralanguage features

数据驱动的数字娱乐最新进展专辑 | Updated：2022-06-06

- Speaker-independent speech emotion recognition by fusion of functional and accompanying paralanguage features
- Speaker-independent speech emotion recognition by fusion of functional and accompanying paralanguage features
- 信息与电子工程前沿（英文） 2013年14卷第7期页码：573-582
- Affiliations：
  
  Department of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang 212013, China
- Author bio：
  
  [ "Qi-rong MAO, E-mail: mao_qr@ujs.edu.cn" ]
- Funds：
  
  Project supported by the National Natural Science Foundation of China (Nos. 61272211 and 61170126), the Natural Science Foundation of Jiangsu Province (No. BK2011521), and the Research Foundation for Talented Scholars of Jiangsu University (No. 10JDG065), China
- DOI：10.1631/jzus.CIDE1310
  中图分类号： TP391.4
- 收稿：2012-12-29，
  
  修回：2013-06-06，
  
  纸质出版：2013-07
- Accepted：
Scan QR Code
Speaker-independent speech emotion recognition by fusion of functional and accompanying paralanguage features[J]. 信息与电子工程前沿（英文）, 2013,14(7):573-582.

MAO Qi-rong, ZHAO Xiao-lei, HUANG Zheng-wei, et al. Speaker-independent speech emotion recognition by fusion of functional and accompanying paralanguage features[J]. Frontiers of Information Technology & Electronic Engineering, 2013, 14(7): 573-582.
Speaker-independent speech emotion recognition by fusion of functional and accompanying paralanguage features[J]. 信息与电子工程前沿（英文）, 2013,14(7):573-582. DOI： 10.1631/jzus.CIDE1310.

MAO Qi-rong, ZHAO Xiao-lei, HUANG Zheng-wei, et al. Speaker-independent speech emotion recognition by fusion of functional and accompanying paralanguage features[J]. Frontiers of Information Technology & Electronic Engineering, 2013, 14(7): 573-582. DOI： 10.1631/jzus.CIDE1310.

摘要

Abstract

Functional paralanguage includes considerable emotion information

and it is insensitive to speaker changes. To improve the emotion recognition accuracy under the condition of speaker-independence

a fusion method combining the functional paralanguage features with the accompanying paralanguage features is proposed for the speaker-independent speech emotion recognition. Using this method

the functional paralanguages

such as laughter

cry

and sigh

are used to assist speech emotion recognition. The contributions of our work are threefold. First

one emotional speech database including six kinds of functional paralanguage and six typical emotions were recorded by our research group. Second

the functional paralanguage is put forward to recognize the speech emotions combined with the accompanying paralanguage features. Third

a fusion algorithm based on confidences and probabilities is proposed to combine the functional paralanguage features with the accompanying paralanguage features for speech emotion recognition. We evaluate the usefulness of the functional paralanguage features and the fusion algorithm in terms of precision

recall

and F1-measurement on the emotional speech database recorded by our research group. The overall recognition accuracy achieved for six emotions is over 67% in the speaker-independent condition using the functional paralanguage features.

关键词

Keywords

references

J.A. Bachorowski , M.J. Smoski , M.J. Owren . The acoustic features of human laughter . J. Acoust. Soc. Am. , 2011 . 110 ( 3 ): 1581 - 1597 . DOI: 10.1121/1.1391244 http://doi.org/10.1121/1.1391244 .

A. Berler , S.E. Shimony . Bayes Networks for Sonar Sensor Fusion . Proc. 13th Conf. on Uncertainty in Artificial Intelligence , 1997 . p.14 - 21 . .

L. Devillers , L. Vidrascu . Real-Life Emotions Detection with Lexical and Paralinguistic Cues on Human- Human Call Center Dialogs . Proc. Interspeech , 2006 . p.801 - 804 . .

Ayadi M. El , M.S. Kamel , F. Karray . Survey on speech emotion recognition: features, classification schemes, and databases . Pattern Recogn. , 2011 . 44 ( 3 ): 572 - 587 . DOI: 10.1016/j.patcog.2010.09.020 http://doi.org/10.1016/j.patcog.2010.09.020 .

S. Fujie , Y. Ejiri , Y. Matsusaka , H. Kikuchi . Recognition of Paralinguistic Information and Its Application to Spoken Dialogue System . Proc. IEEE Workshop on Automatic Speech Recognition and Understanding , 2003 . p.231 - 236 . DOI: 10.1109/ASRU.2003.1318446 http://doi.org/10.1109/ASRU.2003.1318446 .

Y. Hayashi . Recognition of Vocal Expression of Emotions in Japanese: Using the Interjection eh 'Korean' . Proc. Int. Conf. on Phonetic Sciences , 1999 . p.2355 - 2359 . .

C.L. Huang , C.J. Wang . A GA-based feature selection and parameters optimization for support vector machines . Expert Syst. Appl. , 2006 . 31 ( 2 ): 231 - 240 . DOI: 10.1016/j.eswa.2005.09.024 http://doi.org/10.1016/j.eswa.2005.09.024 .

C.W. Huang , Y. Jin , Y. Zhao , Y.H. Yu , L. Zhao . Design and establishment of practical speech emotion database . Techn. Acoust. , 2010 . 29 ( 4 ): 396 - 399 . .

S. Huq , Z. Moussavi . Acoustic breath-phase detection using tracheal breath sounds . Med. Biol. Eng. Comput. , 2012 . 50 ( 3 ): 297 - 308 . DOI: 10.1007/s11517-012-0869-9 http://doi.org/10.1007/s11517-012-0869-9 .

C.T. Ishi , H. Ishiguro , N. Hagita . Evaluation of Prosodic and Voice Quality Features on Automatic Extraction of Paralinguistic Information . Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems , 2006 . p.9 - 15 . DOI: 10.1109/IROS.2006.281786 http://doi.org/10.1109/IROS.2006.281786 .

C.T. Ishi , H. Ishiguro , N. Hagita . Automatic extraction of paralinguistic information using prosodic features related to F0, duration and voice quality . Speech Commun. , 2008 . 50 ( 6 ): 531 - 543 . DOI: 10.1016/j.specom.2008.03.009 http://doi.org/10.1016/j.specom.2008.03.009 .

C. Jones , I.M. Jonsson . Using paralinguistic cues in speech to recognize emotions in older car drivers . LNCS , 2008 . 4868 229 - 240 . DOI: 10.1007/978-3-540-85099-1_20 http://doi.org/10.1007/978-3-540-85099-1_20 .

L.S. Kennedy , D.P.W. Ellis . Laughter Detection in Meetings . Proc. Int. Conf. on Acoustics, Speech, and Signal Processing Meeting Recognition Workshop , 2004 . p.118 - 121 . .

J. Kleckova . Important Nonverbal Attributes for Spontaneous Speech Recognition . 4th Int. Conf. on Systems , 2009 . p.13 - 16 . DOI: 10.1109/ICONS.2009.41 http://doi.org/10.1109/ICONS.2009.41 .

C.G. Li . Paralinguistic Studying . MS Thesis, Heilongjiang University, Harbin, China (in Chinese) , 2004 .

Y.C. Li , B. Wang , J. Wei , C. Qian , Y. Huang . An efficient combination rule of evidence theory . J. Data Acquis. Process. , 2002 . 17 ( 1 ): 33 - 36 . .

Y.X. Li , Q.H. He . Detecting laughter in spontaneous speech by constructing laughter bouts . Int. J. Speech Technol. , 2011 . 14 ( 3 ): 211 - 225 . DOI: 10.1007/s10772-011-9097-1 http://doi.org/10.1007/s10772-011-9097-1 .

K. Maekawam . Production and Perception of 'Paralinguistic' Information . Int. Conf. on Speech Prosody , 2004 . p.367 - 374 . .

Q.R. Mao , X.J. Wang , Y.Z. Zhan . Speech emotion recognition method based on improved decision tree and layered feature selection . Int. J. Human. Rob. , 2010 . 7 ( 2 ): 245 - 261 . DOI: 10.1142/S0219843610002088 http://doi.org/10.1142/S0219843610002088 .

S. Matos , S.S. Birring , I.D. Pavord , D.H. Evans . Detection of cough signals in continuous audio recordings using hidden Markov models . IEEE Trans. Biomed. Eng. , 2006 . 53 ( 6 ): 1078 - 1083 . DOI: 10.1109/TBME.2006.873548 http://doi.org/10.1109/TBME.2006.873548 .

P. Pal , A.N. Iyer , R.E. Yantorno . Emotion Detection from Infant Facial Expressions and Cries . IEEE Int. Conf. on Acoustics, Speech and Signal Processing , 2006 . p.721 - 724 . DOI: 10.1109/ICASSP.2006.1660444 http://doi.org/10.1109/ICASSP.2006.1660444 .

S. Petridis , M. Pantic . Audiovisual Discrimination Between Laughter and Speech . IEEE Int. Conf. on Acoustics, Speech and Signal Processing , 2008 . p.5117 - 5120 . DOI: 10.1109/ICASSP.2008.4518810 http://doi.org/10.1109/ICASSP.2008.4518810 .

P. Pudil , J. Novovicova , J. Kittler . Flating search methods in feature selection . Pattern Recogn. Lett. , 1994 . 15 ( 11 ): 1119 - 1125 . DOI: 10.1016/0167-8655(94)90127-9 http://doi.org/10.1016/0167-8655(94)90127-9 .

S. Sundaramb , S. Narayananc . Automatic acoustic synthesis of human-like laughter . J. Acoust. Soc. Am. , 2007 . 121 ( 1 ): 527 - 535 . DOI: 10.1121/1.2390679 http://doi.org/10.1121/1.2390679 .

D.P. Szameitat , C.J. Darwin , A.J. Szameitat . Formant Characteristics of Human Laughter . Interdisciplinary Workshop on the Phonetics of Laughter , 2007 . p.4 - 5 . .

K.P. Truong , D.A. van Leeuwen . Automatic Detection of Laughter . Proc. 9th European Conf. on Speech Communication and Technology , 2005 . p.485 - 488 . .

K.P. Truong , D.A. van Leeuwen . Automatic discrimination between laughter and speech . Speech Commun. , 2007 . 49 ( 2 ): 144 - 158 . DOI: 10.1016/j.specom.2007.01.001 http://doi.org/10.1016/j.specom.2007.01.001 .

Y.M. Yang , X. Liu . A Re-examination of Text Categorization Methods . Proc. ACM SIGIR Conf. on Research and Development in Information Retrieval , 1999 . p.42 - 49 . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

暂无数据