Mismatched feature detection with finer granularity for emotional speaker recognition

Li CHEN; Ying-chun YANG; Zhao-hui WU

doi:10.1631/jzus.C1400002

Your Location：

Home >

Browse articles >

Mismatched feature detection with finer granularity for emotional speaker recognition

Electrical & Electronic Engineering | Updated：2022-05-19

- Mismatched feature detection with finer granularity for emotional speaker recognition
- 用于情感说话人识别的精细失真特征检测与修正
- Frontiers of Information Technology & Electronic Engineering Vol. 15, Issue 10, Pages: 903-916(2014)
- Affiliations：
  
  College of Computer Science & Technology, Zhejiang University, Hangzhou 310027, China
- Author bio：
  
  [ "Li CHEN, E-mail: stchenli@zju.edu.cn" ]
  Ying-chun YANG, E-mail: yyc@zju.edu.cn
  [ "Zhao-hui WU, E-mail: wzh@zju.edu.cn" ]
- Funds：
- DOI：10.1631/jzus.C1400002
  CLC： TP391.4
- Received：05 January 2014，
  
  Revised：2014-;9-17，
  
  Published：2014-10
- Accepted：
Scan QR Code
Li CHEN, Ying-chun YANG, Zhao-hui WU. Mismatched feature detection with finer granularity for emotional speaker recognition[J]. Frontiers of Information Technology & Electronic Engineering, 2014, 15(10): 903-916.
DOI：

Li CHEN, Ying-chun YANG, Zhao-hui WU. Mismatched feature detection with finer granularity for emotional speaker recognition[J]. Frontiers of Information Technology & Electronic Engineering, 2014, 15(10): 903-916. DOI： 10.1631/jzus.C1400002.

摘要

研究目的

说话人情感变化时其发音器官会发生形变，导致部分语音特征分布较中性条件下发生一定偏移。这些发生偏移的特征使得说话人识别性能大幅下降，称作"失真特征"，需剔除或修正，以提升情感说话人识别系统性能。

创新要点

鉴于不同音素引起的失真特征分布变化存在差异，提出在音素类、高斯符号化和概率高斯符号化三种声学类上的精细失真特征检测模型与修正方法。

研究方法

采用流形分析方法，观测失真特征分布，得到结论：偏离中性特征空间越远，区分说话人能力越差。若基于某项特征的说话人区分能力小于某个阈值，即检测为失真特征（图1）。对于音素类和高斯符号化表示的声学类，采用支持向量机建立可靠–失真特征检测模型；对于概率高斯符号化表征的声学类，采用模糊支持向量机建立可靠–失真特征检测模型。为确保修正后的失真特征逼近真实的中性情形又不损失说话人特性，对检测出的失真特征进行修正时，将失真特征空间映射到可靠特征空间的同时，要使得转换后的失真特征空间和其他说话人的可靠特征空间的距离不会随之减少。

重要结论

情感导致说话人的部分语音特征分布发生变化成为失真特征，通过三种声学类的精细失真特征检测与修正，能够有效处理失真特征，提升系统识别性能。最高的概率高斯符号化下的失真特征修正算法，使得基准的GMM-UBM算法识别率提升6.77%，i-vector算法识别率提升3.32%。

Abstract

The shapes of speakers' vocal organs change under their different emotional states

which leads to the deviation of the emotional acoustic space of short-time features from the neutral acoustic space and thereby the degradation of the speaker recognition performance. Features deviating greatly from the neutral acoustic space are considered as mismatched features

and they negatively affect speaker recognition systems. Emotion variation produces different feature deformations for different phonemes

so it is reasonable to build a finer model to detect mismatched features under each phoneme. However

given the difficulty of phoneme recognition

three sorts of acoustic class recognition—phoneme classes

Gaussian mixture model (GMM) tokenizer

and probabilistic GMM tokenizer—are proposed to replace phoneme recognition. We propose feature pruning and feature regulation methods to process the mismatched features to improve speaker recognition performance. As for the feature regulation method

a strategy of maximizing the between-class distance and minimizing the within-class distance is adopted to train the transformation matrix to regulate the mismatched features. Experiments conducted on the Mandarin affective speech corpus (MASC) show that our feature pruning and feature regulation methods increase the identification rate (IR) by 3.64% and 6.77%

compared with the baseline GMM-UBM (universal background model) algorithm. Also

corresponding IR increases of 2.09% and 3.32% can be obtained with our methods when applied to the state-of-the-art algorithm i-vector.

关键词

Keywords

references

L.M. Arslan , , , J.H.L. Hansen . . Minimum cost based phoneme class detection for improved iterative speech enhancement . . IEEE Int. Conf. on Acoustics, Speech, and Signal Processing , , 1994 . . p.45 - - 48 . . DOI: 10.1109/ICASSP.1994.389722 http://doi.org/10.1109/ICASSP.1994.389722 . .

M. Balasubramanian , , , E.L. Schwartz . . The isomap algorithm and topological stability . . Science , , 2002 . . 295 ( ( 5552 ): ): 7 DOI: 10.1126/science.295.5552.7a http://doi.org/10.1126/science.295.5552.7a . .

H.J. Bao , , , M.X. Xu , , , T.F. Zheng . . Emotion attribute projection for speaker recognition on emotional speech. Proc. 8th Annual Conf. of the Int . . Speech Communication Association, , , 2007 . . p.601 - - 604 . . . .

D. Bitouk , , , R. Verma , , , A. Nenkova . . Class-level spectral features for emotion recognition . . Speech Commun. , , 2010 . . 52 ( ( 7-8 ): ): 613 - - 625 . . DOI: 10.1016/j.specom.2010.02.010 http://doi.org/10.1016/j.specom.2010.02.010 . .

M.C. Brady . . Synthesizing affect with an analog vocal tract: glottal source . . Toward Social Mechanisms of Android Science: a CogSci Workshop , , 2005 . . p.45 - - 49 . . . .

L. Chen , , , Y.C. Yang , , , M. Yao . . Reliability detection by fuzzy SVM with UBM component feature for emotional speaker recognition. Proc. 8th Int. Conf . . on Fuzzy Systems and Knowledge Discovery , , 2011 . . p.458 - - 461 . . DOI: 10.1109/FSKD.2011.6019484 http://doi.org/10.1109/FSKD.2011.6019484 . .

R. Cowie , , , R.R. Cornelius . . Describing the emotional states that are expressed in speech . . Speech Commun. , , 2003 . . 40 ( ( 1-2 ): ): 5 - - 32 . . DOI: 10.1016/S0167-6393(02)00071-7]CrossRefMATH http://doi.org/10.1016/S0167-6393(02)00071-7]CrossRefMATH . .

N. Dehak , , , P. Kenny , , , R. Dehak , , , 等 . . Front-end factor analysis for speaker verification . . IEEE Trans. Audio Speech Lang. Process. , , 2011 . . 19 ( ( 4 ): ): 788 - - 798 . . DOI: 10.1109/tasl.2010.2064307 http://doi.org/10.1109/tasl.2010.2064307 . .

A. Drygajlo , , , M. El-Maliki . . Speaker verification in noisy environments with combined spectral subtraction and missing feature theory. Proc. IEEE Int. Conf . . on Acoustics, Speech and Signal Processing , , 1998 . . p.121 - - 124 . . DOI: 10.1109/ICASSP.1998.674382 http://doi.org/10.1109/ICASSP.1998.674382 . .

Ayadi M. El , , , M.S. Kamel , , , F. Karray . . Survey on speech emotion recognition: features, classification schemes, and databases . . Patt. Recog. , , 2011 . . 44 ( ( 3 ): ): 572 - - 587 . . DOI: 10.1016/j.patcog.2010.09.020]CrossRefMATH http://doi.org/10.1016/j.patcog.2010.09.020]CrossRefMATH . .

J. Gadek . . Influence of upper respiratory system disease on the performance of automatic voice recognition systems . . Comput. Med. Act. , , 2009 . . 65 211 - - 221 . . DOI: 10.1007/978-3-642-04462-5_21 http://doi.org/10.1007/978-3-642-04462-5_21 . .

M.V. Ghiurcau , , , C. Rusu , , , J. Astola . . A study of the effect of emotional state upon text-independent speaker identification . . IEEE Int. Conf. on Acoustics, Speech and Signal Processing , , 2011a . . p.4944 - - 4947 . . DOI: 10.1109/ICASSP.2011.5947465 http://doi.org/10.1109/ICASSP.2011.5947465 . .

M.V. Ghiurcau , , , C. Rusu , , , J. Astola . . Speaker recognition in an emotional environment . . Proc. Signal Processing and Applied Mathematics for Electronics and Communications , , 2011b . . p.81 - - 84 . . . .

T. Huang , , , Y.C. Yang . . Applying pitch-dependent difference detection and modification to emotional speaker recognition . . Proc. 9th Annual Conf. of the Int. Speech Communication Association , , 2008 . . p.2751 - - 2754 . . . .

T. Huang , , , Y.C. Yang . . Learning virtual HD model for bi-model emotional speaker recognition . . Proc. 20th Int. Conf. on Pattern Recognition , , 2010 . . p.1614 - - 1617 . . DOI: 10.1109/ICPR.2010.399 http://doi.org/10.1109/ICPR.2010.399 . .

N.P. Jawarkar , , , R.S. Holambe , , , T.K. Basu . . Textindependent speaker identification in emotional environments: a classifier fusion approach . . Front. Comput. Educ. , , 2012 . . 133 569 - - 576 . . DOI: 10.1007/978-3-642-27552-4_77 http://doi.org/10.1007/978-3-642-27552-4_77 . .

Q. Jin , , , T. Schultz , , , A. Waibel . . Far-field speaker recognition . . IEEE Trans. Audio Speech Lang. Process. , , 2007 . . 15 ( ( 7 ): ): 2023 - - 2032 . . DOI: 10.1109/tasl.2007.902876 http://doi.org/10.1109/tasl.2007.902876 . .

F. Kelly , , , N. Harte . . Effects of long-term ageing on speaker verification . . Proc. European Workshop on Biometrics and ID Management , , 2011 . . p.113 - - 124 . . DOI: 10.1007/978-3-642-19530-3_11 http://doi.org/10.1007/978-3-642-19530-3_11 . .

C.M. Lee , , , S. Yildirim , , , M. Bulut , , , 等 . . Effects of emotion on different phoneme classes . . J. Acoust. Soc. Am. , , 2004 . . 116 2481 DOI: 10.1121/1.4784911 http://doi.org/10.1121/1.4784911 . .

A. Li , , , Q. Fang , , , F. Hu , , , 等 . . Acoustic and articulatory analysis on Mandarin Chinese vowels in emotional speech . . Proc. 7th Int. Symp. on Chinese Spoken Language Processing , , 2010 . . p.38 - - 43 . . DOI: 10.1109/ISCSLP.2010.5684866 http://doi.org/10.1109/ISCSLP.2010.5684866 . .

C.F. Lin , , , S.D. Wang . . Fuzzy support vector machines . . IEEE Trans. Neur. Netw. , , 2002 . . 13 ( ( 2 ): ): 464 - - 471 . . DOI: 10.1109/72.991432 http://doi.org/10.1109/72.991432 . .

J.C. Platt . . Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods . . Adv. Large Margin Classifiers , , 1999 . . 10 ( ( 3 ): ): 61 - - 74 . . . .

D.A. Reynolds . . Channel robust speaker verification via feature mapping . . IEEE Int. Conf. on Acoustics, Speech, and Signal Processing , , 2003 . . p.53 - - 56 . . DOI: 10.1109/ICASSP.2003.1202292 http://doi.org/10.1109/ICASSP.2003.1202292 . .

D.A. Reynolds , , , R.C. Rose . . Robust text-independent speaker identification using Gaussian mixture speaker models . . IEEE Trans. Speech Audio Process. , , 1995 . . 3 ( ( 1 ): ): 72 - - 83 . . DOI: 10.1109/89.365379 http://doi.org/10.1109/89.365379 . .

D.A. Reynolds , , , T.F. Quatieri , , , R.B. Dunn . . Speaker verification using adapted Gaussian mixture models . . Digit. Signal Process. , , 2000 . . 10 ( ( 1-3 ): ): 19 - - 41 . . DOI: 10.1006/dspr.1999.0361 http://doi.org/10.1006/dspr.1999.0361 . .

R.C. Rose , , , E.M. Hofstetter , , , D.A. Reynolds . . Integrated models of signal and background with application to speaker identification in noise . . IEEE Trans. Speech Audio Process. , , 1994 . . 2 ( ( 2 ): ): 245 - - 257 . . DOI: 10.1109/89.279273 http://doi.org/10.1109/89.279273 . .

K. Scherer , , , T. Johnstone , , , T. Bänziger . . Automatic verification of emotionally stressed speakers: the problem of individual differences . . Proc. Int. Conf. on Speech and Computer , , 1998 . . p.233 - - 238 . . . .

I. Shahin . . Speaker identification in emotional talking environments based on CSPHMM2s . . Eng. Appl. Artif. Intell. , , 2013 . . 26 ( ( 7 ): ): 1652 - - 1659 . . DOI: 10.1016/j.engappai.2013.03.013 http://doi.org/10.1016/j.engappai.2013.03.013 . .

Z.Y. Shan , , , Y.C. Yang . . Learning polynomial function based neutral-emotion GMM transformation for emotional speaker recognition . . Proc. 19th Int. Conf. on Pattern Recognition , , 2008 . . p.1 - - 4 . . DOI: 10.1109/icpr.2008.4761647 http://doi.org/10.1109/icpr.2008.4761647 . .

Z.Y. Shan , , , Y.C. Yang , , , R.Z. Ye . . Natural-emotion GMM transformation algorithm for emotional speaker recognition . . Proc. 8th Annual Conf. of the Int. Speech Communication Association , , 2007 . . p.782 - - 785 . . . .

E. Shriberg , , , M. Graciarena , , , H. Bratt , , , 等 . . Effects of vocal effort and speaking style on text-independent speaker verification . . Proc. 9th Annual Conf. of the Int. Speech Communication Association , , 2008 . . p.609 - - 612 . . . .

P.A. Torres-Carrasquillo , , , D.A. Reynolds , , , J.R.Jr. Deller . . Language identification using Gaussian mixture model tokenization . . IEEE Int. Conf. on Acoustics, Speech, and Signal Processing , , 1993 . . p.757 - - 760 . . DOI: 10.1109/ICASSP.2002.5743828 http://doi.org/10.1109/ICASSP.2002.5743828 . .

F. Triefenbach , , , A. Jalalvand , , , B. Schrauwen , , , 等 . . Phoneme recognition with large hierarchical reservoirs. Proc. 24th Annual Conf . . on Neural Information Processing Systems , , 2010 . . p.2307 - - 2315 . . . .

W.F. Twaddell . . On defining the phoneme . . Language , , 1935 . . 11 ( ( 1 ): ): 5 - - 62 . . DOI: 10.2307/522070 http://doi.org/10.2307/522070 . .

Y.C. Yang , , , L. Chen . . Toward emotional speaker recognition: framework and preliminary results. Proc. 7th Chinese Conf . . on Biometric Recognition , , 2012 . . p.235 - - 242 . . DOI: 10.1007/978-3-642-35136-5_29 http://doi.org/10.1007/978-3-642-35136-5_29 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰