Past review, current progress, and challenges ahead on the cocktail party problem

Yan-min QIAN; Chao WENG; Xuan-kai CHANG; Shuai WANG; Dong YU

doi:10.1631/FITEE.1700814

Your Location：

Home >

Browse articles >

Past review, current progress, and challenges ahead on the cocktail party problem

2018 Special Issue on Artificial Intelligence 2.0: Theories and Applications | Updated：2022-05-19

- Past review, current progress, and challenges ahead on the cocktail party problem
- 鸡尾酒会问题的技术回顾、当前进展及未来挑战
- Frontiers of Information Technology & Electronic Engineering Vol. 19, Issue 1, Pages: 40-63(2018)
- Affiliations：
  
  Tencent AI Lab, Tencent, Bellevue 98004, USA
  Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
- Author bio：
  
  Yan-min QIAN, E-mail: yanminqian@tencent.com
- Funds：
- DOI：10.1631/FITEE.1700814
  CLC： TP391.4
- Received：08 December 2017，
  
  Revised：2018-;1-25，
  
  Published：2018-01
- Accepted：
Scan QR Code
Yan-min QIAN, Chao WENG, Xuan-kai CHANG, et al. Past review, current progress, and challenges ahead on the cocktail party problem[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(1): 40-63.
DOI：

Yan-min QIAN, Chao WENG, Xuan-kai CHANG, et al. Past review, current progress, and challenges ahead on the cocktail party problem[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(1): 40-63. DOI： 10.1631/FITEE.1700814.

摘要

鸡尾酒会问题即在多人同时说话的场景下追踪并识别某一个特定说话人的语音。在自动语音识别技术大规模推广应用中，鸡尾酒会问题是亟待解决的关键问题之一。本文回顾了在过去20多年中针对鸡尾酒会问题提出的相关技术。主要讨论在鸡尾酒会问题中扮演中心角色的语音分离问题。介绍了以下内容：传统的单通道情况下的技术，如计算听觉场景分析（computation alauditory scene analysis

CASA）、非负矩阵分解（non-negative matrix factorization

NMF）以及生成式模型建模；传统的多通道情况下的技术，如波束成形和多通道盲源分离；一些基于深度学习的最新技术，如深度聚类（deep clustering

DPCL）、深度吸引网络（deep attractor network

DANet）以及排列不变性训练（permutation invariant training

PIT）。此外，介绍了在鸡尾酒会环境下针对改善多说话人语音识别和说话人识别精度的相关技术。笔者认为，利用一个更加强大的模型来有效地开发和利用来自麦克风阵列、声学训练集合以及语言本身的知识非常重要。更好的优化策略和技术的提出会逐步解决鸡尾酒会问题。

Abstract

The cocktail party problem

i.e.

tracing and recognizing the speech of a specific speaker when multiplespeakers talk simultaneously

is one of the critical problems yet to be solved to enable the wide application ofautomatic speech recognition (ASR) systems. In this overview paper

we review the techniques proposed in thelast two decades in attacking this problem. We focus our discussions on the speech separation problem givenits central role in the cocktail party environment

and describe the conventional single-channel techniques such ascomputational auditory scene analysis (CASA)

non-negative matrix factorization (NMF) and generative models

theconventional multi-channel techniques such as beamforming and multi-channel blind source separation

and the newlydeveloped deep learning-based techniques

such as deep clustering (DPCL)

the deep attractor network (DANet)

andpermutation invariant training (PIT). We also present techniques developed to improve ASR accuracy and speakeridentification in the cocktail party environment. We argue effectively exploiting information in the microphone array

the acoustic training set

and the language itself using a more powerful model. Better optimization objective andtechniques will be the approach to solving the cocktail party problem.

关键词

Keywords

references

O Abdel-Hamid , , , A Mohamed , , , H Jiang , , , 等 . . Convolutional neural networks for speech recognition . . Annual Conf of Int Speech Communication Association , , 2014 . . p.1533 - - 1545 . . . .

X Anguera , , , C Wooters , , , J Hernando . . Acoustic beamforming for speaker diarization of meetings . . IEEE Trans Audio Speech Lang Process , , 2007 . . 15 ( ( 7 ): ): 2011 - - 2022 . . DOI: 10.1109/TASL.2007.902460 http://doi.org/10.1109/TASL.2007.902460 . .

S Applebaum . . Adaptive arrays . . IEEE Trans Antennas Propag , , 1976 . . 24 ( ( 9 ): ): 585 - - 598 . . DOI: 10.1109/TAP.1976.1141417 http://doi.org/10.1109/TAP.1976.1141417 . .

J Barker , , , N Ma , , , A Coy , , , 等 . . Speech fragment decoding techniques for simultaneous speaker identification and speech recognition . . Comput Speech Lang , , 2010 . . 24 ( ( 1 ): ): 94 - - 111 . . DOI: 10.1016/j.csl.2008.05.003 http://doi.org/10.1016/j.csl.2008.05.003 . .

S Behnke . . Discovering hierarchical speech features using convolutional non-negative matrix factorization . . Int Joint Conf on Neural Networks , , 2003 . . p.2758 - - 2763 . . DOI: 10.1109/IJCNN.2003.1224004 http://doi.org/10.1109/IJCNN.2003.1224004 . .

RWJ Bello . . Identifying repeated patterns in music using sparse convolutive non-negative matrix factorization . . 11 th Int Society for Music Information Retrieval Conf , , 2010 . . p.123 - - 128 . . . .

J Benesty , , , J Chen , , , Y Huang , , , 等 . . On microphone-array beamforming from a MIMO acoustic signal processing perspective . . IEEE Trans Audio Speech Lang Process , , 2007 . . 15 ( ( 3 ): ): 1053 - - 1065 . . DOI: 10.1109/TASL.2006.885251 http://doi.org/10.1109/TASL.2006.885251 . .

J Benesty , , , J Chen , , , Y Huang . . Automatic Speech Recognition: a Deep Learning Approach , , : : Springer Berlin Heidelberg, New York USA , , 2008 . . .

M Bi , , , Y Qian , , , K Yu . . Very deep convolutional neural networks for LVCSR . . 16 th Annual Conf of Int Speech Communication Association , , 2015 . . p.3259 - - 3263 . . . .

AS Bregman . . Auditory scene analysis. In: Smelzer NJ, Bates PB (Eds.), International Encyclopedia of the Social and Behavioral Science . . Elsevier, Amsterdam , , 1990 . . .

GJ Brown , , , M Cooke . . Computational auditory scene analysis . . Comput Speech Lang , , 1994 . . 8 ( ( 4 ): ): 297 - - 336 . . DOI: 10.1006/csla.1994.1016 http://doi.org/10.1006/csla.1994.1016 . .

J Capon . . High resolution frequency-wavenumber spectrum analysis . . Proc IEEE , , 1969 . . 57 1408 - - 1418 . . DOI: 10.1109/PROC.1969.7278 http://doi.org/10.1109/PROC.1969.7278 . .

GC Carter , , , AH Nuttall , , , PG Cable . . The smoothed coherence transform . . Proc IEEE , , 1973 . . 61 1497 - - 1498 . . DOI: 10.1109/PROC.1973.9300 http://doi.org/10.1109/PROC.1973.9300 . .

X Chang , , , Y Qian , , , D Yu . . Adaptive permutation invariant training with auxiliary information for monaural multi-talker speech recognition . . Int Conf on Acoustics, Speech, and Signal Processing, in press , , 2018 . . .

J Chen , , , J Benesty , , , Y Huang . . Time delay estimation in room acoustic environments: an overview . . EURASIP J Adv Signal Process , , 2006 . . 026503 DOI: 10.1155/ASP/2006/26503 http://doi.org/10.1155/ASP/2006/26503 . .

N Chen , , , Y Qian , , , K Yu . . Multi-task learning for text-dependent speaker verification . . Annual Conf of Int Speech Communication Association , , 2015 . . p.185 - - 189 . . . .

Z Chen . . Single Channel Auditory Source Separation with Neural Network . . PhD Thesis, Columbia University, New York, USA , , 2017 . . .

Z Chen , , , DP Ellis . . Speech enhancement by sparse, low-rank, and dictionary spectrogram decomposition . . Workshop on Applications of Signal Processing to Audio and Acoustics , , 2013 . . p.1 - - 4 . . DOI: 10.1109/WASPAA.2013.6701883 http://doi.org/10.1109/WASPAA.2013.6701883 . .

Z Chen , , , B McFee , , , DP Ellis . . Speech enhancement by low-rank and convolutive dictionary spectrogram decomposition . . Annual Conf of Int Speech Communication Association , , 2014 . . p.2833 - - 2837 . . . .

Z Chen , , , J Li , , , X Xiao , , , 等 . . Cracking the cocktail party problem by multi-beam deep attractor network . . IEEE Workshop on Automatic Speech Recognition and Understanding , , 2017a . . p.437 - - 444 . . . .

Z Chen , , , Y Luo , , , N Mesgarani . . Deep attractor network for single-microphone speaker separation . . Int Conf on Acoustics, Speech, and Signal Processing , , 2017b . . p.246 - - 250 . . DOI: 10.1109/ICASSP.2017.7952155 http://doi.org/10.1109/ICASSP.2017.7952155 . .

Z Chen , , , J Droppo , , , J Li , , , 等 . . Progressive joint modeling in unsupervised single-channel overlapped speech recognition . . http://arxiv.org/abs/1707.07048 http://arxiv.org/abs/1707.07048 , , 2017c . . .

EC Cherry . . Some experiments on the recognition of speech, with one and with two ears . . J Acoust Soc Am , , 1953 . . 25 ( ( 5 ): ): 975 - - 979 . . DOI: 10.1121/1.1907229 http://doi.org/10.1121/1.1907229 . .

M Cooke , , , JR Hershey , , , SJ Rennie . . Monaural speech separation and recognition challenge . . Comput Speech Lang , , 2010 . . 24 ( ( 1 ): ): 1 - - 15 . . DOI: 10.1016/j.csl.2009.02.006 http://doi.org/10.1016/j.csl.2009.02.006 . .

N Dehak , , , PJ Kenny , , , R Dehak , , , 等 . . Front-end factor analysis for speaker verification . . IEEE Trans Audio Speech Lang Process , , 2011 . . 19 ( ( 4 ): ): 788 - - 798 . . DOI: 10.1109/TASL.2010.2064307 http://doi.org/10.1109/TASL.2010.2064307 . .

S Doclo , , , M Moonen . . Design of far-field and near-field broadband beamformers using eigenfilters . . IEEE Signal Process Lett , , 2003 . . 83 ( ( 12 ): ): 2641 - - 2673 . . DOI: 10.1016/j.sigpro.2003.07.005 http://doi.org/10.1016/j.sigpro.2003.07.005 . .

L Drude , , , R Haeb-Umbach . . Tight integration of spatial and spectral features for BSS with deep clustering embeddings . . Annual Conf of Int Speech Communication Association , , 2017 . . p.2650 - - 2654 . . . .

J Du , , , Y Tu , , , Y Xu , , , 等 . . Speech separation of a target speaker based on deep neural networks . . Int Conf on Signal Processing , , 2014 . . p.473 - - 477 . . DOI: 10.1109/ICOSP.2014.7015050 http://doi.org/10.1109/ICOSP.2014.7015050 . .

DPW Ellis . . Prediction-Driven Computational Auditory Scene Analysis . . PhD Thesis, Massachusetts Institute of Technology, Cambridge, USA. , , 1996 . . .

Y Ephraim , , , D Malah . . Speech enhancement using a minimum mean-square error log-spectral amplitude estimator . . IEEE Trans Audio Speech Lang Process , , 1985 . . 33 ( ( 2 ): ): 443 - - 445 . . DOI: 10.1109/TASSP.1985.1164550 http://doi.org/10.1109/TASSP.1985.1164550 . .

H Erdogan , , , JR Hershey , , , S Watanabe , , , 等 . . Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks . . Int Conf on Acoustics Speech and Signal Processing , , 2015 . . p.708 - - 712 . . DOI: 10.1109/ICASSP.2015.7178061 http://doi.org/10.1109/ICASSP.2015.7178061 . .

H Erdogan , , , J Hershey , , , S Watanabe , , , 等 . . Improved MVDR beamforming using single-channel mask prediction networks . . Annual Conf of Int Speech Communication Association , , 2016 . . p.1981 - - 1985 . . . .

H Erdogan , , , JR Hershey , , , S Watanabe , , , 等 . . Deep recurrent networks for separation and recognition of single-channel speech in nonstationary background audio . . New Era for Robust Speech Recognition , , 2017 . . p.165 - - 186 . . DOI: 10.1007/978-3-319-64680-0\_7 http://doi.org/10.1007/978-3-319-64680-0\_7 . .

S Fischer , , , KU Simmer . . Beamforming microphone arrays for speech acquisition in noisy environments . . Speech Commun , , 1996 . . 20 ( ( 3-4 ): ): 215 - - 227 . . DOI: 10.1016/S0167-6393(96)00054-4 http://doi.org/10.1016/S0167-6393(96)00054-4 . .

OL Frost . . An algorithm for linearly constrained adaptive array processing . . Proc IEEE , , 1972 . . 60 ( ( 8 ): ): 926 - - 935 . . DOI: 10.1109/PROC.1972.8817 http://doi.org/10.1109/PROC.1972.8817 . .

S Gannot , , , D Burshtein , , , E Weinstein . . Signal enhancement using beamforming and nonstationarity with applications to speech . . IEEE Trans Signal Process , , 2001 . . 49 ( ( 8 ): ): 1614 - - 1626 . . DOI: 10.1109/78.934132 http://doi.org/10.1109/78.934132 . .

S Gannot , , , D Burshtein , , , E Weinstein . . Analysis of the power spectral deviation of the general transfer function GSC . . IEEE Trans Signal Process , , 2004 . . 52 ( ( 4 ): ): 1115 - - 1120 . . DOI: 10.1109/TSP.2004.823487 http://doi.org/10.1109/TSP.2004.823487 . .

Z Ghahramani , , , MI Jordan . . Factorial hidden Markov models . . NIPS , , 1996 . . p.472 - - 478 . . . .

JC Hassab , , , RE Boucher . . Performance of the generalized cross correlator in the presence of a strong spectral peak in the signal . . IEEE Trans Audio Speech Lang Process , , 1981 . . 29 ( ( 3 ): ): 549 - - 555 . . DOI: 10.1109/TASSP.1981.1163613 http://doi.org/10.1109/TASSP.1981.1163613 . .

JR Hershey , , , T Kristjansson , , , S Rennie , , , 等 . . Single channel speech separation using factorial dynamics . . NIPS , , 2007 . . p.593 - - 600 . . . .

JR Hershey , , , SJ Rennie , , , PA Olsen , , , 等 . . Super-human multi-talker speech recognition: a graphical modeling approach . . Comput Speech Lang , , 2010 . . 24 ( ( 1 ): ): 45 - - 66 . . DOI: 10.1016/j.csl.2008.11.001 http://doi.org/10.1016/j.csl.2008.11.001 . .

JR Hershey , , , Z Chen , , , J Le Roux , , , 等 . . Deep clustering: discriminative embeddings for segmentation and separation . . Int Conf on Acoustics Speech and Signal Processing , , 2016 . . p.31 - - 35 . . DOI: 10.1109/ICASSP.2016.7471631 http://doi.org/10.1109/ICASSP.2016.7471631 . .

J Heymanna , , , L Drudea , , , R Haeb-Umbacha . . A generic neural acoustic beamforming architecture for robust multi-channel speech processing . . Comput Speech Lang , , 2017 . . 46 374 - - 385 . . DOI: 10.1016/j.csl.2016.11.007 http://doi.org/10.1016/j.csl.2016.11.007 . .

G Hinton , , , L Deng , , , D Yu , , , 等 . . Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups . . IEEE Signal Process Mag , , 2012 . . 29 ( ( 6 ): ): 82 - - 97 . . DOI: 10.1109/MSP.2012.2205597 http://doi.org/10.1109/MSP.2012.2205597 . .

PO Hoyer . . Non-negative matrix factorization with sparseness constraints . . J Mach Learn Res , , 2004 . . 5 1457 - - 1469 . . . .

G Hu , , , D Wang . . Monaural speech segregation based on pitch tracking and amplitude modulation . . IEEE Trans Neur Netw , , 2004 . . 15 ( ( 5 ): ): 1135 - - 1150 . . DOI: 10.1109/TNN.2004.832812 http://doi.org/10.1109/TNN.2004.832812 . .

G Hu , , , D Wang . . Segregation of unvoiced speech from nonspeech interference . . J Acoust Soc Am , , 2008 . . 124 ( ( 2 ): ): 1306 - - 1319 . . DOI: 10.1121/1.2939132 http://doi.org/10.1121/1.2939132 . .

G Hu , , , D Wang . . A tandem algorithm for pitch estimation and voiced speech segregation . . IEEE Trans Audio Speech Lang Process , , 2010 . . 18 ( ( 8 ): ): 2067 - - 2079 . . DOI: 10.1109/TASL.2010.2041110 http://doi.org/10.1109/TASL.2010.2041110 . .

K Hu , , , D Wang . . An unsupervised approach to cochannel speech separation . . IEEE Trans Audio Speech Lang Process , , 2013 . . 21 ( ( 1 ): ): 122 - - 131 . . DOI: 10.1109/TASL.2012.2215591 http://doi.org/10.1109/TASL.2012.2215591 . .

Y Hu , , , PC Loizou . . Subjective comparison and evaluation of speech enhancement algorithms . . Speech Commun , , 2007 . . 49 ( ( 7 ): ): 588 - - 601 . . DOI: 10.1016/j.specom.2006.12.006 http://doi.org/10.1016/j.specom.2006.12.006 . .

Y Hu , , , PC Loizou . . Evaluation of objective quality measures for speech enhancement . . IEEE Trans Audio Speech Lang Process , , 2008 . . 16 ( ( 1 ): ): 229 - - 238 . . DOI: 10.1109/TASL.2007.911054 http://doi.org/10.1109/TASL.2007.911054 . .

Z Huang , , , S Wang , , , Y Qian . . Joint i -vector with end-to-end system for short duration text-independent speaker verification . . Int Conf on Acoustics, Speech, and Signal Processing, in press. , , 2018 . . .

A Hyvarinen , , , J Karhunen , , , E Oja . . Independent Component Analysis , , : : John Wiley & Sons, Inc, New York USA , , 2001 . . .

Y Isik , , , JL Roux , , , Z Chen , , , 等 . . Single-channel multi-speaker separation using deep clustering . . Annual Conf of Int Speech Communication Association , , 2016 . . p.545 - - 549 . . DOI: 10.21437/Interspeech.2016-1176 http://doi.org/10.21437/Interspeech.2016-1176 . .

W Kellermann . . Strategies for combining acoustic echo cancellation and adaptive beamforming microphone arrays . . Int Conf on Acoustics Speech and Signal Processing , , 1997 . . p.219 - - 222 . . DOI: 10.1109/ICASSP.1997.599608 http://doi.org/10.1109/ICASSP.1997.599608 . .

T Kim , , , HT Attias , , , SY Lee , , , 等 . . Blind source separation exploiting higher-order frequency dependencies . . IEEE Trans Audio Speech Lang Process , , 2006 . . 15 ( ( 4 ): ): 70 - - 79 . . DOI: 10.1109/TASL.2006.872618 http://doi.org/10.1109/TASL.2006.872618 . .

U Kjems , , , JB Boldt , , , MS Pedersen , , , 等 . . Role of mask pattern in intelligibility of ideal binary-masked noisy speech . . J Acoust Soc Am , , 2009 . . 126 ( ( 3 ): ): 1415 - - 1426 . . DOI: 10.1121/1.3179673 http://doi.org/10.1121/1.3179673 . .

CK Knapp , , , GC Carter . . The generalized correlation method for estimation of time delay . . IEEE Trans Audio Speech Lang Process , , 1976 . . 24 ( ( 4 ): ): 320 - - 327 . . DOI: 10.1109/TASSP.1976.1162830 http://doi.org/10.1109/TASSP.1976.1162830 . .

M Kolbk , , , D Yu , , , ZH Tan , , , 等 . . Joint separation and denoising of noisy multi-talker speech using recurrent neural networks and permutation invariant training . . IEEE Int Workshop on Machine Learning for Signal Processing. , , 2017a . . .

M Kolbk , , , D Yu , , , ZH Tan , , , 等 . . Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks . . IEEE Trans Audio Speech Lang Process , , 2017b . . 25 ( ( 10 ): ): 1901 - - 1913 . . DOI: 10.1109/TASLP.2017.2726762 http://doi.org/10.1109/TASLP.2017.2726762 . .

T Kristjansson , , , J Hershey , , , P Olsen , , , 等 . . Super-human multi-talker speech recognition: the IBM 2006 speech separation challenge system . . Int Conf on Spoken Language Processing, Paper 1775-Mon1WeS.7. , , 2006 . . .

PK Kuhl . . Human adults and human infants show a "perceptual magnet effect" for the prototypes of speech categories, monkeys do not . . Percept Psychol , , 1991 . . 50 ( ( 2 ): ): 93 - - 107 . . DOI: 10.3758/BF03212211 http://doi.org/10.3758/BF03212211 . .

A Larcher , , , KA Lee , , , B Ma , , , 等 . . Text-dependent speaker verification: classifiers, databases and RSR2015 . . Speech Commun , , 2014 . . 60 56 - - 77 . . DOI: 10.1016/j.specom.2014.03.001 http://doi.org/10.1016/j.specom.2014.03.001 . .

DD Lee , , , HS Seung . . Algorithms for non-negative matrix factorization . . NIPS , , 2001 . . p.556 - - 562 . . . .

TW Lee . . Independent Component Analysis-Theory and Applications . . Kluwer Academic Publishers, Boston, USA. , , 1998 . . .

Y Lei , , , N Scheffer , , , L Ferrer , , , 等 . . A novel scheme for speaker recognition using a phonetically-aware deep neural network . . Int Conf on Acoustics Speech and Signal Processing , , 2014 . . p.1695 - - 1699 . . DOI: 10.1109/ICASSP.2014.6853887 http://doi.org/10.1109/ICASSP.2014.6853887 . .

P Li , , , Y Guan , , , S Wang , , , 等 . . Monaural speech separation based on MAXVQ and CASA for robust speech recognition . . Comput Speech Lang , , 2010 . . 24 ( ( 1 ): ): 30 - - 44 . . DOI: 10.1016/j.csl.2008.05.005 http://doi.org/10.1016/j.csl.2008.05.005 . .

Y Liu , , , Y Qian , , , N Chen , , , 等 . . Deep feature for text-dependent speaker verification . . Speech Commun , , 2015 . . 73 1 - - 13 . . DOI: 10.1016/j.specom.2015.07.003 http://doi.org/10.1016/j.specom.2015.07.003 . .

JM Lovekin , , , RE Yantorno , , , KR Krishnamachari , , , 等 . . Developing usable speech criteria for speaker identification technology . . Int Conf on Acoustics Speech and Signal Processing , , 2001 . . p.421 - - 424 . . DOI: 10.1109/ICASSP.2001.940857 http://doi.org/10.1109/ICASSP.2001.940857 . .

MI Mandel , , , RJ Weiss , , , DPW Ellis . . Model-based expectation maximization source separation and localization . . IEEE Trans Audio Speech Lang Process , , 2010 . . 18 ( ( 2 ): ): 382 - - 394 . . DOI: 10.1109/TASL.2009.2029711 http://doi.org/10.1109/TASL.2009.2029711 . .

JH McDermott . . The cocktail party problem . . Curr Biol , , 2009 . . 19 ( ( 22 ): ): R1024 - - R1027 . . . .

N Mesgarani , , , EF Chang . . Selective cortical representation of attended speaker in multi-talker speech perception . . Nature , , 2012 . . 485 ( ( 7397 ): ): 233 - - 236 . . DOI: 10.1038/nature11020 http://doi.org/10.1038/nature11020 . .

P Mowlaee , , , R Saeidi , , , ZH Tan , , , 等 . . Joint single-channel speech separation and speaker identification . . Int Conf on Acoustics Speech and Signal Processing , , 2010 . . p.4430 - - 4433 . . DOI: 10.1109/ICASSP.2010.5495619 http://doi.org/10.1109/ICASSP.2010.5495619 . .

P Mowlaee , , , R Saeidi , , , MG Christensen , , , 等 . . A joint approach for single-channel speaker identification and speech separation . . IEEE Trans Audio Speech Lang Process , , 2012 . . 20 ( ( 9 ): ): 2586 - - 2601 . . DOI: 10.1109/TASL.2012.2208627 http://doi.org/10.1109/TASL.2012.2208627 . .

A Narayanan , , , D Wang . . Ideal ratio mask estimation using deep neural networks for robust speech recognition . . Int Conf on Acoustics Speech and Signal Processing , , 2013 . . p.7092 - - 7096 . . DOI: 10.1109/ICASSP.2013.6639038 http://doi.org/10.1109/ICASSP.2013.6639038 . .

N Ono . . Stable and fast update rules for independent vector analysis based on auxiliary function technique . . IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. , , 2011 . . DOI: 10.1109/ASPAA.2011.6082320 http://doi.org/10.1109/ASPAA.2011.6082320 . .

V Peddinti , , , D Povey , , , S Khudanpur . . A time delay neural network architecture for efficient modeling of long temporal contexts . . Annual Conf of Int Speech Communication Association , , 2015 . . p.3214 - - 3218 . . . .

MS Pedersen , , , J Larsen , , , U Kjems , , , 等 . . A Survey of Convolutive Blind Source Separation Methods , , : : Springer Press, New York USA. , , 2007 . . .

YM Qian , , , M Bi , , , T Tan , , , 等 . . Very deep convolutional neural networks for noise robust speech recognition . . IEEE Trans Audio Speech Lang Process , , 2016 . . 24 ( ( 12 ): ): 2263 - - 2276 . . DOI: 10.1109/TASLP.2016.2602884 http://doi.org/10.1109/TASLP.2016.2602884 . .

YM Qian , , , XK Chang , , , D Yu . . Single-channel multi-talker speech recognition with permutation invariant training . . http://arxiv.org/abs/1707.06527 http://arxiv.org/abs/1707.06527 , , 2017 . . .

YM Qian , , , T Tan , , , H Hu , , , 等 . . Noise robust speech recognition on Aurora4 by humans and machines . . Int Conf on Acoustics, Speech, and Signal Processing, in press. , , 2018 . . .

B Raj , , , T Virtanen , , , S Chaudhuri , , , 等 . . Non-negative matrix factorization based compensation of music for automatic speech recognition . . Annual Conf of Int Speech Communication Association , , 2010 . . p.717 - - 720 . . . .

SJ Rennie , , , JR Hershey , , , PA Olsen . . Single-channel multitalker speech recognition . . IEEE Signal Process Mag , , 2010 . . 27 ( ( 6 ): ): 66 - - 80 . . DOI: 10.1109/MSP.2010.938081 http://doi.org/10.1109/MSP.2010.938081 . .

DA Reynolds , , , TF Quatieri , , , RB Dunn . . Speaker verification using adapted gaussian mixture models . . Dig Signal Process , , 2000 . . 10 ( ( 1-3 ): ): 19 - - 41 . . DOI: 10.1006/dspr.1999.0361 http://doi.org/10.1006/dspr.1999.0361 . .

AW Rix , , , JG Beerends , , , MP Hollier , , , 等 . . Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs . . Int Conf on Acoustics, Speech, and Signal Processing , , 2001 . . p.749 - - 752 . . DOI: 10.1109/ICASSP.2001.941023 http://doi.org/10.1109/ICASSP.2001.941023 . .

P Roth . . Effective measurements using digital signal analysis . . IEEE Spectr , , 1971 . . 8 ( ( 4 ): ): 62 - - 70 . . . .

TN Sainath , , , A Mohamed , , , B Kingsbury , , , 等 . . Deep convolutional neural networks for LVCSR . . Int Conf on Acoustics Speech and Signal Processing , , 2013 . . p.8614 - - 8618 . . . .

TN Sainath , , , O Vinyals , , , A Senior , , , 等 . . Convolutional, long short-term memory, fully connected deep neural networks . . Int Conf on Acoustics Speech and Signal Processing , , 2015 . . p.4580 - - 4584 . . DOI: 10.1109/ICASSP.2015.7178838 http://doi.org/10.1109/ICASSP.2015.7178838 . .

H Sawada , , , S Araki , , , S Makino . . A two-stage frequency-domain blind source separation method for underdetermined convolutive mixtures . . IEEE Workshop on Applications of Signal Processing to Audio and Acoustics , , 2007 . . p.139 - - 142 . . DOI: 10.1109/ASPAA.2007.4393012 http://doi.org/10.1109/ASPAA.2007.4393012 . .

MN Schmidt , , , RK Olsson . . Single-channel speech separation using sparse non-negative matrix factorization . . Annual Conf of Int Speech Communication Association, Paper 1652-Thu2FoP.10. , , 2006 . . .

B Schuller , , , F Weninger , , , M W llmer , , , 等 . . Non-negative matrix factorization as noise-robust feature extractor for speech recognition . . Int Conf on Acoustics Speech and Signal Processing , , 2010 . . p.4562 - - 4565 . . DOI: 10.1109/ICASSP.2010.5495567 http://doi.org/10.1109/ICASSP.2010.5495567 . .

T Sercu , , , C Puhrsch , , , B Kingsbury , , , 等 . . Very deep multilingual convolutional neural networks for LVCSR . . Int Conf on Acoustics Speech and Signal Processing , , 2016 . . p.4955 - - 4959 . . DOI: 10.1109/ICASSP.2016.7472620 http://doi.org/10.1109/ICASSP.2016.7472620 . .

Y Shao , , , D Wang . . Co-channel speaker identification using usable speech extraction based on multi-pitch tracking . . Int Conf on Acoustics, Speech, and Signal Processing , , 2003 . . p.205 - - 208 . . DOI: 10.1109/ICASSP.2003.1202330 http://doi.org/10.1109/ICASSP.2003.1202330 . .

Y Shao , , , D Wang . . Model-based sequential organization in cochannel speech . . IEEE Trans Audio Speech Lang Process , , 2006 . . 14 ( ( 1 ): ): 289 - - 298 . . DOI: 10.1109/TSA.2005.854106 http://doi.org/10.1109/TSA.2005.854106 . .

M Souden , , , J Benesty , , , S Affes . . On optimal frequency-domain multichannel linear filtering for noise reduction . . IEEE Trans Signal Process , , 2010 . . 18 ( ( 2 ): ): 260 - - 276 . . DOI: 10.1109/TASL.2009.2025790 http://doi.org/10.1109/TASL.2009.2025790 . .

M Souden , , , S Araki , , , K Kinoshita , , , 等 . . A multichannel MMSE-based framework for speech source separation and noise reduction . . IEEE Trans Signal Process , , 2013 . . 21 ( ( 9 ): ): 1913 - - 1928 . . DOI: 10.1109/TASL.2013.2263137 http://doi.org/10.1109/TASL.2013.2263137 . .

C Sydow . . Broadband beamforming for a microphone array . . J Acoust Soc Am , , 1994 . . 96 ( ( 8 ): ): 845 - - 849 . . DOI: 10.1121/1.410323 http://doi.org/10.1121/1.410323 . .

CH Taal , , , RC Hendriks , , , R Heusdens , , , 等 . . A short-time objective intelligibility measure for time-frequency weighted noisy speech . . Int Conf on Acoustics Speech and Signal Processing , , 2010 . . p.4214 - - 4217 . . DOI: 10.1109/ICASSP.2010.5495701 http://doi.org/10.1109/ICASSP.2010.5495701 . .

T Tan , , , Y Qian , , , D Yu . . Knowledge transfer in permutation invariant training for single-channel multi-talker speech recognition . . Int Conf on Acoustics, Speech, and Signal Processing, in press. , , 2018 . . .

Y Tu , , , J Du , , , Y Xu , , , 等 . . Deep neural network based speech separation for robust speech recognition . . Int Conf on Signal Processing , , 2014a . . p.532 - - 536 . . DOI: 10.1109/ICOSP.2014.7015061 http://doi.org/10.1109/ICOSP.2014.7015061 . .

Y Tu , , , J Du , , , Y Xu , , , 等 . . Speech separation based on improved deep neural networks with dual outputs of speech features for both target and interfering speakers . . Int Symp on Chinese Spoken Language Processing , , 2014b . . p.250 - - 254 . . DOI: 10.1109/ISCSLP.2014.6936615 http://doi.org/10.1109/ISCSLP.2014.6936615 . .

E Variani , , , X Lei , , , E McDermott , , , 等 . . Deep neural networks for small footprint text-dependent speaker verification . . Int Conf on Acoustics Speech and Signal Processing , , 2014 . . p.4052 - - 4056 . . DOI: 10.1109/ICASSP.2014.6854363 http://doi.org/10.1109/ICASSP.2014.6854363 . .

E Vincent , , , R Gribonval , , , C F votte . . Performance measurement in blind audio source separation . . IEEE Trans Audio Speech Lang Process , , 2006 . . 14 ( ( 4 ): ): 1462 - - 1469 . . DOI: 10.1109/TSA.2005.858005 http://doi.org/10.1109/TSA.2005.858005 . .

T Virtanen . . Speech recognition using factorial hidden Markov models for separation in the feature space . . Annual Conf of Int Speech Communication Association, Paper 1850-Mon1WeS.5. , , 2006 . . .

T Virtanen . . Monaural sound source separation by nonnegative matrix factorization with temporal continuity and sparseness criteria . . IEEE Trans Audio Speech Lang Process , , 2007 . . 15 ( ( 3 ): ): 1066 - - 1074 . . DOI: 10.1109/TASL.2006.885253 http://doi.org/10.1109/TASL.2006.885253 . .

D Wang . . On ideal binary mask as the computational goal of auditory scene analysis . . In: Divenyi P (Ed.), Speech Separation by Humans and Machines. Springer, Boston, USA , , 2005 . . p.181 - - 197 . . DOI: 10.1007/0-387-22794-6\_12 http://doi.org/10.1007/0-387-22794-6\_12 . .

D Wang , , , GJ Brown . . Computational Auditory Scene Analysis: Principles, Algorithms, and Applications , , : : New York, USA Wiley-IEEE Press , , 2006 . . .

S Wang , , , Y Qian , , , K Yu . . Focal KL-divergence based dilated convolutional neural networks for co-channel speaker identification . . Int Conf on Acoustics, Speech, and Signal Processing, in press. , , 2018 . . .

Y Wang , , , A Narayanan , , , D Wang . . On training targets for supervised speech separation . . IEEE Trans Audio Speech Lang Process , , 2014 . . 22 ( ( 12 ): ): 1849 - - 1858 . . DOI: 10.1109/TASLP.2014.2352935 http://doi.org/10.1109/TASLP.2014.2352935 . .

C Weng , , , D Yu , , , ML Seltzer , , , 等 . . Deep neural networks for single-channel multi-talker speech recognition . . IEEE Trans Audio Speech Lang Process , , 2015 . . 23 ( ( 10 ): ): 1670 - - 1679 . . DOI: 10.1109/TASLP.2015.2444659 http://doi.org/10.1109/TASLP.2015.2444659 . .

F Weninger , , , H Erdogan , , , S Watanabe , , , 等 . . Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR . . Int Conf on Latent Variable Analysis and Signal Separation , , 2015 . . p.91 - - 99 . . DOI: 10.1007/978-3-319-22482-4\_11 http://doi.org/10.1007/978-3-319-22482-4\_11 . .

X Xiao , , , SK Zhao , , , DL Jones , , , 等 . . On time-frequency mask estimation for MVDR beamforming with application in robust speech recognition . . Int Conf on Acoustics Speech and Signal Processing , , 2017 . . p.3246 - - 3250 . . DOI: 10.1109/ICASSP.2017.7952756 http://doi.org/10.1109/ICASSP.2017.7952756 . .

W Xiong , , , J Droppo , , , X Huang , , , 等 . . Achieving human parity in conversational speech recognition . . http://arxiv.org/abs/1610.05256 http://arxiv.org/abs/1610.05256 , , 2016 . . .

Y Xu , , , J Du , , , LR Dai , , , 等 . . An experimental study on speech enhancement based on deep neural networks . . IEEE Signal Process Lett , , 2014 . . 21 ( ( 1 ): ): 65 - - 68 . . DOI: 10.1109/LSP.2013.2291240 http://doi.org/10.1109/LSP.2013.2291240 . .

O Yilmaz , , , S Rickard . . Blind separation of speech mixtures via time-frequency masking . . IEEE Trans Signal Process , , 2004 . . 52 ( ( 7 ): ): 1830 - - 1847 . . DOI: 10.1109/TSP.2004.828896 http://doi.org/10.1109/TSP.2004.828896 . .

D Yu , , , L Deng . . Automatic Speech Recognition: a Deep Learning Approach , , : : New York.USA Springer , , 2014 . . .

D Yu , , , JY Li . . Recent progresses in deep learning based acoustic models . . IEEE/CAA J Automat Sin , , 2017 . . 4 ( ( 3 ): ): 396 - - 409 . . DOI: 10.1109/JAS.2017.7510508 http://doi.org/10.1109/JAS.2017.7510508 . .

D Yu , , , W Xiong , , , J Droppo , , , 等 . . Deep convolutional neural networks with layer-wise context expansion and attention . . Annual Conf of Int Speech Communication Association , , 2016 . . p.17 - - 21 . . DOI: 10.21437/Interspeech.2016-251 http://doi.org/10.21437/Interspeech.2016-251 . .

D Yu , , , X Chang , , , Y Qian . . Recognizing multi-talker speech with permutation invariant training . . Annual Conf of Int Speech Communication Association , , 2017a . . p.2456 - - 2460 . . . .

D Yu , , , M Kolb k , , , ZH Tan , , , 等 . . Permutation invariant training of deep models for speaker-independent multi-talker speech separation . . Int Conf on Acoustics, Speech and Signal Processing , , 2017b . . p.241 - - 245 . . DOI: 10.1109/ICASSP.2017.7952154 http://doi.org/10.1109/ICASSP.2017.7952154 . .

F Yu , , , V Koltun . . Multi-scale context aggregation by dilated convolutions . . http://arxiv.org/abs/1511.07122 http://arxiv.org/abs/1511.07122 , , 2015 . . .

C Zhang , , , K Koishida . . End-to-end text-independent speaker verification with triplet loss on short utterances . . Annual Conf of Int Speech Communication Association , , 2017 . . p.1487 - - 1491 . . DOI: 10.21437/Interspeech.2017-1608 http://doi.org/10.21437/Interspeech.2017-1608 . .

L Zhang , , , Z Chen , , , M Zheng , , , 等 . . Robust non-negative matrix factorization . . Front. Electr. Electron. Eng. China , , 2011 . . 6 ( ( 2 ): ): 192 - - 200 . . DOI: 10.1007/s11460-011-0128-0 http://doi.org/10.1007/s11460-011-0128-0 . .

X Zhao , , , Y Wang , , , D Wang . . Cochannel speaker identification in anechoic and reverberant conditions . . IEEE Trans Audio Speech Lang Process , , 2015a . . 23 ( ( 11 ): ): 1727 - - 1736 . . DOI: 10.1109/TASLP.2015.2447284 http://doi.org/10.1109/TASLP.2015.2447284 . .

X Zhao , , , Y Wang , , , D Wang . . Deep neural networks for cochannel speaker identification . . Int Conf on Acoustics, Speech and Signal Processing , , 2015b . . p.4824 - - 4828 . . DOI: 10.1109/ICASSP.2015.7178887 http://doi.org/10.1109/ICASSP.2015.7178887 . .

Y Zhou , , , Y Qian . . Robust mask estimation by integrating neural network-based and clustering-based approaches for adaptive acoustic beamforming . . Int Conf on Acoustics, Speech, and Signal Processing, in press , , 2018 . . .

K Zmolikova , , , M Delcroix , , , K Kinoshita , , , 等 . . Speaker-aware neural network based beamformer for speaker extraction in speech mixtures . . Annual Conf of Int Speech Communication Association , , 2017 . . p.2655 - - 2659 . . DOI: 10.21437/Interspeech.2017-667 http://doi.org/10.21437/Interspeech.2017-667 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰