基于加权因子自动编码器和潜在特定源生成因子学习的单通道语音分离

陈静静; 毛启容; 秦友才; 钱双庆; 郑志燊

doi:10.1631/FITEE.2000019

Your Location：

Home >

Browse articles >

基于加权因子自动编码器和潜在特定源生成因子学习的单通道语音分离

常规文章 | Updated：2022-05-19

- 基于加权因子自动编码器和潜在特定源生成因子学习的单通道语音分离
- Latent source-specific generative factor learning for monaural speech separation using weighted-factor autoencoder
- 信息与电子工程前沿（英文） 2020年21卷第11期页码：1639-1650
- Affiliations：
  
  School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang 212013, China
  Jiangsu Key Laboratory of Security Technology for Industrial Cyberspace, Zhenjiang 212013, China
- Author bio：
  
  [ "Jing-jing CHEN, E-mail: 2221808071@stmail.ujs.edu.cn" ]
  Qi-rong MAO, E-mail: mao_qr@ujs.edu.cn
  [ "You-cai QIN, E-mail: 2211908026@stmail.ujs.edu.cn" ]
  [ "Shuang-qing QIAN, E-mail: 2211908025@stmail.ujs.edu.cn" ]
  [ "Zhi-shen ZHENG, E-mail: 3160602062@stmail.ujs.edu.cn" ]
- Funds：
  
  Project supported by the Key Project of the National Natural Science Foundation of China (No. U1836220), the National Natural Science Foundation of China (No. 61672267), the Qing Lan Talent Program of Jiangsu Province, China, and the Key Innovation Project of Undergraduate Students in Jiangsu Province, China (No. 201810299045Z)
- DOI：10.1631/FITEE.2000019
  中图分类号： TN912.3
- 收稿：2020-01-13，
  
  修回：2020-;9-;8，
  
  纸质出版：2020-11
- Accepted：
Scan QR Code
陈静静, 毛启容, 秦友才, 等. 基于加权因子自动编码器和潜在特定源生成因子学习的单通道语音分离[J]. 信息与电子工程前沿（英文）, 2020,21(11):1639-1650.

Jing-jing CHEN, Qi-rong MAO, You-cai QIN, et al. Latent source-specific generative factor learning for monaural speech separation using weighted-factor autoencoder[J]. Frontiers of Information Technology & Electronic Engineering, 2020, 21(11): 1639-1650.
陈静静, 毛启容, 秦友才, 等. 基于加权因子自动编码器和潜在特定源生成因子学习的单通道语音分离[J]. 信息与电子工程前沿（英文）, 2020,21(11):1639-1650. DOI： 10.1631/FITEE.2000019.

Jing-jing CHEN, Qi-rong MAO, You-cai QIN, et al. Latent source-specific generative factor learning for monaural speech separation using weighted-factor autoencoder[J]. Frontiers of Information Technology & Electronic Engineering, 2020, 21(11): 1639-1650. DOI： 10.1631/FITEE.2000019.

摘要

通过一系列基于自动编码器的深度学习网络结构，单通道语音分离方法最近取得诸多进展，其使用编码器将输入信号压缩为中间特征，再把这些特征送入解码器重构感兴趣的特定音频源。然而，这些方法既无法为单通道语音分离学习原始输入的生成因子，也无法构造混合语音中的所有音频源。本文提出一个新的加权因子自动编码器模型，在目标函数中引入正则化损失以约束目标源，摒除其他信号源。通过在分离层中引入潜在注意力机制和监督源构造器，加权因子自动编码器可为每一个信号源习得特定于源的生成因子和一组鉴别性特征，从而提升单通道语音分离性能。在基准数据集上的实验表明所提方法优于现有方法。就3个重要指标而言，加权因子自动编码器在相对更具挑战性的任务（与说话人无关的单通道语音分离）上取得巨大成功。

Abstract

Much recent progress in monaural speech separation (MSS) has been achieved through a series of deep learning architectures based on autoencoders

which use an encoder to condense the input signal into compressed features and then feed these features into a decoder to construct a specific audio source of interest. However

these approaches can neither learn generative factors of the original input for MSS nor construct each audio source in mixed speech. In this study

we propose a novel weighted-factor autoencoder (WFAE) model for MSS

which introduces a regularization loss in the objective function to isolate one source without containing other sources. By incorporating a latent attention mechanism and a supervised source constructor in the separation layer

WFAE can learn source-specific generative factors and a set of discriminative features for each source

leading to MSS performance improvement. Experiments on benchmark datasets show that our approach outperforms the existing methods. In terms of three important metrics

WFAE has great success on a relatively challenging MSS case

i.e.

speaker-independent MSS.

关键词

Keywords

references

S Araki , , , H Sawada , , , R Mukai , , , 等 . . Underdetermined blind sparse source separation for arbitrarily arranged multiple sensors . . Signal Process , , 2007 . . 87 ( ( 8 ): ): 1833 - - 1847 . . DOI: 10.1016/j.sigpro.2007.02.003 http://doi.org/10.1016/j.sigpro.2007.02.003 . .

J Benesty , , , JD Chen , , , YT Huang . . Microphone Array Signal Processing , , : : Berlin, Germany Springer , , 2008 . . .

AS Bregman . . Auditory Scene Analysis: the Perceptual Organization of Sound , , : : Cambridge The MIT Press , , 1990 . . .

GJ Brown , , , M Cooke . . Computational auditory scene analysis . . Comput Speech Lang , , 1994 . . 8 ( ( 4 ): ): 297 - - 336 . . DOI: 10.1006/csla.1994.1016 http://doi.org/10.1006/csla.1994.1016 . .

Z Chen , , , Y Luo , , , N Mesgarani . . Deep attractor network for single-microphone speaker separation . . Proc IEEE Int Conf on Acoustics, Speech and Signal Processing , , 2017 . . p.246 - - 250 . . DOI: 10.1109/ICASSP.2017.7952155 http://doi.org/10.1109/ICASSP.2017.7952155 . .

H Erdogan , , , JR Hershey , , , S Watanabe , , , 等 . . Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks . . Proc IEEE Int Conf on Acoustics, Speech and Signal Processing , , 2015 . . p.708 - - 712 . . DOI: 10.1109/ICASSP.2015.7178061 http://doi.org/10.1109/ICASSP.2015.7178061 . .

JS Garofolo , , , LF Lamel , , , WM Fisher , , , 等 . . DARPA TIMIT Acoustic-Phonetic Continous Speech Corpus CD-ROM. NIST Speech Disc 1-1.1 . . NASA STI/Recon Technical Report, NASA, USA , , 1993 . . .

Z Ghahramani , , , MI Jordan . . Factorial hidden Markov models . . Mach Learn , , 1997 . . 29 ( ( 2-3 ): ): 245 - - 273 . . DOI: 10.1023/A:1007425814087 http://doi.org/10.1023/A:1007425814087 . .

JP Gou , , , Z Yi , , , D Zhang , , , 等 . . Sparsity and geometry preserving graph embedding for dimensionality reduction . . IEEE Access , , 2018 . . 6 75748 - - 75766 . . DOI: 10.1109/ACCESS.2018.2884027 http://doi.org/10.1109/ACCESS.2018.2884027 . .

EM Grais , , , MD Plumbley . . Single channel audio source separation using convolutional denoising autoencoders . . Proc IEEE Global Conf on Signal and Information Processing , , 2017 . . p.1265 - - 1269 . . DOI: 10.1109/GlobalSIP.2017.8309164 http://doi.org/10.1109/GlobalSIP.2017.8309164 . .

JR Hershey , , , Z Chen , , , J Le Roux , , , 等 . . Deep clustering: discriminative embeddings for segmentation and separation . . Proc IEEE Int Conf on Acoustics, Speech and Signal Processing , , 2016 . . p.31 - - 35 . . DOI: 10.1109/ICASSP.2016.7471631 http://doi.org/10.1109/ICASSP.2016.7471631 . .

WN Hsu , , , Y Zhang , , , J Glass . . Learning latent representations for speech generation and transformation. 18 th . . Annual Conf of the Int Speech Communication Association , , 2017 . . p.1273 - - 1277 . . . .

K Hu , , , DL Wang . . An unsupervised approach to cochannel speech separation . . IEEE Trans Audio Speech Lang Process , , 2013 . . 21 ( ( 1 ): ): 122 - - 131 . . DOI: 10.1109/TASL.2012.2215591 http://doi.org/10.1109/TASL.2012.2215591 . .

PS Huang , , , M Kim , , , M Hasegawa-Johnson , , , 等 . . Deep learning for monaural speech separation . . Proc IEEE Int Conf on Acoustics, Speech and Signal Processing , , 2014 . . p.1562 - - 1566 . . DOI: 10.1109/ICASSP.2014.6853860 http://doi.org/10.1109/ICASSP.2014.6853860 . .

A Hyvrinen , , , E Oja . . Independent component analysis: algorithms and applications . . Neur Netw , , 2000 . . 13 ( ( 4-5 ): ): 411 - - 430 . . DOI: 10.1016/S0893-6080(00)00026-5 http://doi.org/10.1016/S0893-6080(00)00026-5 . .

E Karamatl , , , AT Cemgil , , , S Krbz . . Weak label supervision for monaural source separation using non-negative denoising variational autoencoders . . Proc 27 th Signal Processing and Communications Applications Conf , , 2019 . . p.1 - - 4 . . DOI: 10.1109/SIU.2019.8806536 http://doi.org/10.1109/SIU.2019.8806536 . .

M Kolbk , , , D Yu , , , ZH Tan , , , 等 . . Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks . . IEEE/ACM Trans Audio Speech Lang Process , , 2017 . . 25 ( ( 10 ): ): 1901 - - 1913 . . DOI: 10.1109/TASLP.2017.2726762 http://doi.org/10.1109/TASLP.2017.2726762 . .

Y Luo , , , N Mesgarani . . Conv-TasNet: surpassing ideal time-frequency magnitude masking for speech separation . . IEEE/ACM Trans Audio Speech Lang Process , , 2019 . . 27 ( ( 8 ): ): 1256 - - 1266 . . DOI: 10.1109/TASLP.2019.2915167 http://doi.org/10.1109/TASLP.2019.2915167 . .

Y Luo , , , Z Chen , , , T Yoshioka . . Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation , , https://arxiv.org/abs/1910.06379 https://arxiv.org/abs/1910.06379 , , 2019 . . .

A Nadas , , , D Nahamoo , , , MA Picheny . . Speech recognition using noise-adaptive prototypes . . IEEE Trans Acoust Speech Signal Process , , 1989 . . 37 ( ( 10 ): ): 1495 - - 1503 . . DOI: 10.1109/29.35387 http://doi.org/10.1109/29.35387 . .

K Osako , , , Y Mitsufuji , , , R Singh , , , 等 . . Supervised monaural source separation based on autoencoders . . Proc IEEE Int Conf on Acoustics, Speech and Signal Processing , , 2017 . . p.11 - - 15 . . DOI: 10.1109/ICASSP.2017.7951788 http://doi.org/10.1109/ICASSP.2017.7951788 . .

V Panayotov , , , GG Chen , , , D Povey , , , 等 . . LibriSpeech: an ASR corpus based on public domain audio books . . Proc IEEE Int Conf on Acoustics, Speech and Signal Processing , , 2015 . . p.5206 - - 5210 . . DOI: 10.1109/ICASSP.2015.7178964 http://doi.org/10.1109/ICASSP.2015.7178964 . .

L Pandey , , , A Kumar , , , V Namboodiri . . Monaural audio source separation using variational autoencoders . . Proc Interspeech , , 2018 . . p.3489 - - 3493 . . DOI: 10.21437/Interspeech.2018-1140 http://doi.org/10.21437/Interspeech.2018-1140 . .

YM Qian , , , C Weng , , , XK Chang , , , 等 . . Past review, current progress, and challenges ahead on the cocktail party problem . . Front Inform Technol Electron Eng , , 2018 . . 19 ( ( 1 ): ): 40 - - 63 . . DOI: 10.1631/FITEE.1700814 http://doi.org/10.1631/FITEE.1700814 . .

A Radford , , , L Metz , , , S Chintala . . Unsupervised representation learning with deep convolutional generative adversarial networks , , https://arxiv.org/abs/1511.06434 https://arxiv.org/abs/1511.06434 , , 2016 . . .

ST Roweis . . One microphone source separation . . Proc 13 th Int Conf on Neural Information Processing Systems , , 2001 . . p.793 - - 799 . . . .

MN Schmidt , , , RK Olsson . . Single-channel speech separation using sparse non-negative matrix factorization . . Proc 9 th Int Conf on Spoken Language Processing , , 2006 . . .

P Smaragdis . . Convolutive speech bases and their application to supervised speech separation . . IEEE Trans Audio Speech Lang Process , , 2007 . . 15 ( ( 1 ): ): 1 - - 12 . . DOI: 10.1109/TASL.2006.876726 http://doi.org/10.1109/TASL.2006.876726 . .

Maaten L van der , , , G Hinton . . Visualizing data using t-SNE . . J Mach Learn Res , , 2008 . . 9 ( ( 11 ): ): 2579 - - 2605 . . . .

E Vincent , , , R Gribonval , , , C Fevotte . . Performance measurement in blind audio source separation . . IEEE Trans Audio Speech Lang Process , , 2006 . . 14 ( ( 4 ): ): 1462 - - 1469 . . DOI: 10.1109/TSA.2005.858005 http://doi.org/10.1109/TSA.2005.858005 . .

DL Wang , , , GJ Brown . . Computational Auditory Scene Analysis: Principles, Algorithms, and Applications , , : : Hoboken, USA Wiley-IEEE Press , , 2006 . . .

YN Wang , , , J Du , , , LR Dai , , , 等 . . Unsupervised single-channel speech separation via deep neural network for different gender mixtures . . Asia-Pacific Signal and Information Processing Association Annual Summit and Conf , , 2016 . . p.1 - - 4 . . DOI: 10.1109/APSIPA.2016.7820736 http://doi.org/10.1109/APSIPA.2016.7820736 . .

YX Wang , , , A Narayanan , , , DL Wang . . On training targets for supervised speech separation . . IEEE/ACM Trans Audio Speech Lang Process , , 2014 . . 22 ( ( 12 ): ): 1849 - - 1858 . . DOI: 10.1109/TASLP.2014.2352935 http://doi.org/10.1109/TASLP.2014.2352935 . .

DS Williamson . . Monaural speech separation using a phase-aware deep denoising auto encoder . . Proc IEEE 28 th Int Workshop on Machine Learning for Signal Processing , , 2018 . . p.1 - - 6 . . DOI: 10.1109/MLSP.2018.8516918 http://doi.org/10.1109/MLSP.2018.8516918 . .

LM Xia , , , H Wang , , , WT Guo . . Gait recognition based on Wasserstein generating adversarial image inpainting network . . J Cent South Univ , , 2019 . . 26 ( ( 10 ): ): 2759 - - 2770 . . DOI: 10.1007/s11771-019-4211-7 http://doi.org/10.1007/s11771-019-4211-7 . .

D Yu , , , M Kolbk , , , ZH Tan , , , 等 . . Permutation invariant training of deep models for speaker-independent multi-talker speech separation . . Proc IEEE Int Conf on Acoustics, Speech and Signal Processing , , 2017 . . p.241 - - 245 . . DOI: 10.1109/ICASSP.2017.7952154 http://doi.org/10.1109/ICASSP.2017.7952154 . .

QJ Zhang , , , L Zhang . . Convolutional adaptive denoising autoencoders for hierarchical feature extraction . . Front Comput Sci , , 2018 . . 12 ( ( 6 ): ): 1140 - - 1148 . . DOI: 10.1007/s11704-016-6107-0 http://doi.org/10.1007/s11704-016-6107-0 . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

MltAuxTSPP: a unified benchmark for deep learning-based traffic state prediction with multi-source auxiliary data

Q-space-coordinate-guided neural networks for high-fidelity diffusion tensor estimation from minimal diffusion-weighted images

CRGT-SA: an interlaced and spatiotemporal deep learning model for network intrusion detection

Federated model with contrastive learning and adaptive control variates for human activity recognition

A comprehensive survey of physical adversarial vulnerabilities in autonomous driving systems