Ensemble enhanced active learning mixture discriminant analysis model and its application for semi-supervised fault classification

Weijun WANG; Yun WANG; Jun WANG; Xinyun FANG; Yuchen HE

doi:10.1631/FITEE.2200053

Your Location：

Home >

Browse articles >

Ensemble enhanced active learning mixture discriminant analysis model and its application for semi-supervised fault classification

Regular Papers | Updated：2022-12-16

- Ensemble enhanced active learning mixture discriminant analysis model and its application for semi-supervised fault classification
- 集成增强主动学习混合判别分析模型及其在半监督故障分类中的应用
- Frontiers of Information Technology & Electronic Engineering Vol. 23, Issue 12, Pages: 1814-1827(2022)
- Affiliations：
  
  1.Key Laboratory of Intelligent Manufacturing Quality Big Data Tracing and Analysis of Zhejiang Province, China Jiliang University, Hangzhou 310018, China
  2.Mechanical and Electrical Engineering Department, Zhejiang Tongji Vocational College of Science and Technology, Hangzhou 311231, China
  3.Suzhou Institute of Metrology, Suzhou 215004, China
- Author bio：
  
  E-mail: S20010811027@cjlu.edu.cn;
  wy@zjtongji.edu.cn;
  blackknight@cjlu.edu.cn;
  fangxy@szjl.com.cn;
  ‡Corresponding authors
- Funds：
- DOI：10.1631/FITEE.2200053
  CLC： TP277
- Received：13 February 2022，
  
  Accepted：09 May 2022，
  
  Published Online：26 July 2022，
  
  Published：2022-12
- Accepted：
Scan QR Code
Weijun WANG, Yun WANG, Jun WANG, et al. Ensemble enhanced active learning mixture discriminant analysis model and its application for semi-supervised fault classification[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(12): 1814-1827.
DOI：

Weijun WANG, Yun WANG, Jun WANG, et al. Ensemble enhanced active learning mixture discriminant analysis model and its application for semi-supervised fault classification[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(12): 1814-1827. DOI： 10.1631/FITEE.2200053.

摘要

Abstract

As an indispensable part of process monitoring

the performance of fault classification relies heavily on the sufficiency of process knowledge. However

data labels are always difficult to acquire because of the limited sampling condition or expensive laboratory analysis

which may lead to deterioration of classification performance. To handle this dilemma

a new semi-supervised fault classification strategy is performed in which enhanced active learning is employed to evaluate the value of each unlabeled sample with respect to a specific labeled dataset. Unlabeled samples with large values will serve as supplementary information for the training dataset. In addition

we introduce several reasonable indexes and criteria

and thus human labeling interference is greatly reduced. Finally

the fault classification effectiveness of the proposed method is evaluated using a numerical example and the Tennessee Eastman process.

关键词

Keywords

references

Abellán J , Masegosa AR , 2010 . Bagging decision trees on data sets with classification noise . Proc 6 th Int Symp on Foundations of Information and Knowledge Systems , p. 248 - 265 . doi: 10.1007/978-3-642-11829-6_17 http://doi.org/10.1007/978-3-642-11829-6_17

Abramson N , Braverman D , Sebestyen G , 1963 . Pattern recognition and machine learning . IEEE Trans Inform Theory , 9 ( 4 ): 257 - 261 . doi: 10.1109/TIT.1963.1057854 http://doi.org/10.1109/TIT.1963.1057854

Araya DB , Grolinger K , ElYamany HF , et al. , 2017 . An ensemble learning framework for anomaly detection in building energy consumption . Energy Build , 144 : 191 - 206 . doi: 10.1016/j.enbuild.2017.02.058 http://doi.org/10.1016/j.enbuild.2017.02.058

Blum A , Chawla S , 2001 . Learning from labeled and unlabeled data using graph mincuts . Proc 18 th Int Conf on Machine Learning , p. 19 - 26 .

Botre C , Mansouri M , Karim MN , et al. , 2017 . Multiscale PLS-based GLRT for fault detection of chemical processes . J Loss Prev Process Ind , 46 : 143 - 153 . doi: 10.1016/j.jlp.2017.01.008 http://doi.org/10.1016/j.jlp.2017.01.008

Bouveyron C , Girard S , 2009 . Robust supervised classification with mixture models: learning from data with uncertain labels . Patt Recogn , 42 ( 11 ): 2649 - 2658 . doi: 10.1016/j.patcog.2009.03.027 http://doi.org/10.1016/j.patcog.2009.03.027

Chapelle O , Sindhwani V , Sathiya Keerthi S , 2006 . Branch and bound for semi-supervised support vector machines . Proc 19 th Int Conf on Neural Information Processing Systems , p. 217 - 224 . doi: 10.5555/2976456.2976484 http://doi.org/10.5555/2976456.2976484

Chen X , Wang ZP , Zhang Z , et al. , 2018 . A semi-supervised approach to bearing fault diagnosis under variable conditions towards imbalanced unlabeled data . Sensors , 18 ( 7 ): 2097 . doi: 10.3390/s18072097 http://doi.org/10.3390/s18072097

Chiang LH , Russell EL , Braatz RD , 2000 . Fault diagnosis in chemical processes using Fisher discriminant analysis, discriminant partial least squares, and principal component analysis . Chemom Intell Lab Syst , 50 ( 2 ): 243 - 252 . doi: 10.1016/S0169-7439(99)00061-1 http://doi.org/10.1016/S0169-7439(99)00061-1

Chiang LH , Kotanchek ME , Kordon AK , 2004 . Fault diagnosis based on Fisher discriminant analysis and support vector machines . Comput Chem Eng , 28 ( 8 ): 1389 - 1401 . doi: 10.1016/j.compchemeng.2003.10.002 http://doi.org/10.1016/j.compchemeng.2003.10.002

Cui XD , Huang J , Chien JT , 2012 . Multi-view and multi-objective semi-supervised learning for HMM-based automatic speech recognition . IEEE Trans Audio Speech Lang Process , 20 ( 7 ): 1923 - 1935 . doi: 10.1109/TASL.2012.2191955 http://doi.org/10.1109/TASL.2012.2191955

Deng XG , Liu XY , Cao YP , et al. , 2022 . Incipient fault detection for dynamic chemical processes based on enhanced CVDA integrated with probability information and fault-sensitive features . J Process Contr , 114 : 29 - 41 . doi: 10.1016/j.jprocont.2022.04.001 http://doi.org/10.1016/j.jprocont.2022.04.001

Dietterich TG , 2000 . An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization . Mach Learn , 40 ( 2 ): 139 - 157 . doi: 10.1023/A:1007607513941 http://doi.org/10.1023/A:1007607513941

Dong YN , Qin SJ , 2018 . A novel dynamic PCA algorithm for dynamic data modeling and process monitoring . J Process Contr , 67 : 1 - 11 . doi: 10.1016/j.jprocont.2017.05.002 http://doi.org/10.1016/j.jprocont.2017.05.002

Downs JJ , Vogel EF , 1993 . A plant-wide industrial process control problem . Comput Chem Eng , 17 ( 3 ): 245 - 255 . doi: 10.1016/0098-1354(93)80018-I http://doi.org/10.1016/0098-1354(93)80018-I

Farajzadeh-Zanjani M , Hallaji E , Razavi-Far R , et al. , 2021 . Adversarial semi-supervised learning for diagnosing faults and attacks in power grids . IEEE Trans Smart Grid , 12 ( 4 ): 3468 - 3478 . doi: 10.1109/TSG.2021.3061395 http://doi.org/10.1109/TSG.2021.3061395

Feng J , Wang J , Han ZY , 2013 . Process monitoring for chemical process based on semi-supervised principal component analysis . Proc 25 th Chinese Control and Decision Conf , p. 4282 - 4286 . doi: 10.1109/CCDC.2013.6561704 http://doi.org/10.1109/CCDC.2013.6561704

Fraley C , Raftery AE , 2002 . Model-based clustering, discriminant analysis, and density estimation . J Am Stat Assoc , 97 ( 458 ): 611 - 631 . doi: 10.1198/016214502760047131 http://doi.org/10.1198/016214502760047131

Ge ZQ , 2016 . Supervised latent factor analysis for process data regression modeling and soft sensor application . IEEE Trans Contr Syst Technol , 24 ( 3 ): 1004 - 1011 . doi: 10.1109/TCST.2015.2473817 http://doi.org/10.1109/TCST.2015.2473817

Ge ZQ , 2017 . Review on data-driven modeling and monitoring for plant-wide industrial processes . Chemom Intell Lab Syst , 171 : 16 - 25 . doi: 10.1016/j.chemolab.2017.09.021 http://doi.org/10.1016/j.chemolab.2017.09.021

Ge ZQ , 2018 . Process data analytics via probabilistic latent variable models: a tutorial review . Ind Eng Chem Res , 57 ( 38 ): 12646 - 12661 . doi: 10.1021/acs.iecr.8b02913 http://doi.org/10.1021/acs.iecr.8b02913

Ge ZQ , Song ZH , Gao FR , 2013 . Review of recent research on data-based process monitoring . Ind Eng Chem Res , 52 ( 10 ): 3543 - 3562 . doi: 10.1021/ie302069q http://doi.org/10.1021/ie302069q

Ge ZQ , Song ZH , Ding SX , et al. , 2017 . Data mining and analytics in the process industry: the role of machine learning . IEEE Access , 5 : 20590 - 20616 . doi: 10.1109/ACCESS.2017.2756872 http://doi.org/10.1109/ACCESS.2017.2756872

Hady MFA , Schwenker F , 2010 . Combining committee-based semi-supervised learning and active learning . J Comput Sci Technol , 25 ( 4 ): 681 - 698 . doi: 10.1007/s11390-010-9357-6 http://doi.org/10.1007/s11390-010-9357-6

Harkat MF , Mansouri M , Nounou MN , et al. , 2019 . Fault detection of uncertain chemical processes using interval partial least squares-based generalized likelihood ratio test . Inform Sci , 490 : 265 - 284 . doi: 10.1016/j.ins.2019.03.068 http://doi.org/10.1016/j.ins.2019.03.068

Hastie T , Tibshirani R , 1996 . Discriminant analysis by Gaussian mixtures . J Roy Stat Soc Ser B , 58 ( 1 ): 155 - 176 . doi: 10.1111/j.2517-6161.1996.tb02073.x http://doi.org/10.1111/j.2517-6161.1996.tb02073.x

He YL , Li K , Zhang N , et al. , 2021 . Fault diagnosis using improved discrimination locality preserving projections integrated with sparse autoencoder . IEEE Trans Instrum Meas , 70 : 3527108 . doi: 10.1109/TIM.2021.3125975 http://doi.org/10.1109/TIM.2021.3125975

Huang CC , Chen T , Yao Y , 2013 . Mixture discriminant monitoring: a hybrid method for statistical process monitoring and fault diagnosis/isolation . Ind Eng Chem Res , 52 ( 31 ): 10720 - 10731 . doi: 10.1021/ie400418c http://doi.org/10.1021/ie400418c

Ipeirotis PG , Provost F , Wang J , 2010 . Quality management on Amazon Mechanical Turk . Proc ACM SIGKDD Workshop on Human Computation , p. 64 - 67 . doi: 10.1145/1837885.1837906 http://doi.org/10.1145/1837885.1837906

Jin YR , Qin CJ , Huang YX , et al. , 2021 . Actual bearing compound fault diagnosis based on active learning and decoupling attentional residual network . Measurement , 173 : 108500 . doi: 10.1016/j.measurement.2020.108500 http://doi.org/10.1016/j.measurement.2020.108500

Kalantar B , Al-Najjar HAH , Pradhan B , et al. , 2019 . Optimized conditioning factors using machine learning techniques for groundwater potential mapping . Water , 11 ( 9 ): 1909 . doi: 10.3390/w11091909 http://doi.org/10.3390/w11091909

Liu J , Song CY , Zhao J , 2018 . Active learning based semi-supervised exponential discriminant analysis and its application for fault classification in industrial processes . Chemom Intell Lab Syst , 180 : 42 - 53 . doi: 10.1016/j.chemolab.2018.07.003 http://doi.org/10.1016/j.chemolab.2018.07.003

Liu J , Song CY , Zhao J , et al. , 2020 . Manifold-preserving sparse graph-based ensemble FDA for industrial label-noise fault classification . IEEE Trans Instrum Meas , 69 ( 6 ): 2621 - 2634 . doi: 10.1109/TIM.2019.2930157 http://doi.org/10.1109/TIM.2019.2930157

Liu JW , Liu Y , Luo XL , 2015 . Semi-supervised learning methods . Chin J Comput , 38 ( 8 ): 1592 - 1617 (in Chinese) . doi: 10.11897/SP.J.1016.2015.01592 http://doi.org/10.11897/SP.J.1016.2015.01592

Liu Y , Ge ZQ , 2018 . Weighted random forests for fault classification in industrial processes with hierarchical clustering model selection . J Process Contr , 64 : 62 - 70 . doi: 10.1016/j.jprocont.2018.02.005 http://doi.org/10.1016/j.jprocont.2018.02.005

MacGregor J , Cinar A , 2012 . Monitoring, fault diagnosis, fault-tolerant control and optimization: data driven methods . Comput Chem Eng , 47 : 111 - 120 . doi: 10.1016/j.compchemeng.2012.06.017 http://doi.org/10.1016/j.compchemeng.2012.06.017

Pu XK , Li CG , 2021 . Probabilistic information-theoretic discriminant analysis for industrial label-noise fault diagnosis . IEEE Trans Ind Inform , 17 ( 4 ): 2664 - 2674 . doi: 10.1109/TII.2020.3001335 http://doi.org/10.1109/TII.2020.3001335

Raina R , Battle A , Lee H , et al. , 2007 . Self-taught learning: transfer learning from unlabeled data . Proc 24 th Int Conf on Machine Learning , p. 759 - 766 . doi: 10.1145/1273496.1273592 http://doi.org/10.1145/1273496.1273592

Raykar VC , Yu SP , Zhao LH , et al. , 2010 . Learning from crowds . J Mach Learn Res , 11 : 1297 - 1322 . doi: 10.5555/1756006.1859894 http://doi.org/10.5555/1756006.1859894

Schwenker F , Trentin E , 2014 . Pattern classification and clustering: a review of partially supervised learning approaches . Patt Recogn Lett , 37 : 4 - 14 . doi: 10.1016/j.patrec.2013.10.017 http://doi.org/10.1016/j.patrec.2013.10.017

Settles B , 2012 . Active Learning . Morgan & Claypool Publishers , USA . doi: 10.2200/S00429ED1V01Y201207AIM018 http://doi.org/10.2200/S00429ED1V01Y201207AIM018

Shao WM , Tian XM , 2017 . Semi-supervised selective ensemble learning based on distance to model for nonlinear soft sensor development . Neurocomputing , 222 : 91 - 104 . doi: 10.1016/j.neucom.2016.10.005 http://doi.org/10.1016/j.neucom.2016.10.005

Shao WM , Ge ZQ , Song ZH , 2019 a. Semi-supervised mixture of latent factor analysis models with application to online key variable estimation . Contr Eng Pract , 84 : 32 - 47 . doi: 10.1016/j.conengprac.2018.11.008 http://doi.org/10.1016/j.conengprac.2018.11.008

Shao WM , Ge ZQ , Song ZH , et al. , 2019 b. Nonlinear industrial soft sensor development based on semi-supervised probabilistic mixture of extreme learning machines . Contr Eng Pract , 91 : 104098 . doi: 10.1016/j.conengprac.2019.07.016 http://doi.org/10.1016/j.conengprac.2019.07.016

Snow R , O’Connor B , Jurafsky D , et al. , 2008 . Cheap and fast—but is it good? Evaluating non-expert annotations for natural language tasks . Proc Conf on Empirical Methods in Natural Language Processing , p. 254 - 263 .

Wang J , Feng J , Han ZY , 2014 . Fault detection for the class imbalance problem in semiconductor manufacturing processes . J Circ Syst Comput , 23 ( 4 ): 1450049 . doi: 10.1142/S0218126614500492 http://doi.org/10.1142/S0218126614500492

Wang JB , Shao WM , Song ZH , 2019 . Semi-supervised variational Bayesian student's t mixture regression and robust inferential sensor application . Contr Eng Pract , 92 : 104155 . doi: 10.1016/j.conengprac.2019.104155 http://doi.org/10.1016/j.conengprac.2019.104155

Wang L , Tian H , Zhang H , 2021 . Soft fault diagnosis of analog circuits based on semi-supervised support vector machine . Analog Integr Circ Signal Process , 108 ( 2 ): 305 - 315 . doi: 10.1007/s10470-021-01851-w http://doi.org/10.1007/s10470-021-01851-w

Yan ZB , Huang CC , Yao Y , 2014 . Semi-supervised mixture discriminant monitoring for chemical batch processes . Chemom Intell Lab Syst , 134 : 10 - 22 . doi: 10.1016/j.chemolab.2014.03.002 http://doi.org/10.1016/j.chemolab.2014.03.002

Yao L , Ge ZQ , 2017 . Locally weighted prediction methods for latent factor analysis with supervised and semisupervised process data . IEEE Trans Autom Sci Eng , 14 ( 1 ): 126 - 138 . doi: 10.1109/TASE.2016.2608914 http://doi.org/10.1109/TASE.2016.2608914

Yin LL , Wang HG , Fan WH , et al. , 2018 . Combining active learning and Fisher discriminant analysis for the semi-supervised process monitoring . IFAC-PapersOnLine , 51 ( 21 ): 147 - 151 . doi: 10.1016/j.ifacol.2018.09.407 http://doi.org/10.1016/j.ifacol.2018.09.407

Yin LL , Wang HG , Fan WH , et al. , 2019 . Incorporate active learning to semi-supervised industrial fault classification . J Process Contr , 78 : 88 - 97 . doi: 10.1016/j.jprocont.2019.04.008 http://doi.org/10.1016/j.jprocont.2019.04.008

Yuen MC , King I , Leung KS , 2011 . A survey of crowdsourcing systems . Proc IEEE 3 rd Int Conf on Privacy, Security, Risk and Trust and IEEE 3 rd Int Conf on Social Computing , p. 766 - 773 . doi: 10.1109/PASSAT/SocialCom.2011.203 http://doi.org/10.1109/PASSAT/SocialCom.2011.203

Zaman SMK , Liang XD , 2021 . An effective induction motor fault diagnosis approach using graph-based semi-supervised learning . IEEE Access , 9 : 7471 - 7482 . doi: 10.1109/ACCESS.2021.3049193 http://doi.org/10.1109/ACCESS.2021.3049193

Zhang N , Xu Y , Zhu QX , et al. , 2022 . Improved locality preserving projections based on heat-kernel and cosine weights for fault classification in complex industrial processes . IEEE Trans Reliab , early access . doi: 10.1109/TR.2021.3139539 http://doi.org/10.1109/TR.2021.3139539

Zheng JH , Wang HJ , Song ZH , et al. , 2019 . Ensemble semi-supervised Fisher discriminant analysis model for fault classification in industrial processes . ISA Trans , 92 : 109 - 117 . doi: 10.1016/j.isatra.2019.02.021 http://doi.org/10.1016/j.isatra.2019.02.021

Zheng JH , Zhu JL , Chen GJ , et al. , 2020 . Dynamic Bayesian network for robust latent variable modeling and fault classification . Eng Appl Artif Intell , 89 : 103475 . doi: 10.1016/j.engappai.2020.103475 http://doi.org/10.1016/j.engappai.2020.103475

Zhong K , Han M , Qiu T , et al. , 2020 . Fault diagnosis of complex processes using sparse kernel local Fisher discriminant analysis . IEEE Trans Neur Netw Learn Syst , 31 ( 5 ): 1581 - 1591 . doi: 10.1109/TNNLS.2019.2920903 http://doi.org/10.1109/TNNLS.2019.2920903

Zou Y , Yu ZD , Liu XF , et al. , 2019 . Confidence regularized self-training . Proc IEEE/CVF Int Conf on Computer Vision , p. 5981 - 5990 . doi: 10.1109/ICCV.2019.00608 http://doi.org/10.1109/ICCV.2019.00608

Views

161

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

Jun Wang

Xinyun Fang

Yuchen He

Weijun Wang

Yunqing Mao

Tianlei Hu

Ke Chen‡

Shengyuan Liu

Related Institution

Key Laboratory of Intelligent Manufacturing Quality Big Data Tracing and Analysis of Zhejiang Province China Jiliang University

Key Lab of Intelligent Computing Based Big Data of Zhejiang Province, Zhejiang University

City Cloud Science and Technology (China) Co., Ltd.

School of Information and Electronics, Beijing Institute of Technology

State Key Laboratory of Millimeter Waves, Southeast University

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰