Generative adversarial network based novelty detection using minimized reconstruction error

Huan-gang WANG; Xin LI; Tao ZHANG

doi:10.1631/FITEE.1700786

Your Location：

Home >

Browse articles >

Generative adversarial network based novelty detection using minimized reconstruction error

2018 Special Issue on Artificial Intelligence 2.0: Theories and Applications | Updated：2022-05-19

- Generative adversarial network based novelty detection using minimized reconstruction error
- 基于最小化重构误差的生成对抗网络异常检测
- Frontiers of Information Technology & Electronic Engineering Vol. 19, Issue 1, Pages: 116-125(2018)
- Affiliations：
  
  Department of Automation, School of Information Science and Technology, Tsinghua University, Beijing 100084, China
- Author bio：
  
  Huan-gang WANG, E-mail: hgwang@tsinghua.edu.cn
- Funds：
- DOI：10.1631/FITEE.1700786
  CLC： TP391
- Received：24 November 2017，
  
  Revised：2018-;1-26，
  
  Published：2018-01
- Accepted：
Scan QR Code
Huan-gang WANG, Xin LI, Tao ZHANG. Generative adversarial network based novelty detection using minimized reconstruction error[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(1): 116-125.
DOI：

Huan-gang WANG, Xin LI, Tao ZHANG. Generative adversarial network based novelty detection using minimized reconstruction error[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(1): 116-125. DOI： 10.1631/FITEE.1700786.

摘要

生成对抗网络是机器学习领域近年来最令人瞩目的进展，它通过在二人零和博弈中达到纳什均衡来训练模型。生成对抗网络由一个生成器和一个判别器构成，二者通过对抗学习机制进行训练。本文引入并调查了生成对抗网络在异常检测中的应用。在训练阶段，生成对抗网络从正常数据中学习；然后，基于过去的未知数据，生成器和判别器可以通过学习到的决策边界，区分异常和正常模式。提出的基于生成对抗网络的异常检测方法在MNIST数字数据集和田纳西-伊斯曼标准数据集上的性能表现极具竞争力。

Abstract

Generative adversarial network (GAN) is the most exciting machine learning breakthrough in recent years

and it trains the learning model by finding the Nash equilibrium of a two-player zero-sum game. GAN is composed of a generator and a discriminator

both trained with the adversarial learning mechanism. In this paper

we introduce and investigate the use of GAN for novelty detection. In training

GAN learns from ordinary data. Then

using previously unknown data

the generator and the discriminator with the designed decision boundaries can both be used to separate novel patterns from ordinary patterns. The proposed GAN-based novelty detection method demonstrates a competitive performance on the MNIST digit database and the Tennessee Eastman (TE) benchmark process compared with the PCA-based novelty detection methods using Hotelling's

$$T^2$$

and squared prediction error statistics.

关键词

Keywords

references

M Abadi , , , D Andersen . . Learning to protect communications with adversarial neural cryptography . . https://arxiv.org/abs/1610.06918 , , 2016 . . .

M Arjovsky , , , S Chintala , , , L Bottou . . Wasserstein generative adversarial networks . . Int Conf on Machine Learning , , 2017 . . p.214 - - 223 . . . .

D Berthelot , , , T Schumm , , , L Metz . . BEGAN: boundary equilibrium generative adversarial networks . . https://arxiv.org/abs/1703.10717 , , 2017 . . .

L Clifton , , , D Clifton , , , P Watkinson , , , 等 . . Identification of patient deterioration in vital-sign data using one-class support vector machines . . Federated Conf on Computer Science and Information Systems , , 2011 . . p.125 - - 131 . . . .

E Denton , , , S Chintala , , , R Fergus , , , 等 . . Deep generative image models using a Laplacian pyramid of adversarial networks . . Advances in Neural Information Processing Systems , , 2015 . . p.1486 - - 1494 . . . .

J Donahue , , , P Krähenbühl , , , T Darrell . . Adversarial feature learning . . https://arxiv.org/abs/1605.09782 , , 2016 . . .

J Downs , , , E Vogel . . A plant-wide industrial process control problem . . Comput Chem Eng , , 1993 . . 17 ( ( 3 ): ): 245 - - 255 . . DOI: 10.1016/0098-1354(93)80018-I http://doi.org/10.1016/0098-1354(93)80018-I . .

V Dumoulin , , , I Belghazi , , , B Poole , , , 等 . . Adversarially learned inference . . https://arxiv.org/abs/1606.00704 , , 2016 . . .

Z Ge , , , Z Song . . Bagging support vector data description model for batch process monitoring . . J Proc Contr , , 2013 . . 23 ( ( 8 ): ): 1090 - - 1096 . . DOI: 10.1016/j.jprocont.2013.06.010 http://doi.org/10.1016/j.jprocont.2013.06.010 . .

Z Ge , , , C Yang , , , Z Song . . Improved kernel PCA-based monitoring approach for nonlinear processes . . Chem Eng Sci , , 2009 . . 64 ( ( 9 ): ): 2245 - - 2255 . . DOI: 10.1016/j.ces.2009.01.050 http://doi.org/10.1016/j.ces.2009.01.050 . .

Z Ge , , , F Gao , , , Z Song . . Batch process monitoring based on support vector data description method . . J Proc Contr , , 2011 . . 21 ( ( 6 ): ): 949 - - 959 . . DOI: 10.1016/j.jprocont.2011.02.004 http://doi.org/10.1016/j.jprocont.2011.02.004 . .

Z Ge , , , Z Song , , , F Gao . . Review of recent research on data-based process monitoring . . Ind Eng Chem Res , , 2013 . . 52 ( ( 10 ): ): 3543 - - 3562 . . DOI: 10.1021/ie302069q http://doi.org/10.1021/ie302069q . .

Z Ge , , , S Demyanov , , , Z Chen , , , 等 . . Generative OpenMax for multi-class open set classification . . https://arxiv.org/abs/1707.07418 , , 2017 . . .

I Goodfellow , , , J Pouget-Abadie , , , M Mirza , , , 等 . . Generative adversarial nets . . Advances in Neural Information Processing Systems , , 2014 . . p.2672 - - 2680 . . . .

A Grover , , , S Ermon . . Boosted generative models . . https://arxiv.org/abs/1702.08484 , , 2017 . . .

V Hautamaki , , , I Karkkainen , , , P Franti . . Outlier detection using k-nearest neighbour graph . . Proc 17 th Int Conf on Pattern Recognition , , 2004 . . p.430 - - 433 . . DOI: 10.1109/ICPR.2004.1334558 http://doi.org/10.1109/ICPR.2004.1334558 . .

Z He , , , S Deng , , , X Xu . . An optimization model for outlier detection in categorical data . . LNCS , , 2005 . . 3644 400 - - 409 . . DOI: 10.1007/11538059_42 http://doi.org/10.1007/11538059_42 . .

H Hoffmann . . Kernel PCA for novelty detection . . Patt Recogn , , 2007 . . 40 ( ( 3 ): ): 863 - - 874 . . DOI: 10.1016/j.patcog.2006.07.009 http://doi.org/10.1016/j.patcog.2006.07.009 . .

A Kadurin , , , A Aliper , , , A Kazennov , , , 等 . . The cornucopia of meaningful leads: applying deep adversarial autoencoders for new molecule development in oncology . . Oncotarget , , 2017a . . 8 ( ( 7 ): ): 10883 DOI: 10.18632/oncotarget.14073 http://doi.org/10.18632/oncotarget.14073 . .

A Kadurin , , , S Nikolenko , , , K Khrabrov , , , 等 . . druGAN: an advanced generative adversarial autoencoder model for de novo generation of new molecules with desired molecular properties in silico . . Mol Pharmaceut , , 2017b . . 14 ( ( 9 ): ): 3098 - - 3104 . . DOI: 10.1021/acs.molpharmaceut.7b00346 http://doi.org/10.1021/acs.molpharmaceut.7b00346 . .

E Keogh , , , S Lonardi , , , C Ratanamahatana . . Towards parameter-free data mining . . Proc 10 th ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining , , 2004 . . p.206 - - 215 . . DOI: 10.1145/1014052.1014077 http://doi.org/10.1145/1014052.1014077 . .

T Kim , , , M Cha , , , H Kim , , , 等 . . Learning to discover cross-domain relations with generative adversarial networks . . https://arxiv.org/abs/1703.05192 , , 2017 . . .

C Ledig , , , L Theis , , , F Huszár , , , 等 . . Photo-realistic single image super-resolution using a generative adversarial network . . https://arxiv.org/abs/1609.04802 , , 2016 . . .

J Li , , , X Liang , , , Y Wei , , , 等 . . Perceptual generative adversarial networks for small object detection . . CVPR , , 2017 . . p.1951 - - 1959 . . DOI: 10.1109/CVPR.2017.211 http://doi.org/10.1109/CVPR.2017.211 . .

Y Li , , , L Maguire . . Selecting critical patterns based on local geometrical and statistical information . . IEEE Trans Patt Anal Mach Intell , , 2011 . . 33 ( ( 6 ): ): 1189 - - 1201 . . DOI: 10.1109/TPAMI.2010.188 http://doi.org/10.1109/TPAMI.2010.188 . .

Y Li , , , S Liu , , , J Yang , , , 等 . . Generative face completion . . CVPR, , , 2017 . . p.5892 - - 5900 . . DOI: 10.1109/CVPR.2017.624 http://doi.org/10.1109/CVPR.2017.624 . .

P Luc , , , C Couprie , , , S Chintala , , , 等 . . Semantic segmentation using adversarial networks . . https://arxiv.org/abs/1611.08408 , , 2016 . . .

S Mahadevan , , , S Shah . . Fault detection and diagnosis in process data using one-class support vector machines . . J Proc Contr , , 2009 . . 19 ( ( 10 ): ): 1627 - - 1639 . . DOI: 10.1016/j.jprocont.2009.07.011 http://doi.org/10.1016/j.jprocont.2009.07.011 . .

X Mao , , , Q Li , , , H Xie , , , 等 . . Least squares generative adversarial networks . . https://arxiv.org/abs/1611.04076 , , 2016 . . .

O Mogren . . C-RNN-GAN: continuous recurrent neural networks with adversarial training . . https://arxiv.org/abs/1611.09904 , , 2016 . . .

A Patcha , , , J Park . . An overview of anomaly detection techniques: existing solutions and latest technological trends . . Comput Netw , , 2007 . . 51 ( ( 12 ): ): 3448 - - 3470 . . DOI: 10.1016/j.comnet.2007.02.001 http://doi.org/10.1016/j.comnet.2007.02.001 . .

M Pimentel , , , D Clifton , , , L Clifton , , , 等 . . A review of novelty detection . . Signal Process , , 2014 . . 99 215 - - 249 . . DOI: 10.1016/j.sigpro.2013.12.026 http://doi.org/10.1016/j.sigpro.2013.12.026 . .

A Radford , , , L Metz , , , S Chintala . . Unsupervised representation learning with deep convolutional generative adversarial networks . . https://arxiv.org/abs/1511.06434 , , 2015 . . .

S Reed , , , Z Akata , , , X Yan , , , 等 . . Generative adversarial text to image synthesis . . Proc 33 rd Int Conf on Machine Learning , , 2016 . . p.1060 - - 1069 . . . .

T Schlegl , , , P Seeböck , , , S Waldstein , , , 等 . . Unsupervised anomaly detection with generative adversarial networks to guide marker discovery . . Int Conf on Information Processing in Medical Imaging , , 2017 . . p.146 - - 157 . . DOI: 10.1007/978-3-319-59050-9_12 http://doi.org/10.1007/978-3-319-59050-9_12 . .

J Springenberg . . Unsupervised and semi-supervised learning with categorical generative adversarial networks . . https://arxiv.org/abs/1511.06390 , , 2015 . . .

C Vondrick , , , H Pirsiavash , , , A Torralba . . Generating videos with scene dynamics . . Advances in Neural Information Processing Systems , , 2016 . . p.613 - - 621 . . . .

J Wu , , , C Zhang , , , T Xue , , , 等 . . Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling . . Advances in Neural Information Processing Systems , , 2016 . . p.82 - - 90 . . . .

Y Xiao , , , H Wang , , , W Xu , , , 等 . . Robust one-class SVM for fault detection . . Chemometr Intell Lab Syst , , 2016 . . 151 15 - - 25 . . DOI: 10.1016/j.chemolab.2015.11.010 http://doi.org/10.1016/j.chemolab.2015.11.010 . .

Z Yang , , , W Chen , , , F Wang , , , 等 . . Improving neural machine translation with conditional sequence generative adversarial nets . . https://arxiv.org/abs/1703.04887 , , 2017 . . .

R Yeh , , , C Chen , , , T Lim , , , 等 . . Semantic image inpainting with perceptual and contextual losses . . https://arxiv.org/abs/1607.07539 , , 2016 . . .

Z Yi , , , H Zhang , , , P Gong , , , 等 . . DualGAN: unsupervised dual learning for image-to-image translation . . https://arxiv.org/abs/1704.02510 , , 2017 . . .

J Yu . . Semiconductor manufacturing process monitoring using Gaussian mixture model and Bayesian method with local and nonlocal information . . IEEE Trans Semicond Manuf , , 2012 . . 25 ( ( 3 ): ): 480 - - 493 . . DOI: 10.1109/TSM.2012.2192945 http://doi.org/10.1109/TSM.2012.2192945 . .

J Yu , , , S Qin . . Multimode process monitoring with Bayesian inference-based finite Gaussian mixture models . . AIChE J , , 2008 . . 54 ( ( 7 ): ): 1811 - - 1829 . . DOI: 10.1002/aic.11515 http://doi.org/10.1002/aic.11515 . .

J Yu , , , S Qin . . Multiway Gaussian mixture model based multiphase batch process monitoring . . Ind Eng Chem Res , , 2009 . . 48 ( ( 18 ): ): 8585 - - 8594 . . DOI: 10.1021/ie900479g http://doi.org/10.1021/ie900479g . .

L Yu , , , W Zhang , , , J Wang , , , 等 . . SeqGAN: sequence generative adversarial nets with policy gradient . . 31 st AAAI Conf on Artificial Intelligence , , 2017 . . p.2852 - - 2858 . . . .

F Zhao , , , J Feng , , , J Zhao , , , 等 . . Robust LSTMautoencoders for face de-occlusion in the wild . . IEEE Trans Image Process , , 2018 . . 27 ( ( 2 ): ): 778 - - 790 . . DOI: 10.1109/TIP.2017.2771408 http://doi.org/10.1109/TIP.2017.2771408 . .

J Zhao , , , M Mathieu , , , Y LeCun . . Energy-based generative adversarial network . . https://arxiv.org/abs/1609.03126 , , 2016 . . .

J Zhu , , , T Park , , , P Isola , , , 等 . . Unpaired image-to-image translation using cycle-consistent adversarial networks . . https://arxiv.org/abs/1703.10593 , , 2017 . . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰