网络安全遇上人工智能：综述

李建华

doi:10.1631/FITEE.1800573

Your Location：

Home >

Browse articles >

网络安全遇上人工智能：综述

网络空间安全专辑 | Updated：2022-05-19

- 网络安全遇上人工智能：综述
- Cyber security meets artificial intelligence: a survey
- 信息与电子工程前沿（英文） 2018年19卷第12期页码：1462-1474
- Affiliations：
  
  School of Cyber Security, Shanghai Jiao Tong University, Shanghai 200240, China
- Author bio：
  
  [ "Jian-hua LI, E-mail: lijh888@sjtu.edu.cn" ]
- Funds：
  
  Project supported by the National Natural Science Foundation of China (Nos. 61431008 and 61571300)
- DOI：10.1631/FITEE.1800573
  中图分类号： TP309
- 收稿：2018-09-16，
  
  修回：2018-12-24，
  
  纸质出版：2018-12
- Accepted：
Scan QR Code
李建华. 网络安全遇上人工智能：综述[J]. 信息与电子工程前沿（英文）, 2018,19(12):1462-1474.

Jian-hua LI. Cyber security meets artificial intelligence: a survey[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(12): 1462-1474.
李建华. 网络安全遇上人工智能：综述[J]. 信息与电子工程前沿（英文）, 2018,19(12):1462-1474. DOI： 10.1631/FITEE.1800573.

Jian-hua LI. Cyber security meets artificial intelligence: a survey[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(12): 1462-1474. DOI： 10.1631/FITEE.1800573.

摘要

网络安全与人工智能技术有着广泛的交叉。一方面，可以将人工智能技术（如深度学习）引入网络安全领域，构建智能模型，实现恶意代码检测、入侵检测和威胁情报感知等。另一方面，人工智能模型面临针对样本、学习过程和决策等的各种威胁。因此，人工智能模型需要网络安全防护技术来对抗各类攻击，实现隐私保护机器学习以及安全的联合深度学习等。本文对人工智能与网络安全交叉研究进行综述，首先总结现有利用人工智能技术对抗网络攻击的研究工作，包括采用传统机器学习技术和深度学习技术在对抗网络攻击方面的应用和效果。然后总结和分析人工智能会遭受的对抗攻击，对现有针对对抗攻击的防御方式进行归类，分析各自特点。最后，从构建加密神经网络和实现安全联合深度学习两个方面阐述现有工作中构建安全人工智能系统的方案。

Abstract

There is a wide range of interdisciplinary intersections between cyber security and artificial intelligence (AI). On one hand

AI technologies

such as deep learning

can be introduced into cyber security to construct smart models for implementing malware classification and intrusion detection and threating intelligence sensing. On the other hand

AI models will face various cyber threats

which will disturb their sample

learning

and decisions. Thus

AI models need specific cyber security defense and protection technologies to combat adversarial machine learning

preserve privacy in machine learning

secure federated learning

etc. Based on the above two aspects

we review the intersection of AI and cyber security. First

we summarize existing research efforts in terms of combating cyber attacks using AI

including adopting traditional machine learning methods and existing deep learning solutions. Then

we analyze the counterattacks from which AI itself may suffer

dissect their characteristics

and classify the corresponding defense methods. Finally

from the aspects of constructing encrypted neural network and realizing a secure federated deep learning

we expatiate the existing research on how to build a secure AI system.

关键词

Keywords

references

A Abeshu , , , N Chilamkurti . . Deep learning: the frontier for distributed attack detection in fog-to-things computing . . IEEE Commun Mag , , 2018 . . 56 ( ( 2 ): ): 169 - - 175 . . DOI: 10.1109/MCOM.2018.1700332 http://doi.org/10.1109/MCOM.2018.1700332 . .

N Akhtar , , , A Mian . . Threat of adversarial attacks on deep learning in computer vision: a survey . . IEEE Access , , 2018 . . 6 14410 - - 14430 . . DOI: 10.1109/ACCESS.2018.2807385 http://doi.org/10.1109/ACCESS.2018.2807385 . .

N Akhtar , , , J Liu , , , A Mian . . Defense against universal adversarial perturbations . . IEEE/CVF Conf on Computer Vision and Pattern Recognition , , 2018 . . p.3389 - - 3398 . . DOI: 10.1109/CVPR.2018.00357 http://doi.org/10.1109/CVPR.2018.00357 . .

K Arulkumaran , , , MP Deisenroth , , , M Brundage , , , 等 . . Deep reinforcement learning: a brief survey . . IEEE Signal Process Mag , , 2017 . . 34 ( ( 6 ): ): 26 - - 38 . . DOI: 10.1109/MSP.2017.2743240 http://doi.org/10.1109/MSP.2017.2743240 . .

RC Aygn , , , AG Yavuz . . A stochastic data discrimination based autoencoder approach for network anomaly detection . . Proc 5 th Signal Processing and Communications Applications Conf , , 2017 . . p.1 - - 4 . . DOI: 10.1109/SIU.2017.7960410 http://doi.org/10.1109/SIU.2017.7960410 . .

K Bonawitz , , , V Ivanov , , , B Kreuter , , , 等 . . Practical secure aggregation for privacy-preserving machine learning . . Proc ACM SIGSAC Conf on Computer and Communications Security , , 2017 . . p.1175 - - 1191 . . DOI: 10.1145/3133956.3133982 http://doi.org/10.1145/3133956.3133982 . .

R Bost , , , RA Popa , , , S Tu , , , 等 . . Machine learning classification over encrypted data . . Network and Distributed System Security Symp , , 2015 . . p.331 - - 364 . . DOI: 10.14722/ndss.2015.23241 http://doi.org/10.14722/ndss.2015.23241 . .

MMU Chowdhury , , , F Hammond , , , G Konowicz , , , 等 . . A few-shot deep learning approach for improved intrusion detection . . Proc 8 th Annual Ubiquitous Computing, Electronics and Mobile Communication Conf , , 2017 . . p.456 - - 462 . . DOI: 10.1109/UEMCON.2017.8249084 http://doi.org/10.1109/UEMCON.2017.8249084 . .

M Cisse , , , Y Adi , , , N Neverova , , , 等 . . Houdini: fooling deep structured prediction models . . https://arxiv.org/abs/1707.05373 https://arxiv.org/abs/1707.05373 , , 2017 . . .

ED Cubuk , , , B Zoph , , , SS Schoenholz , , , 等 . . Intriguing properties of adversarial examples . . https://arxiv.org/abs/1711.02846 https://arxiv.org/abs/1711.02846 , , 2017 . . .

EG Dada . . A hybridized SVM-kNN-pdAPSO approach to intrusion detection system . . Faculty Seminar Series , , 2017 . . p.1 - - 8 . . . .

L Deng , , , D Yu . . Deep learning: methods and applications . . Found Trend Sig Process , , 2014 . . 7 ( ( 3-4 ): ): 197 - - 387 . . DOI: 10.1561/2000000039 http://doi.org/10.1561/2000000039 . .

R Feinman , , , RR Curtin , , , S Shintre , , , 等 . . Detecting adversarial samples from artifacts . . 2017 . . https://arxiv.org/abs/1703.00410 https://arxiv.org/abs/1703.00410 , , . .

W Gao , , , T Morris , , , B Reaves , , , 等 . . On SCADA control system command and response injection and intrusion detection . . eCrime Researchers Summit , , 2010 . . p.1 - - 9 . . DOI: 10.1109/ecrime.2010.5706699 http://doi.org/10.1109/ecrime.2010.5706699 . .

T Gebhart , , , P Schrater . . Adversary detection in neural networks via persistent homology . . https://arxiv.org/abs/1711.10056 https://arxiv.org/abs/1711.10056 , , 2017 . . .

VA Golovko . . Deep learning: an overview and main paradigms . . Opt Memory Neur Netw , , 2017 . . 26 ( ( 1 ): ): 1 - - 17 . . DOI: 10.3103/S1060992X16040081 http://doi.org/10.3103/S1060992X16040081 . .

IJ Goodfellow , , , J Pouget-Abadie , , , M Mirza , , , 等 . . Generative adversarial networks . . https://arxiv.org/abs/1406.2661 https://arxiv.org/abs/1406.2661 , , 2014 . . .

IJ Goodfellow , , , J Shlens , , , C Szegedy . . Explaining and harnessing adversarial examples . . https://arxiv.org/abs/1412.6572 https://arxiv.org/abs/1412.6572 , , 2015 . . .

SX Gu , , , L Rigazio . . Towards deep neural network architectures robust to adversarial examples . . https://arxiv.org/abs/1412.5068 https://arxiv.org/abs/1412.5068 , , 2015 . . .

ZT Guan , , , J Li , , , LF Wu , , , 等 . . Achieving efficient and secure data acquisition for cloud-supported Internet of Things in smart grid . . IEEE Internet Things J , , 2017 . . 4 ( ( 6 ): ): 1934 - - 1944 . . DOI: 10.1109/JIOT.2017.2690522 http://doi.org/10.1109/JIOT.2017.2690522 . .

WG Hatcher , , , W Yu . . A survey of deep learning: platforms, applications and emerging research trends . . IEEE Access , , 2018 . . 6 24411 - - 24432 . . DOI: 10.1109/ACCESS.2018.2830661 http://doi.org/10.1109/ACCESS.2018.2830661 . .

W He , , , J Wei , , , XY Chen , , , 等 . . Adversarial example defenses: ensembles of weak defenses are not strong . . https://arxiv.org/abs/1706.04701 https://arxiv.org/abs/1706.04701 , , 2017 . . .

RT Kokila , , , ST Selvi , , , K Govindarajan . . DDoS detection and analysis in SDN-based environment using support vector machine classifier . . Proc 6 th Int Conf on Advanced Computing , , 2014 . . p.205 - - 210 . . DOI: 10.1109/ICoAC.2014.7229711 http://doi.org/10.1109/ICoAC.2014.7229711 . .

J Korczak , , , M Hernes . . Deep learning for financial time series forecasting in a-trader system . . Proc Federated Conf on Computer Science and Information Systems , , 2017 . . p.905 - - 912 . . DOI: 10.15439/2017F449 http://doi.org/10.15439/2017F449 . .

D Krotov , , , J Hopfield . . Dense associative memory is robust to adversarial inputs . . Neur Comput , , 2018 . . 30 ( ( 12 ): ): 3151 - - 3167 . . DOI: 10.1162/neco_a_01143 http://doi.org/10.1162/neco_a_01143 . .

Y LeCun , , , Y Bengio , , , G Hinton . . Deep learning . . Nature , , 2015 . . 521 ( ( 7553 ): ): 436 - - 444 . . DOI: 10.1038/Nature14539 http://doi.org/10.1038/Nature14539 . .

H Lee , , , S Han , , , J Lee . . Generative adversarial trainer: defense to adversarial perturbations with GAN . . https://arxiv.org/abs/1705.03387 https://arxiv.org/abs/1705.03387 , , 2017 . . .

GL Li , , , J Wu , , , JH Li , , , 等 . . Service popularity-based smart resources partitioning for fog computing-enabled industrial Internet of Things . . IEEE Trans Ind Inform , , 2018 . . 14 ( ( 10 ): ): 4702 - - 4711 . . DOI: 10.1109/TII.2018.2845844 http://doi.org/10.1109/TII.2018.2845844 . .

LZ Li , , , K Ota , , , MX Dong . . Deep learning for smart industry: efficient manufacture inspection system with fog computing . . IEEE Trans Ind Inform , , 2018a . . 14 ( ( 10 ): ): 4665 - - 4673 . . DOI: 10.1109/TII.2018.2842821 http://doi.org/10.1109/TII.2018.2842821 . .

LZ Li , , , K Ota , , , MX Dong . . DeepNFV: a light-weight framework for intelligent edge network functions virtualization . . IEEE Netw, in press , , 2018b . . DOI: 10.1109/MNET.2018.1700394 http://doi.org/10.1109/MNET.2018.1700394 . .

B Liang , , , HC Li , , , MQ Su , , , 等 . . Detecting adversarial image examples in deep networks with adaptive noise reduction . . https://arxiv.org/abs/1705.08378 https://arxiv.org/abs/1705.08378 , , 2017 . . .

G Loukas , , , T Vuong , , , R Heartfield , , , 等 . . Cloud-based cyber-physical intrusion detection for vehicles using deep learning . . IEEE Access , , 2018 . . 6 3491 - - 3508 . . DOI: 10.1109/ACCESS.2017.2782159 http://doi.org/10.1109/ACCESS.2017.2782159 . .

Y Luo , , , X Boix , , , G Roig , , , 等 . . Foveation-based mechanisms alleviate adversarial examples . . https://arxiv.org/abs/1511.06292 https://arxiv.org/abs/1511.06292 , , 2015 . . .

C Lyu , , , KZ Huang , , , HN Liang . . A unified gradient regularization family for adversarial examples . . IEEE Int Conf on Data Mining , , 2015 . . p.301 - - 309 . . DOI: 10.1109/ICDM.2015.84 http://doi.org/10.1109/ICDM.2015.84 . .

CD Manning , , , M Surdeanu , , , J Bauer , , , 等 . . The Stanford CoreNLP natural language processing toolkit . . Proc 52 nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations , , 2014 . . p.55 - - 60 . . DOI: 10.3115/v1/P14-5010 http://doi.org/10.3115/v1/P14-5010 . .

HB McMahan , , , E Moore , , , D Ramage , , , 等 . . Communication-efficient learning of deep networks from decentralized data . . https://arxiv.org/abs/1602.05629 https://arxiv.org/abs/1602.05629 , , 2016 . . .

DY Meng , , , H Chen . . MagNet: a two-pronged defense against adversarial examples . . Proc ACM Conf on Computer and Communications Security , , 2017 . . p.135 - - 147 . . DOI: 10.1145/3133956.3134057 http://doi.org/10.1145/3133956.3134057 . .

WZ Meng , , , WJ Li , , , LF Kwok . . Design of intelligent KNN- based alarm filter using knowledge-based alert verification in intrusion detection . . Secur Commun Netw , , 2015 . . 8 ( ( 18 ): ): 3883 - - 3895 . . DOI: 10.1002/sec.1307 http://doi.org/10.1002/sec.1307 . .

X Meng , , , Z Shan , , , FD Liu , , , 等 . . MCSMGS: malware classification model based on deep learning . . Int Conf on Cyber-Enabled Distributed Computing and Knowledge Discovery , , 2017 . . p.272 - - 275 . . DOI: 10.1109/CyberC.2017.21 http://doi.org/10.1109/CyberC.2017.21 . .

V Mnih , , , K Kavukcuoglu , , , D Silver , , , 等 . . Human-level control through deep reinforcement learning . . Nature , , 2015 . . 518 ( ( 7540 ): ): 529 - - 533 . . DOI: 10.1038/nature14236 http://doi.org/10.1038/nature14236 . .

D Moon , , , H Im , , , I Kim , , , 等 . . DTB-IDS: an intrusion detection system based on decision tree using behavior analysis for preventing APT attacks . . J Supercomput , , 2017 . . 73 ( ( 7 ): ): 2881 - - 2895 . . DOI: 10.1007/s11227-015-1604-8 http://doi.org/10.1007/s11227-015-1604-8 . .

SM Moosavi-Dezfooli , , , A Fawzi , , , P Frossard . . DeepFool: a simple and accurate method to fool deep neural networks . . IEEE Conf on Computer Vision and Pattern Recognition , , 2016 . . p.2574 - - 2582 . . DOI: 10.1109/CVPR.2016.282 http://doi.org/10.1109/CVPR.2016.282 . .

SM Moosavi-Dezfooli , , , A Fawzi , , , O Fawzi , , , 等 . . Universal adversarial perturbations . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2017 . . p.86 - - 94 . . DOI: 10.1109/CVPR.2017.17 http://doi.org/10.1109/CVPR.2017.17 . .

KR Mopuri , , , U Garg , , , RV Babu . . Fast feature fool: a data independent approach to universal adversarial perturbations . . https://arxiv.org/abs/1707.05572 https://arxiv.org/abs/1707.05572 , , 2017 . . .

A Nayebi , , , S Ganguli . . Biologically inspired protection of deep networks from adversarial attacks . . https://arxiv.org/abs/1703.09202 https://arxiv.org/abs/1703.09202 , , 2017 . . .

M Olalere , , , MT Abdullah , , , R Mahmod , , , 等 . . Identification and evaluation of discriminative lexical features of malware URL for real-time classification . . Int Conf on Computer and Communication Engineering , , 2016 . . p.90 - - 95 . . DOI: 10.1109/ICCCE.2016.31 http://doi.org/10.1109/ICCCE.2016.31 . .

K Ota , , , MS Dao , , , V Mezaris , , , 等 . . Deep learning for mobile multimedia: a survey . . ACM Trans Multim Comput Commun Appl , , 2017 . . 13 ( ( 3S ): ): - - Article 34 . . DOI: 10.1145/3092831 http://doi.org/10.1145/3092831 . .

N Papernot , , , P McDaniel , , , S Jha , , , 等 . . The limitations of deep learning in adversarial settings . . IEEE European Symp on Security and Privacy , , 2016 . . p.372 - - 387 . . DOI: 10.1109/EuroSP.2016.36 http://doi.org/10.1109/EuroSP.2016.36 . .

LT Phong , , , Y Aono , , , T Hayashi , , , 等 . . Privacy- preserving deep learning via additively homomorphic encryption . . IEEE Trans Inform Forens Secur , , 2018 . . 13 ( ( 5 ): ): 1333 - - 1345 . . DOI: 10.1109/TIFS.2017.2787987 http://doi.org/10.1109/TIFS.2017.2787987 . .

SQ Ren , , , KM He , , , R Girshick , , , 等 . . Faster R-CNN: towards real-time object detection with region proposal networks . . IEEE Trans Patt Anal Mach Intell , , 2017 . . 39 ( ( 6 ): ): 1137 - - 1149 . . DOI: 10.1109/TPAMI.2016.2577031 http://doi.org/10.1109/TPAMI.2016.2577031 . .

AS Ross , , , F Doshi-Velez . . Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients . . https://arxiv.org/abs/1711.09404 https://arxiv.org/abs/1711.09404 , , 2017 . . .

S Sabour , , , YS Cao , , , F Faghri , , , 等 . . Adversarial manipulation of deep representations . . https://arxiv.org/abs/1511.05122 https://arxiv.org/abs/1511.05122 , , 2015 . . .

N Shahid , , , SA Aleem , , , IH Naqvi , , , 等 . . Support vector machine based fault detection & classification in smart grids . . IEEE Globecom Workshops , , 2012 . . p.1526 - - 1531 . . DOI: 10.1109/GLOCOMW.2012.6477812 http://doi.org/10.1109/GLOCOMW.2012.6477812 . .

R Shokri , , , V Shmatikov . . Privacy-preserving deep learning . . Proc 53 rd Annual Allerton Conf on Communication, Control, and Computing , , 2015 . . p.1310 - - 1321 . . DOI: 10.1109/ALLERTON.2015.7447103 http://doi.org/10.1109/ALLERTON.2015.7447103 . .

AR Syarif , , , W Gata . . Intrusion detection system using hybrid binary PSO and K-nearest neighborhood algorithm . . 11 th Int Conf on Information & Communication Technology and System , , 2017 . . p.181 - - 186 . . DOI: 10.1109/ICTS.2017.8265667 http://doi.org/10.1109/ICTS.2017.8265667 . .

R Vinayakumar , , , KP Soman , , , P Poornachandran , , , 等 . . Detecting Android malware using long short-term memory (LSTM) . . J Int Fuzzy Syst , , 2018 . . 34 ( ( 3 ): ): 1277 - - 1288 . . DOI: 10.3233/JIFS-169424 http://doi.org/10.3233/JIFS-169424 . .

T Vollmer , , , M Manic . . Computationally efficient neural network intrusion security awareness . . Proc 2 nd Int Symp on Resilient Control Systems , , 2009 . . p.25 - - 30 . . DOI: 10.1109/ISRCS.2009.5251357 http://doi.org/10.1109/ISRCS.2009.5251357 . .

TP Vuong , , , G Loukas , , , D Gan , , , 等 . . Decision tree-based detection of denial of service and command injection attacks on robotic vehicles . . IEEE Int Workshop on Information Forensics and Security , , 2015 . . p.1 - - 6 . . DOI: 10.1109/WIFS.2015.7368559 http://doi.org/10.1109/WIFS.2015.7368559 . .

J Wu , , , MX Dong , , , K Ota , , , 等 . . Big data analysis-based secure cluster management for optimized control plane in software-defined networks . . IEEE Trans Netw Serv Manag , , 2018 . . 15 ( ( 1 ): ): 27 - - 38 . . DOI: 10.1109/TNSM.2018.2799000 http://doi.org/10.1109/TNSM.2018.2799000 . .

CH Xie , , , JY Wang , , , ZS Zhang , , , 等 . . Adversarial examples for semantic segmentation and object detection . . IEEE Int Conf on Computer Vision , , 2017 . . p.1378 - - 1387 . . DOI: 10.1109/ICCV.2017.153 http://doi.org/10.1109/ICCV.2017.153 . .

Y Xin , , , LS Kong , , , Z Liu , , , 等 . . Machine learning and deep learning methods for cybersecurity . . IEEE Access , , 2018 . . 6 35365 - - 35381 . . DOI: 10.1109/ACCESS.2018.2836950 http://doi.org/10.1109/ACCESS.2018.2836950 . .

WL Xu , , , D Evans , , , YJ Qi . . Feature squeezing mitigates and detects Carlini/Wagner adversarial examples . . https://arxiv.org/abs/1705.10686 https://arxiv.org/abs/1705.10686 , , 2017 . . .

XY Yuan . . PhD forum: deep learning-based real-time malware detection with multi-stage analysis . . IEEE Int Conf on Smart Computing , , 2017 . . p.1 - - 2 . . DOI: 10.1109/SMARTCOMP.2017.7946997 http://doi.org/10.1109/SMARTCOMP.2017.7946997 . .

GZ Zhao , , , CX Zhang , , , LJ Zheng . . Intrusion detection using deep belief network and probabilistic neural network . . IEEE Int Conf on Computational Science and Engineering and IEEE Int Conf on Embedded and Ubiquitous Computing , , 2017 . . p.639 - - 642 . . DOI: 10.1109/CSE-EUC.2017.119 http://doi.org/10.1109/CSE-EUC.2017.119 . .

DL Zhu , , , H Jin , , , Y Yang , , , 等 . . DeepFlow: deep learning-based malware detection by mining Android application for abnormal usage of sensitive data . . IEEE Symp on Computers and Communications , , 2017 . . p.438 - - 443 . . DOI: 10.1109/ISCC.2017.8024568 http://doi.org/10.1109/ISCC.2017.8024568 . .

M Zolotukhin , , , T Hmlinen , , , T Kokkonen , , , 等 . . Increasing web service availability by detecting application-layer DDoS attacks in encrypted traffic . . Proc 23 rd Int Conf on Telecommunications , , 2016 . . p.1 - - 6 . . DOI: 10.1109/ICT.2016.7500408 http://doi.org/10.1109/ICT.2016.7500408 . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

Deep3DSketch-im: rapid high-fidelity AI 3D model generation by single freehand sketches

Artificial intelligence algorithms for cyberspace security applications: a technological and status review