A network security entity recognition method based on feature template and CNN-BiLSTM-CRF

Ya QIN; Guo-wei SHEN; Wen-bo ZHAO; Yan-ping CHEN; Miao YU; Xin JIN

doi:10.1631/FITEE.1800520

Your Location：

Home >

Browse articles >

A network security entity recognition method based on feature template and CNN-BiLSTM-CRF

Regular Papers | Updated：2022-05-19

- A network security entity recognition method based on feature template and CNN-BiLSTM-CRF
- 一种基于特征模板和CNN-BiLSTM-CRF的网络安全实体识别方法
- Frontiers of Information Technology & Electronic Engineering Vol. 20, Issue 6, Pages: 872-884(2019)
- Affiliations：
  
  College of Computer Science and Technology, Guizhou University, Guiyang 550025, China
  Guizhou Provincial Key Laboratory of Public Big Data, Guiyang 550025, China
  Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100093, China
  National Computer Network Emergency Response Technical Team/Coordination Center of China, Beijing 100029, China
- Author bio：
  
  [ "E-mail: qyamail@163.com" ]
  Guo-wei SHEN, E-mail: gwshen@gzu.edu.cn
- Funds：
- DOI：10.1631/FITEE.1800520
  CLC： TP393.08
- Received：31 August 2018，
  
  Revised：2019-;6-11，
  
  Published：2019-06
- Accepted：
Scan QR Code
Ya QIN, Guo-wei SHEN, Wen-bo ZHAO, et al. A network security entity recognition method based on feature template and CNN-BiLSTM-CRF[J]. Frontiers of Information Technology & Electronic Engineering, 2019, 20(6): 872-884.
DOI：

Ya QIN, Guo-wei SHEN, Wen-bo ZHAO, et al. A network security entity recognition method based on feature template and CNN-BiLSTM-CRF[J]. Frontiers of Information Technology & Electronic Engineering, 2019, 20(6): 872-884. DOI： 10.1631/FITEE.1800520.

摘要

利用海量网络安全威胁情报数据，构建网络安全知识图谱实施深度关联分析和挖掘，可帮助识别安全威胁并提出相应防御措施。这已成为网络安全领域研究热点。本文针对网络安全文本数据，研究实体识别算法，为构建网络安全知识图谱奠定基础。传统方法难以识别网络安全领域的新实体、中英文混合安全实体等，且提取的特征不够充分。本文在神经网络模型基础上，提出基于特征模板的CNN-BiLSTM-CRF网络安全实体识别算法。首先构建人工特征模板，提取局部上下文特征。再利用CNN提取字符特征，与局部上下文特征结合，传入BiLSTM模型提取语义特征。最后利用CRF对安全实体进行标注。结果表明，在大规模网络安全数据集上，该方法优于其它算法，

值达到86%。

Abstract

By network security threat intelligence analysis based on a security knowledge graph (SKG)

multi-source threat intelligence data can be analyzed in a fine-grained manner. This has received extensive attention. It is difficult for traditional named entity recognition methods to identify mixed security entities in Chinese and English in the field of network security

and there are difficulties in accurately identifying network security entities because of insufficient features extracted. In this paper

we propose a novel FT-CNN-BiLSTM-CRF security entity recognition method based on a neural network CNN-BiLSTM-CRF model combined with a feature template (FT). The feature template is used to extract local context features

and a neural network model is used to automatically extract character features and text global features. Experimental results showed that our method can achieve an F-score of 86% on a large-scale network security dataset and outperforms other methods.

关键词

Keywords

references

J Bergstra , , , Y Bengio . . Random search for hyperparameter optimization . . J Mach Learn Res , , 2012 . . 13 ( ( 1 ): ): 281 - - 305 . . . .

JPC Chiu , , , E Nichols . . Named entity recognition with bidirectional LSTM-CNNs . . 2015 . . https://arxiv.org/abs/1511.08308 https://arxiv.org/abs/1511.08308 , , . .

R Collobert , , , J Weston . . A unified architecture for natural language processing: deep neural networks with multitask learning . . Proc ACM 25 th Int Conf on Machine Learning , , 2008 . . p.160 - - 167 . . DOI: 10.1145/1390156.1390177 http://doi.org/10.1145/1390156.1390177 . .

R Collobert , , , J Weston , , , L Bottou , , , 等 . . Natural language processing (almost) from scratch . . J Mach Learn Res , , 2011 . . 12 ( ( 1 ): ): 2493 - - 2537 . . . .

CH Dong , , , JJ Zhang , , , CQ Zong , , , 等 . . Character-based LSTM-CRF with radical-level features for Chinese named entity recognition. In: Lin CY, Xue N, Zhao D, et al. (Eds.), Natural Language Understanding and Intelligent Applications . . Springer, Cham , , 2016 . . p.239 - - 250 . . DOI: 10.1007/978-3-319-50496-4_20 http://doi.org/10.1007/978-3-319-50496-4_20 . .

C Dos Santos , , , V Guimares . . Boosting named entity recognition with neural character embeddings . . Proc 5 th Named Entity Workshop, joint with 53 rd ACL and the 7 th IJCNLP , , 2015 . . p.25 - - 33 . . DOI: 10.18653/v1/w15-3904 http://doi.org/10.18653/v1/w15-3904 . .

YH Feng , , , H Yu , , , G Sun , , , 等 . . Named entity recognition method based on BLSTM . . Comput Sci , , 2018 . . 45 ( ( 2 ): ): 261 - - 268 . . DOI: 10.11896/j.issn.1002-137X.2018.02.045 http://doi.org/10.11896/j.issn.1002-137X.2018.02.045 . .

JR Finkel , , , CD Manning . . Joint parsing and named entity recognition . . Human Language Technologies: the Annual Conf of the North American Chapter of the Association of Computational Linguistics , , 2009 . . p.326 - - 334 . . DOI: 10.3115/1620754.1620802 http://doi.org/10.3115/1620754.1620802 . .

FA Gers , , , A Schmidhuber , , , F Cummins . . Learning to forget: continual prediction with LSTM . . Neur Comput , , 2000 . . 12 ( ( 10 ): ): 2451 - - 2471 . . DOI: 10.1162/089976600300015015 http://doi.org/10.1162/089976600300015015 . .

C Goller , , , A Kuchler . . Learning task-dependent distributed representations by backpropagation through structure . . Proc Int Conf on Neural Networks , , 1996 . . p.347 - - 352 . . DOI: 10.1109/icnn.1996.548916 http://doi.org/10.1109/icnn.1996.548916 . .

J Hammerton . . Named entity recognition with long short-term memory . . Proc 7 th Conf on Natural Language Learning at HLT-NAACL , , 2003 . . p.172 - - 175 . . DOI: 10.3115/1119176.1119202 http://doi.org/10.3115/1119176.1119202 . .

S Hochreiter , , , J Schmidhuber . . Long short-term memory . . Neur Comput , , 1997 . . 9 ( ( 8 ): ): 1735 - - 1780 . . DOI: 10.1162/neco.1997.9.8.1735 http://doi.org/10.1162/neco.1997.9.8.1735 . .

ZH Huang , , , X Wei , , , Y Kai . . Bidirectional LSTM-CRF models for sequence tagging . . 2015 . . https://arxiv.org/abs/1508.01991 https://arxiv.org/abs/1508.01991 , , . .

A Joshi , , , R Lal , , , T Finin , , , 等 . . Extracting cybersecurity related linked data from text . . IEEE 7 th Int Conf on Semantic Computing , , 2013 . . p.252 - - 259 . . DOI: 10.1109/icsc.2013.50 http://doi.org/10.1109/icsc.2013.50 . .

R Koeling . . Chunking with maximum entropy models . . Proc 2 nd Workshop on Learning Language in Logic and the 4 th Conf on Computational Natural Language Learning , , 2000 . . p.139 - - 141 . . DOI: 10.3115/1117601.1117634 http://doi.org/10.3115/1117601.1117634 . .

JD Lafferty , , , A McCallum , , , FCN Pereira . . Conditional random fields: probabilistic models for segmenting and labeling sequence data . . 18 th Int Conf on Machine Learning , , 2001 . . p.282 - - 289 . . . .

G Lample , , , M Ballesteros , , , S Subramanian , , , 等 . . Neural architectures for named entity recognition . . Proc NAACL- HLT , , 2016 . . p.260 - - 270 . . DOI: 10.18653/v1/N16-1030 http://doi.org/10.18653/v1/N16-1030 . .

Y LCun , , , L Bottou , , , Y Bengio , , , 等 . . Gradient-based learning applied to document recognition . . Proc IEEE , , 1998 . . 86 ( ( 11 ): ): 2278 - - 2324 . . DOI: 10.1109/5.726791 http://doi.org/10.1109/5.726791 . .

JH Li . . Overview of the technologies of threat intelligence sensing, sharing and analysis in cyber space . . Chin J Network Inform Secur , , 2016 . . 2 ( ( 2 ): ): 16 - - 29 . . DOI: 10.11959/j.issn.2096-109x.2016.00028 http://doi.org/10.11959/j.issn.2096-109x.2016.00028 . .

W Liu , , , Y Li , , , H Duan , , , 等 . . Knowledge graph construction techniques . . J Comput Res Dev , , 2016 . . 53 ( ( 3 ): ): 582 - - 600 . . DOI: 10.7544/issn1000-1239.2016.20148228 http://doi.org/10.7544/issn1000-1239.2016.20148228 . .

G Luo , , , XJ Huang , , , CY Li , , , 等 . . Joint named entity recognition and disambiguation . . Proc Conf on Empirical Methods in Natural Language Processing , , 2015 . . p.879 - - 888 . . DOI: 10.18653/v1/d15-1104 http://doi.org/10.18653/v1/d15-1104 . .

XZ Ma , , , E Hovy . . End-to-end sequence labeling via bi- directional LSTM-CNNs-CRF . . 2016 . . DOI: 10.18653/v1/p16-1101 http://doi.org/10.18653/v1/p16-1101 . .

T Mikolov , , , K Chen , , , G Corrado , , , 等 . . Efficient estimation of word representations in vector space . . 2013a . . https://arxiv.org/abs/1301.3781 https://arxiv.org/abs/1301.3781 , , . .

T Mikolov , , , I Sutskever , , , K Chen , , , 等 . . Distributed representations of words and phrases and their compositionality . . 2013b . . https://arxiv.org/abs/1310.4546 https://arxiv.org/abs/1310.4546 , , . .

A Passos , , , V Kumar , , , A McCallum . . Lexicon infused phrase embeddings for named entity resolution . . Proc 18 th Conf on Computational Language Learning , , 2014 . . p.78 - - 86 . . DOI: 10.3115/v1/w14-1609 http://doi.org/10.3115/v1/w14-1609 . .

NY Peng , , , M Dredze . . Named entity recognition for Chinese social media with jointly trained embeddings . . Proc Conf on Empirical Methods in Natural Language Processing , , 2015 . . p.548 - - 554 . . DOI: 10.18653/v1/d15-1064 http://doi.org/10.18653/v1/d15-1064 . .

J Pennington , , , R Socher , , , C Manning . . Glove: global vectors for word representation . . Proc Conf on Empirical Methods in Natural Language Processing , , 2014 . . p.1532 - - 1543 . . DOI: 10.3115/v1/d14-1162 http://doi.org/10.3115/v1/d14-1162 . .

V Pham , , , T Bluche , , , C Kermorvant , , , 等 . . Dropout improves recurrent neural networks for handwriting recognition . . 14 th Int Conf on Frontiers in Handwriting Recognition , , 2014 . . p.285 - - 290 . . DOI: 10.1109/icfhr.2014.55 http://doi.org/10.1109/icfhr.2014.55 . .

QQ Qiu , , , DQ Miao , , , ZF Zhang . . Named entity recognition on Chinese microblog . . Comput Sci , , 2013 . . 40 ( ( 6 ): ): 196 - - 198 . . DOI: 10.3969/j.issn.1002-137X.2013.06.042 http://doi.org/10.3969/j.issn.1002-137X.2013.06.042 . .

LR Rabiner . . A tutorial on hidden Markov models and selected applications in speech recognition . . Proc IEEE , , 1989 . . 77 ( ( 2 ): ): 257 - - 286 . . DOI: 10.1109/5.18626 http://doi.org/10.1109/5.18626 . .

BZ Tang , , , HX Cao , , , XL Wang , , , 等 . . Evaluating word representation features in biomedical named entity recognition tasks . . Biomed Res Int , , 2014 . . 240403 DOI: 10.1155/2014/240403 http://doi.org/10.1155/2014/240403 . .

YM Yang . . An evaluation of statistical approaches to text categorization . . Inform Retriev , , 1999 . . 1 ( ( 1-2 ): ): 69 - - 90 . . DOI: 10.1023/A:1009982220290 http://doi.org/10.1023/A:1009982220290 . .

HK Yu , , , HP Zhang , , , Q Liu , , , 等 . . Chinese named entity identification using cascaded hidden Markov model . . J Commun , , 2006 . . 27 ( ( 2 ): ): 87 - - 94 . . DOI: 10.3321/j.issn:1000-436X.2006.02.013 http://doi.org/10.3321/j.issn:1000-436X.2006.02.013 . .

XY Zhang , , , T Wang , , , HW Chen . . Research on named entity recognition . . Comput Sci , , 2005 . . 32 ( ( 4 ): ): 44 - - 48 . . DOI: 10.3969/j.issn.1002-137X.2005.04.014 http://doi.org/10.3969/j.issn.1002-137X.2005.04.014 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Neural mesh refinement

Accurate estimation of 6-DoF tooth pose in 3D intraoral scans for dental applications using deep learning

Attention-based efficient robot grasp detection network

Adaptive neural network based boundary control of a flexible marine riser system with output constraints

Adaptive tracking control of high-order MIMO nonlinear systems with prescribed performance

Related Author

Yiyi LIAO

Lu YU

Xiang GAO

Zhiwei ZHU

Wanghui DING

Kaiwei SUN

Mengfei YU

Hangzheng LIN

Related Institution

Zhejiang Provincial Key Laboratory of Information Processing, Communication and Networking (IPCAN)

College of Information Science and Electronic Engineering, Zhejiang University

Hangzhou Dental Hospital

Angel Align Inc

Zhejiang University–University of Illinois at Urbana-Champaign Institute, Zhejiang University

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰