基于中文维基的大规模命名实体识别语料自动生成方法

Zhou Jie; Li Bi-cheng; Chen Gang

doi:10.1631/FITEE.1500067

Your Location：

Home >

Browse articles >

基于中文维基的大规模命名实体识别语料自动生成方法

计算机与自动化 | Updated：2022-06-06

- 基于中文维基的大规模命名实体识别语料自动生成方法
- Automatically building large-scale named entity recognition corpora from Chinese Wikipedia*
- 信息与电子工程前沿（英文） 2015年16卷第11期页码：940-956
- Affiliations：
  
  Department of Signal Analysis and Information Processing, Zhengzhou Information Science and Technology Institute, Zhengzhou 450002, China
- Author bio：
  
  †E-mail:zhoujie.nlp@gmail.com
- Funds：
  
  *Project supported by the National Natural Science Foundation of China (No. 14BXW028)
- DOI：10.1631/FITEE.1500067
  中图分类号：
- 收稿：2015-03-07，
  
  修回：2015-10-15，
  
  录用：2015-08-09，
  
  纸质出版：2015-11
- Accepted：
Scan QR Code
基于中文维基的大规模命名实体识别语料自动生成方法[J]. 信息与电子工程前沿（英文）, 2015,16(11):940-956.

Zhou Jie, Li Bi-cheng, Chen Gang. Automatically building large-scale named entity recognition corpora from Chinese Wikipedia*[J]. Frontiers of Information Technology & Electronic Engineering, 2015, 16(11): 940-956.
基于中文维基的大规模命名实体识别语料自动生成方法[J]. 信息与电子工程前沿（英文）, 2015,16(11):940-956. DOI： 10.1631/FITEE.1500067.

Zhou Jie, Li Bi-cheng, Chen Gang. Automatically building large-scale named entity recognition corpora from Chinese Wikipedia*[J]. Frontiers of Information Technology & Electronic Engineering, 2015, 16(11): 940-956. DOI： 10.1631/FITEE.1500067.

摘要

目的

命名实体识别作为自然语言处理领域一项重要的基础性工作，当前主流方法是基于有监督的机器学习方法。该类方法依赖于特定语种和领域的标注语料，而语料的标注过程需耗费大量的人力、物力。本文提出一种基于中文维基的大规模命名实体识别（NER）语料自动生成方法。利用该方法能自动抽取并标记中文维基中的句子，从而为中文NER任务提供有效的语料支持。

创新点

本文根据中文维基的特点设计出四类启发式规则，并结合有监督的命名实体分类器，实现中文维基条目的命名实体类型的准确、全面识别；为避免缺失的维基链接引发的标注缺失，本文利用出链接的边界信息发现维基文档中的隐式指称项，并利用实体链接技术识别歧义指称项的实体类型；本文提出一种基于核心条目扩展的标注语料选择方法，实现测试数据的领域自适应。

方法

本文方法的整体流程如原文图2所示。该方法主要包括显式指称项的实体分类、隐式指称项的类型识别和标注语料选择三个主要步骤。在显式指称项的实体分类中，为实现准确、全面的实体类型识别，采用基于启发式规则与有监督实体分类器相结合的方法；在隐式指称项的类型识别中，提出一种新方法发现维基文档中的隐式指称项并识别歧义指称项的实体类型；在标注语料选择中，提出一种基于核心条目扩展的方法，实现测试数据的领域自适应。

结论

根据实验结果，采用本文方法能自动生成大规模的中文NER语料。此外，将生成语料与标准语料结合时，训练获得的NER模型性能更优。

Abstract

Named entity recognition (NER) is a core component in many natural language processing applications. Most NER systems rely on supervised machine learning methods

which depend on time-consuming and expensive annotations in different languages and domains. This paper presents a method for automatically building silver-standard NER corpora from Chinese Wikipedia. We refine novel and language-dependent features by exploiting the text and structure of Chinese Wikipedia. To reduce tagging errors caused by entity classification

we design four types of heuristic rules based on the characteristics of Chinese Wikipedia and train a supervised NE classifier

and a combined method is used to improve the precision and coverage. Then

we realize type identification of implicit mention by using boundary information of outgoing links. By selecting the sentences related with the domains of test data

we can train better NER models. In the experiments

large-scale NER corpora containing 2.3 million sentences are built from Chinese Wikipedia. The results show the effectiveness of automatically annotated corpora

and the trained NER models achieve the best performance when combining our silver-standard corpora with gold-standard corpora.

关键词

Keywords

references

F Alotaibi , M Lee . Mapping Arabic Wikipedia into the named entities taxonomy . 2012 . Proc. 24th Int. Conf. on Computational Linguistics . 43 - 52 . .

J An , S Lee , GG Lee . Automatic acquisition of named entity tagged corpus from World Wide Web . 2003 . Proc. 41st Annual Meeting on Association for Computational Linguistics . 165 - 168 . DOI: 10.3115/1075178.1075207 http://doi.org/10.3115/1075178.1075207 .

S Auer , C Bizer , G Kobilarov , 等 . DBpedia: a nucleus for a Web of open data . LNCS , 2007 . 4825 722 - 735 . DOI: 10.1007/978-3-540-76298-0_52 http://doi.org/10.1007/978-3-540-76298-0_52 .

D Balasuriya , N Ringland , J Nothman , 等 . Named entity recognition in Wikipedia . 2009 . Proc. Workshop on the People’s Web Meets NLP, ACL-IJCNLP . 10 - 18 . .

R Bunescu , M Paşca . Using encyclopedic knowledge for named entity disambiguation . 2006 . Proc. 11th Conf. of the European Chapter of the Association for Computational Linguistics . 9 - 16 . .

J Carletta . Assessing agreement on classification tasks: the kappa statistic . Comput Ling , 1996 . 22 ( 2 ): 249 - 254 . .

M Ciaramita , Y Altun . Named-entity recognition in novel domains with external lexical knowledge . 2005 . Proc. Human Language Technologies in Advances in Structured Learning for Text and Speech Processing Workshop . 209 - 212 . .

W Dakka , S Cucerzan . Augmenting Wikipedia with named entity tags . 2008 . Proc. Int. Joint Conf. on Natural Language Processing . 545 - 552 . .

K Darwish . Named entity recognition using cross-lingual resources: Arabic as an example . 2013 . Proc. 51st Annual Meeting of the Association for Computational Linguistics . 1558 - 1567 . .

M Ehrmann , M Turchi . Building multilingual named entity annotated corpora exploiting parallel corpora . 2010 . Proc. Workshop on Annotation and Exploitation of Parallel Corpora . 24 - 33 . .

O Etzioni , M Cafarella , D Downey , 等 . Unsupervised named-entity extraction from the Web: an experimental study . Artif Intell , 2005 . 165 ( 1 ): 91 - 134 . DOI: 10.1016/j.artint.2005.03.001 http://doi.org/10.1016/j.artint.2005.03.001 .

R Fu , B Qin , T Liu . Generating Chinese named entity data from a parallel corpus . 2011 . Proc. 5th Int. Joint Conf. on Natural Language Processing . 264 - 272 . .

E Gabrilovich , S Markovitch . Computing semantic relatedness using Wikipedia-based explicit semantic analysis . 2007 . Proc. 20th Int. Joint Conf. on Artificial Intelligence . 1606 - 1611 . .

H Guo , H Zhu , Z Guo , 等 . Domain adaptation with latent semantic association for named entity recognition . 2009 . Proc. Human Language Technologies: the Annual Conf. of the North American Chapter of the ACL . 281 - 289 . .

R Higashinaka , K Sadamitsu , K Saito , 等 . Creating an extended named entity dictionary from Wikipedia . 2012 . Proc. 24th Int. Conf. on Computational Linguistics . 1163 - 1178 . .

H Ji , R Grishman , HT Dang . Overview of the TAC2011 Knowledge Base Population Track . 2011 . Proc. Text Analysis Conf . .

J Jiang , CX Zhai . Exploiting domain structure for named entity recognition . 2006 . Proc. Main Conf. on Human Language Technology Conf. of the North American Chapter of the Association of Computational Linguistics . 74 - 81 . DOI: 10.3115/1220835.1220845 http://doi.org/10.3115/1220835.1220845 .

J Jiang , CX Zhai . A two-stage approach to domain adaptation for statistical classifiers . 2007 . Proc. 16th ACM Conf. on Information and Knowledge Management . 401 - 410 . DOI: 10.1145/1321440.1321498 http://doi.org/10.1145/1321440.1321498 .

W Liao , S Veeramachaneni . A simple semi-supervised algorithm for named entity recognition . 2009 . Proc. NAACL HLT Workshop on Semi-Supervised Learning for Natural Language Processing . 58 - 65 . .

H Liu , Y Chen . Computing semantic relatedness between named entities using Wikipedia . 2010 . Proc. Int. Conf. on Artificial Intelligence and Computational Intelligence . 388 - 392 . DOI: 10.1109/AICI.2010.88 http://doi.org/10.1109/AICI.2010.88 .

X Liu , S Zhang , F Wei , 等 . Recognizing named entities in Tweets . 2011 . Proc. 49th Annual Meeting of the Association for Computational Linguistics . 359 - 367 . .

O Medelyan , D Milne , C Legg , 等 . Mining meaning from Wikipedia . Int J Human-Comput Stud , 2009 . 67 ( 9 ): 716 - 754 . DOI: 10.1016/j.ijhcs.2009.05.004 http://doi.org/10.1016/j.ijhcs.2009.05.004 .

P Mika , M Ciaramita , H Zaragoza , 等 . Learning to tag and tagging to learn: a case study on Wikipedia . IEEE Intell Syst , 2008 . 23 ( 5 ): 26 - 33 . DOI: 10.1109/MIS.2008.85 http://doi.org/10.1109/MIS.2008.85 .

D Nadeau , PD Turney , S Matwin . Unsupervised named entity recognition: generating gazetteers and resolving ambiguity . LNCS , 2006 . 4013 266 - 277 . DOI: 10.1007/11766247_23 http://doi.org/10.1007/11766247_23 .

V Nastase , M Strube . Transforming Wikipedia into a large scale multilingual concept network . Artif Intell , 2013 . 194 62 - 85 . DOI: 10.1016/j.artint.2012.06.008 http://doi.org/10.1016/j.artint.2012.06.008 .

DM Nemeskey , E Simon . Automatically generated NE tagged corpora for English and Hungarian . 2012 . Proc. 4th Named Entity Workshop . 38 - 46 . .

Y Ni , L Zhang , Z Qiu , 等 . Enhancing the open-domain classification of named entity using linked open data . 2010 . Proc. 9th Int. Semantic Web Conf . 566 - 581 . .

J Nothman , JR Curran , T Murphy . Transforming Wikipedia into named entity training data . 2008 . Proc. Australian Language Technology Workshop . 124 - 132 . .

J Nothman , N Ringland , W Radford , 等 . Learning multilingual named entity recognition from Wikipedia . Artif Intell , 2013 . 194 151 - 175 . DOI: 10.1016/j.artint.2012.03.006 http://doi.org/10.1016/j.artint.2012.03.006 .

L Ratinov , D Roth . Design challenges and misconceptions in named entity recognition . 2009 . Proc. 13th Conf. on Computational Natural Language Learning . 147 - 155 . DOI: 10.3115/1596374.1596399 http://doi.org/10.3115/1596374.1596399 .

AE Richman , P Schone . Mining Wiki resources for multilingual named entity recognition . 2008 . Proc. 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies . 1 - 9 . .

A Toral , S Ferrández , M Monachini , 等 . Web 2.0, language resources and standards to automatically build a multilingual named entity lexicon . Lang Res Eval , 2012 . 46 ( 3 ): 383 - 419 . DOI: 10.1007/s10579-011-9148-x http://doi.org/10.1007/s10579-011-9148-x .

T Zesch , C Müller , I Gurevych . Extracting lexical semantic knowledge from Wikipedia and Wiktionary . 2008 . Proc. Conf. on Language Resources and Evaluation . 1646 - 1651 . .

W Zhang , L Sun , X Zhang . A entity relation extraction method based on Wikipedia and pattern clustering . J Chin Inform Process , 2012 . 26 ( 2 ): 75 - 81 . (in Chinese) .

J Zhou , X Dai , C Yin , 等 . Automatic recognition of Chinese organization name based on cascaded conditional random fields . Acta Electron Sin , 2006 . 34 ( 5 ): 804 - 809 . (in Chinese) .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

暂无数据