一种易用的实体识别消歧系统评测框架

陈辉; 魏宝刚; 李一鸣; 刘勇怀; 朱文浩

doi:10.1631/FITEE.1500473

Your Location：

Home >

Browse articles >

一种易用的实体识别消歧系统评测框架

常规文章 | Updated：2022-05-19

- 一种易用的实体识别消歧系统评测框架
- Antenna-in-package system integrated with meander line antenna based on LTCC technology
- 信息与电子工程前沿（英文） 2017年18卷第2期页码：195-205
- Affiliations：
  
  1College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
  2Department of Computer Science, Aberystwyth University, Ceredigion SY23 3DB, UK
  3School of Computer Engineering and Science, Shanghai University, Shanghai 200000, China
- Author bio：
  
  E-mail:chenhuicn@126.com
  E-mail:wbg@zju.edu.cn
- Funds：
  
  Project supported by the National Natural Science Foundation of China (No. 61572434), the China Knowledge Centre for Engineering Sciences and Technology (No. CKC-EST-2015-2-5), and the Specialized Research Fund for the Doctoral Program of Higher Education (SRFDP), China (No. 20130101110-136)
- DOI：10.1631/FITEE.1500473
  中图分类号： TP391.1
- 收稿：2015-12-26，
  
  录用：2016-3-13，
  
  纸质出版：2017-02
- Accepted：
Scan QR Code
陈辉, 魏宝刚, 李一鸣, 等. 一种易用的实体识别消歧系统评测框架[J]. 信息与电子工程前沿（英文）, 2017,18(2):195-205.

Hui CHEN, Bao-gang WEI, Yi-ming LI, et al. Antenna-in-package system integrated with meander line antenna based on LTCC technology[J]. Frontiers of Information Technology & Electronic Engineering, 2017, 18(2): 195-205.
陈辉, 魏宝刚, 李一鸣, 等. 一种易用的实体识别消歧系统评测框架[J]. 信息与电子工程前沿（英文）, 2017,18(2):195-205. DOI： 10.1631/FITEE.1500473.

Hui CHEN, Bao-gang WEI, Yi-ming LI, et al. Antenna-in-package system integrated with meander line antenna based on LTCC technology[J]. Frontiers of Information Technology & Electronic Engineering, 2017, 18(2): 195-205. DOI： 10.1631/FITEE.1500473.

摘要

实体识别消歧是知识库扩充和信息抽取的重要技术之一。近些年该领域诞生了很多研究成果，提出了许多实体识别消歧系统。但由于缺乏对这些系统的完善评测对比，该领域依然处于良莠淆杂的状态。因此很有必要设计一个评测框架对各个系统进行统一评测。本文提出一个实体识别消歧系统的统一评测框架，用于公平地比较各个实体识别消歧系统的效果。该框架代码开源，可以采用新的系统、数据集、评测机制扩展。通过该框架评测实体系统，可以分析得到系统各个模块的优劣之处。本文分析对比了几个公开的实体识别消歧系统，并总结出了一些有用的结论。

Abstract

Entity recognition and disambiguation (ERD) is a crucial technique for knowledge base population and information extraction. In recent years

numerous papers have been published on this subject

and various ERD systems have been developed. However

there are still some confusions over the ERD field for a fair and complete comparison of these systems. Therefore

it is of emerging interest to develop a unified evaluation framework. In this paper

we present an easy-to-use evaluation framework (EUEF)

which aims at facilitating the evaluation process and giving a fair comparison of ERD systems. EUEF is well designed and released to the public as an open source

and thus could be easily extended with novel ERD systems

datasets

and evaluation metrics. It is easy to discover the advantages and disadvantages of a specific ERD system and its components based on EUEF. We perform a comparison of several popular and publicly available ERD systems by using EUEF

and draw some interesting conclusions after a detailed analysis.

关键词

Keywords

references

C Bizer , , , J Lehmann , , , G Kobilarov , , , 等 . . DBpedia—a crystallization point for the Web of Data . . Web Semant. Sci. Serv. Agents World Wide Web , , 2009 . . 7 ( ( 3 ): ): 154 - - 165 . . DOI: 10.1016/j.websem.2009.07.002 http://doi.org/10.1016/j.websem.2009.07.002 . .

J Carletta . . Assessing agreement on classification tasks: the kappa statistic . . Comput. Ling. , , 1996 . . 22 ( ( 2 ): ): 249 - - 254 . . . .

M Cornolti , , , P Ferragina , , , M Ciaramita . . A framework for benchmarking entity-annotation systems . . 2013 . . Proc. 22nd Int. Conf. on World Wide Web . . 249 - - 260 . . . .

JR Finkel , , , T Grenager , , , C Manning . . Incorporating non-local information into information extraction systems by Gibbs sampling . . 2005 . . Proc. 43rd Annual Meeting on Association for Computational Linguistics . . 363 - - 370 . . DOI: 10.3115/1219840.1219885 http://doi.org/10.3115/1219840.1219885 . .

B Hachey , , , J Nothman , , , W Radford . . Cheap and easy entity evaluation . . 2014 . . Proc. 52nd Annual Meeting of the Association for Computational Linguistics . . 464 - - 469 . . . .

J Hoffart , , , MA Yosef , , , I Bordino , , , 等 . . Robust disambiguation of named entities in text . . 2011 . . Proc. Conf. on Empirical Methods in Natural Language Processing . . 782 - - 792 . . . .

H Ji , , , J Nothman , , , B Hachey , , , 等 . . Overview of TAC-KBP2014 entity discovery and linking tasks . . 2014 . . Proc. Text Analysis Conf. . . . .

H Ji , , , J Nothman , , , B Hachey , , , 等 . . Overview of TAC-KBP2015 tri-lingual entity discovery and linking . . 2015 . . Proc. Text Analysis Conf. . . . .

X Ling , , , S Singh , , , DS Weld . . Design challenges for entity linking . . Trans. Assoc. Comput. Ling. , , 2015 . . 3 315 - - 328 . . . .

D Milne , , , IH Witten . . Learning to link with Wikipedia . . 2008 . . Proc. 17th ACM Conf. on Information and Knowledge Management . . 509 - - 518 . . DOI: 10.1145/1458082.1458150 http://doi.org/10.1145/1458082.1458150 . .

D Milne , , , IH Witten . . An open-source toolkit for mining Wikipedia . . Artif. Intell. , , 2013 . . 194 222 - - 239 . . DOI: 10.1016/j.artint.2012.06.007 http://doi.org/10.1016/j.artint.2012.06.007 . .

L Ratinov , , , D Roth . . Design challenges and misconceptions in named entity recognition . . 2009 . . Proc. 13th Conf. on Computational Natural Language Learning . . 147 - - 155 . . DOI: 10.3115/1596374.1596399 http://doi.org/10.3115/1596374.1596399 . .

L Ratinov , , , D Roth , , , D Downey , , , 等 . . Local and global algorithms for disambiguation to Wikipedia . . 2011 . . Proc. 49th Annual Meeting of the Association for Computational Linguistics: Human Language . . 1375 - - 1384 . . . .

ES Ristad , , , PN Yianilos . . Learning string-edit distance . . IEEE Trans. Patt. Anal. Mach. Intell. , , 1998 . . 20 ( ( 5 ): ): 522 - - 532 . . DOI: 10.1109/34.682181 http://doi.org/10.1109/34.682181 . .

G Rizzo , , , M van Erp , , , R Troncy . . Benchmarking the extraction and disambiguation of named entities on the semantic web . . 2014 . . Proc. 9th Int. Conf. on Language Resources and Evaluation . . . .

W Shen , , , J Wang , , , J Han . . Entity linking with a knowledge base: issues, techniques, and solutions . . IEEE Trans. Knowl. Data Eng. , , 2015 . . 27 ( ( 2 ): ): 443 - - 460 . . DOI: 10.1109/TKDE.2014.2327028 http://doi.org/10.1109/TKDE.2014.2327028 . .

VI Spitkovsky , , , AX Chang . . A cross-lingual dictionary for English Wikipedia concepts . . 2012 . . 8th Int. Conf. on Language Resources and Evaluation . . 3168 - - 3175 . . . .

R Usbeck , , , M Röder , , , AC Ngonga Ngomo , , , 等 . . GERBIL: general entity annotator benchmarking framework . . 2015 . . Proc. 24th Int. Conf. on World Wide Web . . 1133 - - 1143 . . DOI: 10.1145/2736277.2741626 http://doi.org/10.1145/2736277.2741626 . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

A review on cyber security named entity recognition