Temporality-enhanced knowledge memory network for factoid question answering

Xin-yu DUAN; Si-liang TANG; Sheng-yu ZHANG; Yin ZHANG; Zhou ZHAO; Jian-ru XUE; Yue-ting ZHUANG; Fei WU

doi:10.1631/FITEE.1700788

Your Location：

Home >

Browse articles >

Temporality-enhanced knowledge memory network for factoid question answering

2018 Special Issue on Artificial Intelligence 2.0: Theories and Applications | Updated：2022-05-19

- Temporality-enhanced knowledge memory network for factoid question answering
- 时序增强的知识记忆网络在问答中的应用
- Frontiers of Information Technology & Electronic Engineering Vol. 19, Issue 1, Pages: 104-115(2018)
- Affiliations：
  
  College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
  School of Information Management, Wuhan University, Wuhan 430000, China
  Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an 710049, China
- Author bio：
  
  Si-liang TANG, E-mail:siliang@zju.edu.cn
  Fei WU, E-mail:wufei@zju.edu.cn
- Funds：
- DOI：10.1631/FITEE.1700788
  CLC： TP391
- Received：25 November 2017，
  
  Revised：2018-;1-25，
  
  Published：2018-01
- Accepted：
Scan QR Code
Xin-yu DUAN, Si-liang TANG, Sheng-yu ZHANG, et al. Temporality-enhanced knowledge memory network for factoid question answering[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(1): 104-115.
DOI：

Xin-yu DUAN, Si-liang TANG, Sheng-yu ZHANG, et al. Temporality-enhanced knowledge memory network for factoid question answering[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(1): 104-115. DOI： 10.1631/FITEE.1700788.

摘要

问答系统旨在为人类以自然语言提出的问题提供具体答案。如何对问题做出有效回答是该领域的热点问题。在问答系统研究中，对于给定问题与其相应答案，现有方法通常侧重于模拟问答语料间语义相关性或句法关系。当一个问题包含的答案线索非常少时，这些模型大多受到影响。本文设计了一个名为时序增强型知识记忆网络（temporality-enhanced knowledge memory network

TE-KMN）的架构，并将该模型应用于一个名为Quiz Bowl的知识竞赛问答数据集。与多数现有方法不同，该模型不仅对文本内容进行编码，也对问题中连续语句间能够逐步揭示答案的时序线索进行编码。此外，该模型通过协同使用外部知识，能够更好理解给定问题。实验结果表明，该方法性能优于目前几种最先进方法。

Abstract

Question answering is an important problem that aims to deliver specific answers to questions posed by humans in natural language. How to efficiently identify the exact answer with respect to a given question has become an active line of research. Previous approaches in factoid question answering tasks typically focus on modeling the semantic relevance or syntactic relationship between a given question and its corresponding answer. Most of these models suffer when a question contains very little content that is indicative of the answer. In this paper

we devise an architecture named the temporality-enhanced knowledge memory network (TE-KMN) and apply the model to a factoid question answering dataset from a trivia competition called quiz bowl. Unlike most of the existing approaches

our model encodes not only the content of questions and answers

but also the temporal cues in a sequence of ordered sentences which gradually remark the answer. Moreover

our model collaboratively uses external knowledge for a better understanding of a given question. The experimental results demonstrate that our method achieves better performance than several state-of-the-art methods.

关键词

Keywords

references

D Bahdanau , , , K Cho , , , Y Bengio . . Neural machine translation by jointly learning to align and translate . . https://arxiv.org/abs/1409.0473 , , 2014 . . arXiv:1409.0473. . .

JW Bao , , , N Duan , , , M Zhou , , , 等 . . Knowledge-based question answering as machine translation . . 52 nd Annual Meeting of the Association for Computational Linguistics , , 2014 . . p.967 - - 976 . . DOI: 10.3115/v1/p14-1091 http://doi.org/10.3115/v1/p14-1091 . .

A Barrón-Cedeño , , , S Filice , , , GDS Martino , , , 等 . . Thread-level information for comment classification in community question answering . . 53 rd Annual Meeting of the Association for Computational Linguistics , , 2015 . . p.687 - - 693 . . DOI: 10.3115/v1/p15-2113 http://doi.org/10.3115/v1/p15-2113 . .

MW Bilotti , , , JL Elsas , , , JG Carbonell , , , 等 . . Rank learning for factoid question answering with linguistic and semantic constraints . . 19 th ACM Conf on Information and Knowledge Management , , 2010 . . p.459 - - 468 . . DOI: 10.1145/1871437.1871498 http://doi.org/10.1145/1871437.1871498 . .

JL Boyd-Graber , , , B Satinoff , , , H He , , , 等 . . Besting the quiz master: crowdsourcing incremental classification games . . Joint Conf on Empirical Methods in Natural Language Processing and Computational Natural Language Learning , , 2012 . . p.1290 - - 1301 . . . .

CE Carr . . Processing of temporal information in the brain . . Ann Rev Neurosci , , 1993 . . 16 ( ( 1 ): ): 223 - - 243 . . DOI: 10.1016/S0166-4115(96)80051-3 http://doi.org/10.1016/S0166-4115(96)80051-3 . .

D Chen , , , J Bolton , , , CD Manning . . A thorough examination of the CNN/daily mail reading comprehension task . . 54 th Annual Meeting of the Association for Computational Linguistics , , 2016 . . p.2358 - - 2367 . . DOI: 10.18653/v1/p16-1223 http://doi.org/10.18653/v1/p16-1223 . .

K Cho , , , B van Merrienboer , , , Ç Gülçehre , , , 等 . . Learning phrase representations using RNN encoder-decoder for statistical machine translation . . Conf on Empirical Methods in Natural Language Processing , , 2014 . . p.1724 - - 1734 . . DOI: 10.3115/v1/d14-1179 http://doi.org/10.3115/v1/d14-1179 . .

J Chorowski , , , D Bahdanau , , , D Serdyuk , , , 等 . . Attention-based models for speech recognition . . Advances in Neural Information Processing Systems , , 2015 . . p.577 - - 585 . . . .

S Ding , , , G Cong , , , C Lin , , , 等 . . Using conditional random fields to extract contexts and answers of questions from online forums . . 46 th Annual Meeting of the Association for Computational Linguistics , , 2008 . . p.710 - - 718 . . . .

A Figueroa , , , J Atkinson . . Maximum entropy context models for ranking biographical answers to open-domain definition questions . . 25 th AAAI Conf on Artificial Intelligence , , 2011 . . p.1173 - - 1179 . . . .

CJ Fillmore . . Frame semantics and the nature of language . . Ann New York Acad Sci , , 1976 . . 280 ( ( 1 ): ): 20 - - 32 . . DOI: 10.1111/j.1749-6632.1976.tb25467.x http://doi.org/10.1111/j.1749-6632.1976.tb25467.x . .

M Ghazvininejad , , , C Brockett , , , M Chang , , , 等 . . A knowledge-grounded neural conversation model . . https://arxiv.org/abs/1702.01932 , , 2017 . . arXiv:1702.01932 . .

A Graves , , , S Fernández , , , J Schmidhuber . . Bidirectional LSTM networks for improved phoneme classification and recognition . . 15 th Int Conf on Artificial Neural Networks , , 2005 . . p.799 - - 804 . . . .

J Huang , , , M Zhou , , , D Yang . . Extracting chatbot knowledge from online discussion forums . . 20 th Int Joint Conf on Artifical Intelligence , , 2007 . . p.423 - - 428 . . . .

RB Ivry . . The representation of temporal information in perception and motor control . . Curr Opin Neurobiol , , 1996 . . 6 ( ( 6 ): ): 851 - - 857 . . DOI: 10.1016/s0959-4388(96)80037-7 http://doi.org/10.1016/s0959-4388(96)80037-7 . .

M Iyyer , , , JL Boyd-Graber , , , LMB Claudino , , , 等 . . A neural network for factoid question answering over paragraphs . . Conf on Empirical Methods in Natural Language Processing , , 2014 . . p.633 - - 644 . . DOI: 10.3115/v1/d14-1070 http://doi.org/10.3115/v1/d14-1070 . .

M Iyyer , , , V Manjunatha , , , JL Boyd-Graber , , , 等 . . Deep unordered composition rivals syntactic methods for text classification . . 53 rd Annual Meeting of the Association for Computational Linguistics , , 2015 . . p.1681 - - 1691 . . DOI: 10.3115/v1/p15-1162 http://doi.org/10.3115/v1/p15-1162 . .

J Jeon , , , WB Croft , , , JH Lee . . Finding similar questions in large question and answer archives . . 14 th ACM Int Conf on Information and Knowledge Management , , 2005 . . p.84 - - 90 . . DOI: 10.1145/1099554.1099572 http://doi.org/10.1145/1099554.1099572 . .

SR Joty , , , A Barrón-Cedeño , , , GDS Martino , , , 等 . . Global thread-level inference for comment classification in community question answering . . Conf on Empirical Methods in Natural Language Processing , , 2015 . . p.573 - - 578 . . DOI: 10.18653/v1/d15-1068 http://doi.org/10.18653/v1/d15-1068 . .

P Jurczyk , , , E Agichtein . . Discovering authorities in question answer communities by using link analysis . . 16 th ACM Conf on Information and Knowledge Management , , 2007 . . p.919 - - 992 . . DOI: 10.1145/1321440.1321575 http://doi.org/10.1145/1321440.1321575 . .

N Kalchbrenner , , , E Grefenstette , , , P Blunsom . . A convolutional neural network for modeling sentences . . 52 nd Annual Meeting of the Association for Computational Linguistics , , 2014 . . p.655 - - 665 . . DOI: 10.3115/v1/p14-1062 http://doi.org/10.3115/v1/p14-1062 . .

QV Le , , , T Mikolov . . Distributed representations of sentences and documents . . 31 st Int Conf on Machine Learning , , 2014 . . p.1188 - - 1196 . . . .

B Li , , , MR Lyu , , , I King . . Communities of Yahoo! Answers and Baidu Zhidao: complementing or competing? . . Int Joint Conf on Neural Networks , , 2012 . . p.1 - - 8 . . DOI: 10.1109/ijcnn.2012.6252435 http://doi.org/10.1109/ijcnn.2012.6252435 . .

T Luong , , , H Pham , , , CD Manning . . Effective approaches to attention-based neural machine translation . . Conf on Empirical Methods in Natural Language Processing , , 2015 . . p.1412 - - 1421 . . DOI: 10.18653/v1/d15-1166 http://doi.org/10.18653/v1/d15-1166 . .

X Ma , , , EH Hovy . . End-to-end sequence labeling via bidirectional LSTM-CNNs-CRF . . 54 th Annual Meeting of the Association for Computational Linguistics , , 2016 . . p.1064 - - 1074 . . DOI: 10.18653/v1/p16-1101 http://doi.org/10.18653/v1/p16-1101 . .

T Mikolov , , , M Karafiát , , , L Burget , , , 等 . . Recurrent neural network based language model . . 11 th Annual Conf of the Int Speech Communication Association , , 2010 . . p.1045 - - 1048 . . . .

T Mikolov , , , K Chen , , , G Corrado , , , 等 . . Efficient estimation of word representations in vector space . . https://arxiv.org/abs/1301.3781 , , 2013 . . arXiv:1301.3781 . .

M Minsky . . Society of mind: a response to four reviews . . Artif Intell , , 1991 . . 48 ( ( 3 ): ): 371 - - 396 . . DOI: 10.1016/0004-3702(91)90036-J http://doi.org/10.1016/0004-3702(91)90036-J . .

N Nakashole , , , TM Mitchell . . A knowledge-intensive model for prepositional phrase attachment . . 53 rd Annual Meeting of the Association for Computational Linguistics , , 2015 . . p.365 - - 375 . . DOI: 10.3115/v1/p15-1036 http://doi.org/10.3115/v1/p15-1036 . .

R Navigli , , , P Velardi . . Learning word-class lattices for definition and hypernym extraction . . 48 th Annual Meeting of the Association for Computational Linguistics , , 2010 . . p.1318 - - 1327 . . . .

B Pan , , , H Li , , , Z Zhao , , , 等 . . MEMEN: multi-layer embedding with memory networks for machine comprehension . . https://arxiv.org/abs/1707.09098 , , 2017 . . arXiv:1707.09098 . .

Y Pan . . Heading toward artificial intelligence 2.0 . . Engineering , , 2016 . . 2 ( ( 4 ): ): 409 - - 413 . . DOI: 10.1016/J.ENG.2016.04.018 http://doi.org/10.1016/J.ENG.2016.04.018 . .

X Qiu , , , X Huang . . Convolutional neural tensor network architecture for community-based question answering . . 24 th Int Joint Conf on Artificial Intelligence , , 2015 . . p.1305 - - 1311 . . . .

AM Rush , , , S Chopra , , , J Weston . . A neural attention model for abstractive sentence summarization . . Conf on Empirical Methods in Natural Language Processing , , 2015 . . p.379 - - 389 . . DOI: 10.18653/v1/d15-1044 http://doi.org/10.18653/v1/d15-1044 . .

J Schweppe , , , R Rummer . . Attention, working memory, and long-term memory in multimedia learning: an integrated perspective based on process models of working memory . . Ed Psychol Rev , , 2013 . . 26 ( ( 2 ): ): 285 - - 306 . . DOI: 10.1007/s10648-013-9242-2 http://doi.org/10.1007/s10648-013-9242-2 . .

C Shah , , , J Pomerantz . . Evaluating and predicting answer quality in community QA . . 33 rd Int ACM SIGIR Conf on Research and Development in Information Retrieval , , 2010 . . p.411 - - 418 . . DOI: 10.1145/1835449.1835518 http://doi.org/10.1145/1835449.1835518 . .

Y Shen , , , W Rong , , , Z Sun , , , 等 . . Question/Answer matching for CQA system via combining lexical and sequential information . . 29 th AAAI Conf on Artificial Intelligence , , 2015 . . p.275 - - 281 . . . .

S Sukhbaatar , , , A Szlam , , , J Weston , , , 等 . . End-to-end memory networks . . Advances in Neural Information Processing Systems , , 2015 . . p.2440 - - 2448 . . . .

R Sun , , , J Jiang , , , YF Tan , , , 等 . . Using syntactic and semantic relation analysis in question answering . . 14 th Text REtrieval Conf. , , 2005 . . .

I Sutskever , , , O Vinyals , , , QV Le . . Sequence to sequence learning with neural networks . . Advances in Neural Information Processing Systems , , 2014 . . p.3104 - - 3112 . . . .

B Wang , , , X Wang , , , C Sun , , , 等 . . Modeling semantic relevance for question-answer pairs in web social communities . . 48 th Annual Meeting of the Association for Computational Linguistics , , 2010 . . p.1230 - - 1238 . . . .

M Wang . . A Survey of Answer Extraction Techniques in Factoid Question Answering . . https://cs.stanford.edu/people/mengqiu/publication/LSII-LitReview.pdf , , 2006 . . .

X Wei , , , H Huang , , , L Nie , , , 等 . . I know what you want to express: sentence element inference by incorporating external knowledge base . . IEEE Trans Knowl Data Eng , , 2017 . . 29 ( ( 2 ): ): 344 - - 358 . . DOI: 10.1109/TKDE.2016.2622705 http://doi.org/10.1109/TKDE.2016.2622705 . .

X Xue , , , J Jeon , , , WB Croft . . Retrieval models for question and answer archives . . 31 st Int ACM SIGIR Conf on Research and Development in Information Retrieval , , 2008 . . p.475 - - 482 . . DOI: 10.1145/1390334.1390416 http://doi.org/10.1145/1390334.1390416 . .

B Yang , , , TM Mitchell . . Leveraging knowledge bases in LSTMs for improving machine reading . . 55 th Annual Meeting of the Association for Computational Linguistics , , 2017 . . p.1436 - - 1446 . . DOI: 10.18653/v1/p17-1132 http://doi.org/10.18653/v1/p17-1132 . .

Y Yang , , , Y Zhuang , , , F Wu , , , 等 . . Harmonizing hierarchical manifolds for multimedia document semantics understanding and cross-media retrieval . . IEEE Trans Multimed , , 2008 . . 10 ( ( 3 ): ): 437 - - 446 . . DOI: 10.1109/TMM.2008.917359 http://doi.org/10.1109/TMM.2008.917359 . .

X Yao , , , BV Durme . . Information extraction over structured data: question answering with freebase . . 52 nd Annual Meeting of the Association for Computational Linguistics , , 2014 . . p.956 - - 966 . . DOI: 10.3115/v1/p14-1090 http://doi.org/10.3115/v1/p14-1090 . .

S Zheng , , , H Bao , , , J Zhao , , , 等 . . A novel hierarchical convolutional neural network for question answering over paragraphs . . IEEE/WIC/ACM Int Conf on Web Intelligence and Intelligent Agent Technology , , 2015 . . p.60 - - 66 . . DOI: 10.1109/WI-IAT.2015.20 http://doi.org/10.1109/WI-IAT.2015.20 . .

G Zhou , , , L Cai , , , J Zhao , , , 等 . . Phrase-based translation model for question retrieval in community question answer archives . . 49 th Annual Meeting of the Association for Computational Linguistics , , 2011 . . p.653 - - 662 . . . .

Y Zhuang , , , Y Yang , , , F Wu . . Mining semantic correlation of heterogeneous multimedia data for cross-media retrieval . . IEEE Trans Multimed , , 2008 . . 10 ( ( 2 ): ): 221 - - 229 . . DOI: 10.1109/TMM.2007.911822 http://doi.org/10.1109/TMM.2007.911822 . .

Y Zhuang , , , F Wu , , , C Chen , , , 等 . . Challenges and opportunities: from big data to knowledge in AI 2.0 . . Front Inform Technol Electron Eng , , 2017 . . 18 ( ( 1 ): ): 3 - - 14 . . DOI: 10.1631/FITEE.1601883 http://doi.org/10.1631/FITEE.1601883 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰