基于注意机制编码解码模型的答案选择方法

聂原平; 韩毅; 黄九鸣; 焦波; 李爱平

doi:10.1631/FITEE.1601232

Your Location：

Home >

Browse articles >

基于注意机制编码解码模型的答案选择方法

常规文章 | Updated：2022-05-19

- 基于注意机制编码解码模型的答案选择方法
- Attention-based encoder-decoder model for answer selection in question answering
- 信息与电子工程前沿（英文） 2017年18卷第4期页码：535-544
- Affiliations：
  
  1College of Computer, National University of Defense Technology, Changsha 410073, China
  2Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100093, China
  3Luoyang Electronic Equipment Test Center, Luoyang 471003, China
- Author bio：
  
  E-mail:yuanpingnie@nudt.edu.cn
- Funds：
  
  Project supported by the National Basic Research Program (973) of China (Nos. 2013CB329601 and 2013CB329604) and the National Natural Science Foundation of China (Nos. 61372191, 61202362, and 61472433)
- DOI：10.1631/FITEE.1601232
  中图分类号： TP391.4
- 收稿：2016-03-15，
  
  录用：2016-10-15，
  
  纸质出版：2017-04
- Accepted：
Scan QR Code
聂原平, 韩毅, 黄九鸣, 等. 基于注意机制编码解码模型的答案选择方法[J]. 信息与电子工程前沿（英文）, 2017,18(4):535-544.

Yuan-ping NIE, Yi HAN, Jiu-ming HUANG, et al. Attention-based encoder-decoder model for answer selection in question answering[J]. Frontiers of Information Technology & Electronic Engineering, 2017, 18(4): 535-544.
聂原平, 韩毅, 黄九鸣, 等. 基于注意机制编码解码模型的答案选择方法[J]. 信息与电子工程前沿（英文）, 2017,18(4):535-544. DOI： 10.1631/FITEE.1601232.

Yuan-ping NIE, Yi HAN, Jiu-ming HUANG, et al. Attention-based encoder-decoder model for answer selection in question answering[J]. Frontiers of Information Technology & Electronic Engineering, 2017, 18(4): 535-544. DOI： 10.1631/FITEE.1601232.

摘要

问答技术的重要挑战之一就是解决问题与答案之间的语义空白。机器翻译模型已经被证明能有效的提升解决问题与答案之间的语义空白。本文提出了一种基于注意机制的深度神经网络模型来解决问答系统中的答案选择任务。该模型采用了基于双向长短时记忆（Long short-term memory

LSTM）的编码解码模型，编码解码模型是一个被证明再机器翻译领域取得了突出的成绩。我们还在模型中应用了注意力机制来提升模型的效果。本文在一个公开数据集上验证了实验的有效性，同时通过结合该模型显著提高了问答系统的性能在TREC 2015 liveQA的任务中。

Abstract

One of the key challenges for question answering is to bridge the lexical gap between questions and answers because there may not be any matching word between them. Machine translation models have been shown to boost the performance of solving the lexical gap problem between question-answer pairs. In this paper

we introduce an attention-based deep learning model to address the answer selection task for question answering. The proposed model employs a bidirectional long short-term memory (LSTM) encoder-decoder

which has been demonstrated to be effective on machine translation tasks to bridge the lexical gap between questions and answers. Our model also uses a step attention mechanism which allows the question to focus on a certain part of the candidate answer. Finally

we evaluate our model using a benchmark dataset and the results show that our approach outperforms the existing approaches. Integrating our model significantly improves the performance of our question answering system in the TREC 2015 LiveQA task.

关键词

Keywords

references

D Bahdanau , , , K Cho , , , Y Bengio . . Neural machine translation by jointly learning to align and translate . . ArXiv:1409.0473 , , 2014 . . .

A Berger , , , R Caruana , , , D Cohn , , , 等 . . Bridging the lexical chasm: statistical approaches to answer-finding . . 2000 . . 23rd Annual Int. ACM SIGIR Conf. on Research and Development in Information Retrieval . . 192 - - 199 . . DOI: 10.1145/345508.345576 http://doi.org/10.1145/345508.345576 . .

K Cho , , , B van Merriënboer , , , C Gulcehre , , , 等 . . Learning phrase representations using RNN encoder-decoder for statistical machine translation . . ArXiv:1406.1078 , , 2014 . . .

H Cui , , , R Sun , , , K Li , , , 等 . . Question answering passage retrieval using dependency relations . . 2005 . . Proc. 28th Annual Int. ACM SIGIR Conf. on Research and Development in Information Retrieval . . 400 - - 407 . . DOI: 10.1145/1076034.1076103 http://doi.org/10.1145/1076034.1076103 . .

C dos Santos , , , L Barbosa , , , D Bogdanova , , , 等 . . Learning hybrid representations to retrieve semantically equivalent questions . . 2015 . . Proc. 53rd Annual Meeting of the Association for Computational Linguistics and 7th Int. Joint Conf. on Natural Language Processing . . 694 - - 699 . . DOI: 10.3115/v1/P15-2114 http://doi.org/10.3115/v1/P15-2114 . .

A Echihabi , , , D Marcu . . A noisy-channel approach to question answering . . 2003 . . Proc. 41st Annual Meeting of the Association for Computational Linguistics . . 16 - - 23 . . DOI: 10.3115/1075096.1075099 http://doi.org/10.3115/1075096.1075099 . .

M Feng , , , B Xiang , , , MR Glass , , , 等 . . Applying deep learning to answer selection: a study and an open task . . ArXiv:1508.01585 , , 2015 . . .

A Graves , , , A Mohamed , , , GE Hinton . . Speech recognition with deep recurrent neural networks . . 2013 . . IEEE Int. Conf. on Acoustics, Speech and Signal Processing . . 6645 - - 6649 . . DOI: 10.1109/ICASSP.2013.6638947 http://doi.org/10.1109/ICASSP.2013.6638947 . .

M Heilman , , , NA Smith . . Tree edit models for recognizing textual entailments, paraphrases, and answers to questions . . 2010 . . Human Language Technologies: Annual Conf. of the North American Chapter of the Association for Computational Linguistics . . 1011 - - 1019 . . . .

S Hochreiter , , , J Schmidhuber . . Long short-term memory . . Neur. Comput. , , 1997 . . 9 ( ( 8 ): ): 1735 - - 1780 . . DOI: 10.1162/neco.1997.9.8.1735 http://doi.org/10.1162/neco.1997.9.8.1735 . .

M Iyyer , , , JL Boyd-Graber , , , LMB Claudino , , , 等 . . A neural network for factoid question answering over paragraphs . . 2014 . . Proc. Conf. on Empirical Methods in Natural Language Processing . . 633 - - 644 . . DOI: 10.3115/v1/D14-1070 http://doi.org/10.3115/v1/D14-1070 . .

J Jeon , , , WB Croft , , , JH Lee . . Finding similar questions in large question and answer archives . . 2005 . . Proc. 14th ACM Int. Conf. on Information and Knowledge Management . . 84 - - 90 . . DOI: 10.1145/1099554.1099572 http://doi.org/10.1145/1099554.1099572 . .

N Kalchbrenner , , , P Blunsom . . Recurrent continuous translation models . . 2013 . . Proc. Conf. on Empirical Methods in Natural Language Processing . . 1700 - - 1709 . . . .

Y Kim . . Convolutional neural networks for sentence classification . . ArXiv:1408.5882 , , 2014 . . .

V Punyakanok , , , D Roth , , , WT Yih . . Mapping dependencies trees: an application to question answering . . 2004 . . Proc. 8th Int. Symp. on Artificial Intelligence and Mathematics . . 1 - - 10 . . . .

S Riezler , , , A Vasserman , , , I Tsochantaridis , , , 等 . . Statistical machine translation for query expansion in answer retrieval . . 2007 . . Annual Meeting of the Association for Computational Linguistics . . 464 - - 471 . . . .

SE Robertson , , , S Walker , , , S Jones , , , 等 . . Okapi at TREC-3 . . 1995 . . Overview of 3rd Text REtrieval Conf. . . 109 - - 126 . . . .

AM Rush , , , S Chopra , , , J Weston . . A neural attention model for abstractive sentence summarization . . ArXiv: 1509.00685 , , 2015 . . .

A Severyn , , , A Moschitti . . Automatic feature engineering for answer selection and extraction . . 2013 . . Proc. Conf. on Empirical Methods in Natural Language Processing . . 458 - - 467 . . . .

A Severyn , , , A Moschitti . . Learning to rank short text pairs with convolutional deep neural networks . . 2015 . . Proc. 38th Int. ACM SIGIR Conf. on Research and Development in Information Retrieval . . 373 - - 382 . . DOI: 10.1145/2766462.2767738 http://doi.org/10.1145/2766462.2767738 . .

R Soricut , , , E Brill . . Automatic question answering using the web: beyond the factoid . . Inform. Retr. , , 2006 . . 9 ( ( 2 ): ): 191 - - 206 . . DOI: 10.1007/s10791-006-7149-y http://doi.org/10.1007/s10791-006-7149-y . .

M Surdeanu , , , M Ciaramita , , , H Zaragoza . . Learning to rank answers to non-factoid questions from web collections . . Comput. Ling. , , 2011 . . 37 ( ( 2 ): ): 351 - - 383 . . DOI: 10.1162/COLI_a_00051 http://doi.org/10.1162/COLI_a_00051 . .

I Sutskever , , , O Vinyals , , , QV Le . . Sequence to sequence learning with neural networks . . 2014 . . Advances in Neural Information Processing Systems . . 3104 - - 3112 . . . .

D Wang , , , E Nyberg . . A long short-term memory model for answer sentence selection in question answering . . 2015 . . Proc. 53rd Annual Meeting of the Association for Computational Linguistics and 7th Int. Joint Conf. on Natural Language Processing . . 707 - - 712 . . DOI: 10.3115/v1/P15-2116 http://doi.org/10.3115/v1/P15-2116 . .

M Wang , , , CD Manning . . Probabilistic tree-edit models with structured latent variables for textual entailment and question answering . . 2010 . . Proc. 23rd Int. Conf. on Computational Linguistics . . 1164 - - 1172 . . . .

M Wang , , , NA Smith , , , T Mitamura . . What is the jeopardy model? A quasi-synchronous grammar for QA . . 2007 . . Proc. Joint Conf. on Empirical Methods in Natural Language Processing and Computational Natural Language Learning . . 22 - - 32 . . . .

K Xu , , , J Ba , , , R Kiros , , , 等 . . Show, attend and tell: neural image caption generation with visual attention . . ArXiv:1502.03044 , , 2015 . . .

X Xue , , , J Jeon , , , WB Croft . . Retrieval models for question and answer archives . . 2008 . . Proc. 31st Annual Int. ACM SIGIR Conf. on Research and Development in Information Retrieval . . 475 - - 482 . . DOI: 10.1145/1390334.1390416 http://doi.org/10.1145/1390334.1390416 . .

X Yao , , , Durme B van , , , C Callison-Burch , , , 等 . . Answer extraction as sequence tagging with tree edit distance . . 2013a . . Proc. NAACL-HLT . . 858 - - 867 . . . .

X Yao , , , B van Durme , , , C Callisonburch , , , 等 . . Semi-Markov phrase-based monolingual alignment . . 2013b . . Proc. Conf. on Empirical Methods in Natural Language Processing . . 590 - - 600 . . . .

W Yih , , , M Chang , , , C Meek , , , 等 . . Question answering using enhanced lexical semantic models . . 2013 . . Proc. 51st Annual Meeting of the Association for Computational Linguistics . . 1744 - - 1753 . . . .

W Yih , , , X He , , , C Meek . . Semantic parsing for singlerelation question answering . . 2014 . . Proc. 52nd Annual Meeting of the Association for Computational Linguistics . . 643 - - 648 . . DOI: 10.3115/v1/P14-2105 http://doi.org/10.3115/v1/P14-2105 . .

L Yu , , , KM Hermann , , , P Blunsom , , , 等 . . Deep learning for answer sentence selection . . ArXiv:1412.1632 , , 2014 . . .

G Zhou , , , L Cai , , , J Zhao , , , 等 . . Phrase-based translation model for question retrieval in community question answer archives . . 2011 . . Proc. 49th Annual Meeting of the Association for Computational Linguistics . . 653 - - 662 . . . .

G Zhou , , , F Liu , , , Y Liu , , , 等 . . Statistical machine translation improves question retrieval in community question answering via matrix factorization . . 2013 . . Proc. 51st Annual Meeting of the Association for Computational Linguistics . . 852 - - 861 . . . .

G Zhou , , , Y Zhou , , , T He , , , 等 . . Learning semantic representation with neural networks for community question answering retrieval . . Knowl.-Based Syst. , , 2016 . . 93 75 - - 83 . . DOI: 10.1016/j.knosys.2015.11.002 http://doi.org/10.1016/j.knosys.2015.11.002 . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

MltAuxTSPP: a unified benchmark for deep learning-based traffic state prediction with multi-source auxiliary data

Q-space-coordinate-guided neural networks for high-fidelity diffusion tensor estimation from minimal diffusion-weighted images

CRGT-SA: an interlaced and spatiotemporal deep learning model for network intrusion detection

Federated model with contrastive learning and adaptive control variates for human activity recognition

A comprehensive survey of physical adversarial vulnerabilities in autonomous driving systems