From Eliza to XiaoIce: challenges and opportunities with social chatbots

Heung-yeung SHUM; Xiao-dong HE; Di LI

doi:10.1631/FITEE.1700826

Your Location：

Home >

Browse articles >

From Eliza to XiaoIce: challenges and opportunities with social chatbots

2018 Special Issue on Artificial Intelligence 2.0: Theories and Applications | Updated：2022-05-19

- From Eliza to XiaoIce: challenges and opportunities with social chatbots
- 从Eliza到小冰：社交对话机器人的机遇和挑战
- Frontiers of Information Technology & Electronic Engineering Vol. 19, Issue 1, Pages: 10-26(2018)
- Affiliations：
  
  Microsoft Corporation, Redmond, WA 98052, USA
- Author bio：
  
  Heung-yeung SHUM, E-mail: hshum@microsoft.com
  [ "Xiao-dong HE, E-mail:xiaohe@microsoft.com" ]
  [ "Di LI, E-mail:lidi@microsoft.com" ]
- Funds：
- DOI：10.1631/FITEE.1700826
  CLC： TP391
- Received：10 December 2017，
  
  Revised：2018-;1-;8，
  
  Published：2018-01
- Accepted：
Scan QR Code
Heung-yeung SHUM, Xiao-dong HE, Di LI. From Eliza to XiaoIce: challenges and opportunities with social chatbots[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(1): 10-26.
DOI：

Heung-yeung SHUM, Xiao-dong HE, Di LI. From Eliza to XiaoIce: challenges and opportunities with social chatbots[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(1): 10-26. DOI： 10.1631/FITEE.1700826.

摘要

会话系统经过数十年研究与开发，从20世纪六七十年代的Eliza和Parry，到航空旅行信息系统（Airline Travel Information System

ATIS）项目中的自动任务完成系统，从智能个人助理Siri，再到社交对话机器人微软小冰，出现了多种形式。社交对话机器人的吸引力在于其不仅具有回应用户不同请求的能力，还具有与用户建立情感联系的能力。其中，后者通过满足用户对于沟通、情感及社会归属感的感性需求来完成。社交对话机器人的设计必须专注于用户参与度，同时也须考虑智商和情商。为了吸引用户和聊天机器人交流，我们将社交对话机器人的成功程度以每次会话中交流回合数（conversation-turns per session

CPS）来衡量。以小冰为例，在本文中我们讨论了从核心对话、视觉感知到可扩展技巧等一系列社交对话机器人构建中的重要技术，展示了小冰动态识别用户感情的能力，并在长时间交互中以适当的人际关系反应吸引用户。作为第一代与人工智能共生的人类，感情丰富且功能强大的社交对话机器人将很快变成我们生活中不可或缺的一部分。

Abstract

Conversational systems have come a long way since their inception in the 1960s. After decades of research and development

we have seen progress from Eliza and Parry in the 1960s and 1970s

to task-completion systems as in the Defense Advanced Research Projects Agency (DARPA) communicator program in the 2000s

to intelligent personal assistants such as Siri

in the 2010s

to today's social chatbots like XiaoIce. Social chatbots' appeal lies not only in their ability to respond to users' diverse requests

but also in being able to establish an emotional connection with users. The latter is done by satisfying users' need for communication

affection

as well as social belonging. To further the advancement and adoption of social chatbots

their design must focus on user engagement and take both intellectual quotient (IQ) and emotional quotient (EQ) into account. Users should want to engage with a social chatbot; as such

we define the success metric for social chatbots as conversation-turns per session (CPS). Using XiaoIce as an illustrative example

we discuss key technologies in building social chatbots from core chat to visual awareness to skills. We also show how XiaoIce can dynamically recognize emotion and engage the user throughout long conversations with appropriate interpersonal responses. As we become the first generation of humans ever living with artificial intelligenc (AI)

we have a responsibility to design social chatbots to be both useful and empathetic

so they will become ubiquitous and help society as a whole.

关键词

Keywords

references

F Alam , , , M Danieli , , , G Riccardi . . Annotating and modeling empathy in spoken conversations . . Comput Speech Lang , , 2017 . . 50 40 - - 61 . . DOI: 10.1016/j.csl.2017.12.003 http://doi.org/10.1016/j.csl.2017.12.003 . .

G Andreani , , , G di Fabbrizio , , , M Gilbert , , , 等 . . Letos DISCOH: collecting an annotated open corpus with dialogue acts and reward signals for natural language helpdesks . . Proc IEEE Spoken Language Technology Workshop , , 2006 . . p.218 - - 221 . . DOI: 10.1109/SLT.2006.326794 http://doi.org/10.1109/SLT.2006.326794 . .

D Bahdanau , , , K Cho , , , Y Bengio . . Neural machine translation by jointly learning to align and translate . . https://arxiv.org/abs/1409.0473 , , 2014 . . .

M Beldoch . . Sensitivity to expression of emotional meaning in three modes of communication . . In: Davitz JR (Ed.), The Communication of Emotional Meaning. McGraw-Hill, New York , , 1964 . . p.31 - - 42 . . . .

Y Bengio , , , R Ducharme , , , P Vincent , , , 等 . . A neural probabilistic language model . . Proc Neural Information Processing Systems , , 2003 . . p.1137 - - 1155 . . . .

HM Chen , , , MS Sun , , , CC Tu , , , 等 . . Neural sentiment classification with user and product attention . . Proc Conf on Empirical Methods in Natural Language Processing , , 2016 . . p.1650 - - 1659 . . . .

KM Colby . . Artificial Paranoia: a Computer Simulation of Paranoid Processes . . Pergamon Press INC. Maxwell House, New York, NY, England , , 1975 . . .

DA Dahl , , , M Bates , , , M Brown , , , 等 . . Expanding the scope of the ATIS task: the ATIS-3 corpus . . Proc Workshop on Human Language Technology , , 1994 . . p.43 - - 48 . . DOI: 10.3115/1075812.1075823 http://doi.org/10.3115/1075812.1075823 . .

L Deng , , , JY Li , , , JT Huang , , , 等 . . Recent advances in deep learning for speech research at Microsoft . . Proc IEEE Int Conf on Acoustics, Speech and Signal Processing , , 2013 . . p.8604 - - 8608 . . DOI: 10.1109/ICASSP.2013.6639345 http://doi.org/10.1109/ICASSP.2013.6639345 . .

AM Elkahky , , , Y Song , , , XD He . . A multi-view deep learning approach for cross domain user modeling in recommendation systems . . Proc 24 th Int Conf on World Wide Web , , 2015 . . p.278 - - 288 . . DOI: 10.1145/2736277.2741667 http://doi.org/10.1145/2736277.2741667 . .

H Fang , , , S Gupta , , , F Iandola , , , 等 . . From captions to visual concepts and back . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2015 . . p.1473 - - 1482 . . DOI: 10.1109/CVPR.2015.7298754 http://doi.org/10.1109/CVPR.2015.7298754 . .

P Fung , , , D Bertero , , , Y Wan , , , 等 . . Towards empathetic human-robot interactions . . Proc 17 th Int Conf on Intelligent Text and Computational Linguistics , , 2016 . . .

C Gan , , , Z Gan , , , XD He , , , 等 . . StyleNet: generating attractive visual captions with styles . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2017 . . p.3137 - - 3146 . . DOI: 10.1109/CVPR.2017.108 http://doi.org/10.1109/CVPR.2017.108 . .

H Gardner . . Frames of Mind: the Theory of Multiple Intelligences . . Basic Books, New York , , 1983 . . DOI: 10.2307/3324261 http://doi.org/10.2307/3324261 . .

J Glass , , , G Flammia , , , D Goodine , , , 等 . . Multilingual spoken-language understanding in the MIT Voyager system . . Speech Commun , , 1995 . . 17 ( ( 1 ): ): 1 - - 18 . . DOI: 10.1016/0167-6393(95)00008-C http://doi.org/10.1016/0167-6393(95)00008-C . .

D Goleman . . Emotional Intelligence: Why It Can Matter More than IQ . . Bloomsbury, Inc., New York, NY, England , , 1995 . . .

D Goleman . . Working with Emotional Intelligence . . Bloomsbury, Inc., New York, NY, England , , 1998 . . .

G Güzeldere , , , S Franchi . . Dialogues with colorful "personalities" of early AI . . Stanford Human Rev , , 1995 . . 4 ( ( 2 ): ): 161 - - 169 . . . .

KM He , , , YX Zhang , , , SQ Ren , , , 等 . . Deep residual learning for image recognition . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2016 . . p.770 - - 778 . . DOI: 10.1109/CVPR.2016.90 http://doi.org/10.1109/CVPR.2016.90 . .

XD He , , , L Deng . . Speech-centric information processing: an optimization-oriented approach . . Proc IEEE , , 2013 . . 101 ( ( 5 ): ): 116 - - 1135 . . , , DOI: 10.1109/JPROC.2012.2236631 http://doi.org/10.1109/JPROC.2012.2236631 . .

XD He , , , L Deng . . Deep learning for image-to-text generation: a technical overview . . IEEE Signal Process Mag , , 2017 . . 34 ( ( 6 ): ): 109 - - 116 . . DOI: 10.1109/MSP.2017.2741510 http://doi.org/10.1109/MSP.2017.2741510 . .

CT Hemphill , , , JJ Godfrey , , , GR Doddington . . The ATIS spoken language systems pilot corpus . . Proc Workshop on Speech and Natural Language , , 1990 . . p.96 - - 101 . . DOI: 10.3115/116580.116613 http://doi.org/10.3115/116580.116613 . .

G Hinton , , , L Deng , , , D Yu , , , 等 . . Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups . . IEEE Signal Process Mag , , 2012 . . 29 ( ( 6 ): ): 82 - - 97 . . DOI: 10.1109/MSP.2012.2205597 http://doi.org/10.1109/MSP.2012.2205597 . .

S Hochreiter , , , J Schmidhuber . . Long short-term memory . . Neur Comput , , 1997 . . 9 ( ( 8 ): ): 1735 - - 1780 . . DOI: 10.1162/neco.1997.9.8.1735 http://doi.org/10.1162/neco.1997.9.8.1735 . .

PS Huang , , , XD He , , , JF Gao , , , 等 . . Learning deep structured semantic models for web search using click through data . . Proc 22 nd ACM Int Conf on Information & Knowledge Management , , 2013 . . p.2333 - - 2338 . . DOI: 10.1145/2505515.2505665 http://doi.org/10.1145/2505515.2505665 . .

A Karpathy , , , FF Li . . Deep visual-semantic alignments for generating image descriptions . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2015 . . p.3128 - - 3137 . . DOI: 10.1109/CVPR.2015.7298932 http://doi.org/10.1109/CVPR.2015.7298932 . .

A Krizhevsky , , , I Sutskever , , , GE Hinton . . ImageNet classification with deep convolutional neural networks . . Proc 25 th Int Conf on Neural Information Processing Systems , , 2012 . . p.1097 - - 1105 . . . .

E Levin , , , S Narayanan , , , R Pieraccini , , , 等 . . The ATT-DARPA ommunicator mixed-initiative spoken dialog system . . 6 th Int Conf on Spoken Language Processing , , 2000 . . .

JW Li , , , M Galley , , , C Brockett , , , 等 . . A persona-based neural conversation model . . Proc 54 th Annual Meeting of the Association for Computational Linguistics , , 2016 . . p.944 - - 1003 . . . .

X Li , , , LL Mou , , , R Yan , , , 等 . . Stalematebreaker: a proactive content-introducing approach to automatic human-computer conversation . . Proc 25 th Int Joint Conf on Artificial Intelligence , , 2016 . . p.2845 - - 2851 . . . .

XD Liu , , , JF Gao , , , XD He , , , 等 . . Representation learning using multi-task deep neural networks for semantic classification and information retrieval . . Proc Annual Conf on North American Chapter of the ACL , , 2015 . . p.912 - - 921 . . . .

ZD Lu , , , H Li . . A deep architecture for matching short texts . . Proc Int Conf on Neural Information Processing Systems , , 2013 . . p.1367 - - 1375 . . . .

AH Maslow . . A theory of human motivation . . Psychol Rev , , 1943 . . 50 ( ( 4 ): ): 370 - - 396 . . . .

A Mathews , , , LX Xie , , , XM He . . SentiCap: generating image descriptions with sentiments . . Proc 30 th AAAI Conf on Artificial Intelligence , , 2016 . . p.3574 - - 3580 . . . .

G Mesnil , , , X He , , , L Deng , , , 等 . . Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding . . Interspeech , , 2013 . . p.3771 - - 3775 . . . .

G Mesnil , , , Y Dauphin , , , KS Yao , , , 等 . . Using recurrent neural networks for slot filling in spoken language understanding . . IEEE/ACM Trans Audio Speech Lang Process , , 2015 . . 23 ( ( 3 ): ): 530 - - 539 . . DOI: 10.1109/TASLP.2014.2383614 http://doi.org/10.1109/TASLP.2014.2383614 . .

T Mikolov , , , I Sutskever , , , K Chen , , , 等 . . Distributed representations of words and phrases and their compositionality . . Proc 26 th Int Conf on Neural Information Processing Systems , , 2013 . . p.3111 - - 3119 . . . .

E Mower , , , MJ Matarić , , , S Narayanan . . A framework for automatic human emotion classification using emotion profiles . . IEEE Trans Audio Speech Lang Process , , 2011 . . 19 ( ( 5 ): ): 1057 - - 1070 . . DOI: 10.1109/TASL.2010.2076804 http://doi.org/10.1109/TASL.2010.2076804 . .

KR Murphy . . A critique of emotional intelligence: what are the problems and how can they be fixed . . Pers Psychol- , , 2007 . . 60 ( ( 1 ): ): 235 - - 238 . . DOI: 10.1111/j.1744-6570.2007.00071_2.x http://doi.org/10.1111/j.1744-6570.2007.00071_2.x . .

PJ Price . . Evaluation of spoken language systems: the ATIS domain . . Proc Workshop on Speech and Natural Language , , 1990 . . p.91 - - 95 . . DOI: 10.3115/116580.116612 http://doi.org/10.3115/116580.116612 . .

Y Qian , , , YC Fan , , , WP Hu , , , 等 . . On the training aspects of deep neural network (DNN) for parametric TTS synthesis . . Proc IEEE Int Conf on Acoustics, Speech and Signal Processing , , 2014 . . p.3829 - - 3833 . . DOI: 10.1109/ICASSP.2014.6854318 http://doi.org/10.1109/ICASSP.2014.6854318 . .

A Raux , , , B Langner , , , D Bohus , , , 等 . . Let's go public! Taking a spoken dialog system to the real world . . 9 th European Conf on Speech Communication and Technology , , 2005 . . p.885 - - 888 . . . .

AI Rudnicky , , , EH Thayer , , , PC Constantinides , , , 等 . . Creating natural dialogs in the Carnegie Mellon communicator system . . 6 th European Conf on Speech Communication and Technology , , 1999 . . .

R Sarikaya . . The technology behind personal digital assistantsjan-overview of the system architecture and key components . . IEEE Signal Process Mag , , 2017 . . 34 ( ( 1 ): ): 67 - - 81 . . DOI: 10.1109/MSP.2016.2617341 http://doi.org/10.1109/MSP.2016.2617341 . .

R Sarikaya , , , PA Crook , , , A Marin , , , 等 . . An overview of end-to-end language understanding and dialog management for personal digital assistants . . Proc IEEE Spoken Language Technology Workshop , , 2016 . . p.391 - - 397 . . DOI: 10.1109/SLT.2016.7846294 http://doi.org/10.1109/SLT.2016.7846294 . .

S Seneff , , , E Hurley , , , R Lau , , , 等 . . Galaxy-Ⅱ: a reference architecture for conversational system development . . 5 th Int Conf on Spoken Language Processing , , 1998 . . .

IV Serban , , , T Klinger , , , G Tesauro , , , 等 . . Multiresolution recurrent neural networks: an application to dialogue response generation . . AAAI , , 2017 . . p.3288 - - 3294 . . . .

BA Shawar , , , E Atwell . . Different measurements metrics to evaluate a chatbot system . . Proc Workshop on Bridging the Gap: Academic and Industrial Research in Dialog Technologies , , 2007 . . p.89 - - 96 . . . .

SM Shieber . . Lessons from a restricted Turing test . . Commun ACM , , 1994 . . 37 ( ( 6 ): ): 70 - - 78 . . DOI: 10.1145/175208.175217 http://doi.org/10.1145/175208.175217 . .

R Socher , , , A Perelygin , , , JY Wu , , , 等 . . Recursive deep models for semantic compositionality over a sentiment treebank . . Proc Conf on Empirical Methods in Natural Language Processing , , 2013 . . p.1631 - - 1642 . . . .

R Song . . Image to poetry by cross-modality understanding with unpaired data . . Personal Communication , , 2018 . . .

A Sordoni , , , M Galley , , , M Auli , , , 等 . . A neural network approach to context-sensitive generation of conversational responses . . Proc Annual Conf on North American Chapter of the ACL , , 2015 . . p.196 - - 205 . . . .

I Sutskever , , , O Vinyals , , , QVV Le . . Sequence to sequence learning with neural networks . . NIPS , , 2014 . . p.1 - - 9 . . DOI: 10.1007/s10107-014-0839-0 http://doi.org/10.1007/s10107-014-0839-0 . .

R Tokuhisa , , , K Inui , , , Y Matsumoto . . Emotion classification using massive examples extracted from the web . . Proc 22 nd Int Conf on Computational Linguistics , , 2008 . . p.881 - - 888 . . . .

G Tur , , , R de Mori . . Spoken Language Understanding: Systems for Extracting Semantic Information from Speech . . John Wiley and Sons, New York, NY , , 2011 . . .

G Tur , , , L Deng . . Intent determination and spoken utterance classification. In: Tur G, de Mori R (Eds.), Spoken Language Understanding: Systems for Extracting Semantic Information from Speech . . John Wiley and Sons, New York, NY , , 2011 . . .

A Turing . . Computing machinery and intelligence . . Mind , , 1950 . . 59 433 - - 460 . . . .

A van den Oord , , , S Dieleman , , , HG Zen , , , 等 . . WaveNet: a generative model for raw audio . . 9 th ISCA Speech Synthesis Workshop , , 2016 . . p.125 . .

O Vinyals , , , QV Le . . A neural conversational model . . Proc 31 st Int Conf on Machine Learning , , 2015 . . .

O Vinyals , , , A Toshev , , , S Bengio , , , 等 . . Show and tell: a neural image caption generator . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2015 . . p.3156 - - 3164 . . DOI: 10.1109/CVPR.2015.7298935 http://doi.org/10.1109/CVPR.2015.7298935 . .

M Walker , , , J Aberdeen , , , J Boland , , , 等 . . DARPA Communicator dialog travel planning systems: the June 2000 data collection . . Proc 7 th European Conf on Speech Communication and Technology , , 2001 . . .

M Walker , , , AI Rudnicky , , , JS Aberdeen , , , 等 . . DARPA Communicator evaluation: progress from 2000 to 2001 . . Proc Int Conf on Spoken Language Processing , , 2002 . . p. 273 - - 276 . . . .

RS Wallace . . The anatomy of A.L.I.C.E. In: Epstein R, Roberts G, Beber G (Eds.), Parsing the Turing Test: Philosophical and Methodological Issues in the Quest for the Thinking Computer . . Springer, Dordrecht , , 2009 . . p.181 - - 210 . . DOI: 10.1007/978-1-4020-6710-5_13 http://doi.org/10.1007/978-1-4020-6710-5_13 . .

HN Wang , , , XD He , , , MW Chang , , , 等 . . Personalized ranking model adaptation for web search . . Proc 36 th Int ACM SIGIR Conf on Research and Development in Information Retrieval , , 2013 . . p.323 - - 332 . . DOI: 10.1145/2484028.2484068 http://doi.org/10.1145/2484028.2484068 . .

YY Wang , , , L Deng , , , A Acero . . Semantic frame-based spoken language understanding. In: Tur G, de Mori R (Eds.), Spoken Language Understanding: Systems for Extracting Semantic Information from Speech . . John Wiley and Sons, New York, NY , , 2011 . . .

ZY Wang , , , HX Wang , , , JR Wen , , , 等 . . An inference approach to basic level of categorization . . Proc 24 th ACM Int Conf on Information and Knowledge Management , , 2015 . . p.653 - - 662 . . DOI: 10.1145/2806416.2806533 http://doi.org/10.1145/2806416.2806533 . .

J Weizenbaum . . ELIZA-a computer program for the study of natural language communication between man and machine . . Commun ACM , , 1966 . . 9 ( ( 1 ): ): 36 - - 45 . . DOI: 10.1145/365153.365168 http://doi.org/10.1145/365153.365168 . .

TH Wen , , , D Vandyke , , , N Mrksic , , , 等 . . A network-based end-to-end trainable task-oriented dialogue system . . Proc 15 th Conf on European Chapter of the Association for Computational Linguistics , , 2016 . . p.438 - - 449 . . . .

JD Williams , , , S Young . . Partially observable Markov decision processes for spoken dialog systems . . Comput Speech Lang , , 2007 . . 21 ( ( 2 ): ): 393 - - 422 . . DOI: 10.1016/j.csl.2006.06.008 http://doi.org/10.1016/j.csl.2006.06.008 . .

W Xiong , , , J Droppo , , , XD Huang , , , 等 . . Achieving human parity in conversational speech recognition . . IEEE/ACM Trans Audio Speech Lang Process, in press , , 2016 . . DOI: 10.1109/TASLP.2017.2756440 http://doi.org/10.1109/TASLP.2017.2756440 . .

R Yan , , , YP Song , , , H Wu . . Learning to respond with deep neural networks for retrieval-based human-computer conversation system . . Proc 39 th Int ACM SIGIR Conf on Research and Development in Information Retrieval , , 2016 . . p.55 - - 64 . . , , DOI: 10.1145/2911451.2911542 http://doi.org/10.1145/2911451.2911542 . .

ZC Yang , , , XD He , , , JF Gao , , , 等 . . Stacked attention networks for image question answering . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2016a . . p.21 - - 29 . . DOI: 10.1109/CVPR.2016.10 http://doi.org/10.1109/CVPR.2016.10 . .

ZC Yang , , , DY Yang , , , C Dyer , , , 等 . . Hierarchical attention networks for document classification . . Proc 15 th Annual Conf on North American Chapter of the Association for Computational Linguistics: Human Language Technologies , , 2016b . . p.1480 - - 1489 . . . .

Z Yu , , , ZY Xu , , , AW Black , , , 等 . . Chatbot evaluation and database expansion via crowdsourcing . . Proc RE-WOCHAT Workshop of LREC , , 2016 . . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Visual knowledge in the big model era: retrospect and prospect

Quant 4.0: engineering quantitative investment with automated, explainable, and knowledge-driven artificial intelligence

Accurate estimation of 6-DoF tooth pose in 3D intraoral scans for dental applications using deep learning

Parallel cognition: hybrid intelligence for human-machine interaction and management

Smart grid dispatch powered by deep learning: a survey

Related Author

Yunhe PAN

Yi YANG

Wenguan WANG

Heung-Yeung SHUM

Lionel M. NI

Saizhuo WANG

Jian GUO

Hangzheng LIN

Related Institution

College of Computer Science and Technology, Zhejiang University

The Hong Kong University of Science and Technology

IDEA Research, International Digital Economy Academy

Hangzhou Dental Hospital

Angel Align Inc

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰