Coordinated control of an intelligent wheelchair based on a brain-computer interface and speech recognition

Hong-tao WANG; Yuan-qing LI; Yuan-qing YU

doi:10.1631/jzus.C1400150

Your Location：

Home >

Browse articles >

Coordinated control of an intelligent wheelchair based on a brain-computer interface and speech recognition

Special Part Issue on Brain-Machine Interface (BMI) and Cyborg Intelligence | Updated：2022-05-19

- Coordinated control of an intelligent wheelchair based on a brain-computer interface and speech recognition
- 基于脑机接口与语音协同控制的智能轮椅
- Frontiers of Information Technology & Electronic Engineering Vol. 15, Issue 10, Pages: 832-838(2014)
- Affiliations：
  
  1.School of Information Engineering, Wuyi University, Jiangmen 529020, China
  2.School of Automation Science and Engineering, South China University of Technology, Guangzhou 510640, China
  3.Engineering Research Center for Massive Biometric Information Processing, Jiangmen 529020, China
- Author bio：
  
  Yuan-qing LI, E-mail:auyqli@scut.edu.cn
- Funds：
- DOI：10.1631/jzus.C1400150
  CLC： TP399;R318
- Received：26 April 2014，
  
  Revised：2014-;9-17，
  
  Published：2014-10
- Accepted：
Scan QR Code
Hong-tao WANG, Yuan-qing LI, Yuan-qing YU. Coordinated control of an intelligent wheelchair based on a brain-computer interface and speech recognition[J]. Frontiers of Information Technology & Electronic Engineering, 2014, 15(10): 832-838.
DOI：

Hong-tao WANG, Yuan-qing LI, Yuan-qing YU. Coordinated control of an intelligent wheelchair based on a brain-computer interface and speech recognition[J]. Frontiers of Information Technology & Electronic Engineering, 2014, 15(10): 832-838. DOI： 10.1631/jzus.C1400150.

摘要

研究目的

面向重症瘫痪（特别是伴随失语症）病人，充分发掘其有限对外交流信息的途径（脑电、语音等），利用脑机接口与语音识别协同控制的方式实现智能轮椅控制。

创新要点

采用脑电(事件关联电位、运动想象)和语音（单音，如"啊"）实现了轮椅的前进、左转、右转和停止功能。

研究方法

设计了轮椅控制硬件平台（图1）和协同控制算法（图4）。其中，事件关联电位脑机接口（P300）子算法识别受试者是否注视"S"闪烁键（闪烁键界面如图3所示），若受试者有控制意图，则转化成相应的启动指令；运动想象脑机接口子算法实时提取受试者脑电信号，并将其左/右手运动想象意图转化为左/右转控制指令；语音识别子算法识别受试者主动发出的单音（如"啊"），并转化为停止指令。

重要结论

受试者仅用视觉刺激，想象左/右手及单音（如"啊"）就能控制轮椅的前进、左转、右转、停止和启动。五位健康受试者参与了评估实验，达到了良好的控制效果，特别是最佳时间比和最佳路径比两项指标令人满意。

Abstract

An intelligent wheelchair is devised

which is controlled by a coordinated mechanism based on a brain-computer interface (BCI) and speech recognition. By performing appropriate activities

users can navigate the wheelchair with four steering behaviors (start

stop

turn left

and turn right). Five healthy subjects participated in an indoor experiment. The results demonstrate the efficiency of the coordinated control mechanism with satisfactory path and time optimality ratios

and show that speech recognition is a fast and accurate supplement for BCI-based control systems. The proposed intelligent wheelchair is especially suitable for patients suffering from paralysis (especially those with aphasia) who can learn to pronounce only a single sound (e.g.

'ah').

关键词

Keywords

references

M. Abraham , , , J. Ormrod . . Artificial speaking device for aphasic children . . Proc. 1st Joint BMES/EMBS Conf. , , 1999 . . p.665 - - 671 . . DOI: 10.1109/IEMBS.1999.802749 http://doi.org/10.1109/IEMBS.1999.802749 . .

R. Barea , , , L. Boquete , , , M. Mazp , , , 等 . . System for assisted mobility using eye movements based on electrooculography . . IEEE Trans. Neur. Syst. Rehabil. Eng. , , 2002 . . 10 ( ( 4 ): ): 209 - - 218 . . DOI: 10.1109/TNSRE.2002.806829 http://doi.org/10.1109/TNSRE.2002.806829 . .

G. Blanchard , , , B. Blankertz . . BCI competition 2003-data set IIa: spatial patterns of self-controlled brain rhythm modulations . . IEEE Trans. Biomed. Eng. , , 2004 . . 51 ( ( 6 ): ): 1062 - - 1066 . . DOI: 10.1109/TBME.2004.826691 http://doi.org/10.1109/TBME.2004.826691 . .

M.B. Bromberg . . Quality of life in amyotrophic lateral sclerosis . . Phys. Med. Rehabil. Clin. N. Amer. , , 2008 . . 19 ( ( 3 ): ): 591 - - 605 . . DOI: 10.1016/j.pmr.2008.02.005 http://doi.org/10.1016/j.pmr.2008.02.005 . .

S.R. Eddy . . Hidden Markov models . . Curr. Opin. Struct. Biol. , , 1996 . . 6 ( ( 3 ): ): 361 - - 365 . . DOI: 10.1016/S0959-440X(96)80056-X http://doi.org/10.1016/S0959-440X(96)80056-X . .

S. Furui . . Speaker-independent isolated word recognition using dynamic features of speech spectrum . . IEEE Trans. Acoust. Speech Signal Process. , , 1986 . . 34 ( ( 1 ): ): 52 - - 59 . . DOI: 10.1109/TASSP.1986.1164788 http://doi.org/10.1109/TASSP.1986.1164788 . .

D.D. Huang , , , K. Qian , , , D.Y. Fei , , , 等 . . Electroencephalography (EEG)-based brain-computer interface (BCI): a 2-D virtual wheelchair control based on event-related desynchronization/synchronization and state control . . IEEE Trans. Neur. Syst. Rehabil. Eng. , , 2012 . . 20 ( ( 3 ): ): 379 - - 388 . . DOI: 10.1109/TNSRE.2012.2190299 http://doi.org/10.1109/TNSRE.2012.2190299 . .

X.L. Huo , , , J. Wang , , , M. Ghovanloo . . A magnetoinductive sensor based wireless tongue-computer interface . . IEEE Trans. Neur. Syst. Rehabil. Eng. , , 2008 . . 16 ( ( 5 ): ): 497 - - 504 . . DOI: 10.1109/TNSRE.2008.2003375 http://doi.org/10.1109/TNSRE.2008.2003375 . .

I. Iturrate , , , J.M. Antelis , , , A. Kubler , , , 等 . . A noninvasive brain-actuated wheelchair based on a P300 neurophysiological protocol and automated navigation . . IEEE Trans. Robot. , , 2009 . . 25 ( ( 3 ): ): 614 - - 627 . . DOI: 10.1109/TRO.2009.2020347 http://doi.org/10.1109/TRO.2009.2020347 . .

K.F. Lee , , , H.W. Hon . . Speaker-independent phone recognition using hidden Markov models . . IEEE Trans. Acoust. Speech Signal Process. , , 1989 . . 37 ( ( 11 ): ): 1641 - - 1648 . . DOI: 10.1109/29.46546 http://doi.org/10.1109/29.46546 . .

Y.Q. Li , , , C.T. Guan . . Joint feature re-extraction and classification using an iterative semi-supervised support vector machine algorithm . . Mach. Learn. , , 2008 . . 71 ( ( 1 ): ): 33 - - 53 . . DOI: 10.1007/s10994-007-5039-1 http://doi.org/10.1007/s10994-007-5039-1 . .

Y.Q. Li , , , J. Long , , , T. Yu , , , 等 . . An EEG-based BCI system for 2-D cursor control by combining mu/beta rhythm and P300 potential . . IEEE Trans. Biomed. Eng. , , 2010 . . 57 ( ( 10 ): ): 2495 - - 2505 . . DOI: 10.1109/TBME.2010.2055564 http://doi.org/10.1109/TBME.2010.2055564 . .

Y.Q. Li , , , J.H. Pan , , , F. Wang , , , 等 . . A hybrid BCI system combining P300 and SSVEP and its application to wheelchair control . . IEEE Trans. Biomed. Eng. , , 2013 . . 60 ( ( 11 ): ): 3156 - - 3166 . . DOI: 10.1109/TBME.2013.2270283 http://doi.org/10.1109/TBME.2013.2270283 . .

T. Liu , , , Y.Q. Li , , , H.T. Wang . . An assistive system based on ultrasonic sensors for brain-controlled wheelchair to avoid obstacles . . Proc. 31st Chinese Control Conf. , , 2012 . . p.3741 - - 3744 . . . .

J.Y. Long , , , Y.Q. Li , , , H.T. Wang , , , 等 . . A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair . . IEEE Trans. Neur. Syst. Rehabil. Eng. , , 2012 . . 20 ( ( 5 ): ): 720 - - 729 . . DOI: 10.1109/TNSRE.2012.2197221 http://doi.org/10.1109/TNSRE.2012.2197221 . .

Y. Luo , , , H. Huang , , , Y. Zhang , , , 等 . . A novel endpoint detection method and its application on humancomputer interaction of intelligent wheelchair . . J. Chongqing Univ. Posts Telecommun. (Nat.. Sci. Ed.) , , 2011 . . 23 ( ( 4 ): ): 487 - - 491 . . . .

J.D.R. Millán , , , F. Galán , , , D. Vanhooydonck , , , 等 . . Asynchronous non-invasive brain-actuated control of an intelligent wheelchair . . Proc. Annual Int. Conf. of the IEEE Engineering in Medicine and Biology Society , , 2009 . . p.3361 - - 3364 . . DOI: 10.1109/IEMBS.2009.5332828 http://doi.org/10.1109/IEMBS.2009.5332828 . .

R.C. Panicker , , , S. Puthusserypady , , , Y. Sun . . An asynchronous P300 BCI with SSVEP-based control state detection . . IEEE Trans. Biomed. Eng. , , 2011 . . 58 ( ( 6 ): ): 1781 - - 1788 . . DOI: 10.1109/TBME.2011.2116018 http://doi.org/10.1109/TBME.2011.2116018 . .

N. Peixoto , , , H.G. Nik , , , H. Charkhkar . . Voice controlled wheelchairs: fine control by humming . . Comput. Methods Programs Biomed. , , 2013 . . 112 ( ( 1 ): ): 156 - - 165 . . DOI: 10.1016/j.cmpb.2013.06.009 http://doi.org/10.1016/j.cmpb.2013.06.009 . .

L.R. Rabiner , , , S.E. Levinson , , , M.M. Sondhi . . On the application of vector quantization and hidden Markov models to speaker-independent, isolated word recognition . . Bell Syst. Tech. J. , , 1983 . . 62 ( ( 4 ): ): 1075 - - 1105 . . DOI: 10.1002/j.1538-7305.1983.tb03115.x http://doi.org/10.1002/j.1538-7305.1983.tb03115.x . .

R. Raya , , , J.O. Roa , , , E. Rocon , , , 等 . . Wearable inertial mouse for children with physical and cognitive impairments . . Sens. Actuat. A , , 2010 . . 162 ( ( 2 ): ): 248 - - 259 . . DOI: 10.1016/j.sna.2010.04.019 http://doi.org/10.1016/j.sna.2010.04.019 . .

B. Rebsamen , , , E. Burdet , , , C.T. Guan , , , 等 . . A brain-controlled wheelchair based on P300 and path guidance . . Proc. 1st IEEE/RAS-EMBS Int. Conf. on Biomedical Robotics and Biomechatronics , , 2006 . . p.1101 - - 1106 . . DOI: 10.1109/BIOROB.2006.1639239 http://doi.org/10.1109/BIOROB.2006.1639239 . .

B. Rebsamen , , , C.T. Guan , , , H.H. Zhang , , , 等 . . A brain controlled wheelchair to navigate in familiar environments . . IEEE Trans. Neur. Syst. Rehabil. Eng. , , 2010 . . 18 ( ( 6 ): ): 590 - - 598 . . DOI: 10.1109/TNSRE.2010.2049862 http://doi.org/10.1109/TNSRE.2010.2049862 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Binary neural networks for speech recognition

A bidirectional brain-computer interface for effective epilepsy control

Coordinated standoff tracking of moving targets using differential geometry

Related Author

Yan-min QIAN

Xu XIANG

Yu QI

Fei-qiang MA

Ting-ting GE

Yue-ming WANG

Jun-ming ZHU

Jian-min ZHANG

Related Institution

Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University

SpeechLab, Department of Computer Science and Engineering, Shanghai Jiao Tong University

Qiushi Academy for Advanced Studies, Zhejiang University

Department of Computer Science, Zhejiang University

Second Affiliated Hospital of Zhejiang University College of Medicine

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰