Finding misplaced items using a mobile robot in a smart home environment

Qi WANG; Zhen FAN; Wei-hua SHENG; Sen-lin ZHANG; Mei-qin LIU

doi:10.1631/FITEE.1800275

Your Location：

Home >

Browse articles >

Finding misplaced items using a mobile robot in a smart home environment

Regular Papers | Updated：2022-05-19

- Finding misplaced items using a mobile robot in a smart home environment
- 智能家居环境中基于移动机器人的失物搜索
- Frontiers of Information Technology & Electronic Engineering Vol. 20, Issue 8, Pages: 1036-1048(2019)
- Affiliations：
  
  College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China
  School of Electrical and Computer Engineering, Oklahoma State University, Stillwater OK 74078, USA
- Author bio：
  
  Mei-qin LIU, E-mail: liumeiqin@zju.edu.cn
- Funds：
- DOI：10.1631/FITEE.1800275
  CLC： TP242.6
- Received：03 May 2018，
  
  Revised：2019-;8-;9，
  
  Published：2019-08
- Accepted：
Scan QR Code
Qi WANG, Zhen FAN, Wei-hua SHENG, et al. Finding misplaced items using a mobile robot in a smart home environment[J]. Frontiers of Information Technology & Electronic Engineering, 2019, 20(8): 1036-1048.
DOI：

Qi WANG, Zhen FAN, Wei-hua SHENG, et al. Finding misplaced items using a mobile robot in a smart home environment[J]. Frontiers of Information Technology & Electronic Engineering, 2019, 20(8): 1036-1048. DOI： 10.1631/FITEE.1800275.

摘要

智能家居可提供补充信息以帮助家居服务机器人。本文利用智能家居环境中机器人所获取的历史轨迹数据，提出一种机器人失物搜索（MIF）系统。首先，提出一种多传感器融合方法定位和跟踪用户。其次，利用人物历史轨迹所包含的知识，提出一种路径规划方法生成机器人搜索路径。再次，应用一种基于卷积神经网络的实时物体检测单元检测目标丢失物品。基于搭建的智能家居测试平台进行MIF系统相关实验。最后，实验结果验证了所提方案的准确性和有效性。

Abstract

Smart homes can provide complementary information to assist home service robots. We present a robotic misplaced item finding (MIF) system

which uses human historical trajectory data obtained in a smart home environment. First

a multi-sensor fusion method is developed to localize and track a resident. Second

a path-planning method is developed to generate the robot movement plan

which considers the knowledge of the human historical trajectory. Third

a real-time object detector based on a convolutional neural network is applied to detect the misplaced item. We present MIF experiments in a smart home testbed and the experimental results verify the accuracy and efficiency of our solution.

关键词

Keywords

references

SJ Ahern , , , J Carter , , , P Wilson . . Autonomous robot retrieval system . . SAI Intelligent Systems Conf , , 2015 . . p.280 - - 282 . . DOI: 10.1109/IntelliSys.2015.7361155 http://doi.org/10.1109/IntelliSys.2015.7361155 . .

JA Castellanos , , , J Neira , , , JD Tards . . Multisensor fusion for simultaneous localization and map building . . IEEE Trans Rob Autom , , 2001 . . 17 ( ( 6 ): ): 908 - - 914 . . DOI: 10.1109/70.976024 http://doi.org/10.1109/70.976024 . .

YS Choi , , , T Deyle , , , T Chen , , , 等 . . A list of household objects for robotic retrieval prioritized by people with ALS . . IEEE Int Conf on Rehabilitation Robotics , , 2009 . . p.510 - - 517 . . DOI: 10.1109/ICORR.2009.5209484 http://doi.org/10.1109/ICORR.2009.5209484 . .

S Ekvall , , , D Kragic , , , P Jensfelt . . Object detection and mapping for service robot tasks . . Robotica , , 2007 . . 25 ( ( 2 ): ): 175 - - 187 . . DOI: 10.1017/S0263574706003237 http://doi.org/10.1017/S0263574706003237 . .

T Fujii , , , H Ueda , , , M Minoh . . A looking-for-objects service in ubiquitous home . . J Nat Inst Inform Commun Technol , , 2007 . . 54 ( ( 3 ): ): 175 - - 181 . . . .

SM Huynh , , , D Parry , , , ACM Fong , , , 等 . . Home localization system for misplaced objects . . IEEE Int Conf on Consumer Electronics , , 2014 . . p.462 - - 463 . . DOI: 10.1109/ICCE.2014.6776087 http://doi.org/10.1109/ICCE.2014.6776087 . .

T Kollar , , , N Roy . . Utilizing object-object and object-scene context when planning to find things . . IEEE Int Conf on Robotics and Automation , , 2009 . . p.2168 - - 2173 . . DOI: 10.1109/ROBOT.2009.5152831 http://doi.org/10.1109/ROBOT.2009.5152831 . .

M Komatsuzaki , , , K Tsukada , , , I Siio , , , 等 . . IteMinder: finding items in a room using passive RFID tags and an autonomous robot (poster) . . Proc 13 th Int Conf on Ubiquitous Computing , , 2011 . . p.599 - - 600 . . DOI: 10.1145/2030112.2030232 http://doi.org/10.1145/2030112.2030232 . .

YC Lin , , , ST Wei , , , SA Yang , , , 等 . . Planning on searching occluded target object with a mobile robot manipulator . . IEEE Int Conf on Robotics and Automation , , 2015 . . p.3110 - - 3115 . . DOI: 10.1109/ICRA.2015.7139626 http://doi.org/10.1109/ICRA.2015.7139626 . .

Q Meng , , , MH Lee . . Design issues for assistive robotics for the elderly . . Adv Eng Inform , , 2006 . . 20 ( ( 2 ): ): 171 - - 186 . . DOI: 10.1016/j.aei.2005.10.003 http://doi.org/10.1016/j.aei.2005.10.003 . .

PIR . . Passive Infrared Sensor , , 2018 . . https://en.wikipedia.org/wiki/Passive_infrared_sensor https://en.wikipedia.org/wiki/Passive_infrared_sensor , , . .

K Ravi , , , KJ Bhavani , , , M Vinayak , , , 等 . . Identification of misplaced objects . . Indian J Sci Technol , , 2016 . . 9 ( ( 17 ): ): 1 - - 4 . . DOI: 10.17485/ijst/2016/v9i17/92991 http://doi.org/10.17485/ijst/2016/v9i17/92991 . .

J Redmon , , , S Divvala , , , R Girshick , , , 等 . . You only look once: unified, real-time object detection . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2016 . . p.779 - - 788 . . DOI: 10.1109/CVPR.2016.91 http://doi.org/10.1109/CVPR.2016.91 . .

JG Rogers , , , HI Christensen . . Robot planning with a semantic map . . IEEE Int Conf on Robotics and Automation , , 2013 . . p.2239 - - 2244 . . DOI: 10.1109/ICRA.2013.6630879 http://doi.org/10.1109/ICRA.2013.6630879 . .

ROS . . Robot Operating System , , 2018 . . http://www.ros.org/ http://www.ros.org/ , , . .

C Sadhu , , , MH Abhiram , , , B Chandan , , , 等 . . Cognitive learning enabled real time object search robot . . Int Conf on Control, Automation, Robotics and Embedded System , , 2013 . . p.1 - - 6 . . DOI: 10.1109/CARE.2013.6733739 http://doi.org/10.1109/CARE.2013.6733739 . .

D Schwarz , , , M Schwarz , , , J Stckler , , , 等 . . Cosero, find my keys . . Object localization and retrieval using bluetooth low energy tags. Proc 18 th RoboCup Int Symp , , 2014 . . p.195 - - 206 . . DOI: 10.1007/978-3-319-18615-3_16 http://doi.org/10.1007/978-3-319-18615-3_16 . .

M Susi , , , V Renaudin , , , G Lachapelle . . Motion mode recognition and step detection algorithms for mobile phone users . . Sensors , , 2013 . . 13 ( ( 2 ): ): 1539 - - 1562 . . DOI: 10.3390/s130201539 http://doi.org/10.3390/s130201539 . .

R Tesoriero , , , JA Gallud , , , MD Lozano , , , 等 . . Tracking autonomous entities using RFID technology . . IEEE Trans Consum Electron , , 2009 . . 55 ( ( 2 ): ): 650 - - 655 . . DOI: 10.1109/TCE.2009.5174435 http://doi.org/10.1109/TCE.2009.5174435 . .

Q Wang . . Youtube Video on Robotic Misplaced Item Finding System , , 2018 . . https://youtu.be/jEDKbr71pis https://youtu.be/jEDKbr71pis , , . .

Q Wang , , , SL Zhang , , , MQ Liu , , , 等 . . Retrieval of misplaced items using a mobile robot via visual object recognition . . IEEE 7 th Conf on CYBER Technology in Automation, Control, and Intelligent Systems , , 2017 . . p.1188 - - 1193 . . DOI: 10.1109/CYBER.2017.8446158 http://doi.org/10.1109/CYBER.2017.8446158 . .

D Xu , , , Q Huang , , , HW Liu . . Object detection on robot operation system . . IEEE 11 th Conf on Industrial Electronics and Applications , , 2016 . . p.1155 - - 1159 . . DOI: 10.1109/ICIEA.2016.7603758 http://doi.org/10.1109/ICIEA.2016.7603758 . .

Xunfei . . Voice Dictation , , 2018 . . http://www.xfyun.cn/services/voicedictation http://www.xfyun.cn/services/voicedictation , , . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Cloud-assisted cognition adaptation for service robots in changing home environments

Related Author

Qi WANG

Zhen FAN

Weihua SHENG

Senlin ZHANG

Meiqin LIU

Related Institution

School of Electrical and Computer Engineering, Oklahoma State University, Stillwater

Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰