A deep Q-learning network based active object detection model with a novel training algorithm for service robots

Shaopeng LIU; Guohui TIAN; Yongcheng CUI; Xuyang SHAO

doi:10.1631/FITEE.2200109

Your Location：

Home >

Browse articles >

A deep Q-learning network based active object detection model with a novel training algorithm for service robots

Regular Papers | Updated：2023-01-11

- A deep Q-learning network based active object detection model with a novel training algorithm for service robots
- 基于深度Q学习网络与新训练算法的服务机器人主动物品检测模型
- Frontiers of Information Technology & Electronic Engineering Vol. 23, Issue 11, Pages: 1673-1683(2022)
- Affiliations：
  
  School of Control Science and Engineering, Shandong University, Jinan 250061, China
- Author bio：
  
  †E-mail: shaopeng.liu66@mail.sdu.edu.cn;
  ‡Corresponding author
- Funds：
- DOI：10.1631/FITEE.2200109
  CLC： TP242
- Received：20 March 2022，
  
  Accepted：29 July 2022，
  
  Published Online：24 September 2022，
  
  Published：0 November 2022
- Accepted：
Scan QR Code
Shaopeng LIU, Guohui TIAN, Yongcheng CUI, et al. A deep Q-learning network based active object detection model with a novel training algorithm for service robots[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(11): 1673-1683.
DOI：

Shaopeng LIU, Guohui TIAN, Yongcheng CUI, et al. A deep Q-learning network based active object detection model with a novel training algorithm for service robots[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(11): 1673-1683. DOI： 10.1631/FITEE.2200109.

摘要

本文研究了主动物品检测（AOD）问题。AOD是服务机器人在家庭环境中完成服务任务的重要组成部分，通过适当的移动动作引导机器人接近目标物品。目前基于强化学习的AOD模型存在训练效率低和测试精度差的问题。因此，本文提出一种基于深度Q学习网络的AOD模型，并设计了一种新的模型训练算法。该模型旨在拟合各种动作Q值，包括状态空间、特征提取和多层感知机。与现有研究不同，本文针对所提AOD模型设计了一种基于记忆的训练算法，以提高模型训练效率和测试精度。此外，提出一种最终状态生成方法判断训练过程中AOD任务何时停止。本文所提方法在AOD数据集上进行了充分的对比实验和消融实验。实验结果表明所提方法优于其他同类方法，所设计的训练算法比原始训练算法更高效。

Abstract

This paper focuses on the problem of active object detection (AOD). AOD is important for service robots to complete tasks in the family environment

and leads robots to approach the target object by taking appropriate moving actions. Most of the current AOD methods are based on reinforcement learning with low training efficiency and testing accuracy. Therefore

an AOD model based on a deep Q-learning network (DQN) with a novel training algorithm is proposed in this paper. The DQN model is designed to fit the Q-values of various actions

and includes state space

feature extraction

and a multilayer perceptron. In contrast to existing research

a novel training algorithm based on memory is designed for the proposed DQN model to improve training efficiency and testing accuracy. In addition

a method of generating the end state is presented to judge when to stop the AOD task during the training process. Sufficient comparison experiments and ablation studies are performed based on an AOD dataset

proving that the presented method has better performance than the comparable methods and that the proposed training algorithm is more effective than the raw training algorithm.

关键词

Keywords

references

Ammirato P , Poirson P , Park E , et al. , 2017 . A dataset for developing and benchmarking active vision . Proc IEEE Int Conf on Robotics and Automation , p. 1378 - 1385 . doi: 10.1109/ICRA.2017.7989164 http://doi.org/10.1109/ICRA.2017.7989164

Ammirato P , Berg AC , Košecká J , 2018 . Active vision dataset benchmark . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition Workshops , p. 2046 - 2049 . doi: 10.1109/CVPRW.2018.00277 http://doi.org/10.1109/CVPRW.2018.00277

Dos Reis DH , Welfer D , De Souza Leite Cuadros MA , et al. , 2019 . Mobile robot navigation using an object recognition software with RGBD images and the YOLO algorithm . Appl Artif Intell , 33 ( 14 ): 1290 - 1305 . doi: 10.1080/08839514.2019.1684778 http://doi.org/10.1080/08839514.2019.1684778

Duan KW , Bai S , Xie LX , et al. , 2019 . CenterNet: keypoint triplets for object detection . Proc IEEE/CVF Int Conf on Computer Vision , p. 6568 - 6577 . doi: 10.1109/ICCV.2019.00667 http://doi.org/10.1109/ICCV.2019.00667

Han XN , Liu HP , Sun FC , et al. , 2019 . Active object detection with multistep action prediction using deep Q-network . IEEE Trans Ind Inform , 15 ( 6 ): 3723 - 3731 . doi: 10.1109/TII.2019.2890849 http://doi.org/10.1109/TII.2019.2890849

He KM , Zhang XY , Ren SQ , et al. , 2016 . Deep residual learning for image recognition . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 770 - 778 . doi: 10.1109/CVPR.2016.90 http://doi.org/10.1109/CVPR.2016.90

Liu SP , Tian GH , Zhang Y , et al. , 2022a . Active object detection based on a novel deep Q-learning network and long-term learning strategy for the service robot . IEEE Trans Ind Electron , 69 ( 6 ): 5984 - 5993 . doi: 10.1109/TIE.2021.3090707 http://doi.org/10.1109/TIE.2021.3090707

Liu SP , Tian GH , Zhang Y , et al. , 2022b . Service planning oriented efficient object search: a knowledge-based framework for home service robot . Exp Syst Appl , 187 : 115853 . doi: 10.1016/j.eswa.2021.115853 http://doi.org/10.1016/j.eswa.2021.115853

Mnih V , Kavukcuoglu K , Silver D , et al. , 2015 . Human-level control through deep reinforcement learning . Nature , 518 ( 7540 ): 529 - 533 . doi: 10.1038/nature14236 http://doi.org/10.1038/nature14236

Mousavian A , Toshev A , Fišer M , et al. , 2019 . Visual representations for semantic target driven navigation . Proc IEEE Int Conf on Robotics and Automation , p. 8846 - 8852 . doi: 10.1109/ICRA.2019.8793493 http://doi.org/10.1109/ICRA.2019.8793493

Paletta L , Pinz A , 2000 . Active object recognition by view integration and reinforcement learning . Robot Autom Syst , 31 ( 1-2 ): 71 - 86 . doi: 10.1016/S0921-8890(99)00079-2 http://doi.org/10.1016/S0921-8890(99)00079-2

Pu SL , Zhao W , Chen WJ , et al. , 2021 . Unsupervised object detection with scene-adaptive concept learning . Front Inform Technol Electron Eng , 22 ( 5 ): 638 - 651 . doi: 10.1631/FITEE.2000567 http://doi.org/10.1631/FITEE.2000567

Ren SQ , He KM , Girshick R , et al. , 2017 . Faster R-CNN: towards real-time object detection with region proposal networks . IEEE Trans Patt Anal Mach Intell , 39 ( 6 ): 1137 - 1149 . doi: 10.1109/TPAMI.2016.2577031 http://doi.org/10.1109/TPAMI.2016.2577031

Schmid JF , Lauri M , Frintrop S , 2019 . Explore, approach, and terminate: evaluating subtasks in active visual object search based on deep reinforcement learning . Proc IEEE/RSJ Int Conf on Intelligent Robots and Systems , p. 5008 - 5013 . doi: 10.1109/IROS40897.2019.8967805 http://doi.org/10.1109/IROS40897.2019.8967805

Shuai W , Chen XP , 2019 . KeJia: towards an autonomous service robot with tolerance of unexpected environmental changes . Front Inform Technol Electron Eng , 20 ( 3 ): 307 - 317 . doi: 10.1631/FITEE.1900096 http://doi.org/10.1631/FITEE.1900096

Singh A , Sha J , Narayan KS , et al. , 2014 . BigBIRD: a large-scale 3D database of object instances . Proc IEEE Int Conf on Robotics and Automation , p. 509 - 516 . doi: 10.1109/ICRA.2014.6906903 http://doi.org/10.1109/ICRA.2014.6906903

van Hasselt H , Guez A , Silver D , 2016 . Deep reinforcement learning with double Q-learning . Proc AAAI Conf on Artificial Intelligence , p. 2094 - 2100 . doi: 10.1609/aaai.v30i1.10295 http://doi.org/10.1609/aaai.v30i1.10295

Wan SH , Goudos S , 2020 . Faster R-CNN for multi-class fruit detection using a robotic vision system . Comput Netw , 168 : 107036 . doi: 10.1016/j.comnet.2019.107036 http://doi.org/10.1016/j.comnet.2019.107036

Wang Q , Fan Z , Sheng WH , et al. , 2019 . Finding misplaced items using a mobile robot in a smart home environment . Front Inform Technol Electron Eng , 20 ( 8 ): 1036 - 1048 . doi: 10.1631/FITEE.1800275 http://doi.org/10.1631/FITEE.1800275

Xu QL , Fang F , Gauthier N , et al. , 2021 . Towards efficient multiview object detection with adaptive action prediction . Proc IEEE Int Conf on Robotics and Automation , p. 13423 - 13429 . doi: 10.1109/ICRA48506.2021.9561388 http://doi.org/10.1109/ICRA48506.2021.9561388

Zhang H , Liu HP , Guo D , et al. , 2017 . From foot to head: active face finding using deep Q-learning . Proc IEEE Int Conf on Image Processing , p. 1862 - 1866 . doi: 10.1109/ICIP.2017.8296604 http://doi.org/10.1109/ICIP.2017.8296604

Zhou XY , Zhuo JC , Krähenbühl P , 2019 . Bottom-up object detection by grouping extreme and center points . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 850 - 859 . doi: 10.1109/CVPR.2019.00094 http://doi.org/10.1109/CVPR.2019.00094

Views

151

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

Zhen Liang

Tao-ran Wu

Wan-wei Liu

Bai Xue

Wen-jing Yang

Ji Wang

Zheng-bin Pang

Related Institution

Institute for Quantum Information &amp

State Key Laboratory of High Performance Computing, National University of Defense Technology

State Key Laboratory of Computer Science Institute of Software, Chinese Academy of Sciences

University of Chinese Academy of Sciences

College of Computer Science and Technology, National University of Defense Technology

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰