基于深度Q学习网络与新训练算法的服务机器人主动物品检测模型

刘少鹏; 田国会; 崔永成; 邵旭阳

doi:10.1631/FITEE.2200109

Your Location：

Home >

Browse articles >

基于深度Q学习网络与新训练算法的服务机器人主动物品检测模型

常规文章 | Updated：2023-01-11

- 基于深度Q学习网络与新训练算法的服务机器人主动物品检测模型
- A deep Q-learning network based active object detection model with a novel training algorithm for service robots
- 信息与电子工程前沿（英文） 2022年23卷第11期页码：1673-1683
- Affiliations：
  
  School of Control Science and Engineering, Shandong University, Jinan 250061, China
- Author bio：
  
  †E-mail: shaopeng.liu66@mail.sdu.edu.cn;
  ‡Corresponding author
- Funds：
  
  National Natural Science Foundation of China(U1813215;62273203)
- DOI：10.1631/FITEE.2200109
  中图分类号： TP242
- 收稿：2022-03-20，
  
  录用：2022-07-29，
  
  网络出版：2022-09-24，
  
  纸质出版：2022-11-0
- Accepted：
Scan QR Code
刘少鹏, 田国会, 崔永成, 等. 基于深度Q学习网络与新训练算法的服务机器人主动物品检测模型[J]. 信息与电子工程前沿（英文）, 2022,23(11):1673-1683.

Shaopeng LIU, Guohui TIAN, Yongcheng CUI, et al. A deep Q-learning network based active object detection model with a novel training algorithm for service robots[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(11): 1673-1683.
刘少鹏, 田国会, 崔永成, 等. 基于深度Q学习网络与新训练算法的服务机器人主动物品检测模型[J]. 信息与电子工程前沿（英文）, 2022,23(11):1673-1683. DOI： 10.1631/FITEE.2200109.

Shaopeng LIU, Guohui TIAN, Yongcheng CUI, et al. A deep Q-learning network based active object detection model with a novel training algorithm for service robots[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(11): 1673-1683. DOI： 10.1631/FITEE.2200109.

摘要

本文研究了主动物品检测（AOD）问题。AOD是服务机器人在家庭环境中完成服务任务的重要组成部分，通过适当的移动动作引导机器人接近目标物品。目前基于强化学习的AOD模型存在训练效率低和测试精度差的问题。因此，本文提出一种基于深度Q学习网络的AOD模型，并设计了一种新的模型训练算法。该模型旨在拟合各种动作Q值，包括状态空间、特征提取和多层感知机。与现有研究不同，本文针对所提AOD模型设计了一种基于记忆的训练算法，以提高模型训练效率和测试精度。此外，提出一种最终状态生成方法判断训练过程中AOD任务何时停止。本文所提方法在AOD数据集上进行了充分的对比实验和消融实验。实验结果表明所提方法优于其他同类方法，所设计的训练算法比原始训练算法更高效。

Abstract

This paper focuses on the problem of active object detection (AOD). AOD is important for service robots to complete tasks in the family environment

and leads robots to approach the target object by taking appropriate moving actions. Most of the current AOD methods are based on reinforcement learning with low training efficiency and testing accuracy. Therefore

an AOD model based on a deep Q-learning network (DQN) with a novel training algorithm is proposed in this paper. The DQN model is designed to fit the Q-values of various actions

and includes state space

feature extraction

and a multilayer perceptron. In contrast to existing research

a novel training algorithm based on memory is designed for the proposed DQN model to improve training efficiency and testing accuracy. In addition

a method of generating the end state is presented to judge when to stop the AOD task during the training process. Sufficient comparison experiments and ablation studies are performed based on an AOD dataset

proving that the presented method has better performance than the comparable methods and that the proposed training algorithm is more effective than the raw training algorithm.

关键词

Keywords

references

Ammirato P , Poirson P , Park E , et al. , 2017 . A dataset for developing and benchmarking active vision . Proc IEEE Int Conf on Robotics and Automation , p. 1378 - 1385 . doi: 10.1109/ICRA.2017.7989164 http://doi.org/10.1109/ICRA.2017.7989164

Ammirato P , Berg AC , Košecká J , 2018 . Active vision dataset benchmark . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition Workshops , p. 2046 - 2049 . doi: 10.1109/CVPRW.2018.00277 http://doi.org/10.1109/CVPRW.2018.00277

Dos Reis DH , Welfer D , De Souza Leite Cuadros MA , et al. , 2019 . Mobile robot navigation using an object recognition software with RGBD images and the YOLO algorithm . Appl Artif Intell , 33 ( 14 ): 1290 - 1305 . doi: 10.1080/08839514.2019.1684778 http://doi.org/10.1080/08839514.2019.1684778

Duan KW , Bai S , Xie LX , et al. , 2019 . CenterNet: keypoint triplets for object detection . Proc IEEE/CVF Int Conf on Computer Vision , p. 6568 - 6577 . doi: 10.1109/ICCV.2019.00667 http://doi.org/10.1109/ICCV.2019.00667

Han XN , Liu HP , Sun FC , et al. , 2019 . Active object detection with multistep action prediction using deep Q-network . IEEE Trans Ind Inform , 15 ( 6 ): 3723 - 3731 . doi: 10.1109/TII.2019.2890849 http://doi.org/10.1109/TII.2019.2890849

He KM , Zhang XY , Ren SQ , et al. , 2016 . Deep residual learning for image recognition . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 770 - 778 . doi: 10.1109/CVPR.2016.90 http://doi.org/10.1109/CVPR.2016.90

Liu SP , Tian GH , Zhang Y , et al. , 2022a . Active object detection based on a novel deep Q-learning network and long-term learning strategy for the service robot . IEEE Trans Ind Electron , 69 ( 6 ): 5984 - 5993 . doi: 10.1109/TIE.2021.3090707 http://doi.org/10.1109/TIE.2021.3090707

Liu SP , Tian GH , Zhang Y , et al. , 2022b . Service planning oriented efficient object search: a knowledge-based framework for home service robot . Exp Syst Appl , 187 : 115853 . doi: 10.1016/j.eswa.2021.115853 http://doi.org/10.1016/j.eswa.2021.115853

Mnih V , Kavukcuoglu K , Silver D , et al. , 2015 . Human-level control through deep reinforcement learning . Nature , 518 ( 7540 ): 529 - 533 . doi: 10.1038/nature14236 http://doi.org/10.1038/nature14236

Mousavian A , Toshev A , Fišer M , et al. , 2019 . Visual representations for semantic target driven navigation . Proc IEEE Int Conf on Robotics and Automation , p. 8846 - 8852 . doi: 10.1109/ICRA.2019.8793493 http://doi.org/10.1109/ICRA.2019.8793493

Paletta L , Pinz A , 2000 . Active object recognition by view integration and reinforcement learning . Robot Autom Syst , 31 ( 1-2 ): 71 - 86 . doi: 10.1016/S0921-8890(99)00079-2 http://doi.org/10.1016/S0921-8890(99)00079-2

Pu SL , Zhao W , Chen WJ , et al. , 2021 . Unsupervised object detection with scene-adaptive concept learning . Front Inform Technol Electron Eng , 22 ( 5 ): 638 - 651 . doi: 10.1631/FITEE.2000567 http://doi.org/10.1631/FITEE.2000567

Ren SQ , He KM , Girshick R , et al. , 2017 . Faster R-CNN: towards real-time object detection with region proposal networks . IEEE Trans Patt Anal Mach Intell , 39 ( 6 ): 1137 - 1149 . doi: 10.1109/TPAMI.2016.2577031 http://doi.org/10.1109/TPAMI.2016.2577031

Schmid JF , Lauri M , Frintrop S , 2019 . Explore, approach, and terminate: evaluating subtasks in active visual object search based on deep reinforcement learning . Proc IEEE/RSJ Int Conf on Intelligent Robots and Systems , p. 5008 - 5013 . doi: 10.1109/IROS40897.2019.8967805 http://doi.org/10.1109/IROS40897.2019.8967805

Shuai W , Chen XP , 2019 . KeJia: towards an autonomous service robot with tolerance of unexpected environmental changes . Front Inform Technol Electron Eng , 20 ( 3 ): 307 - 317 . doi: 10.1631/FITEE.1900096 http://doi.org/10.1631/FITEE.1900096

Singh A , Sha J , Narayan KS , et al. , 2014 . BigBIRD: a large-scale 3D database of object instances . Proc IEEE Int Conf on Robotics and Automation , p. 509 - 516 . doi: 10.1109/ICRA.2014.6906903 http://doi.org/10.1109/ICRA.2014.6906903

van Hasselt H , Guez A , Silver D , 2016 . Deep reinforcement learning with double Q-learning . Proc AAAI Conf on Artificial Intelligence , p. 2094 - 2100 . doi: 10.1609/aaai.v30i1.10295 http://doi.org/10.1609/aaai.v30i1.10295

Wan SH , Goudos S , 2020 . Faster R-CNN for multi-class fruit detection using a robotic vision system . Comput Netw , 168 : 107036 . doi: 10.1016/j.comnet.2019.107036 http://doi.org/10.1016/j.comnet.2019.107036

Wang Q , Fan Z , Sheng WH , et al. , 2019 . Finding misplaced items using a mobile robot in a smart home environment . Front Inform Technol Electron Eng , 20 ( 8 ): 1036 - 1048 . doi: 10.1631/FITEE.1800275 http://doi.org/10.1631/FITEE.1800275

Xu QL , Fang F , Gauthier N , et al. , 2021 . Towards efficient multiview object detection with adaptive action prediction . Proc IEEE Int Conf on Robotics and Automation , p. 13423 - 13429 . doi: 10.1109/ICRA48506.2021.9561388 http://doi.org/10.1109/ICRA48506.2021.9561388

Zhang H , Liu HP , Guo D , et al. , 2017 . From foot to head: active face finding using deep Q-learning . Proc IEEE Int Conf on Image Processing , p. 1862 - 1866 . doi: 10.1109/ICIP.2017.8296604 http://doi.org/10.1109/ICIP.2017.8296604

Zhou XY , Zhuo JC , Krähenbühl P , 2019 . Bottom-up object detection by grouping extreme and center points . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 850 - 859 . doi: 10.1109/CVPR.2019.00094 http://doi.org/10.1109/CVPR.2019.00094

浏览量

151

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

暂无数据