用于规划快速变化无人机群的动态值迭代网络

李伟; 杨波威; 宋广华; 姜晓红

doi:10.1631/FITEE.1900712

Your Location：

Home >

Browse articles >

用于规划快速变化无人机群的动态值迭代网络

常规文章 | Updated：2022-05-19

- 用于规划快速变化无人机群的动态值迭代网络
- Dynamic value iteration networks for the planning of rapidly changing UAV swarms
- 信息与电子工程前沿（英文） 2021年22卷第5期页码：687-696
- Affiliations：
  
  School of Aeronautics and Astronautics, Zhejiang University, Hangzhou 310027, China
  School of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
- Author bio：
  
  [ "Wei LI, E-mail: li2ui2@zju.edu.cn" ]
  Bowei YANG, E-mail: boweiy@zju.edu.cn
  [ "Guanghua SONG, E-mail: ghsong@zju.edu.cn" ]
  [ "Xiaohong JIANG, E-mail: jiangxh@zju.edu.cn" ]
- Funds：
  
  Project supported by the National Natural Science Foundation of China (No. 61501399), the SAIC MOTOR (No. 1925), and the National Key R & D Program of China (No. 2018AAA0102302)
- DOI：10.1631/FITEE.1900712
  中图分类号： TP183;TP393.1
- 收稿：2019-12-19，
  
  修回：2020-10-20，
  
  网络出版：2021-01-12，
  
  纸质出版：2021-05
- Accepted：
Scan QR Code
李伟, 杨波威, 宋广华, 等. 用于规划快速变化无人机群的动态值迭代网络[J]. 信息与电子工程前沿（英文）, 2021,22(5):687-696.

Wei LI, Bowei YANG, Guanghua SONG, et al. Dynamic value iteration networks for the planning of rapidly changing UAV swarms[J]. Frontiers of Information Technology & Electronic Engineering, 2021, 22(5): 687-696.
李伟, 杨波威, 宋广华, 等. 用于规划快速变化无人机群的动态值迭代网络[J]. 信息与电子工程前沿（英文）, 2021,22(5):687-696. DOI： 10.1631/FITEE.1900712.

Wei LI, Bowei YANG, Guanghua SONG, et al. Dynamic value iteration networks for the planning of rapidly changing UAV swarms[J]. Frontiers of Information Technology & Electronic Engineering, 2021, 22(5): 687-696. DOI： 10.1631/FITEE.1900712.

摘要

在无人机自组网（UANET）中，稀疏且高速移动的无人机节点会动态改变无人机自组网的拓扑结构，这可能会导致无人机自组网服务性能问题。为规划快速变化的无人机群，本文提出一种动态值迭代网络（DVIN）模型，该模型利用无人机自组网的连接信息，采用场景式Q学习方法训练，生成状态值传播函数，使无人机节点能够自适应调节至新的物理位置。然后，评估了动态值迭代网络模型的性能，并将其与非支配排序遗传算法NSGA-II和穷举法比较。仿真结果表明，动态值迭代网络模型显著缩短了无人机节点路径规划的决策时间，且平均成功率更高。

Abstract

In an unmanned aerial vehicle ad-hoc network (UANET)

sparse and rapidly mobile unmanned aerial vehicles (UAVs)/nodes can dynamically change the UANET topology. This may lead to UANET service performance issues. In this study

for planning rapidly changing UAV swarms

we propose a dynamic value iteration network (DVIN) model trained using the episodic Q-learning method with the connection information of UANETs to generate a state value spread function

which enables UAVs/nodes to adapt to novel physical locations. We then evaluate the performance of the DVIN model and compare it with the non-dominated sorting genetic algorithm Ⅱ and the exhaustive method. Simulation results demonstrate that the proposed model significantly reduces the decision-making time for UAV/node path planning with a high average success rate.

关键词

Keywords

references

M Abadi , , , P Barham , , , JM Chen , , , 等 . . TensorFlow: a system for large-scale machine learning . . Proc 12 th USENIX Conf on Operating Systems Design and Implementation , , 2016 . . p.265 - - 283 . . . .

I Bekmezci , , , OK Sahingoz , , , Temel . . Flying ad-hoc networks (FANETs): a survey . . Ad Hoc Netw , , 2013 . . 11 ( ( 3 ): ): 1254 - - 1270 . . DOI: 10.1016/j.adhoc.2012.12.004 http://doi.org/10.1016/j.adhoc.2012.12.004 . .

R Bellman . . Dynamic programming . . Science , , 1966 . . 153 ( ( 3731 ): ): 34 - - 37 . . DOI: 10.1126/science.153.3731.34 http://doi.org/10.1126/science.153.3731.34 . .

DP Bertsekas . . Dynamic Programming and Optimal Control . . Athena Scientific, Belmont, USA , , 1995 . . .

YL Boureau , , , F Bach , , , Y LeCun , , , 等 . . Learning midlevel features for recognition . . Proc IEEE Computer Society Conf on Computer Vision and Pattern Recognition , , 2010 . . p.2559 - - 2566 . . DOI: 10.1109/CVPR.2010.5539963 http://doi.org/10.1109/CVPR.2010.5539963 . .

I Buck , , , T Foley , , , D Horn , , , 等 . . Brook for GPUs: stream computing on graphics hardware . . ACM Trans Graph , , 2004 . . 23 ( ( 3 ): ): 777 - - 786 . . DOI: 10.1145/1015706.1015800 http://doi.org/10.1145/1015706.1015800 . .

U Challita , , , W Saad , , , C Bettstetter . . Deep reinforcement learning for interference-aware path planning of cellular-connected UAVs . . Proc IEEE Int Conf on Communications , , 2018 . . p. 1 - - 7 . . DOI: 10.1109/ICC.2018.8422706 http://doi.org/10.1109/ICC.2018.8422706 . .

F Cruz , , , P Wppen , , , A Fazrie , , , 等 . . Action selection methods in a robotic reinforcement learning scenario . . Proc IEEE Latin American Conf on Computational Intelligence , , 2019 . . p. 1 - - 6 . . DOI: 10.1109/LA-CCI.2018.8625243 http://doi.org/10.1109/LA-CCI.2018.8625243 . .

K Deb , , , A Pratap , , , S Agarwal , , , 等 . . A fast and elitist multiobjective genetic algorithm: NSGA-Ⅱ . . IEEE Trans Evol Comput , , 2002 . . 6 ( ( 2 ): ): 182 - - 197 . . DOI: 10.1109/4235.996017 http://doi.org/10.1109/4235.996017 . .

RR Fontes . . Emulando Redes Sem Fio Com MininetWiFi , , 2019 . . https://github.com/ramonfontes/mn-wifi-book-pt/blob/master/preview-book.pdf https://github.com/ramonfontes/mn-wifi-book-pt/blob/master/preview-book.pdf , , . .

RR Fontes , , , S Afzal , , , SHB Brito , , , 等 . . Mininet-WiFi: emulating software-defined wireless networks . . Proc 11 th Int Conf on Network and Service Management , , 2015 . . p. 384 - - 389 . . DOI: 10.1109/CNSM.2015.7367387 http://doi.org/10.1109/CNSM.2015.7367387 . .

V Franois-Lavet , , , P Henderson , , , R Islam , , , 等 . . An introduction to deep reinforcement learning . . Found Trends Mach Learn , , 2018 . . 11 ( ( 3-4 ): ): 219 - - 354 . . DOI: 10.1561/2200000071 http://doi.org/10.1561/2200000071 . .

F Koohifar , , , A Kumbhar , , , I Guvenc . . Receding horizon multi-UAV cooperative tracking of moving RF source . . IEEE Commun Lett , , 2017 . . 21 ( ( 6 ): ): 1433 - - 1436 . . DOI: 10.1109/LCOMM.2016.2603977 http://doi.org/10.1109/LCOMM.2016.2603977 . .

A Krizhevsky , , , I Sutskever , , , GE Hinton . . ImageNet classification with deep convolutional neural networks . . Commun ACM , , 2017 . . 60 ( ( 6 ): ): 84 - - 90 . . DOI: 10.1145/3065386 http://doi.org/10.1145/3065386 . .

J Lee , , , BY Kang , , , DW Kim . . Fast genetic algorithm for robot path planning . . Electron Lett , , 2013 . . 49 ( ( 23 ): ): 1449 - - 1451 . . DOI: 10.1049/el.2013.3143 http://doi.org/10.1049/el.2013.3143 . .

V Mnih , , , K Kavukcuoglu , , , D Silver , , , 等 . . Human-level control through deep reinforcement learning . . Nature , , 2015 . . 518 ( ( 7540 ): ): 529 - - 533 . . DOI: 10.1038/nature14236 http://doi.org/10.1038/nature14236 . .

V Mnih , , , AP Badia , , , L Mirza , , , 等 . . Asynchronous methods for deep reinforcement learning . . Proc 33 rd Int Conf on Machine Learning , , 2016 . . p. 1928 - - 1937 . . . .

SF Niu , , , SH Chen , , , HY Guo , , , 等 . . Generalized value iteration networks: life beyond lattices . . Proc 32 nd AAAI Conf on Artificial Intelligence , , 2018 . . p.6246 - - 6253 . . . .

V Roberge , , , M Tarbouchi , , , G Labonte . . Comparison of parallel genetic algorithm and particle swarm optimization for real-time UAV path planning . . IEEE Trans Ind Inform , , 2013 . . 9 ( ( 1 ): ): 132 - - 141 . . DOI: 10.1109/TⅡ.2012.2198665 http://doi.org/10.1109/TⅡ.2012.2198665 . .

S Schaal . . Is imitation learning the route to humanoid robots . . Trends Cogn Sci , , 1999 . . 3 ( ( 6 ): ): 233 - - 242 . . DOI: 10.1016/s1364-6613(99)01327-3 http://doi.org/10.1016/s1364-6613(99)01327-3 . .

A Tamar , , , Y Wu , , , G Thomas , , , 等 . . Value iteration networks . . Proc 26 th Int Joint Conf on Artificial Intelligence , , 2017 . . p.4949 - - 4953 . . DOI: 10.24963/ijcai.2017/700 http://doi.org/10.24963/ijcai.2017/700 . .

M Tokic , , , G Palm . . Value-difference based exploration: adaptive control between epsilon-greedy and softmax . . Proc 34 th Annual German Conf on Advances in Artificial Intelligence , , 2011 . . p.335 - - 346 . . DOI: 10.1007/978-3-642-24455-1_33 http://doi.org/10.1007/978-3-642-24455-1_33 . .

CJCH Watkins , , , P Dayan . . Q-learning . . Mach Learn , , 1992 . . 8 ( ( 3-4 ): ): 279 - - 292 . . DOI: 10.1007/BF00992698 http://doi.org/10.1007/BF00992698 . .

CY Zhang , , , P Patras , , , H Haddadi . . Deep learning in mobile and wireless networking: a survey . . IEEE Commun Surv Tutor , , 2019 . . 21 ( ( 3 ): ): 2224 - - 2287 . . DOI: 10.1109/COMST.2019.2904897 http://doi.org/10.1109/COMST.2019.2904897 . .

T Zhang , , , Q Li , , , CS Zhang , , , 等 . . Current trends in the development of intelligent unmanned autonomous systems . . Front Inform Technol Electron Eng , , 2017 . . 18 ( ( 1 ): ): 68 - - 85 . . DOI: 10.1631/FITEE.1601650 http://doi.org/10.1631/FITEE.1601650 . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

An optimized formula for the two-point resistance of a cobweb resistance network and its potential application

A distributed variable density path search and simplification method for industrial manipulators with end-effector’s attitude constraints

A review of cooperative path planning of an unmanned aerial vehicle group