Dynamic value iteration networks for the planning of rapidly changing UAV swarms

Wei LI; Bowei YANG; Guanghua SONG; Xiaohong JIANG

doi:10.1631/FITEE.1900712

Your Location：

Home >

Browse articles >

Dynamic value iteration networks for the planning of rapidly changing UAV swarms

Regular Papers | Updated：2022-05-19

- Dynamic value iteration networks for the planning of rapidly changing UAV swarms
- 用于规划快速变化无人机群的动态值迭代网络
- Frontiers of Information Technology & Electronic Engineering Vol. 22, Issue 5, Pages: 687-696(2021)
- Affiliations：
  
  School of Aeronautics and Astronautics, Zhejiang University, Hangzhou 310027, China
  School of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
- Author bio：
  
  [ "Wei LI, E-mail: li2ui2@zju.edu.cn" ]
  Bowei YANG, E-mail: boweiy@zju.edu.cn
  [ "Guanghua SONG, E-mail: ghsong@zju.edu.cn" ]
  [ "Xiaohong JIANG, E-mail: jiangxh@zju.edu.cn" ]
- Funds：
- DOI：10.1631/FITEE.1900712
  CLC： TP183;TP393.1
- Received：19 December 2019，
  
  Revised：2020-10-20，
  
  Published Online：12 January 2021，
  
  Published：2021-05
- Accepted：
Scan QR Code
Wei LI, Bowei YANG, Guanghua SONG, et al. Dynamic value iteration networks for the planning of rapidly changing UAV swarms[J]. Frontiers of Information Technology & Electronic Engineering, 2021, 22(5): 687-696.
DOI：

Wei LI, Bowei YANG, Guanghua SONG, et al. Dynamic value iteration networks for the planning of rapidly changing UAV swarms[J]. Frontiers of Information Technology & Electronic Engineering, 2021, 22(5): 687-696. DOI： 10.1631/FITEE.1900712.

摘要

在无人机自组网（UANET）中，稀疏且高速移动的无人机节点会动态改变无人机自组网的拓扑结构，这可能会导致无人机自组网服务性能问题。为规划快速变化的无人机群，本文提出一种动态值迭代网络（DVIN）模型，该模型利用无人机自组网的连接信息，采用场景式Q学习方法训练，生成状态值传播函数，使无人机节点能够自适应调节至新的物理位置。然后，评估了动态值迭代网络模型的性能，并将其与非支配排序遗传算法NSGA-II和穷举法比较。仿真结果表明，动态值迭代网络模型显著缩短了无人机节点路径规划的决策时间，且平均成功率更高。

Abstract

In an unmanned aerial vehicle ad-hoc network (UANET)

sparse and rapidly mobile unmanned aerial vehicles (UAVs)/nodes can dynamically change the UANET topology. This may lead to UANET service performance issues. In this study

for planning rapidly changing UAV swarms

we propose a dynamic value iteration network (DVIN) model trained using the episodic Q-learning method with the connection information of UANETs to generate a state value spread function

which enables UAVs/nodes to adapt to novel physical locations. We then evaluate the performance of the DVIN model and compare it with the non-dominated sorting genetic algorithm Ⅱ and the exhaustive method. Simulation results demonstrate that the proposed model significantly reduces the decision-making time for UAV/node path planning with a high average success rate.

关键词

Keywords

references

M Abadi , , , P Barham , , , JM Chen , , , 等 . . TensorFlow: a system for large-scale machine learning . . Proc 12 th USENIX Conf on Operating Systems Design and Implementation , , 2016 . . p.265 - - 283 . . . .

I Bekmezci , , , OK Sahingoz , , , Temel . . Flying ad-hoc networks (FANETs): a survey . . Ad Hoc Netw , , 2013 . . 11 ( ( 3 ): ): 1254 - - 1270 . . DOI: 10.1016/j.adhoc.2012.12.004 http://doi.org/10.1016/j.adhoc.2012.12.004 . .

R Bellman . . Dynamic programming . . Science , , 1966 . . 153 ( ( 3731 ): ): 34 - - 37 . . DOI: 10.1126/science.153.3731.34 http://doi.org/10.1126/science.153.3731.34 . .

DP Bertsekas . . Dynamic Programming and Optimal Control . . Athena Scientific, Belmont, USA , , 1995 . . .

YL Boureau , , , F Bach , , , Y LeCun , , , 等 . . Learning midlevel features for recognition . . Proc IEEE Computer Society Conf on Computer Vision and Pattern Recognition , , 2010 . . p.2559 - - 2566 . . DOI: 10.1109/CVPR.2010.5539963 http://doi.org/10.1109/CVPR.2010.5539963 . .

I Buck , , , T Foley , , , D Horn , , , 等 . . Brook for GPUs: stream computing on graphics hardware . . ACM Trans Graph , , 2004 . . 23 ( ( 3 ): ): 777 - - 786 . . DOI: 10.1145/1015706.1015800 http://doi.org/10.1145/1015706.1015800 . .

U Challita , , , W Saad , , , C Bettstetter . . Deep reinforcement learning for interference-aware path planning of cellular-connected UAVs . . Proc IEEE Int Conf on Communications , , 2018 . . p. 1 - - 7 . . DOI: 10.1109/ICC.2018.8422706 http://doi.org/10.1109/ICC.2018.8422706 . .

F Cruz , , , P Wppen , , , A Fazrie , , , 等 . . Action selection methods in a robotic reinforcement learning scenario . . Proc IEEE Latin American Conf on Computational Intelligence , , 2019 . . p. 1 - - 6 . . DOI: 10.1109/LA-CCI.2018.8625243 http://doi.org/10.1109/LA-CCI.2018.8625243 . .

K Deb , , , A Pratap , , , S Agarwal , , , 等 . . A fast and elitist multiobjective genetic algorithm: NSGA-Ⅱ . . IEEE Trans Evol Comput , , 2002 . . 6 ( ( 2 ): ): 182 - - 197 . . DOI: 10.1109/4235.996017 http://doi.org/10.1109/4235.996017 . .

RR Fontes . . Emulando Redes Sem Fio Com MininetWiFi , , 2019 . . https://github.com/ramonfontes/mn-wifi-book-pt/blob/master/preview-book.pdf https://github.com/ramonfontes/mn-wifi-book-pt/blob/master/preview-book.pdf , , . .

RR Fontes , , , S Afzal , , , SHB Brito , , , 等 . . Mininet-WiFi: emulating software-defined wireless networks . . Proc 11 th Int Conf on Network and Service Management , , 2015 . . p. 384 - - 389 . . DOI: 10.1109/CNSM.2015.7367387 http://doi.org/10.1109/CNSM.2015.7367387 . .

V Franois-Lavet , , , P Henderson , , , R Islam , , , 等 . . An introduction to deep reinforcement learning . . Found Trends Mach Learn , , 2018 . . 11 ( ( 3-4 ): ): 219 - - 354 . . DOI: 10.1561/2200000071 http://doi.org/10.1561/2200000071 . .

F Koohifar , , , A Kumbhar , , , I Guvenc . . Receding horizon multi-UAV cooperative tracking of moving RF source . . IEEE Commun Lett , , 2017 . . 21 ( ( 6 ): ): 1433 - - 1436 . . DOI: 10.1109/LCOMM.2016.2603977 http://doi.org/10.1109/LCOMM.2016.2603977 . .

A Krizhevsky , , , I Sutskever , , , GE Hinton . . ImageNet classification with deep convolutional neural networks . . Commun ACM , , 2017 . . 60 ( ( 6 ): ): 84 - - 90 . . DOI: 10.1145/3065386 http://doi.org/10.1145/3065386 . .

J Lee , , , BY Kang , , , DW Kim . . Fast genetic algorithm for robot path planning . . Electron Lett , , 2013 . . 49 ( ( 23 ): ): 1449 - - 1451 . . DOI: 10.1049/el.2013.3143 http://doi.org/10.1049/el.2013.3143 . .

V Mnih , , , K Kavukcuoglu , , , D Silver , , , 等 . . Human-level control through deep reinforcement learning . . Nature , , 2015 . . 518 ( ( 7540 ): ): 529 - - 533 . . DOI: 10.1038/nature14236 http://doi.org/10.1038/nature14236 . .

V Mnih , , , AP Badia , , , L Mirza , , , 等 . . Asynchronous methods for deep reinforcement learning . . Proc 33 rd Int Conf on Machine Learning , , 2016 . . p. 1928 - - 1937 . . . .

SF Niu , , , SH Chen , , , HY Guo , , , 等 . . Generalized value iteration networks: life beyond lattices . . Proc 32 nd AAAI Conf on Artificial Intelligence , , 2018 . . p.6246 - - 6253 . . . .

V Roberge , , , M Tarbouchi , , , G Labonte . . Comparison of parallel genetic algorithm and particle swarm optimization for real-time UAV path planning . . IEEE Trans Ind Inform , , 2013 . . 9 ( ( 1 ): ): 132 - - 141 . . DOI: 10.1109/TⅡ.2012.2198665 http://doi.org/10.1109/TⅡ.2012.2198665 . .

S Schaal . . Is imitation learning the route to humanoid robots . . Trends Cogn Sci , , 1999 . . 3 ( ( 6 ): ): 233 - - 242 . . DOI: 10.1016/s1364-6613(99)01327-3 http://doi.org/10.1016/s1364-6613(99)01327-3 . .

A Tamar , , , Y Wu , , , G Thomas , , , 等 . . Value iteration networks . . Proc 26 th Int Joint Conf on Artificial Intelligence , , 2017 . . p.4949 - - 4953 . . DOI: 10.24963/ijcai.2017/700 http://doi.org/10.24963/ijcai.2017/700 . .

M Tokic , , , G Palm . . Value-difference based exploration: adaptive control between epsilon-greedy and softmax . . Proc 34 th Annual German Conf on Advances in Artificial Intelligence , , 2011 . . p.335 - - 346 . . DOI: 10.1007/978-3-642-24455-1_33 http://doi.org/10.1007/978-3-642-24455-1_33 . .

CJCH Watkins , , , P Dayan . . Q-learning . . Mach Learn , , 1992 . . 8 ( ( 3-4 ): ): 279 - - 292 . . DOI: 10.1007/BF00992698 http://doi.org/10.1007/BF00992698 . .

CY Zhang , , , P Patras , , , H Haddadi . . Deep learning in mobile and wireless networking: a survey . . IEEE Commun Surv Tutor , , 2019 . . 21 ( ( 3 ): ): 2224 - - 2287 . . DOI: 10.1109/COMST.2019.2904897 http://doi.org/10.1109/COMST.2019.2904897 . .

T Zhang , , , Q Li , , , CS Zhang , , , 等 . . Current trends in the development of intelligent unmanned autonomous systems . . Front Inform Technol Electron Eng , , 2017 . . 18 ( ( 1 ): ): 68 - - 85 . . DOI: 10.1631/FITEE.1601650 http://doi.org/10.1631/FITEE.1601650 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

An optimized formula for the two-point resistance of a cobweb resistance network and its potential application

A distributed variable density path search and simplification method for industrial manipulators with end-effector’s attitude constraints

A review of cooperative path planning of an unmanned aerial vehicle group

Related Author

Yu GUAN

Xiaoyu JIANG

Yanpeng ZHENG

Zhaolin JIANG

Jin WANG

Shengjie LI

Haiyun ZHANG

Guodong LU

Related Institution

School of Information Science and Engineering, Linyi University

School of Automation and Electrical Engineering, Linyi University

School of Mathematics and Statistics, Linyi University

State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University

Engineering Research Center for Design Engineering and Digital Twin of Zhejiang Province, School of Mechanical Engineering, Zhejiang University

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰