Federated deep reinforcement learning based computation offloading in a low Earth orbit satellite edge computing system

Min JIA; Jian WU; Xinyu WANG; Qing GUO

doi:10.1631/FITEE.2400448

Your Location：

Home >

Browse articles >

Federated deep reinforcement learning based computation offloading in a low Earth orbit satellite edge computing system

Regular Papers | Updated：2025-06-09

- Federated deep reinforcement learning based computation offloading in a low Earth orbit satellite edge computing system
  Enhanced Publication
- 基于联邦深度强化学习的低轨卫星边缘计算系统计算卸载
- Frontiers of Information Technology & Electronic Engineering Vol. 26, Issue 5, Pages: 805-815(2025)
- Affiliations：
  
  School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin 150006, China
- Author bio：
  
  ‡ Corresponding author
  20B905014@stu.hit.edu.cn
  wang_xinyu@hit.edu.cn
  qguo@hit.edu.cn
- Funds：
- DOI：10.1631/FITEE.2400448
  CLC： TN929.5
- Received：28 May 2024，
  
  Revised：2024-10-15，
  
  Published Online：27 December 2024，
  
  Published：2025-05
- Accepted：
Scan QR Code
Min JIA, Jian WU, Xinyu WANG, et al. Federated deep reinforcement learning based computation offloading in a low Earth orbit satellite edge computing system[J]. Frontiers of Information Technology & Electronic Engineering, 2025, 26(5): 805-815.
DOI：

Min JIA, Jian WU, Xinyu WANG, et al. Federated deep reinforcement learning based computation offloading in a low Earth orbit satellite edge computing system[J]. Frontiers of Information Technology & Electronic Engineering, 2025, 26(5): 805-815. DOI： 10.1631/FITEE.2400448.

摘要

最近研究表明系统容量对蜂窝网络非常重要。本文考虑最大化蜂窝网络下行链路和上行链路的加权和速率，其中每个小区由一个全双工基站和半双工用户组成。联邦学习可以在没有集中数据的情况下训练模型，实现对用户数据的隐私保护。将移动边缘计算服务器放置在低轨卫星上，可形成低轨卫星边缘计算系统，大大提高卫星的处理能力。因此，本文将联邦学习和移动边缘计算结合，提出一种基于联邦学习的计算卸载算法，在保证用户数据安全的同时最大化加权和速率。采用具有出色全局搜索能力的深度强化学习算法解决子信道分配和功率分配问题。仿真结果表明，与基准算法相比，该算法实现了最大的加权和速率，并具有良好收敛性能。

Abstract

Recent studies have shown that system capacity is very important for cellular networks. In this paper

we consider maximizing the weighted sum-rate of the cellular network downlink and uplink

where each cell consists of a full-duplex (FD) base station (BS) and half-duplex (HD) users. Federated learning (FL) can train models in the absence of centralized data

which can achieve privacy protection of user data. A low Earth orbit (LEO) satellite edge computing system (LSECS) can be formed by placing the mobile edge computing (MEC) servers on LEO satellites

which greatly increases the processing capacities of the satellites. Therefore

we consider a combination of FL and MEC and propose an FL-based computation offloading algorithm to maximize the weighted sum-rate while ensuring the security of user data. We consider solving the sub-channel assignment and power allocation problems using deep reinforcement learning (DRL) algorithms with excellent global search capabilities. The simulation results show that our proposed algorithm achieves the maximum weighted sum-rate compared with the baseline algorithms and excellent convergence.

关键词

Keywords

references

Alkhrijah Y , Camp J , Rajan D , 2023 . Multi-band full duplex MAC protocol (MB-FDMAC) . IEEE J Sel Areas Commun , 41 ( 9 ): 2864 - 2878 . https://doi.org/10.1109/JSAC.2023.3287546 https://doi.org/10.1109/JSAC.2023.3287546

Chen H , Xiao M , Pang ZB , 2022 . Satellite-based computing networks with federated learning . IEEE Wirel Commun , 29 ( 1 ): 78 - 84 . https://doi.org/10.1109/MWC.008.00353 https://doi.org/10.1109/MWC.008.00353

Chen XM , Xu ZB , Shang L , 2023 . Satellite Internet of Things: challenges, solutions, and development trends . Front Inform Technol Electron Eng , 24 ( 7 ): 935 - 944 . https://doi.org/10.1631/FITEE.2200648 https://doi.org/10.1631/FITEE.2200648

Dai XY , Zhao C , Wang X , et al. , 2022 . Image-based traffic signal control via world models . Front Inform Technol Electron Eng , 23 ( 12 ): 1795 - 1813 . https://doi.org/10.1631/FITEE.2200323 https://doi.org/10.1631/FITEE.2200323

El Houda ZA , Moudoud H , Brik B , 2024 . Federated deep reinforcement learning for efficient jamming attack mitigation in O-RAN . IEEE Trans Veh Technol , 73 ( 7 ): 9334 - 9343 . https://doi.org/10.1109/TVT.2024.3359998 https://doi.org/10.1109/TVT.2024.3359998

Fawaz H , Lahoud S , Helou ME , et al. , 2023 . Queue-aware resource allocation in full-duplex multi-cellular wireless networks . IEEE J Sel Areas Commun , 41 ( 9 ): 2852 - 2863 . https://doi.org/10.1109/JSAC.2023.3287541 https://doi.org/10.1109/JSAC.2023.3287541

Fu H , Si WJ , Kim IM , 2023 . Deep learning-based joint pilot design and channel estimation for OFDM systems . IEEE Trans Commun , 71 ( 8 ): 4577 - 4590 . https://doi.org/10.1109/TCOMM.2023.3280937 https://doi.org/10.1109/TCOMM.2023.3280937

Gao YF , Ji Z , Zhao KL , et al. , 2024 . Game-based computation offloading and power allocation for LEO constellation networks in distributed and dynamic environment . IEEE Int Things J , 11 ( 4 ): 7040 - 7058 . https://doi.org/10.1109/JIOT.2023.3314650 https://doi.org/10.1109/JIOT.2023.3314650

Han DJ , Hosseinalipour S , Love DJ , et al. , 2024 . Cooperative federated learning over ground-to-satellite integrated networks: joint local computation and data offloading . IEEE J Sel Areas Commun , 42 ( 5 ): 1080 - 1096 . https://doi.org/10.1109/JSAC.2024.3365901 https://doi.org/10.1109/JSAC.2024.3365901

He ZY , Xu W , Shen H , et al. , 2023 . Full-duplex communication for ISAC: joint beamforming and power optimization . IEEE J Sel Areas Commun , 41 ( 9 ): 2920 - 2936 . https://doi.org/10.1109/JSAC.2023.3287540 https://doi.org/10.1109/JSAC.2023.3287540

Jia M , Wu J , Zhang L , et al. , 2023 . Joint optimization communication and computing resource for LEO satellites with edge computing . Chin J Electron , 32 ( 5 ): 1011 - 1021 . https://doi.org/10.23919/cje.2022.00.314 https://doi.org/10.23919/cje.2022.00.314

Jia M , Wu J , Guo Q , et al. , 2024 . Service-oriented SAGIN with pervasive intelligence for resource-constrained users . IEEE Netw , 38 ( 2 ): 79 - 86 . https://doi.org/10.1109/MNET.2024.3353414 https://doi.org/10.1109/MNET.2024.3353414

Kamal M , Rashid I , Iqbal W , et al. , 2023 . Privacy and security federated reference architecture for Internet of Things . Front Inform Technol Electron Eng , 24 ( 4 ): 481 - 508 . https://doi.org/10.1631/FITEE.2200368 https://doi.org/10.1631/FITEE.2200368

Kang YH , Zhu YF , Wang D , et al. , 2024 . Joint server selection and handover design for satellite-based federated learning using mean-field evolutionary approach . IEEE Trans Netw Sci Eng , 11 ( 2 ): 1655 - 1667 . https://doi.org/10.1109/TNSE.2023.3328776 https://doi.org/10.1109/TNSE.2023.3328776

Liao Y , Yang ZJ , Yin ZS , et al. , 2023 . DQN-based adaptive MCS and SDM for 5G massive MIMO-OFDM downlink . IEEE Commun Lett , 27 ( 1 ): 185 - 189 . https://doi.org/10.1109/LCOMM.2022.3210928 https://doi.org/10.1109/LCOMM.2022.3210928

Lim B , Vu M , 2023 . Distributed multi-agent deep Q-learning for load balancing user association in dense networks . IEEE Wirel Commun Lett , 12 ( 7 ): 1120 - 1124 . https://doi.org/10.1109/LWC.2023.3250492 https://doi.org/10.1109/LWC.2023.3250492

Liu PX , Jiang JM , Zhu GX , et al. , 2022 . Training time minimization for federated edge learning with optimized gradient quantization and bandwidth allocation . Front Inform Technol Electron Eng , 23 ( 8 ): 1247 - 1263 . https://doi.org/10.1631/FITEE.2100538 https://doi.org/10.1631/FITEE.2100538

Lv ZH , Xiu WQ , 2020 . Interaction of edge-cloud computing based on SDN and NFV for next generation IoT . IEEE Int Things J , 7 ( 7 ): 5706 - 5712 . https://doi.org/10.1109/JIOT.2019.2942719 https://doi.org/10.1109/JIOT.2019.2942719

Razmi N , Matthiesen B , Dekorsy A , et al. , 2022 . Ground-assisted federated learning in LEO satellite constellations . IEEE Wirel Commun Lett , 11 ( 4 ): 717 - 721 . https://doi.org/10.1109/LWC.2022.3141120 https://doi.org/10.1109/LWC.2022.3141120

Salim S , Moustafa N , Hassanian M , et al. , 2024 . Deep-federated-learning-based threat detection model for extreme satellite communications . IEEE Int Things J , 11 ( 3 ): 3853 - 3867 . https://doi.org/10.1109/JIOT.2023.3301626 https://doi.org/10.1109/JIOT.2023.3301626

Sultan R , Shamseldeen A , 2024 . Uplink-downlink cochannel interference cancellation in RIS-aided full-duplex networks . IEEE Syst J , 18 ( 2 ): 1220 - 1223 . https://doi.org/10.1109/JSYST.2024.3379438 https://doi.org/10.1109/JSYST.2024.3379438

Sun YW , Duan BY , Su X , et al. , 2023 . Performance analysis on reconfigurable intelligent surface and network-controlled repeater in 3GPP release-18 . Front Inform Technol Electron Eng , 24 ( 12 ): 1815 - 1828 . https://doi.org/10.1631/FITEE.2300321 https://doi.org/10.1631/FITEE.2300321

Tang FX , Wen C , Chen XH , et al. , 2023 . Federated learning for intelligent transmission with space-air-ground integrated network toward 6G . IEEE Netw , 37 ( 2 ): 198 - 204 . https://doi.org/10.1109/MNET.104.2100615 https://doi.org/10.1109/MNET.104.2100615

Teklu MB , Choi DY , Meng WX , 2024 . Resource efficient full-duplex mode of transmissions under imperfect CSI . IEEE Trans Broadcast , 70 ( 1 ): 87 - 98 . https://doi.org/10.1109/TBC.2023.3323929 https://doi.org/10.1109/TBC.2023.3323929

Tran DD , Sharma SK , Ha VN , et al. , 2023 . Multi-agent DRL approach for energy-efficient resource allocation in URLLC-enabled grant-free NOMA systems . IEEE Open J Commun Soc , 4 : 1470 - 1486 . https://doi.org/10.1109/OJCOMS.2023.3291689 https://doi.org/10.1109/OJCOMS.2023.3291689

Uddin R , Kumar SAP , 2023 . SDN-based federated learning approach for satellite-IoT framework to enhance data security and privacy in space communication . IEEE J Radio Freq Identif , 7 : 424 - 440 . https://doi.org/10.1109/JRFID.2023.3279329 https://doi.org/10.1109/JRFID.2023.3279329

Vishnoi V , Budhiraja I , Gupta S , et al. , 2023 . A deep reinforcement learning scheme for sum rate and fairness maximization among D2D pairs underlaying cellular network with NOMA . IEEE Trans Veh Technol , 72 ( 10 ): 13506 - 13522 . https://doi.org/10.1109/TVT.2023.3276647 https://doi.org/10.1109/TVT.2023.3276647

Wang Q , Chen XM , Qi Q , 2024 . Energy-efficient design of satellite-terrestrial computing in 6G wireless networks . IEEE Trans Commun , 72 ( 3 ): 1759 - 1772 . https://doi.org/10.1109/TCOMM.2023.3334813 https://doi.org/10.1109/TCOMM.2023.3334813

Wang ZJ , Gao WF , Li GH , et al. , 2024 . Path planning for unmanned aerial vehicle via off-policy reinforcement learning with enhanced exploration . IEEE Trans Emerg Top Comput Intell , 8 ( 3 ): 2625 - 2639 . https://doi.org/10.1109/TETCI.2024.3369485 https://doi.org/10.1109/TETCI.2024.3369485

Wu J , Jia M , Zhang NT , et al. , 2024 . Multi-agent deep reinforcement learning-based computation offloading in LEO satellite edge computing system . IEEE Commun Lett , 28 ( 10 ): 2352 - 2356 . https://doi.org/10.1109/LCOMM.2024.3440489 https://doi.org/10.1109/LCOMM.2024.3440489

Xiao Y , Song YQ , Liu J , 2023 . Multi-agent deep reinforcement learning based resource allocation for ultra-reliable low-latency Internet of Controllable Things . IEEE Trans Wirel Commun , 22 ( 8 ): 5414 - 5430 . https://doi.org/10.1109/TWC.2022.3233853 https://doi.org/10.1109/TWC.2022.3233853

Xu HT , Han SY , Li XH , et al. , 2023 . Anomaly traffic detection based on communication-efficient federated learning in space-air-ground integration network . IEEE Trans Wirel Commun , 22 ( 12 ): 9346 - 9360 . https://doi.org/10.1109/TWC.2023.3270179 https://doi.org/10.1109/TWC.2023.3270179

Xu X , Li RP , Zhao ZF , et al. , 2024 . The gradient convergence bound of federated multi-agent reinforcement learning with efficient communication . IEEE Trans Wirel Commun , 23 ( 1 ): 507 - 528 . https://doi.org/10.1109/TWC.2023.3279268 https://doi.org/10.1109/TWC.2023.3279268

Yu B , Qian C , Lee J , et al. , 2023 . Realizing high power full duplex in millimeter wave system: design, prototype and results . IEEE J Sel Areas Commun , 41 ( 9 ): 2893 - 2906 . https://doi.org/10.1109/JSAC.2023.3287609 https://doi.org/10.1109/JSAC.2023.3287609

Zhao D , Zheng Z , Qi PF , et al. , 2024 . Resource allocation in multi-user cellular networks: a Transformer-based deep reinforcement learning approach . China Commun , 21 ( 5 ): 77 - 96 . https://doi.org/10.23919/JCC.ea.2021-0665.202401 https://doi.org/10.23919/JCC.ea.2021-0665.202401

Views

180

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

A survey of energy-efficient strategies for federated learning in mobile edge computing

Black-box adversarial attacks on deep reinforcement learning-based proportional–integral–derivative controllers for load frequency control

Prototypical clustered federated learning for heart rate prediction

FedMcon: an adaptive aggregation method for federated learning via meta controller^&

Significance extraction based on data augmentation for reinforcement learning

Related Author

Kang YAN

Nina SHU

Tao WU

Chunsheng LIU

Panlong YANG

Xuguo JIAO

Xin WANG

Zhenyong ZHANG

Related Institution

School of Electronic Engineering, National University of Defense Technology

Department of Computing, Hong Kong Polytechnic University

School of Computer Science and Technology, University of Science and Technology of China

State Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University

School of Information and Control Engineering, Qingdao University of Technology

Map

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰