MDLB：一种基于强化学习的元数据动态负载均衡机制

武兆琪; 卫今; 张帆; 郭威; 谢光伟

doi:10.1631/FITEE.1900121

Your Location：

Home >

Browse articles >

MDLB：一种基于强化学习的元数据动态负载均衡机制

常规文章 | Updated：2022-05-19

- MDLB：一种基于强化学习的元数据动态负载均衡机制
- MDLB: a metadata dynamic load balancing mechanism based on reinforcement learning
- 信息与电子工程前沿（英文） 2020年21卷第7期页码：1034-1046
- Affiliations：
  
  National Digital Switching System Engineering & Technological R & D Center, Zhengzhou 450002, China
  School of Computer Science, Fudan University, Shanghai 200433, China
  Data Arena Institute, Fudan University, Shanghai 200433, China
- Author bio：
  
  Fan ZHANG, E-mail: 17034203@qq.com
- Funds：
  
  Project supported by the National Natural Science Foundation of China (Nos. 61572520 and 61521003)
- DOI：10.1631/FITEE.1900121
  中图分类号： TP338.8
- 收稿：2019-03-01，
  
  修回：2020-;6-;2，
  
  纸质出版：2020-07
- Accepted：
Scan QR Code
武兆琪, 卫今, 张帆, 等. MDLB：一种基于强化学习的元数据动态负载均衡机制[J]. 信息与电子工程前沿（英文）, 2020,21(7):1034-1046.

Zhao-qi WU, Jin WEI, Fan ZHANG, et al. MDLB: a metadata dynamic load balancing mechanism based on reinforcement learning[J]. Frontiers of Information Technology & Electronic Engineering, 2020, 21(7): 1034-1046.
武兆琪, 卫今, 张帆, 等. MDLB：一种基于强化学习的元数据动态负载均衡机制[J]. 信息与电子工程前沿（英文）, 2020,21(7):1034-1046. DOI： 10.1631/FITEE.1900121.

Zhao-qi WU, Jin WEI, Fan ZHANG, et al. MDLB: a metadata dynamic load balancing mechanism based on reinforcement learning[J]. Frontiers of Information Technology & Electronic Engineering, 2020, 21(7): 1034-1046. DOI： 10.1631/FITEE.1900121.

摘要

随着信息和数据量增长，面向对象的存储系统已被广泛应用到很多领域，包括Google文件系统、AmazonS3、Hadoop分布式文件系统和Ceph。其中元数据负载均衡在提高整个系统输入/输出性能方面起着重要作用，元数据负载不平衡会导致服务器出现严重的系统性能瓶颈问题。然而现有元数据负载平衡策略缺乏良好动态性和适用性，如基于子树分割或者哈希的负载策略。提出一种基于强化学习的动态负载平衡机制（MDLB）。采用Q_learning算法，所提基于强化学习机制由3个模块组成，即策略选择网络、负载均衡网络和参数更新网络。实验结果表明MDLB算法可根据元数据服务器的性能动态调节负载，在数据量骤变情况下仍具有很好适应性。

Abstract

With the growing amount of information and data

object-oriented storage systems have been widely used in many applications

including the Google File System

Amazon S3

Hadoop Distributed File System

and Ceph

in which load balancing of metadata plays an important role in improving the input/output performance of the entire system. Unbalanced load on the metadata server leads to a serious bottleneck problem for system performance. However

most existing metadata load balancing strategies

which are based on subtree segmentation or hashing

lack good dynamics and adaptability. In this study

we propose a metadata dynamic load balancing (MDLB) mechanism based on reinforcement learning (RL). We learn that the Q_learning algorithm and our RL-based strategy consist of three modules

i.e.

the policy selection network

load balancing network

and parameter update network. Experimental results show that the proposed MDLB algorithm can adjust the load dynamically according to the performance of the metadata servers

and that it has good adaptability in the case of sudden change of data volume.

关键词

Keywords

references

SI Amari , , , T Ozeki , , , R Karakida , , , 等 . . Dynamics of learning in MLP: natural gradient and singularity revisited . . Neur Comput , , 2018 . . 30 ( ( 1 ): ): 1 - - 33 . . DOI: 10.1162/neco_a_01029 http://doi.org/10.1162/neco_a_01029 . .

F Azzedin . . Towards a scalable HDFS architecture . . Proc Int Conf on Collaboration Technologies and Systems , , 2013 . . p.155 - - 161 . . DOI: 10.1109/CTS.2013.6567222 http://doi.org/10.1109/CTS.2013.6567222 . .

T Chen , , , N Xiao , , , F Liu . . Adaptive metadata load balancing for object storage systems . . J Softw , , 2013 . . 24 ( ( 2 ): ): 331 - - 342 . . DOI: 10.3724/SP.J.1001.2013.04177 http://doi.org/10.3724/SP.J.1001.2013.04177 . .

M Fortunato , , , MG Azar , , , B Piot , , , 等 . . Noisy networks for exploration . . 2017 . . https://arxiv.org/abs/1706.10295 https://arxiv.org/abs/1706.10295 , , . .

S Ghemawat , , , H Gobioff , , , ST Leung . . The Google file system . . Proc 19 th ACM Symp on Operating Systems Principles , , 2003 . . p.29 - - 43 . . DOI: 10.1145/945445.945450 http://doi.org/10.1145/945445.945450 . .

M Hausknecht , , , P Stone . . Deep recurrent Q-learning for partially observable MDPs . . 2015 . . https://arxiv.org/abs/1507.06527 https://arxiv.org/abs/1507.06527 , , . .

M Hessel , , , J Modayil , , , H van Hasselt , , , 等 . . Rainbow: combining improvements in deep reinforcement learning . . 2018 . . https://arxiv.org/abs/1710.02298 https://arxiv.org/abs/1710.02298 , , . .

D Hitz , , , J Lau , , , M Malcolm . . File system design for an NFS file server appliance . . Proc USENIX Winter Technical Conf , , 1994 . . p.9 - - 19 . . . .

J Howard , , , M Kazar , , , S Menees , , , 等 . . Scale and performance in a distributed file system . . ACM SIGOPS Oper Syst Rev , , 1987 . . 21 ( ( 5 ): ): 1 - - 2 . . DOI: 10.1145/37499.37500 http://doi.org/10.1145/37499.37500 . .

Y Hua , , , YF Zhu , , , H Jiang , , , 等 . . Scalable and adaptive metadata management in ultra large-scale file systems . . Proc 28 th Int Conf on Distributed Computing Systems , , 2008 . . p.403 - - 410 . . DOI: 10.1109/ICDCS.2008.32 http://doi.org/10.1109/ICDCS.2008.32 . .

S Kanai , , , Y Fujiwara , , , Y Yamanaka , , , 等 . . Sigsoftmax: reanalysis of the softmax bottleneck . . Proc 32 nd Int Conf on Neural Information Processing Systems , , 2018 . . p.284 - - 294 . . . .

TG Lewis , , , CR Cook . . Hashing for dynamic and static internal tables . . Computer , , 1988 . . 21 ( ( 10 ): ): 45 - - 56 . . DOI: 10.1109/2.7056 http://doi.org/10.1109/2.7056 . .

WJ Li , , , W Xue , , , JW Shu , , , 等 . . Dynamic hashing: adaptive metadata management for petabyte-scale file systems . . Proc 23 rd IEEE/14 th NASA Goddard Conf on Mass Storage Systems and Technologies , , 2006 . . p.159 - - 164 . . . .

Z Liu , , , XM Zhou . . A metadata management method based on directory path . . J Softw , , 2007 . . 18 ( ( 2 ): ): 236 - - 245 . . . .

A Manindra , , , PS Thiagarajan . . Lazy rectangular hybrid automata . . Proc Int Workshop on Hybrid Systems: Computation and Control , , 2004 . . p.1 - - 15 . . DOI: 10.1007/978-3-540-24743-2_1 http://doi.org/10.1007/978-3-540-24743-2_1 . .

V Mnih , , , K Kavukcuoglu , , , D Silver , , , 等 . . Playing Atari with deep reinforcement learning . . 2013 . . https://arxiv.org/abs/1312.5602 https://arxiv.org/abs/1312.5602 , , . .

MR Palankar , , , A Iamnitchi , , , M Ripeanu , , , 等 . . Amazon S3 for science grids: a viable solution . . Proc Int Workshop on Data-Aware Distributed Computing , , 2008 . . p.55 - - 64 . . DOI: 10.1145/1383519.1383526 http://doi.org/10.1145/1383519.1383526 . .

R Patgiri , , , D Dev , , , A Ahmed . . dMDS: uncover the hidden issues of metadata server design . . In: Sa PK, Sahoo MN, Murugappan M, et al. (Eds.), Progress in Intelligent Computing Techniques: Theory, Practice, and Applications. Springer, Singapore , , 2017 . . p.531 - - 541 . . DOI: 10.1007/978-981-10-3373-5_53 http://doi.org/10.1007/978-981-10-3373-5_53 . .

M Satyanarayanan , , , JJ Kistler , , , P Kumar , , , 等 . . Coda: a highly available file system for a distributed workstation environment . . IEEE Trans Comput , , 1990 . . 39 ( ( 4 ): ): 447 - - 459 . . DOI: 10.1109/12.54838 http://doi.org/10.1109/12.54838 . .

C Schindelhauer , , , G Schomaker . . Weighted distributed hash tables . . Proc 7 th Annual ACM Symp on Parallelism in Algorithms and Architectures , , 2005 . . p.218 - - 227 . . DOI: 10.1145/1073970.1074008 http://doi.org/10.1145/1073970.1074008 . .

K Shvachko , , , HR Kuang , , , S Radia , , , 等 . . The Hadoop distributed file system . . Proc IEEE 26 th Symp on Mass Storage Systems and Technologies , , 2010 . . p.1 - - 10 . . DOI: 10.1109/MSST.2010.5496972 http://doi.org/10.1109/MSST.2010.5496972 . .

I Stoica , , , R Morris , , , D Karger , , , 等 . . Chord: a scalable peer-to-peer lookup service for Internet applications . . Proc Conf on Applications, Technologies, Architectures, and Protocols for Computer Communications , , 2001 . . p.149 - - 160 . . DOI: 10.1145/383059.383071 http://doi.org/10.1145/383059.383071 . .

RS Sutton , , , AG Barto . . Reinforcement Learning: an Introduction (2 nd Ed.) , , : : London, UK MIT Press , , 2018 . . .

GH Tzeng , , , JJ Huang . . Multiple Attribute Decision Making: Methods and Applications . . Chapman and Hall, New York, USA , , 2011 . . .

H van Hasselt , , , A Guez , , , D Silver . . Deep reinforcement learning with double Q-learning . . Proc 30 th AAAI Conf on Artificial Intelligence , , 2016 . . p.2094 - - 2100 . . . .

FY Wang , , , M Nelson , , , S Oral , , , 等 . . Performance and scalability evaluation of the Ceph parallel file system . . Proc 8 th Parallel Data Storage Workshop , , 2013 . . p.14 - - 19 . . DOI: 10.1145/2538542.2538562 http://doi.org/10.1145/2538542.2538562 . .

JD Wang , , , T Zhang , , , JK Song , , , 等 . . A survey on learning to hash . . IEEE Trans Patt Anal Mach Intell , , 2018 . . 40 ( ( 4 ): ): 769 - - 790 . . DOI: 10.1109/TPAMI.2017.2699960 http://doi.org/10.1109/TPAMI.2017.2699960 . .

ZY Wang , , , T Schaul , , , M Hessel , , , 等 . . Dueling network architectures for deep reinforcement learning . . Proc 33 rd Int Conf on Machine Learning , , 2016 . . p.1995 - - 2003 . . . .

CX Wu , , , R Burns . . Handling heterogeneity in shared-disk file systems . . Proc ACM/IEEE Conf on Supercomputing, Article 7 , , 2003 . . DOI: 10.1109/SC.2003.10045 http://doi.org/10.1109/SC.2003.10045 . .

JJ Wu , , , PF Liu , , , YC Chung . . Metadata partitioning for large-scale distributed storage systems . . Proc IEEE 3 rd Int Conf on Cloud Computing , , 2010 . . p.212 - - 219 . . DOI: 10.1109/CLOUD.2010.24 http://doi.org/10.1109/CLOUD.2010.24 . .

J Xiong , , , RF Tang , , , SN Wu , , , 等 . . An efficient metadata distribution policy for cluster file systems . . Proc IEEE Int Conf on Cluster Computing , , 2005 . . p.1 - - 10 . . DOI: 10.1109/CLUSTR.2005.347060 http://doi.org/10.1109/CLUSTR.2005.347060 . .

HY Zhang , , , K Wang . . Research of dynamic load balancing based on stimulated annealing algorithm . . Int J Embed Syst , , 2018 . . 10 ( ( 3 ): ): 188 - - 195 . . DOI: 10.1504/IJES.2018.091777 http://doi.org/10.1504/IJES.2018.091777 . .

LJ Zhang , , , Y Cui , , , GC Luo , , , 等 . . Dynamic load balance algorithm for big-data distributed storage . . Comput Sci , , 2017 . . 44 ( ( 5 ): ): 178 - - 183 . . DOI: 10.11896/j.issn.1002-137X.2017.05.032 http://doi.org/10.11896/j.issn.1002-137X.2017.05.032 . .

S Zhong , , , J Chen , , , YR Yang . . Sprite: a simple, cheat-proof, credit-based system for mobile ad-hoc networks . . Proc 22 nd Annual Joint Conf of the IEEE Computer and Communications Societies , , 2003 . . p.1987 - - 1997 . . DOI: 10.1109/INFCOM.2003.1209220 http://doi.org/10.1109/INFCOM.2003.1209220 . .

YF Zhu , , , H Jiang , , , J Wang , , , 等 . . HBA: distributed metadata management for large cluster-based storage systems . . IEEE Trans Parall Distrib Syst , , 2008 . . 19 ( ( 6 ): ): 750 - - 763 . . DOI: 10.1109/TPDS.2007.70788 http://doi.org/10.1109/TPDS.2007.70788 . .

浏览量

323

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

Jiu fusion artificial intelligence (JFA): a two-stage reinforcement learning model with hierarchical neural networks and human knowledge for Tibetan Jiu chess

Efficient learning of robust multigait quadruped locomotion for minimizing the cost of transport

Reinforcement learning based privacy-preserving consensus tracking control of nonstrict-feedback discrete-time multi-agent systems

Domain adaptation in reinforcement learning: a comprehensive and systematic study

Multi-agent reinforcement learning behavioral control for nonlinear second-order systems