DDUC: an erasure-coded system with decoupled data updating and coding

Yaofeng TU; Rong XIAO; Yinjun HAN; Zhenghua CHEN; Hao JIN; Xuecheng QI; Xinyuan SUN

doi:10.1631/FITEE.2200466

Your Location：

Home >

Browse articles >

DDUC: an erasure-coded system with decoupled data updating and coding

Special Feature on Design and Application of New Storage Systems | Updated：2023-05-31

- DDUC: an erasure-coded system with decoupled data updating and coding
- DDUC：数据更新与编码解耦的纠删码系统
- Frontiers of Information Technology & Electronic Engineering Vol. 24, Issue 5, Pages: 716-730(2023)
- Affiliations：
  
  1.State Key Laboratory of Mobile Network and Mobile Multimedia Technology, Shenzhen 518000, China
  2.ZTE Corporation, Nanjing 210000, China
- Author bio：
  
  E-mail: tu.yaofeng@zte.com.cn
  xiao.rong1@zte.com.cn
  ‡ Corresponding author
  chen.zhenghua@zte.com.cn
  jin.hao1@zte.com.cn
  qi.xuecheng@zte.com.cn
  sun.xinyuan@zte.com.cn
- Funds：
- DOI：10.1631/FITEE.2200466
  CLC： TP333
- Received：15 October 2022，
  
  Accepted：12 February 2023，
  
  Published：0 May 2023
- Accepted：
Scan QR Code
Yaofeng TU, Rong XIAO, Yinjun HAN, et al. DDUC: an erasure-coded system with decoupled data updating and coding[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(5): 716-730.
DOI：

Yaofeng TU, Rong XIAO, Yinjun HAN, et al. DDUC: an erasure-coded system with decoupled data updating and coding[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(5): 716-730. DOI： 10.1631/FITEE.2200466.

摘要

在分布式存储系统中，常用的数据冗余方法包括副本和纠删码（erasure code，EC）。相较于副本，EC具有更好的存储效率，但是在更新方面的开销更大。此外，并发更新带来的一致性和可靠性问题给EC应用带来了新的挑战。许多研究工作都致力于优化EC技术，包括算法优化、数据更新方法创新等，但并发更新的一致性和可靠性问题尚未得到很好解决。本文介绍了一种将数据更新与EC编码解耦的存储系统，命名为DDUC，并提出了一种副本与校验块结合的放置策略。对于（

，

）的EC系统，按照

组

+1的副本进行数据布局，并将同一条带的冗余数据块都放置在校验节点上，使得校验节点可以自主地执行本地EC编码。基于上述策略，实现了一种两阶段数据更新方法，在第一阶段按照副本模式进行数据更新，在第二阶段由校验节点独立完成EC编码。这样在保证高并发性能的同时，解决了并发更新导致的数据可靠性降低的问题。同时利用PMem硬件的字节寻址和8字节原子写特性实现了一种轻量级的日志机制，在提升性能的同时保证了数据的一致性。实验结果表明，和当前主流的存储系统Ceph相比，本文所提出的存储系统并发访问性能提升至1.70–3.73倍，时延仅为Ceph的3.4%–5.9%。

Abstract

In distributed storage systems

replication and erasure code (EC) are common methods for data redundancy. Compared with replication

EC has better storage efficiency

but suffers higher overhead in update. Moreover

consistency and reliability problems caused by concurrent updates bring new challenges to applications of EC. Many works focus on optimizing the EC solution

including algorithm optimization

novel data update method

and so on

but lack the solutions for consistency and reliability problems. In this paper

we introduce a storage system that decouples data updating and EC encoding

namely

decoupled data updating and coding (DDUC)

and propose a data placement policy that combines replication and parity blocks. For the (

) EC system

the data are placed as

groups of

+1 replicas

and redundant data blocks of the same stripe are placed in the parity nodes

so that the parity nodes can autonomously perform local EC encoding. Based on the above policy

a two-phase data update method is implemented in which data are updated in replica mode in phase 1

and the EC encoding is done independently by parity nodes in phase 2. This solves the problem of data reliability degradation caused by concurrent updates while ensuring high concurrency performance. It also uses persistent memory (PMem) hardware features of the byte addressing and eight-byte atomic write to implement a lightweight logging mechanism that improves performance while ensuring data consistency. Experimental results show that the concurrent access performance of the proposed storage system is 1.70–3.73 times that of the state-of-the-art storage system Ceph

and the latency is only 3.4%–5.9% that of Ceph.

关键词

Keywords

references

Aguilera MK , Janakiraman R , Xu LH , 2005a . On the erasure recoverability of MDS codes under concurrent updates . Proc IEEE Int Symp on Information Theory , p. 1358 - 1362 . doi: 10.1109/ISIT.2005.1523564 http://doi.org/10.1109/ISIT.2005.1523564

Aguilera MK , Janakiraman R , Xu LH , 2005b . Using erasure codes efficiently for storage in a distributed system . Int Conf on Dependable Systems and Networks , p. 336 - 345 . doi: 10.1109/DSN.2005.96 http://doi.org/10.1109/DSN.2005.96

Chan JCW , Ding Q , Lee PPC , et al. , 2014 . Parity logging with reserved space: towards efficient updates and recovery in erasure-coded clustered storage . Proc 12th USENIX Conf on File and Storage Technologies , p. 163 - 176 .

Ghemawat S , Gobioff H , Leung ST , 2003 . The Google file system . Proc 19th ACM Symp on Operating Systems Principles , p. 29 - 43 . doi: 10.1145/945445.945450 http://doi.org/10.1145/945445.945450

Gong GW , Shen ZR , Wu SZ , et al. , 2021 . Optimal rack-coordinated updates in erasure-coded data centers . 40th IEEE Conf on Computer Communications , p. 1 - 10 . doi: 10.1109/INFOCOM42981.2021.9488813 http://doi.org/10.1109/INFOCOM42981.2021.9488813

Huang C , Simitci H , Xu YK , et al. , 2012 . Erasure coding in Windows Azure Storage . Proc USENIX Annual Technical Conf , p. 2 .

Huang JZ , Xia J , Qin X , et al. , 2019 . Optimization of small updates for erasure-coded in-memory stores . Comput J , 62 ( 6 ): 869 - 883 . doi: 10.1093/comjnl/bxz003 http://doi.org/10.1093/comjnl/bxz003

Jiang TY , Zhang GY , Huang ZC , et al. , 2021 . FusionRAID: achieving consistent low latency for commodity SSD arrays . 19th USENIX Conf on File and Storage Technologies , p. 355 - 370 .

Konwar KM , Prakash N , Lynch N , et al. , 2017 . A layered architecture for erasure-coded consistent distributed storage . Proc ACM Symp on Principles of Distributed Computing , p. 63 - 72 . doi: 10.1145/3087801.3087832 http://doi.org/10.1145/3087801.3087832

Li HB , Zhang YM , Zhang ZM , et al. , 2017 . PARIX: speculative partial writes in erasure-coded systems . Proc USENIX Annual Technical Conf , p. 581 - 587 .

Liu YJ , Wei B , Wu JG , et al. , 2021 . Erasure-coded multi-block updates based on hybrid writes and common XORs first . 39th Int Conf on Computer Design , p. 472 - 479 .

Meng YL , Zhang LL , Xu D , et al. , 2019 . A dynamic erasure code based on block code . Proc Int Conf on Embedded Wireless Systems and Networks , p. 379 - 383 .

Ousterhout J , Agrawal P , Erickson D , et al. , 2010 . The case for RAMClouds: scalable high-performance storage entirely in DRAM . ACM SIGOPS Oper Syst Rev , 43 ( 4 ): 92 - 105 . doi: 10.1145/1713254.1713276 http://doi.org/10.1145/1713254.1713276

Peter K , Reinefeld A , 2012 . Consistency and fault tolerance for erasure-coded distributed storage systems . Proc 5th Int Workshop on Data-Intensive Distributed Computing , p. 23 - 32 . doi: 10.1145/2286996.2287002 http://doi.org/10.1145/2286996.2287002

Pless V , 1998 . Introduction to the Theory of Error-Correcting Codes ( 3 rd Ed.). John Wiley & Sons , Hoboken, USA . doi: 10.1002/9781118032749 http://doi.org/10.1002/9781118032749

Pu WJ , Chen NJ , Zhong QW , 2020 . SDCUP: software-defined-control based erasure-coded collaborative data update mechanism . IEEE Access , 8 : 180646 - 180660 . doi: 10.1109/ACCESS.2020.3028381 http://doi.org/10.1109/ACCESS.2020.3028381

Rizzo L , 1997 . Effective erasure codes for reliable computer communication protocols . ACM SIGCOMM Comput Commun Rev , 27 ( 2 ): 24 - 36 . doi: 10.1145/263876.263881 http://doi.org/10.1145/263876.263881

Shen ZR , Lee PPC , 2018 . Cross-rack-aware updates in erasure-coded data centers . Proc 47th Int Conf on Parallel Processing , Article 80 . doi: 10.1145/3225058.3225065 http://doi.org/10.1145/3225058.3225065

Wang F , Tang YJ , Xie YW , et al. , 2019 . XORInc: optimizing data repair and update for erasure-coded systems with XOR-based in-network computation . Proc 35th Symp on Mass Storage Systems and Technologies , p. 244 - 256 . doi: 10.1109/MSST.2019.00005 http://doi.org/10.1109/MSST.2019.00005

Wang YJ , Pei XQ , Ma XK , et al. , 2018 . TA-update: an adaptive update scheme with tree-structured transmission in erasure-coded storage systems . IEEE Trans Parall Distrib Syst , 29 ( 8 ): 1893 - 1906 . doi: 10.1109/TPDS.2017.2717981 http://doi.org/10.1109/TPDS.2017.2717981

Weatherspoon H , Kubiatowicz JD , 2002 . Erasure coding vs . replication: a quantitative comparison. 1st Int Workshop on Peer-to-Peer Systems , p. 328 - 337 . doi: 10.1007/3-540-45748-8_31 http://doi.org/10.1007/3-540-45748-8_31

Xiong YL , Zhou J , Su L , et al. , 2021 . ECCH: erasure coded consistent hashing for distributed storage systems . IEEE Int Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking , p. 177 - 184 . doi: 10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00036 http://doi.org/10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00036

Views

125

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

ShortTail: taming tail latency for erasure-code-based in-memory systems

SA-RSR: a read-optimal data recovery strategy for XOR-coded distributed storage systems

A multiscale-contour-based interpolation framework for generating a time-varying quasi-dense point cloud sequence

A Chebyshev/Legendre polynomial interpolation approach for fingerprint orientation estimation smoothing and prediction

Related Author

Guangyan ZHANG

Jing HUANG

Zhiyue LI

Yun TENG

Xingjun ZHANG

Ningjing LIANG

Yunfei LIU

Changjiang ZHANG

Related Institution

College of Computer Science and Technology, Jilin University

Department of Computer Science and Technology, Tsinghua University

Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University

Beijing National Research Center for Information Science and Technology (Tsinghua University)

School of Computer Science and Technology, Xi’an Jiaotong University

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰