SAPER-AI accelerator: a systolic array-based power-efficient reconfigurable AI accelerator

Fahad Bin MUSLIM; Kashif INAYAT; Muhammad Zain SIDDIQI; Safiullah KHAN; Tayyeb MAHMOOD; Ihtesham ul ISLAM

doi:10.1631/FITEE.2400867

Your Location：

Home >

Browse articles >

SAPER-AI accelerator: a systolic array-based power-efficient reconfigurable AI accelerator

Regular Papers | Updated：2025-11-05

- SAPER-AI accelerator: a systolic array-based power-efficient reconfigurable AI accelerator
- SAPER-AI加速器：一种基于脉动阵列的低能耗可重构人工智能加速器
- Frontiers of Information Technology & Electronic Engineering Vol. 26, Issue 9, Pages: 1624-1636(2025)
- Affiliations：
  
  1.Faculty of Computer Science and Engineering, GIK Institute, Topi 23460, Pakistan
  2.Barcelona Supercomputing Center, Barcelona 1-3 08034, Spain
  3.Department of Electronics Engineering, Incheon National University, Incheon 22006, Republic of Korea
  4.Department of Computing and Mathematics, Manchester Metropolitan University, Manchester M15 6BX, UK
  5.Nextwave Inc., Daejeon 34134, Republic of Korea
  6.Department of Computer Software Engineering, National University of Sciences and Technology, Islamabad H-12, Pakistan
- Author bio：
  
  ‡ Corresponding author
- Funds：
- DOI：10.1631/FITEE.2400867
  CLC： TP39
- Received：21 September 2024，
  
  Revised：2025-01-24，
  
  Published Online：16 September 2025，
  
  Published：2025-09
- Accepted：
Scan QR Code
Fahad Bin MUSLIM, Kashif INAYAT, Muhammad Zain SIDDIQI, et al. SAPER-AI accelerator: a systolic array-based power-efficient reconfigurable AI accelerator[J]. Frontiers of Information Technology & Electronic Engineering, 2025, 26(9): 1624-1636.
DOI：

Fahad Bin MUSLIM, Kashif INAYAT, Muhammad Zain SIDDIQI, et al. SAPER-AI accelerator: a systolic array-based power-efficient reconfigurable AI accelerator[J]. Frontiers of Information Technology & Electronic Engineering, 2025, 26(9): 1624-1636. DOI： 10.1631/FITEE.2400867.

摘要

深度学习加速器对于满足现代神经网络日益增长的计算需求至关重要。基于脉动阵列的加速器由二维网格状的处理元件组成，这些处理元件协同工作以加速矩阵乘法运算。此类加速器的能效至关重要，尤其是在边缘人工智能场景下。本文提出的SAPER-AI加速器是一种脉动阵列加速器，采用统一功耗格式来定义其功耗意图，几乎无需优化其微架构。该加速器以粗粒度方式关闭处理元件的行和列，从而使脉动阵列微架构能适应现代深度学习工作负载不断变化的计算需求。分析表明，在最佳情况下，32×32和64×64脉动阵列设计的能效分别提升10%至25%。此外，更大规模的脉动阵列的功率延迟积改善约6%。进一步，基于MobileNet和ResNet50模型的性能比较表明，脉动阵列在处理ResNet50工作负载时通常表现更佳。这是因为ResNet50所呈现的更为规整的卷积计算更受脉动阵列青睐，且随着脉动阵列规模增大，这一性能差距会进一步扩大。

Abstract

Deep learning (DL) accelerators are critical for handling the growing computational demands of modern neural networks. Systolic array (SA)-based accelerators consist of a 2D mesh of processing elements (PEs) working cooperatively to accelerate matrix multiplication. The power efficiency of such accelerators is of primary importance

especially considering the edge AI regime. This work presents the SAPER-AI accelerator

an SA accelerator with power intent specified via a unified power format representation in a simplified manner with negligible microarchitectural optimization effort. Our proposed accelerator switches off rows and columns of PEs in a coarse-grained manner

thus leading to SA microarchitecture complying with the varying computational requirements of modern DL workloads. Our analysis demonstrates enhanced power efficiency ranging between 10% and 25% for the best case 32×32 and 64×64 SA designs

respectively. Additionally

the power delay product (PDP) exhibits a progressive improvement of around 6% for larger SA sizes. Moreover

a performance comparison between the MobileNet and ResNet50 models indicates generally better SA performance for the ResNet50 workload. This is due to the more regular convolutions portrayed by ResNet50 that are more favored by SAs

with the performance gap widening as the SA size increases.

关键词

Keywords

references

Amirshahi A , Ansaloni G , Atienza D , 2023a . Accelerator-driven data arrangement to minimize transformers run-time on multi-core architectures . 15 th Workshop on Parallel Programming and Run-Time Management Techniques for Many-Core Architectures and 13 th Workshop on Design Tools and Architectures for Multicore Embedded Computing Platforms , p. 2:1 - 2:13 . https://doi.org/10.4230/OASIcs.parma-ditam.2024.2 https://doi.org/10.4230/OASIcs.parma-ditam.2024.2

Amirshahi A , Klein JAH , Ansaloni G , et al. , 2023b . TiC-SAT: tightly-coupled systolic accelerator for transformers . Proc 28 th Asia and South Pacific Design Automation Conf , p. 657 - 663 . https://doi.org/10.1145/3566097.3567867 https://doi.org/10.1145/3566097.3567867

Bobda C , Mbongue JM , Chow P , et al. , 2022 . The future of FPGA acceleration in datacenters and the cloud . ACM Trans Reconfig Technol Syst , 15 ( 3 ): 34 . https://doi.org/10.1145/3506713 https://doi.org/10.1145/3506713

Chadha R , Bhasker J , 2012 . An ASIC Low Power Primer: Analysis, Techniques and Specification . Springer Science & Business Media , New York, NY, USA . https://doi.org/10.1007/978-1-4614-4271-4 https://doi.org/10.1007/978-1-4614-4271-4

Chang SW , Kim DS , 2024 . Scalable Transformer accelerator with variable systolic array for multiple models in voice assistant applications . Electronics , 13 ( 23 ): 4683 . https://doi.org/10.3390/electronics13234683 https://doi.org/10.3390/electronics13234683

Chen YH , Emer J , Sze V , 2016 . Eyeriss: a spatial architecture for energy-efficient dataflow for convolutional neural networks . ACM SIGARCH Comput Archit News , 44 ( 3 ): 367 - 379 . https://doi.org/10.1145/3007787.3001177 https://doi.org/10.1145/3007787.3001177

Chen YH , Yang TJ , Emer J , et al. , 2019 . Eyeriss v2: a flexible accelerator for emerging deep neural networks on mobile devices . IEEE J Emerg Sel Top Circ Syst , 9 ( 2 ): 292 - 308 . https://doi.org/10.1109/JETCAS.2019.2910232 https://doi.org/10.1109/JETCAS.2019.2910232

Chen YJ , Luo T , Liu SL , et al. , 2014 . DaDianNao: a machine-learning supercomputer . 47 th Annual IEEE/ACM Int Symp on Microarchitecture , p. 609 - 622 . https://doi.org/10.1109/MICRO.2014.58 https://doi.org/10.1109/MICRO.2014.58

Chen YJ , Chen TS , Xu ZW , et al. , 2016 . DianNao family: energy-efficient hardware accelerators for machine learning . Commun ACM , 59 ( 11 ): 105 - 112 . https://doi.org/10.1145/2996864 https://doi.org/10.1145/2996864

Gao MY , Pu J , Yang X , et al. , 2017 . TETRIS: scalable and efficient neural network acceleration with 3D memory . ACM SIGARCH Comput Archit News , 45 ( 1 ): 751 - 764 . https://doi.org/10.1145/3093337.303770 https://doi.org/10.1145/3093337.303770

Genc H , Kim S , Amid A , et al. , 2021 . Gemmini: enabling systematic deep-learning architecture evaluation via full-stack integration . Proc 58 th Annual Design Automation Conf , p. 769 - 774 . https://doi.org/10.1109/DAC18074.2021.9586216 https://doi.org/10.1109/DAC18074.2021.9586216

Guo C , Xue FC , Leng JW , et al. , 2024 . Accelerating sparse DNNs based on tiled GEMM . IEEE Trans Comput , 73 ( 5 ): 1275 - 1289 . https://doi.org/10.1109/TC.2024.3365942 https://doi.org/10.1109/TC.2024.3365942

Hoefler T , Alistarh D , Ben-Nun T , et al. , 2021 . Sparsity in deep learning: pruning and growth for efficient inference and training in neural networks . J Mach Learn Res , 22 ( 1 ): 124 .

Inayat K , Muslim FB , Iqbal J , et al. , 2023 . Power-intent systolic array using modified parallel multiplier for machine learning acceleration . Sensors , 23 ( 9 ): 4297 . https://doi.org/10.3390/s23094297 https://doi.org/10.3390/s23094297

Inayat K , Muslim FB , Mahmood T , et al. , 2024 . FPGA-assisted design space exploration of parameterized AI accelerators: a Quickloop approach . J Syst Archit , 155 : 103260 . https://doi.org/10.1016/j.sysarc.2024.103260 https://doi.org/10.1016/j.sysarc.2024.103260

Jouppi NP , Young C , Patil N , et al. , 2017 . In-datacenter performance analysis of a tensor processing unit . Proc 44 th Annual Int Symp on Computer Architecture , p. 1 - 12 . https://doi.org/10.1145/3079856.3080246 https://doi.org/10.1145/3079856.3080246

Kim B , Lee S , Trivedi AR , et al. , 2020 . Energy-efficient acceleration of deep neural networks on realtime-constrained embedded edge devices . IEEE Access , 8 : 216259 - 216270 . https://doi.org/10.1109/ACCESS.2020.3038908 https://doi.org/10.1109/ACCESS.2020.3038908

Lai CT , Zhang W , 2024 . gem5-NVDLA: a simulation framework for compiling, scheduling and architecture evaluation on AI system-on-chips . ACM Trans Des Autom Electr Systt , 29 ( 5 ): 84 . https://doi.org/10.1145/3661997 https://doi.org/10.1145/3661997

Lee J , Kim C , Kang S , et al. , 2018 . An Energy-Efficient Unified Deep Neural Network Accelerator with Fully-Variable Weight Precision for Mobile Deep Learning Applications . https://old.hotchips.org/hc30/3posters/An_EnergyEfficient_Unified_Deep_Neural_Network_Accelerator.pdf https://old.hotchips.org/hc30/3posters/An_EnergyEfficient_Unified_Deep_Neural_Network_Accelerator.pdf [Accessed on Sept. 20, 2024 ] .

Li WQ , Liu TY , Xiao ZY , et al. , 2023 . TCADer: a tightly coupled accelerator design framework for heterogeneous system with hardware/software co-design . J Syst Archit , 136 : 102822 . https://doi.org/10.1016/j.sysarc.2023.102822 https://doi.org/10.1016/j.sysarc.2023.102822

Loh J , Dudchenko L , Viga J , et al. , 2025 . Towards hardware supported domain generalization in DNN-based edge computing devices for health monitoring . IEEE Trans Biomed Circ Syst , 19 ( 1 ): 5 - 15 . https://doi.org/10.1109/TBCAS.2024.3418085 https://doi.org/10.1109/TBCAS.2024.3418085

Lym S , Erez M , 2020 . FlexSA: flexible systolic array architecture for efficient pruned DNN model training . https://doi.org/10.48550/arXiv.2004.13027 https://doi.org/10.48550/arXiv.2004.13027

Moghaddasi I , Nam BG , 2024 . Enhancing computation-efficiency of deep neural network processing on edge devices through serial/parallel systolic computing . Mach Learn Knowl Extr , 6 ( 3 ): 1484 - 1493 . https://doi.org/10.3390/make6030070 https://doi.org/10.3390/make6030070

Moons B , De Brabandere B , Van Gool L , et al. , 2016 . Energy-efficient ConvNets through approximate computing . IEEE Winter Conf on Applications of Computer Vision , p. 1 - 8 . https://doi.org/10.1109/WACV.2016.7477614 https://doi.org/10.1109/WACV.2016.7477614

Moons B , Uytterhoeven R , Dehaene W , et al. , 2017a . 14.5 envision: a 0.26-to-10TOPS/W subword-parallel dynamic-voltage-accuracy-frequency-scalable convolutional neural network processor in 28nm FDSOIF . IEEE Int Solid-State Circuits Conf , p. 246 - 247 . https://doi.org/10.1109/ISSCC.2017.7870353 https://doi.org/10.1109/ISSCC.2017.7870353

Moons B , Uytterhoeven R , Dehaene W , et al. , 2017b . DVAFS: trading computational accuracy for energy through dynamic-voltage-accuracy-frequency-scaling . Design, Automation & Test in Europe Conf & Exhibition , p. 488 - 493 . https://doi.org/10.23919/DATE.2017.7927038 https://doi.org/10.23919/DATE.2017.7927038

Muslim FB , Inayat K , Khan S , 2024 . LPCHISEL: automatic power intent generation for a chisel-based ASIC design . Comput Electr Eng , 115 : 109115 . https://doi.org/10.1016/j.compeleceng.2024.109115 https://doi.org/10.1016/j.compeleceng.2024.109115

Park M , Hwang S , Cho H , 2024 . BIRD: bi-directional input reuse dataflow for enhancing depthwise convolution performance on systolic arrays . IEEE Trans Comput , 73 ( 12 ): 2708 - 2721 . https://doi.org/10.1109/TC.2024.3449103 https://doi.org/10.1109/TC.2024.3449103

Qamar A , Muslim FB , Iqbal J , et al. , 2017 . LP-HLS: automatic power-intent generation for high-level synthesis based hardware implementation flow . Microproc Microsyst , 50 : 26 - 38 . https://doi.org/10.1016/j.micpro.2017.02.002 https://doi.org/10.1016/j.micpro.2017.02.002

Ryu S , Kim H , Yi W , et al. , 2019 . BitBlade: area and energy-efficient precision-scalable neural network accelerator with bitwise summation . Proc 56 th ACM/IEEE Design Automation Conf , p. 1 - 6 . https://doi.org/10.1145/3316781.3317784 https://doi.org/10.1145/3316781.3317784

Seshadri K , Akin B , Laudon J , et al. , 2022 . An evaluation of edge TPU accelerators for convolutional neural networks . IEEE Int Symp on Workload Characterization , p. 79 - 91 . https://doi.org/10.1109/IISWC55918.2022.00017 https://doi.org/10.1109/IISWC55918.2022.00017

Song J , Cho Y , Park JS , et al. , 2019 . 7.1 an 11.5 TOPS/W 1024-MAC butterfly structure dual-core sparsity-aware neural processing unit in 8nm flagship mobile SoC . IEEE Int Solid-State Circuits Conf , p. 130 - 132 . https://doi.org/10.1109/ISSCC.2019.8662476 https://doi.org/10.1109/ISSCC.2019.8662476

Xu R , Ma S , Wang YH , et al. , 2021a . Configurable multi-directional systolic array architecture for convolutional neural networks . ACM Trans Archit Code Optim , 18 ( 4 ): 42 . https://doi.org/10.1145/3460776 https://doi.org/10.1145/3460776

Xu R , Ma S , Wang YH , et al. , 2021b . Heterogeneous systolic array architecture for compact CNNs hardware accelerators . IEEE Trans Parallel Distrib Syst , 33 ( 11 ): 2860 - 2871 . https://doi.org/10.1109/TPDS.2021.3129647 https://doi.org/10.1109/TPDS.2021.3129647

Xu R , Ma S , Guo Y , et al. , 2023 . A survey of design and optimization for systolic array-based DNN accelerators . ACM Comput Surv , 56 ( 1 ): 20 . https://doi.org/10.1145/3604802 https://doi.org/10.1145/3604802

Yüzügüler AC , Sönmez C , Drumond M , et al. , 2023 . Scale-out systolic arrays . ACM Trans Archit Code Optim , 20 ( 2 ): 27 . https://doi.org/10.1145/3572917 https://doi.org/10.1145/3572917

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Map

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰