LDformer: a parallel neural network model for long-term power forecasting

Ran TIAN; Xinmei LI; Zhongyu MA; Yanxing LIU; Jingxia WANG; Chu WANG

doi:10.1631/FITEE.2200540

Your Location：

Home >

Browse articles >

LDformer: a parallel neural network model for long-term power forecasting

Regular Papers | Updated：2023-09-21

- LDformer: a parallel neural network model for long-term power forecasting
- Ldformer：面向长期电力预测的并行神经网络模型
- Frontiers of Information Technology & Electronic Engineering Vol. 24, Issue 9, Pages: 1287-1301(2023)
- Affiliations：
  
  College of Computer Science & Engineering, Northwest Normal University, Lanzhou 730070, China
- Author bio：
  
  ‡Corresponding author
  2020211978@nwnu.edu.cn
  mazybg@nwnu.edu.cn
  lyanxing@nwnu.edu.cn
  2020222004@nwnu.edu.cn
  2020221992@nwnu.edu.cn
- Funds：
- DOI：10.1631/FITEE.2200540
  CLC： TP183
- Received：03 November 2022，
  
  Revised：2023-02-13，
  
  Accepted：13 February 2023，
  
  Published：0 September 2023
- Accepted：
Scan QR Code
Ran TIAN, Xinmei LI, Zhongyu MA, et al. LDformer: a parallel neural network model for long-term power forecasting[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(9): 1287-1301.
DOI：

Ran TIAN, Xinmei LI, Zhongyu MA, et al. LDformer: a parallel neural network model for long-term power forecasting[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(9): 1287-1301. DOI： 10.1631/FITEE.2200540.

摘要

准确的长期电力预测对电网决策运行和用户用电管理非常重要，可保证电力系统的可靠供电和电网经济的可靠运行。然而，大多数时间序列预测模型在数据量大、预测精度高的长时间序列预测任务中表现不佳。为了应对这一挑战，提出名为LDformer的并行时间序列预测模型。首先，将Informer与长短期记忆网络相结合，以获得时间序列的深度表达能力。其次，提出并行编码器模块提高模型鲁棒性，并将卷积层与注意力机制相结合，以避免注意力机制中的值冗余。最后，提出结合UniDrop的概率稀疏注意力机制，以减少计算开销并减轻序列中一些关键连接丢失的风险。在5个真实数据集上的实验结果显示，在不同的长时间序列预测任务中，LDformer大部分结果都优于最先进的基线结果。

Abstract

Accurate long-term power forecasting is important in the decision-making operation of the power grid and power consumption management of customers to ensure the power system’s reliable power supply and the grid economy’s reliable operation. However

most time-series forecasting models do not perform well in dealing with long-time-series prediction tasks with a large amount of data. To address this challenge

we propose a parallel time-series prediction model called LDformer. First

we combine Informer with long short-term memory (LSTM) to obtain deep representation abilities in the time series. Then

we propose a parallel encoder module to improve the robustness of the model and combine convolutional layers with an attention mechanism to avoid value redundancy in the attention mechanism. Finally

we propose a probabilistic sparse (ProbSparse) self-attention mechanism combined with UniDrop to reduce the computational overhead and mitigate the risk of losing some key connections in the sequence. Experimental results on five datasets show that LDformer outperforms the state-of-the-art methods for most of the cases when handling the different long-time-series prediction tasks.

关键词

Keywords

references

Cao JS , Wang JH , 2019 . Stock price forecasting model based on modified convolution neural network and financial time series analysis . Int J Commun Syst , 32 ( 12 ): e3987 . https://doi.org/10.1002/dac.3987 https://doi.org/10.1002/dac.3987

Chakraborty T , Chattopadhyay S , Ghosh I , 2019 . Forecasting dengue epidemics using a hybrid methodology . Phys A Stat Mech Appl , 527 : 121266 . https://doi.org/10.1016/j.physa.2019.121266 https://doi.org/10.1016/j.physa.2019.121266

Chen JF , Wang WM , Huang CM , 1995 . Analysis of an adaptive time-series autoregressive moving-average (ARMA) model for short-term load forecasting . Electr Power Syst Res , 34 ( 3 ): 187 - 196 . https://doi.org/10.1016/0378-7796(95)00977-1 https://doi.org/10.1016/0378-7796(95)00977-1

Chen XY , Sun LJ , 2022 . Bayesian temporal factorization for multidimensional time series prediction . IEEE Trans Patt Anal Mach Intell , 44 ( 9 ): 4659 - 4673 . https://doi.org/10.1109/TPAMI.2021.3066551 https://doi.org/10.1109/TPAMI.2021.3066551

Chuku C , Simpasa A , Oduor J , 2019 . Intelligent forecasting of economic growth for developing economies . Int Econ , 159 : 74 - 93 . https://doi.org/10.1016/j.inteco.2019.06.001 https://doi.org/10.1016/j.inteco.2019.06.001

Ciechulski T , Osowski S , 2021 . High precision LSTM model for short-time load forecasting in power systems . Energies , 14 ( 11 ): 2983 . https://doi.org/10.3390/en14112983 https://doi.org/10.3390/en14112983

Ding M , Zhou H , Xie H , et al. , 2021 . A time series model based on hybrid-kernel least-squares support vector machine for short-term wind power forecasting . ISA Trans , 108 : 58 - 68 . https://doi.org/10.1016/j.isatra.2020.09.002 https://doi.org/10.1016/j.isatra.2020.09.002

Gai SX , Zeng XQ , Yuan TF , 2021 . Parking volume forecast of railway station garages based on passenger behaviour analysis using the LSTM network . J Adv Transp , 2021 : 6688609 . https://doi.org/10.1155/2021/6688609 https://doi.org/10.1155/2021/6688609

Guo SN , Lin YF , Li SJ , et al. , 2019 . Deep spatial-temporal 3D convolutional neural networks for traffic data forecasting . IEEE Trans Intell Transp Syst , 20 ( 10 ): 3913 - 3926 . https://doi.org/10.1109/TITS.2019.2906365 https://doi.org/10.1109/TITS.2019.2906365

Han ZY , Zhao J , Leung H , et al. , 2021 . A review of deep learning models for time series prediction . IEEE Sens J , 21 ( 6 ): 7833 - 7848 . https://doi.org/10.1109/JSEN.2019.2923982 https://doi.org/10.1109/JSEN.2019.2923982

Hosseini MK , Talebpour A , 2019 . Traffic prediction using time-space diagram: a convolutional neural network approach . Transp Res Rec J Transp Res Board , 2673 ( 7 ): 425 - 435 . https://doi.org/10.1177/0361198119841291 https://doi.org/10.1177/0361198119841291

Hu R , Chiu YC , Hsieh CW , 2020 . Crowding prediction on mass rapid transit systems using a weighted bidirectional recurrent neural network . IET Intell Transp Syst , 14 ( 3 ): 196 - 203 . https://doi.org/10.1049/iet-its.2018.5542 https://doi.org/10.1049/iet-its.2018.5542

Karevan Z , Suykens JAK , 2020 . Transductive LSTM for time-series prediction: an application to weather forecasting . Neur Netw , 125 : 1 - 9 . https://doi.org/10.1016/j.neunet.2019.12.030 https://doi.org/10.1016/j.neunet.2019.12.030

Khan MH , Muhammad NS , El-Shafie A , 2020 . Wavelet based hybrid ANN-ARIMA models for meteorological drought forecasting . J Hydrol , 590 : 125380 . https://doi.org/10.1016/j.jhydrol.2020.125380 https://doi.org/10.1016/j.jhydrol.2020.125380

Khandelwal U , He H , Qi P , et al. , 2018 . Sharp nearby, fuzzy far away: how neural language models use context . Proc 56 th Annual Meeting of the Association for Computational Linguistics , p. 284 - 294 . https://doi.org/10.18653/v1/P18-1027 https://doi.org/10.18653/v1/P18-1027

Kim SH , Lee G , Kwon GY , et al. , 2018 . Deep learning based on multi-decomposition for short-term load forecasting . Energies , 11 ( 12 ): 3433 . https://doi.org/10.3390/en11123433 https://doi.org/10.3390/en11123433

Li SY , Jin XY , Xuan Y , et al. , 2019 . Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting . Proc 33 rd Int Conf on Neural Information Processing Systems , Article 471 . https://doi.org/10.5555/3454287.3454758 https://doi.org/10.5555/3454287.3454758

Marcjasz G , Uniejewski B , Weron R , 2020 . Probabilistic electricity price forecasting with NARX networks: combine point or probabilistic forecasts? Int J Forecast , 36 ( 2 ):‍ 466 - 479 . https://doi.org/10.1016/j.ijforecast.2019.07.002 https://doi.org/10.1016/j.ijforecast.2019.07.002

Miao KC , Han TT , Yao YQ , et al. , 2020 . Application of LSTM for short term fog forecasting based on meteorological elements . Neurocomputing , 408 : 285 - 291 . https://doi.org/10.1016/j.neucom.2019.12.129 https://doi.org/10.1016/j.neucom.2019.12.129

Min K , Kim D , Park J , et al. , 2019 . RNN-based path prediction of obstacle vehicles with deep ensemble . IEEE Trans Veh Technol , 68 ( 10 ): 10252 - 10256 . https://doi.org/10.1109/TVT.2019.2933232 https://doi.org/10.1109/TVT.2019.2933232

Nóbrega JP , Oliveira ALI , 2019 . A sequential learning method with Kalman filter and extreme learning machine for regression and time series forecasting . Neurocomputing , 337 : 235 - 250 . https://doi.org/10.1016/j.neucom.2019.01.070 https://doi.org/10.1016/j.neucom.2019.01.070

Ran XD , Shan ZG , Fang YF , et al. , 2019 . An LSTM-based method with attention mechanism for travel time prediction . Sensors , 19 ( 4 ): 861 . https://doi.org/10.3390/s19040861 https://doi.org/10.3390/s19040861

Syriopoulos T , Tsatsaronis M , Karamanos I , 2021 . Support vector machine algorithms: an application to ship price forecasting . Comput Econ , 57 ( 1 ): 55 - 8 . https://doi.org/10.1007/s10614-020-10032-2 https://doi.org/10.1007/s10614-020-10032-2

Vaswani A , Shazeer N , Parmar N , et al. , 2017 . Attention is all you need . Proc 31 st Int Conf on Neural Information Processing Systems , p. 6000 - 6010 . https://doi.org/10.5555/3295222.3295349 https://doi.org/10.5555/3295222.3295349

Viccione G , Guarnaccia C , Mancini S , et al. , 2020 . On the use of ARIMA models for short-term water tank levels forecasting . Water Supply , 20 ( 3 ): 787 - 799 . https://doi.org/10.2166/ws.2019.190 https://doi.org/10.2166/ws.2019.190

Wang C , Tian R , Hu J , et al. , 2023 . A trend graph attention network for traffic prediction . Inform Sci , 623 : 275 - 292 . https://doi.org/10.1016/j.ins.2022.12.048 https://doi.org/10.1016/j.ins.2022.12.048

Wang ZP , Qu JF , Fang XY , et al. , 2020 . Prediction of early stabilization time of electrolytic capacitor based on ARIMA-Bi_LSTM hybrid model . Neurocomputing , 403 : 63 - 79 . https://doi.org/10.1016/j.neucom.2020.03.054 https://doi.org/10.1016/j.neucom.2020.03.054

Wu N , Green B , Ben X , et al. , 2020 . Deep transformer models for time series forecasting: the influenza prevalence case . Proc 37 th Int Conf on Machine Learning , p. 1 - 10 .

Wu Z , Wu LJ , Meng Q , et al. , 2021 . UniDrop: a simple yet effective technique to improve transformer without extra cost . Proc Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , p. 3865 - 3878 . https://doi.org/10.18653/v1/2021.naacl-main.302 https://doi.org/10.18653/v1/2021.naacl-main.302

Xiao S , Yan JC , Farajtabar M , et al. , 2019 . Learning time series associated event sequences with recurrent point process networks . IEEE Trans Neur Netw Learn Syst , 30 ( 10 ): 3124 - 3136 . https://doi.org/10.1109/TNNLS.2018.2889776 https://doi.org/10.1109/TNNLS.2018.2889776

Xie YY , Lou YS , 2019 . Hydrological time series prediction by ARIMA-SVR combined model based on wavelet transform . Proc 3 rd Int Conf on Innovation in Artificial Intelligence , p. 243 - 247 . https://doi.org/10.1145/3319921.3319959 https://doi.org/10.1145/3319921.3319959

Xu DW , Wang YD , Jia LM , et al. , 2017 . Real-time road traffic state prediction based on ARIMA and Kalman filter . Front Inform Technol Electron Eng , 18 ( 2 ): 287 - 302 . https://doi.org/10.1631/FITEE.1500381 https://doi.org/10.1631/FITEE.1500381

Zhang D , Ling JW , Wei ZH , et al. , 2018 . Long-term traffic speed prediction based on multiscale spatio-temporal feature learning network . IEEE Trans Intell Transp Syst , 20 ( 10 ): 3700 - 3709 . https://doi.org/10.1109/TITS.2018.2878068 https://doi.org/10.1109/TITS.2018.2878068

Zhang L , Wang G , Giannakis GB , 2019 . Real-time power system state estimation and forecasting via deep unrolled neural networks . IEEE Trans Signal Process , 67 ( 15 ): 4069 - 4077 . https://doi.org/10.1109/TSP.2019.2926023 https://doi.org/10.1109/TSP.2019.2926023

Zhang TJ , Song S , Li SG , et al. , 2019 . Research on gas concentration prediction models based on LSTM multidimensional time series . Energies , 12 ( 1 ): 161 . https://doi.org/10.3390/en12010161 https://doi.org/10.3390/en12010161

Zheng JH , Huang MF , 2020 . Traffic flow forecast through time series analysis based on deep learning . IEEE Access , 8 : 82562 - 82570 . https://doi.org/10.1109/ACCESS.2020.2990738 https://doi.org/10.1109/ACCESS.2020.2990738

Zhou HY , Zhang SH , Peng JQ , et al. , 2021 . Informer: beyond efficient transformer for long sequence time-series forecasting . Proc 35 th AAAI Conf on Artificial Intelligence , p. 1 - 15 . https://doi.org/10.1609/aaai.v35i12.17325 https://doi.org/10.1609/aaai.v35i12.17325

Views

207

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

TibetanGoTinyNet: a lightweight U-Net style network for zero learning of Tibetan Go

Exploring nonlinear spatiotemporal effects for personalized next point-of-interest recommendation

ECGID: a human identification method based on adaptive particle swarm optimization and the bidirectional LSTM model

Video summarization with a graph convolutional attention network

Related Author

Junzhi YU

Yandong CHEN

Licheng WU

Yanyin ZHANG

Xiali LI

Zhimin LV

Xi SUN

Yefei ZHANG

Related Institution

Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance, Ministry of Education, Minzu University of China

School of Information Engineering, Minzu University of China

Department of Advanced Manufacturing and Robotics, College of Engineering, Peking University

Collaborative Innovation Center of Steel Technology, University of Science and Technology Beijing

School of Electronics and Information Engineering, Hangzhou Dianzi University

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰