Energies, Vol. 19, Pages 596: Enhanced Transformer for Multivariate Load Forecasting: Timestamp Embedding and Convolution-Augmented Attention
Energies doi: 10.3390/en19030596
Authors:
Wanxing Sheng
Xiaoyu Yang
Dongli Jia
Keyan Liu
Zhenhao Wang
Rongheng Lin
Aiming at the insufficient capture of temporal dependence and weak coupling of external factors in multivariate load forecasting, this paper proposes a Transformer model integrating timestamp-based positional embedding and convolution-augmented attention. The model enhances temporal modeling capability through timestamp-based positional embedding, optimizes local contextual representation via convolution-augmented attention, and achieves deep fusion of load data with external factors such as temperature, humidity, and electricity price. Experiments based on the 2018 full-year load dataset for a German region show that the proposed model outperforms single-factor and multi-factor LSTMs in both short-term (24 h) and long-term (cross-month) forecasting. The research results verify the model’s accuracy and stability in multivariate load forecasting, providing technical support for smart grid load dispatching.