Energies, Vol. 18, Pages 6162: Short-Term Wind Power Forecasting with Transformer-Based Models Enhanced by Time2Vec and Efficient Attention
Energies doi: 10.3390/en18236162
Authors:
Djayr Alves Bispo Junior
Gustavo de Novaes Pires Leite
Enrique Lopez Droguett
Othon Vinicius Cavalcanti de Souza
Lucas Albuquerque Lisboa
George Darmiton da Cunha Cavalcanti
Alvaro Antonio Villa Ochoa
Alexandre Carlos Araújo da Costa
Olga de Castro Vilela
Leonardo José de Petribú Brennand
Guilherme Ferretti Rissi
Giovanni Moura de Holanda
Tsang Ing Ren
Accurate wind power forecasting is essential to optimize wind farm operations and ensure the stable integration of renewable energy into the grid. This study explores Transformer-based architectures to address the challenges of wind variability and temporal dependencies in short-term forecasting. A sensitivity analysis on model architecture is conducted, incorporating Time2Vec—a temporal encoding technique that captures complex temporal patterns. In addition, we replace the standard FullAttention mechanism with ProbSparse Attention, FlowAttention and FlashAttention, resulting in the Informer, Flowformer and Flashformer models, to improve computational efficiency while maintaining predictive accuracy. The novelty of this work lies in applying FlashAttention within the context of wind power forecasting and integrating Time2Vec into the Informer, Flowformer and Flashformer models. We propose four architectures—T2V-Transformer, T2V-Informer, T2V-Flowformer, and T2V-Flashformer—and compare them against benchmark models: Multi-Layer Perceptron (MLP), Long Short-Term Memory (LSTM), and DLinear. Real-world data from a wind farm in the Northeast of Brazil is used under two forecasting scenarios. In Scenario A, T2V-Transformer, T2V-Informer and T2V-Flashformer achieved Improvement over Reference RMSE (IoR-RMSE) scores of 17.73%, 17.59% and 16.67%, respectively. In Scenario B, T2V-Flowformer and T2V-Flashformer reached 27.84% and 27.45%, respectively. These results confirm the effectiveness of the proposed models in advancing short-term wind power forecasting.