Short-Term Load Forecasting Utilizing a Combination Model: A Brief Review

To deliver electricity to customers safely and economically, power companies encounter numerous economic and technical challenges in their operations. Power flow analysis, planning, and control of power systems stand out among these issues. Over the last several years, one of the most developing study topics in this vital and demanding discipline has been electricity short-term load forecasting (STLF). Power system dispatching, emergency analysis, power flow analysis, planning, and maintenance all require it. This study emphasizes new research on long short-term memory (LSTM) algorithms related to particle swarm optimization (PSO) inside this area of short-term load forecasting. The paper presents an in-depth overview of hybrid networks that combine LSTM and PSO and have been effectively used for STLF. In the future, the integration of LSTM and PSO in the development of comprehensive prediction methods and techniques for multi-heterogeneous models is expected to offer significant opportunities. With an increased dataset, the utilization of advanced multi-models for comprehensive power load prediction is anticipated to achieve higher accuracy.

Combined model; LSTM; Particle swarm optimization; STLF

As one of the frequently used artificial networks, the LSTM neural network, as a specialized recurrent neural network (RNN), can be successfully trained and forecasted based on historical data, resulting in superior prediction outcomes. In recent years, LSTM has been widely used in the field of power load forecasting due to its ability to effectively learn time series and nonlinear data correlations (Stratigakos et al., 2021; Wei and Pan, 2021; Bedi and Toshniwal, 2019; Imani and Ghassemian, 2019; Wang et al., 2019; Tang et al., 2019; Choi, Ryu, and Kim, 2018; Kim, Kim, and Choi, 2018; Bouktif et al., 2018; Zheng, Yuan, and Chen, 2017; Di Persio and Honchar, 2017; Sri et al., 2017). As a result, several researchers have created LSTM neural network versions, which are often a combination of neural networks (LSTM) and learning algorithms (PSO). In terms of accuracy, computational cost, and time requirements, hybrid models beat classical models.

The following sections make up this paper. The first section provides a summary of the technology, its origins, benefits, and drawbacks, as well as the study's objective. The second section gives a brief summary of the LSTM and PSO approaches. The third section contains a summary of the literature review technique and stages, as well as PSO and LSTM, classical and mixed neural network algorithms that have been effectively employed to STLF. Finally, the review's findings are presented in Section 4.

Brief Description of PSO and LSTM

2.1. PSO

     Human social behavior influenced the concept of particle swarm optimization. Kennedy and Eberhart (Garcia-Gonzalo and Fernandez-Martinez, 2012) suggested it in 1994. There are n particles in the solution space, each of which represents a dimension of the solution space. This is how it represents the problem: the particles explore the solution space in search of the optimal solution. According to Kennedy, the three concepts of evaluation (learning by self-experience), comparison (learning by comparing experiences), and imitation (learning by adapting to ideal trends) encapsulate the learning processes involved (Marini and Walczak, 2015). Each particle makes its own choices while being impacted by its surroundings. When a particle attempts to replicate its best-known solution, it may unintentionally discover an even better option, potentially affecting its neighbors negatively. This process links all particles to an optimal point, akin to the cultural inclination within human society.

2.2. LSTM neural network
       M.I. Jordan and Jeffrey Elman proposed the Recurrent Neural Network (RNN) in the late 1980s (Yu et al., 2019). Because it overcomes the inability of standard neural networks to simulate time, it is frequently used in industries such as speech recognition. While a basic RNN can manage certain short-term dependencies, it is not capable of dealing with long-term ones. At the close of the twentieth century, Sepp Hochreiter and Jürgen Schmidhuber created an LSTM neural network to cope with this challenge, which was later improved in practical applications Guo et al. (2021b). The "memory" capabilities of the LSTM neural network in coping with timing-related difficulties are substantially superior to that of the RNN neural network due to its unique gate structure. Figure 1 depicts the building of the LSTM memory block.

Figure 1 The memory block architecture of the LSTM (Zheng, Yuan, and Chen, 2017)
        In Figure 1,  represents the input at time  and  respectively, represent the forget gate, input gate, and output gate. At time  is the input node state; at time  and and C are the unit states; at time  and  and h are the outputs; tanh is the hyperbolic tangent function, while  is the activation function.

Different Variants of PSO and LSTM: An Overview

    This section introduces and analyzes some recent work on PSO combined with LSTM neural network models for STLF. There are three categories to the work that has been done thus far, as shown in Table 1. Next, Table 2 provides a detailed list of the various models and their applications as mentioned in the related literature. This table displays the type of load each model addresses, the specific name of the proposed model, and its classification, further enriching our understanding of the application of PSO combined with LSTM.

Table 1 Job categorization as of now

Type	Related Literature
Classical PSO combined with LSTM neural network	(Zou et al., 2020), (Cao et al., 2021)
Variants of PSO Algorithm Combined with LSTM Neural Network	(Xudong, Shuo, and Qingwu, 2020), (Wei and Pan, 2021), (Guo et al., 2021a), (Chang et al., 2020)
Using PSO and LSTM models, as well as other models and approaches, create comprehensive load forecasting models	(Yuan et al., 2019), (Shang et al., 2021)

Table 2 Models and types proposed in the literature

Literature	Load type	Proposed model	Type of model
(Zou et al., 2020)	The daily energy load of the community	PSO-LSTM	Classical PSO combined with LSTM neural network
(Cao et al., 2021)	Heat pump power load	PSO-LSTM	Classical PSO combined with LSTM neural network
(Xudong, Shuo, and Qingwu, 2020)	Building heating and cooling load	IPSO-LSTM	Variants of PSO Algorithm Combined with LSTM Neural Network
(Wei and Pan, 2021)	Grid data daily electricity load	ACMPSO-LSTM	Variants of PSO Algorithm Combined with LSTM Neural Network
(Guo et al., 2021a)	Grid data daily electricity load	GPSO-LSTM	Variants of PSO Algorithm Combined with LSTM Neural Network
(Chang et al., 2020)	Loads under VPP jurisdiction	IPSO-LSTM	Variants of PSO Algorithm Combined with LSTM Neural Network
(Yuan et al., 2019)	Wind energy load	Beta-PSO-LSTM	Using PSO and LSTM models, as well as other models and approaches, create comprehensive load forecasting models.
(Shang et al., 2021)	Public electricity load	PSO-KFCM&CNN-LSTM	Using PSO and LSTM models, as well as other models and approaches, create comprehensive load forecasting models.

3.1. Classical PSO combined with LSTM neural network

        In (Zou et al., 2020), Luyao Zou et al. proposed a combined strategy related to the LSTM as well as PSO to address the challenge of reducing non-renewable power use throughout the community. The prior day's energy use was predicted using an LSTM-based model. The community's battery and P2P sharing system, which uses PSO to fix the imbalance between load and solar power production, is evaluated in the second scheduling stage. The LSTM-based prediction technique outperforms the Auto Regressive Integrated Moving Average (ARIMA) in precision as well as model training time, as evaluated by root mean square error (RMSE), total training length, and mean square error (MSE). However, for hourly day-ahead demand forecasting with LSTM, this study only addresses the demand of 17 families from a single neighborhood, which may impair the forecast's accuracy. Future studies will address energy scheduling in different communities as well as the interests of each family, which will need more datasets.

        Cao et al. (2021) proposed an improved power management solution that incorporates LSTM as well as PSO. LSTM was performed to anticipate the power consumption of a heat pump based on prior data utilizing thermal pump data from the United Kingdom. Then, for simulation comparison, construct the models as given in Table 3. The assessment measures of the prediction impact in these models are mean absolute percentage error (MAPE) and root mean square error (RMSE). With a MAPE of 1.59 percent and an RMSE of 0.15 percent, the LSTM model has the greatest performance and outperforms other models, according to the data. The LSTM neural network STLF model was then combined with a PSO algorithm-based home energy management system to create a fully optimal home energy management system (HEMS). This model's accuracy was shown to be exceptional, with an average percent difference of 2.06 percent between projected and real HEMS power prices, which is higher than previous comparable research in the area. However, the study's predictive accuracy can be improved once again. Individual end-user forecasting results might be used in future studies to analyze the community's overall forecasting accuracy and the impact of integrating load forecasting with HEMS on the communities.

Table 3 The outcomes for the optimal parameters of several algorithms (Cao et al., 2021) 

Algorithms	LSTM	BPNN	SARIMA	RF	ES
MAPE (%)	1.59	6.4	13.1	12.24	10.72
RMSE	0.15	0.17	0.34	0.31	0.27

3.2. Variants of PSO Algorithm Combined with LSTM Neural Network

Liu Xudong et al. (Xudong, Shuo, and Qingwu, 2020), proposed a prediction model for building cooling and heating loads using an improved PSO (IPSO) method and an LSTM neural network model. This research evaluates the prediction outputs of IPSO-LSTM, Support Vector Regression (SVR), Media Loss Rate (MLR), and Extreme Learning Machine (ELM) models using datasets from the UCI machine learning collection. The findings reveal that IPSO-LSTM can anticipate heating and cooling loads better than other algorithms, that its prediction performance is better than other algorithms, and that its R-squared index is closer to 1 than other algorithms. However, a significant number of trials show that the forecast accuracy of various cooling load models is not particularly good. Future studies will primarily concentrate on two key areas: substituting a more suitable model for predicting cooling load and identifying building factors more closely linked to cooling load.
        In (Wei and Pan, 2021), Wei Tengfei et al. proposed a fresh STLF model (ACMPSO-LSTM) that enhances LSTM neural networks by using an adaptive cauchy mutated particle swarm optimizer (ACMPSO) algorithm. This study used data (including temperature, humidity, day type and actual whole point load) from China's Zhejiang Power Grid to build LSTM, PSO-LSTM, as well as ACMPSO-LSTM models. The average prediction errors of LSTM, PSO-LSTM, and ACMPSO-LSTM for one week are 3.92 percent, 3.24 percent, and 2.59 percent, respectively, according to the data. When compared to LSTM and PSO-LSTM, the ACMPSO-LSTM model improves prediction accuracy by 33.9% and 20.1%,  respectively. The ACMPSO algorithm is more adaptable than the PSO technique, and it can find better model parameters and increase the prediction effect. In addition, the model considers the impacts of temperature, humidity, and day type. In the future, the model can also be applied to wind speed forecasting, photovoltaic power generation forecasting, and forecasting in other fields.

Guo et al. (2021a) proposed a fresh strategy for STLF that combines feature correlation analysis correction and global particle swarm optimization (GPSO) with a recurrent neural network (RNN) using LSTM. This research uses load data, date type data, and corresponding daily weather data from a region in southern China from 2012 to 2014 as a data set and then performs exploratory data analysis (EDA) and corrects relevant influencing factors to ensure the integrity and standardization of the data, before selecting features based on correlation set sequence as input. Following that, the GPSO-LSTM model, an Elman RNN model and a back-propagation neural network (BPNN), were created. The GPSO-LSTM model, according to simulation findings, has a lower error than the BPNN and Elman techniques. On normal days, holiday, and critical days, the performance is notably better, particularly on weekdays where the prediction impact is most pronounced. The new model demonstrated a mean absolute percentage error (MAPE) of 1.18 percent and a normalized root mean square error (NRMSE) of 2.4 percent. However, EDA cannot totally eradicate the inaccuracies in the data set utilized in this work. In the future, better equipment will be used to obtain more specific and comprehensive relevant data right away for forecasting.

Chang et al. (2020) proposed an energy co-optimization management approach based on model predictive control (MPC) for a virtual power plant's (VPP's) energy storage system (ESS). LSTM neural networks are being used to collect one-hour predictions for load, wind, and solar electricity output within the VPP's jurisdiction. In the MPC architecture, the optimal schedule is subsequently addressed using an improved particle swarm optimization (PSO) approach. This study takes a year's worth of data from a specific location as a data set and assesses the effectiveness of the proposed algorithm primarily on two fronts: distributed generation usage efficiency and public grid impact reduction. According to the studies, the method increases distributed generating utilization while smoothing the power exchange graph and lowering its effect on the power system. The technique uses monitor outcomes as feedback, which makes the optimal solution a feedback circuit and reduces the influence of prediction error on efficiency, especially when the prediction data is inaccurate. However, wind energy and photovoltaic power generation are affected by weather factors. In future studies, the prediction model will need to take into account associated weather aspects in order to increase forecast accuracy.

3.3. Using PSO and LSTM models, as well as other models and approaches, create comprehensive load forecasting models

Yuan et al. (2019) proposed a combined model (Beta-PSO-LSTM) for the prediction interval of wind power (PIWP) based on the beta distribution function of an LSTM neural network and PSO. In this research, one wind farm collected 1200 wind energy-related data every 10 minutes in March 2016, 1100 for training, and 100 for prediction. The LSTM model is utilized to forecast wind power. Following that, six models were created for simulation comparison, as shown in Table 4. The freshly presented model gives the maximum value in terms of performance measures, according to the data. When utilizing the fresh model to calculate the wind power prediction interval, it is possible to construct forecast intervals with more coverage and better performance. It is useful for wind power sequence uncertainty modeling. Wind power, on the other hand, has fluctuating and intermittent features in the functioning of power systems, and is particularly susceptible to the effect of meteorological conditions. In future development, the forecasting model should consider significant meteorological elements to increase forecasting accuracy. Due to the unpredictability of wind power, the wind power forecast interval may also be applied in the ideal scheduling issue of the water-heating wind system.

Table 4 Six PIWP models' performance (Yuan et al., 2019)

In (Shang et al., 2021), Chuan Shang et al. proposed an STLF method using PSO-KFCM (KFCM, Kernel Fuzzy c-means) and CNN-LSTM (CNN, Convolutional Neural Network) models based on daily load curve clustering. The study makes use of a public electricity load dataset from New South Wales (NSW), Australia. Then, for simulation, the classic LSTM model, eXtreme Gradient Boosting (XGBoost), Multilayer Perceptron (MLP), Bidirectional Recurrent Neural Network (Bi-RNN), Gate Recurrent Unit (GRU), and the newly suggested model are set up. Table 5 depicts the comprehensive evaluation parameters for the six models. According to simulation findings, the suggested novel model obtains a MAPE of 1.34 percent on the whole test dataset. The new model's MAPE is 3.02 percent lower than the regular LSTM model. This complete technology considers previous load data as well as influencing elements (meteorology, date type, economics, and so on), normalizes the historical load data, and conducts fuzzy mapping on the influencing factors using the Pearson Correlation Coefficient (PCC). When compared to the usual fuzzy c-means strategy for clustering daily load curves, the PSO-KFCM approach significantly improved clustering quality (31.9%). The advantages of strong feature extraction are combined with the capacity to analyze vast time series in this hybrid prediction. The CNN-LSTM model and the PSO-KFCM approach described in this study, on the other hand, are not restricted to STLF alone. In the future, it might be used for bearing problem diagnostics, signal pattern identification, intelligent visual sorting, and other deep learning applications.

Table 5 The figure below compares the six models over the test set in terms of maximum MAPE, lowest MAPE, and average MAPE (Shang et al., 2021)

Models	Max MAPE	Min MAPE	Average MAPE
MLP Model	3.82%	1.88%	2.73%
GRU Model	6.88%	2.59%	4.65%
Bi-RNN Model	4.51%	1.83%	3.39%
LSTM Model	5.46%	3.06%	4.36%
XGboost Model	3.08%	1.15%	2.16%
Proposed	2.40%	0.48%	1.34%

    This paper presents novel work on combining PSO with an LSTM algorithm for Short-Term Load Forecasting (STLF), now employed for effective short-term load anticipation. According to the results of various studies, LSTM-based forecasting algorithms have the ability to offer a viable solution for the difficult problem of time series forecasting. To handle this tough and intriguing challenge, PSO, a random search algorithm with global learning potential, is also integrated with LSTM. The described technologies have proven their capacity to estimate electrical demands, lowering power system running costs and enhancing operational efficiency. Furthermore, as smart energy meter infrastructure improves and Internet of Things technology advances more detailed and comprehensive relevant data will be available in real-time in the future. Therefore, the comprehensive prediction method and technology of the multi-heterogeneous model currently have a broad space for development. In the follow-up research work, on the basis of the PSO combined with the LSTM model, other learning technologies or models can be combined to form a comprehensive load forecasting model combined with smart grid and IoT technologies. Based on acquiring more data, advanced multi-models are used for integrated forecasting of power loads.

Bedi, J., Toshniwal, D., 2019. Deep Learning Framework to Forecast Electricity Demand. Applied Energy, Volume 238, pp. 1312–1326

Berawi, M.A., 2022. Innovative Digital Technology and Economy Capacity Development. International Journal of Technology, Volume 13(7), pp. 1369–1372

Bouktif, S., Fiaz, A., Ouni, A., Serhani, M., 2018. Optimal Deep Learning LSTM Model for Electric Load Forecasting using Feature Selection and Genetic Algorithm: Comparison with Machine Learning Approaches. Energies, Volume 11(7), p. 1636

Cao, Z., Han, X., Lyons, W., O'Rourke, F., 2021. Energy Management Optimisation Using a Combined Long Short-Term Memory Recurrent Neural Network – Particle Swarm Optimisation Model. Journal of Cleaner Production, Volume 326, p. 129246

Chang, W., Dong, W., Zhao, L., Yang, Q., 2020. Model Predictive Control based Energy Collaborative Optimization Management for Energy Storage System of Virtual Power Plant. In: 19^th International Symposium on Distributed Computing and Applications for Business Engineering and Science (DCABES), pp. 112–115

Chafi, Z.S., Afrakhte, H., 2021. Short-Term Load Forecasting Using Neural Network and Particle Swarm Optimization (PSO) Algorithm. Mathematical Problems in Engineering, Volume 2021, pp. 1–10

Choi, H., Ryu, S., Kim, H., 2018. Short-Term Load Forecasting based on ResNet and LSTM. In: 2018 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm), pp. 1–6

Di Persio, L., Honchar, O., 2017. Analysis of Recurrent Neural Networks for Short-Term Energy Load Forecasting. AIP Conference Proceedings, Volume 1906(1), pp. 190006

El-Hadad, R., Tan, Y.F., Tan, W.N., 2022. Anomaly Prediction in Electricity Consumption Using a Combination of Machine Learning Techniques. International Journal of Technology, Volume 13(6), pp. 1317–1325

Garcia-Gonzalo, E. & Fernandez-Martinez, J. L., 2012. A brief historical review of particle swarm optimization (PSO). Journal of Bioinformatics and Intelligent Control, 1(1), pp. 3-16.

Guo, F., Liu, D., Zhang, Z., Tang, F., 2021a. GPSO- LSTM short- Term Load Forecasting Based on Feature Correlation Analysis and Correction. Electrical Measurement ? Instrumentation, Volume 58, pp. 39–48

Guo, W., Che, L., Shahidehpour, M., Wan, X., 2021b. Machine-Learning based methods in short-term load forecasting. The Electricity Journal, Volume 34(1), p. 106884

Imani, M., Ghassemian, H., 2019. Lagged Load Wavelet Decomposition and LSTM Networks for Short-Term Load Forecasting. In: 4^th International Conference on Pattern Recognition and Image Analysis (IPRIA), pp. 6–12

Kim, N., Kim, M., Choi, J., 2018. LSTM Based Short-term Electricity Consumption Forecast with Daily Load Profile Sequences. In: IEEE 7^th Global Conference on Consumer Electronics (GCCE), pp. 136–137

Marini, F. & Walczak, B., 2015. Particle swarm optimization (PSO). A tutorial. Chemometrics and Intelligent Laboratory Systems, 149, pp. 153-165.

Ruzic, S., Vuckovic, A., Nikolic, N., 2003. Weather Sensitive Method for Short Term Load Forecasting in Electric Power Utility of Serbia. IEEE Transactions on Power Systems, Volume 18(4), pp. 1581–1586

Shang, C., Gao, J., Liu, H., Liu, F., 2021. Short-Term Load Forecasting Based on PSO-KFCM Daily Load Curve Clustering and CNN-LSTM Model. IEEE Access, Volume 9, pp. 50344-50357

Sri, H., Rao, P., Kammardi, P., Shekar, S., Kathavate, S., Gowranga, K., 2017. A smart adaptive LSTM technique for electrical load forecasting at source. In: 2^nd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT), pp. 1717–1721

Stratigakos, A., Bachoumis, A., Vita, V., Zafiropoulos, E., 2021. Short-Term Net Load Forecasting with Singular Spectrum Analysis and LSTM Neural Networks. Energies, Volume 14(14), p. 4107

Tang, X., Dai, Y., Wang, T., Chen, Y., 2019. Short?Term Power Load Forecasting Based on Multi?Layer Bidirectional Recurrent Neural Network. IET Generation, Transmission & Distribution,  Volume 13(17), pp. 3847–3854

Ul-Asar, A., Hassnain, S., Khan, A., 2007. Short Term Load Forecasting Using Particle Swarm Optimization Based ANN Approach. In: International Joint Conference on Neural Networks, pp. 1476–1481

Wang, S., Wang, X., Wang, S., Wang, D., 2019. Bi-directional Long Short-Term Memory Method Based on Attention Mechanism and Rolling Update for Short-Term Load Forecasting. International Journal of Electrical Power & Energy Systems, Volume 109, pp. 470–479

Wei, T., Pan, T., 2021. Short-term Power Load Forecasting Based on LSTM Neural Network Optimized by Improved PSO. Journal of System Simulation, Volume 33, pp. 1866-1874

Xudong, L., Shuo, L., Qingwu, F., 2020. Prediction of Building Heating and Cooling Load Based on IPSO-LSTM Neural Network. In: 2020 Chinese Automation Congress (CAC), pp. 1085–1090.

Yu, Y., Si, X., Hu, C. & Zhang, J., 2019. A review of recurrent neural networks: LSTM cells and network architectures. Neural computation, 31(7), pp. 1235-1270.

Yuan, X., Chen, C., Jiang, M., Yuan, Y., 2019. Prediction Interval of Wind Power Using Parameter Optimized Beta Distribution-Based LSTM Model. Applied Soft Computing, Volume 82, pp. 105550

Zaytsev, A., Dmitriev, N., Rodionov, D., Magradze, T., 2021. Assessment of the Innovative Potential of Alternative Energy in the Context of the Transition to the Circular Economy. International Journal of Technology, Volume 12(7), pp. 1328–1338

Zeng, N., Zhang, H., Liu, W., Liang, J., Alsaadi, F., 2017. A Switching Delayed PSO Optimized Extreme Learning Machine for Short-Term Load Forecasting. Neurocomputing, Volume 240, pp. 175–182

Zheng, H., Yuan, J., Chen, L., 2017. Short-Term Load Forecasting Using EMD-LSTM Neural Networks with a Xgboost Algorithm for Feature Importance Evaluation. Energies, Volume 10(8), p. 1168

Zou, L., Munir, M., Kim, K., Hong, C., 2020. Day-ahead Energy Sharing Schedule for the P2P Prosumer Community Using LSTM and Swarm Intelligence. In: International Conference on Information Networking (ICOIN), pp. 396–401