Performance Analysis of Ensemble Deep Learning NARX System for Estimating the Earthquake Occurrences in the Subduction Zone of Java Island

Earthquake is a natural hazard causing significant damage and loss of lives. In recent years, there has been a growing interest in the development of earthquake early warning system, using machine learning methods. One of the promising method is the use of Neural Network-Based Nonlinear AutoRegressive with eXogenous inputs (NN-based NARX) system, which has gained attention for the potential to improve the prediction accuracy and the robustness of earthquake early warning system. NN-based NARX system is composed of an effective recurrent neural network in modeling the time-series data. Therefore, this research aimed to investigate the performance of Ensemble Deep Learning NARX system, regarding earthquake occurrences estimation in the subduction zone, including Sunda Strait, Southern Java, and Bali Region. Ensemble Deep Learning NARX system was developed as the predictor to improve the performance characteristics of NN-based NARX system in determining earthquake occurrences in the subduction zone of Java Island. The proposed Ensemble model combined multiple NARX system, each trained on a different subset of earthquake data, using the diversity and complementarity of individual model. The results showed that Ensemble Deep Learning NARX system outperforms individual model and traditional method, yielding a significantly improved estimation performance. The mean square error (MSE) of the testing data set was 5.97x10^-23, 8.97x10^-24, 9.73x10^-26 for Sunda Strait, Southern Java, and Bali Region, respectively. The research provided valuable insights for seismic hazard assessment, facilitating the development of proactive measures for earthquake mitigation and preparedness in the regions.

Deep learning; Earthquake; Ensemble; NARX neural network; Subduction

The subduction zone is characterized by a complex tectonic setting resulting from the convergence of the Indo-Australian and the Eurasian Plates. This geological configuration includes the oceanic lithosphere subduction in the Indo-Australian Plate beneath the Eurasian Plate (Supendi et al., 2020). The subduction zone is identified by dynamic plate boundaries (Dokht, Gu, and Sacchi, 2018), which is prominent in Indonesia extending from the south-western coast of Sumatra to southern Java, Bali, and the Nusa Tenggara, as shown in Figure 1 (Hochstein and Sudarman, 2008).

        Neural network has the capability to diagnose faults in steam turbines and power transformers of thermal power plants (Dhini et al., 2022; Dhini, Kusumoputro, and Surjandari, 2017). Furthermore,  it serves as an integral component for inner loop control of a small helicopter and altitude control of quadcopter maneuvers (Heryanto and Kusumo, 2021; Suprijono and Kusumoputro, 2017). Neural network is also applied in the classification of electroencephalography (Nurfirdausi et al., 2022) and in predicting the quality of experience in communication network (Tan, Lim, and Diong, 2022). Deep learning neural network is a promising, showing higher accuracy of the prediction results in the time-series analysis (Lara-Benítez, Carranza-García, and Riquelme, 2021), wastewater flow rate (Kang et al., 2020), and for predictive maintenance (Makridis, Kyriazis, and Plitsos, 2020). Consequently, research on earthquake prediction using various neural network learning mechanisms has been developed (Haris et al., 2018).

Figure 1 Tectonic settings in the subduction zone are depicted in section 1 for Sunda Strait, section 2 for Southern Java, and section 3 for Bali Region.

NARX (Nonlinear AutoRegressive with eXternal Input) is a recurrent neural network capable of modeling time-series data with external inputs (Louzazni, Mosalam, and Cotfas, 2021), temperature prediction (Gao et al., 2023), and torque combustion engines (Ricci et al., 2023). Compared to conventional earthquake precursors, NARX neural network can identify and account for minor thermal anomalies resulting from natural climate fluctuations (Nekoee and Shah-Hosseini, 2020). Moreover, a previous research introduces a NARX-based recurrent neural network model that is designed to accurately predict the response of large structures of a 38-story high-rise building structure at a 1:20 scale, exposed to seismic excitations and ambient vibrations (Perez-Ramirez et al., 2019).

In the research on earthquake estimation, a machine-learning method for estimating magnitude through various parameters is conducted (Sheng, Li, and Lu, 2021; Beroza, Segou, and Mostafa-Mousavi, 2021; Mousavi and Beroza, 2020; Schäfer and Wenzel, 2019; Ochoa, Niño, and Vargas, 2018). An end-to-end method for earthquake detection, including phase detection, association, and location tasks, has also been carried out using deep learning method (Zhu et al., 2022). To further enhance a single deep learning neural network, Ensemble Deep Learning Neural Network is developed. This ensemble system is constructed by combining multiple deep learning model to enhance the general predictive performance and robustness of a system. Furthermore, ensemble has superior performance in addressing more intricate problems, in tasks such as image classification (Hameed et al., 2020), object detection (Xu et al., 2020), and natural language processing (Al-Makhadmeh and Tolba, 2020). It has also enhanced single deep learning neural network in the context of time series classification problems (Liu et al., 2020; Fawaz et al., 2019) and for detecting cyber-attacks (Al-Abassi et al., 2020).

The methodology for the performance analysis of Ensemble Deep Learning NARX system to estimate earthquake occurrences includes:

1.   Ensemble Design: Select appropriate network architectures, hyperparameters, and training algorithms for Ensemble Deep Learning NARX model.

2.   Ensemble Formation: Combine multiple NARX model to create Ensemble, each trained on distinct subsets of earthquake data.

3.   Model Training: Train Ensemble Deep Learning NARX model using the collected and preprocessed earthquake data.

4.    Performance Evaluation: Evaluate the trained model using performance metrics mean square error.

5.   Comparative Analysis: Compare the performance of Ensemble Deep Learning NARX system with other architecture model.

6.   Interpretation and Discussion: Interpret and discuss the results obtained from the performance analysis, showing the strengths and limitations of Ensemble Deep Learning NARX system for estimating earthquake occurrences.

2.1. Data Pre-processing

Earthquake data were sourced from the catalog of the Indonesia Meteorological Climatological and Geophysical Agency (BMKG). Data for earthquake in Sunda Strait (Figure 2a), Southern Java region (Figure 2b), and Bali Region (Figure 2c) with a magnitude greater than 4.0 were collected between 2011 and 2022. After data pre-processing, the data were divided into two sets, namely training and testing. The training set was used to train the model, while the testing set was used to evaluate the performance on unseen data. As a standard practice, a larger portion of the data was allocated for training while a smaller was reserved for testing. To ensure an unbiased selection, this research randomly assigned 80% of the data for training and designated the remaining 20% for testing.

Figure 2 Frequency of earthquake occurrence: (a) Sunda Strait; (b) Southern Java; and (c) Bali region

2.2. Single Deep Learning NARX System

NARX neural network, characterized by a straightforward structure with a single hidden layer, incorporates time-delayed inputs and outputs, which influence time-series. The structure of NARX neural network is relatively straightforward, featuring one hidden layer. Meanwhile, Deep Learning NARX neural network builds on the basic NARX architecture by adding multiple hidden layers and non-linear activation functions. The use of deep learning empowers the model to discern more intricate patterns in the time-series data. Furthermore, the architecture can be automatically optimized through various techniques. Deep Learning NARX neural network represents a more advanced and adaptable architecture compared to the basic NARX network. It has the capacity to discern complex patterns, potentially achieving superior performance in time-series prediction tasks. In this research, the training results were obtained using Single Deep Learning NARX architecture, as shown in Table 1. To further enhance these results, the implementation of Ensemble Deep Learning NARX system was proposed.

Table 1 Training performance earthquake data using Single Deep Learning NARX

Table 2 Testing performance earthquake data using Single Deep Learning NARX

2.3. Ensemble Deep Learning NARX System

Proposing a novel model, Ensemble Deep Learning NARX System combines the strengths of deep learning and ensemble methods to enhance predictive modeling. Individual model in the ensemble contribute the prediction, which is aggregated to obtain the final prediction. This method leverages the diversity of the ensemble members to overcome the limitations of individual model, thereby achieving more accurate and reliable predictions.

Individual NARX model is combined to form ensembles, where the predictions of each model are aggregated using the average update error. In the back propagation process, the average error value updates the weights and biases in each model. This procedure continues to reach the smallest error value or designated iteration limit. As presented in Figure 3a, Ensemble Deep Learning NARX uses an open-loop architecture during the training phase and transitions to a closed-loop architecture for testing purposes. In the closed-loop architecture presented in Figure 3b, the output of each NARX network is feedback as input to the other networks in the ensemble. This feedback mechanism allows network to learn from the errors and others. The ensemble pseudo code is provided below for reference:

1.   Initialize architecture, define the time delay, window size, or dimension, and set the number of an ensemble, hidden layers, and neurons in each layer.

2.   The Y output of each architecture is averaged for the results in to find the  is an error factor in the output layer, where t_k is the target (Equation 1):

3.      Back propagation process to update the weights faor each architecture.

4.      Calculate the error in each hidden unit  (Equation 2)

Where,  is a sum of the errors of hidden unit

  is the output value in each hidden unit

5.      Calculate all changes in the weight of the output and hidden layer

6.      The new weights are used for the next process to reach convergence or maximum epochs

Figure 3 Architecture (a) Open Loop Ensemble Deep Learning NN-NARX; and (b) Close Loop Ensemble Deep Learning NN-NARX

3.1. Training Performance

The results showed that the ensemble model architecture used showed small error values. The initial training phase used earthquake data from Sunda Strait, with the parameters, and architecture presented in Table 3. Moreover, the next training phases used data from Southern Java and Bali, accompanied by the parameters and architecture outlined in Tables 4 and 5. The Mean Square Error (MSE) values presented in these tables, pertaining to data from distinct earthquake regions, show the superiority of the ensemble model method compared to individual NARX model, implying enhanced predictive performance. Furthermore, the application of the 'update average error' method effectively addressed the challenge of bias in the ensemble model. This method continuously updated and adjusted the weights based on the performance of individual model. Open Loop model successfully captured and assimilated the underlying patterns and dependencies inherent in the training data. During the testing phase, the best model obtained was used, and Closed Loop architecture was applied in the training process. As shown in Table 6, the results for Sunda Strait, Southern Java, and Bali data reflected Closed Loop model ability to produce low MSE values, showing its capacity to accurately estimate and predict complex time-series data.

The capacity of the model to capture intricate patterns and dependencies in the data contributed to predictions that were more precise and dependable. Closed Loop structure offers adaptability in fine-tuning model parameters to correspond with the data patterns. The ensemble architecture for different earthquake datasets shows distinct parameters. However, both architectures yielded exceptional MSE values consistently in the training and testing phases. The performance analysis of the model also yielded valuable insights. The use of the 'update average error' method was instrumental in refining and augmenting the predictive capacities of the model.

Table 3 Training performance earthquake data Sunda Strait

Table 4 Training performance earthquake data Southern Java

Table 5 Training performance earthquake data Bali

Table 6 Testing performance earthquake data

By incorporating the average error from each NARX network in ensemble, the model delivered predictions with greater resilience and accuracy. This method effectively reduced the impact of outliers or underperforming network, resulting in an enhancement of performance. Furthermore, MSE values achieved showed the model capacity to minimize disparities between predicted and actual values. These consistently low error values served as a testament to the effectiveness of the model in capturing the patterns and dynamic data. A comparative performance analysis of the model with earthquake data from two different regions also showed outstanding results. The precision in estimating earthquake occurrences showed significant importance for earthquake prediction, where precise and dependable time series forecasting played an essential role in decision-making.

3.2. N-Step Prediction 

        The next phase includes forecasting earthquake occurrences for the upcoming 12 months in 2023. The predictions are based on the model selected during testing. After examining the comparison graph between observed and predicted data, it was discovered that differences persisted in the number of earthquake in Sunda Strait (Figure 4a), Southern Java (Figure 4b), and Bali (Figure 4c). However, there was a partial correspondence with the original data.

The results from N-step ahead predictions showed a significant outcome. Although disparities in the number of earthquake persisted, the predicted data showed a partial correspondence with the original dataset. This discovery showed the complexity and inherent variability in earthquake occurrences in the research regions. Despite the disparities, the model ability to capture certain aspects of the seismic activity pattern signified a promising step toward improving earthquake prediction. Moreover, further     investigation and refinement of the model hold the potential to enhance its predictive accuracy and contribute to the development of more effective early warning system in earthquake-prone regions.

In conclusion, this research showed the effectiveness of the proposed method in accurately predicting earthquake time-series. Ensemble model, comprising multiple NARX neural network trained on distinct subsets of earthquake data, showed superior prediction performance and high robustness when contrasted with individual model. The use of weight and bias updates through algorithms grounded in the average error of each deep learning architecture showed the potential to enhance accuracy. However, the extent of improvement was closely related to the selection of appropriate parameters, such as a number of ensemble, the number of hidden layers, and the configuration of neurons. In future research in this field, several key regions required attention. Ensemble method should be optimized by exploring various strategies to enhance the model performance. This included the use of heterogeneous ensemble methods or the amalgamation of different types of deep learning model. An exploration of the influence of different hyperparameters and architectural configurations on ensemble performance could yield valuable insights for model refinement. Furthermore, extending the application of Ensemble Deep Learning NARX system to augment the precision of earthquake prediction, potentially incorporating parameters, could significantly expand the research scope. The results and methodology presented in this research provide a foundation for future advancements in Ensemble Deep Learning NARX system and the potential applications in various fields.

This research was supported by the Faculty of Engineering, Universitas Indonesia, through the Seed Funding Professor Program 2022/2023 under Grant NKB-1934/UN2.F4.D /PPM.00. 00/2022.

Al-Abassi, A., Karimipour, H., Dehghantanha, A., Parizi, R.M., 2020. An Ensemble Deep Learning-Based Cyber-Attack Detection in Industrial Control System. IEEE Access, Volume 8, pp. 83965–83973

Al-Makhadmeh, Z., Tolba, A., 2020. Automatic Hate Speech Detection Using Killer Natural Language Processing Optimizing Ensemble Deep Learning Approach. Computing, Volume 102, pp. 501–522

Beroza, G.C., Segou, M., Mostafa-Mousavi, S., 2021., Machine Learning And Earthquake Forecasting Next Steps. Nature Communications, Volume 12, p. 4761

Dhini, A., Kusumoputro, B., Surjandari, I., 2017. Neural Network Based System For Detecting and Diagnosing Faults in Steam Turbine of Thermal Power Plant. In: 2017 IEEE 8^th International Conference on Awareness Science and Technology (iCAST), pp. 149–154

Dhini, A., Surjandari, I., Kusumoputro, B., Kusiak, A., 2022. Extreme Learning Machine–Radial Basis Function (ELM-RBF) Networks For Diagnosing Faults in a Steam Turbine. Journal of Industrial and Production Engineering, Volume 39(7), pp. 572–580

Dokht, R.M., Gu, Y.J., Sacchi, M.D., 2018. Migration Imaging of The Java Subduction Zones. Journal of Geophysical Research: Solid Earth, Volume 123(2), pp. 1540–1558

Fawaz, H.I., Forestier, G., Weber, J., Idoumghar, L., Muller, P.-A., 2019. Deep neural network ensembles for time series classification. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–6

Gao, M., Wu, Q., Li, J., Wang, B., Zhou, Z., Liu, C., Wang, D., 2023. Temperature Prediction of Solar Greenhouse Based on NARX Regression Neural Network. Scientific Reports, Volume 13(1), p. 1563

Hameed, Z., Zahia, S., Garcia-Zapirain, B., Javier-Aguirre, J., Maria-Vanegas, A., 2020. Breast Cancer Histopathology Image Classification Using an Ensemble of Deep Learning Models. Sensors, Volume 20(16), p. 4373

Haris, A., Murdianto, B., Susattyo, R., Riyanto, A., 2018. Transforming Seismic Data into Lateral Sonic Properties using Artificial Neural Network: A Case Study of Real Data Set. International Journal of Technology, Volume 9(3), pp. 472–478

Heryanto, M.A., Kusumoputro, B., 2021. Attitude and Altitude Control of Quadcopter Maneuvers using Neural Network–Based Direct Inverse Control. International Journal of Technology, Volume 12(4), pp. 843–853

Hochstein, M.P., Sudarman, S., 2008. History of geothermal exploration in Indonesia from 1970 to 2000. Geothermics, Volume 37(3), pp. 220–266

Kang, H., Yang, S., Huang, J., Oh, J., 2020. Time Series Prediction of Wastewater Flow Rate by Bidirectional LSTM Deep Learning. International Journal of Control, Automation and Systems, Volume 18, pp. 3023–3030

Lara-Benítez, P., Carranza-García, M., Riquelme, J.C., 2021. An Experimental Review on Deep Learning Architectures for Time Series Forecasting. International Journal of Neural Systems, Volume 31(3), p. 2130001

Liu, H., Yu, C., Wu, H., Duan, Z., Yan, G., 2020. A New Hybrid Ensemble Deep Reinforcement Learning Model For Wind Speed Short Term Forecasting. Energy, Volume 202, p. 117794

Louzazni, M., Mosalam, H., Cotfas, D.T., 2021. Forecasting of Photovoltaic Power by Means of Non-Linear Auto-Regressive Exogenous Artificial Neural Network and Time Series Analysis. Electronics, Volume 10(16), p. 1953

Makridis, G., Kyriazis, D., Plitsos, S., 2020. Predictive Maintenance Leveraging Machine Learning for Time-Series Forecasting in The Maritime Industry. In: 2020 IEEE 23^rd International Conference on Intelligent Transportation Systems (ITSC), 2020. IEEE, pp. 1–8

Mousavi, S.M., Beroza, G.C., 2020. A Machine-Learning Approach For Earthquake Magnitude Estimation. Geophysical Research Letters, Volume 47(1), p. e2019GL085976

Nekoee, M., Shah-Hosseini, R., 2020. Thermal Anomaly Detection Using NARX Neural Network Method to Estimate The Earthquake Occurrence Time. Earth Observation and Geomatics Engineering, Volume 4(2), pp. 98–108

Nurfirdausi, A.F., Apsari, R.A., Wijaya, S.K., Prajitno, P., Ibrahim, N., 2022. Wavelet Decomposition and Feedforward Neural Network for Classification of Acute Ischemic Stroke based on Electroencephalography. International Journal of Technology, Volume 13(8), pp. 1745–1754

Ochoa, L.H., Niño, L.F., Vargas, C.A., 2018. Fast Magnitude Determination Using a Single Seismological Station Record Implementing Machine Learning Techniques. Geodesy and Geodynamics, Volume 9(1), pp. 34–41

Perez-Ramirez, C.A., Amezquita-Sanchez, J.P., Valtierra-Rodriguez, M., Adeli, H., Dominguez-Gonzalez, A., Romero-Troncoso, R.J., 2019. Recurrent Neural Network Model With Bayesian Training and Mutual Information For Response Prediction of Large Buildings. Engineering Structures, Volume 178, pp. 603–615

Ricci, F., Petrucci, L., Mariani, F., Grimaldi, C.N., 2023. NARX Technique to Predict Torque in Internal Combustion Engines. Information, Volume 14(7), p. 417

Schäfer, A.M., Wenzel, F., 2019. Global Megathrust Earthquake Hazard—Maximum Magnitude Assessment Using Multi-Variate Machine Learning. Frontiers in Earth Science, Volume 7, p. 443496

Sheng, X., Li, S., Lu, J., 2021. The Application of Machine Learning in Determining Earthquake Magnitude as an Early Warning. In: IOP Conference Series: Earth and Environmental Science, Volume 783(1), p. 012089

Supendi, P., Nugraha, A.D., Widiyantoro, S., Abdullah, C., Rawlinson, N., Cummins, P., Harris, C., Roosmawati, N., Miller, M., 2020. Fate of Forearc Lithosphere At Arc-Continent Collision Zones: Evidence From Local Earthquake Tomography Of The Sunda-Banda Arc Transition, Indonesia. Geophysical Research Letters, Volume 47(6), p. e2019GL086472

Suprijono, H., Kusumoputro, B., 2017. Direct Inverse Control Based On Neural Network For Unmanned Small Helicopter Attitude And Altitude Control. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), Volume 9(2), pp. 99–102

Tan, K.H., Lim, H.S., Diong, K.S., 2022. Modelling and Predicting Quality-of-Experience of Online Gaming Users in 5G Networks. International Journal of Technology, Volume 13(5), pp. 1035–1044

Xu, J., Wang, W., Wang, H., Guo, J., 2020. Multi-Model Ensemble with Rich Spatial Information for Object Detection. Pattern Recognition, Volume 99, p. 107098

Zhu, W., Tai, K.S., Mousavi, S.M., Bailis, P., Beroza, G.C., 2022. An End-To-End Earthquake Detection Method For Joint Phase Picking and Association Using Deep Learning. Journal of Geophysical Research: Solid Earth, Volume 127(3), p. e2021JB023283