Analyzing Spatio-Temporal Machine Learning Models through Input Perturbation

The increasing complexity of social science data and phenomena necessitates using advanced analytical techniques to capture nonlinear relationships that traditional linear models often overlook. This chapter explores the application of machine learning (ML) models in social science research, focusing on their ability to manage nonlinear interactions in multidimensional datasets. Nonlinear relationships are central to understanding social behaviors, socioeconomic factors, and psychological processes. Machine learning models, including decision trees, neural networks, random forests, and support vector machines, provide a flexible framework for capturing these intricate patterns. The chapter begins by examining the limitations of linear models and introduces essential machine learning techniques suited for nonlinear modeling. A discussion follows on how these models automatically detect interactions and threshold effects, offering superior predictive power and robustness against noise compared to traditional methods. The chapter also covers the practical challenges of model evaluation, validation, and handling imbalanced data, emphasizing cross-validation and performance metrics tailored to the nuances of social science datasets. Practical recommendations are offered to researchers, highlighting the balance between predictive accuracy and model interpretability, ethical considerations, and best practices for communicating results to diverse stakeholders. This chapter demonstrates that while machine learning models provide robust solutions for modeling nonlinear relationships, their successful application in social sciences requires careful attention to data quality, model selection, validation, and ethical considerations. Machine learning holds transformative potential for understanding complex social phenomena and informing data-driven psychology, sociology, and political science policy-making.

Enhancing wastewater treatment efficiency through machine learning-driven effluent quality prediction: A plant-level analysis

Systematic literature review of machine learning based software development effort estimation models

The ongoing Russia–Ukraine conflict has exacerbated the global crisis of natural gas supply, particularly in Europe. During the winter season, major importers of liquefied natural gas (LNG), such as South Korea and Japan, were directly affected by fluctuating spot LNG prices. This study aimed to use machine learning (ML) to predict the Japan Korea Marker (JKM), a spot LNG price index, to reduce price fluctuation risks for LNG importers such as the Korean Gas Corporation (KOGAS). Hence, price prediction models were developed based on long short-term memory (LSTM), artificial neural network (ANN), and support vector machine (SVM) algorithms, which were used for time series data prediction. Eighty-seven variables were collected for JKM prediction, of which eight were selected for modeling. Four scenarios (scenarios A, B, C, and D) were devised and tested to analyze the effect of each variable on the performance of the models. Among the eight variables, JKM, national balancing point (NBP), and Brent price indexes demonstrated the largest effects on the performance of the ML models. In contrast, the variable of LNG import volume in China had the least effect. The LSTM model showed a mean absolute error (MAE) of 0.195, making it the best-performing algorithm. However, the LSTM model demonstrated a decreased in performance of at least 57% during the COVID-19 period, which raises concerns regarding the reliability of the test results obtained during that time. The study compared the ML models’ prediction performances with those of the traditional statistical model, autoregressive integrated moving averages (ARIMA), to verify their effectiveness. The comparison results showed that the LSTM model’s performance deviated by an MAE of 15–22%, which can be attributed to the constraints of the small dataset size and conceptual structural differences between the ML and ARIMA models. However, if a sufficiently large dataset can be secured for training, the ML model is expected to perform better than the ARIMA. Additionally, separate tests were conducted to predict the trends of JKM fluctuations and comprehensively validate the practicality of the ML models. Based on the test results, LSTM model, identified as the optimal ML algorithm, achieved a performance of 53% during the regular period and 57% d during the abnormal period (i.e., COVID-19). Subject matter experts agreed that the performance of the ML models could be improved through additional studies, ultimately reducing the risk of price fluctuations when purchasing spot LNG.

This paper investigates the application of machine learning models, including Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), and a hybrid LSTM-CNN model, for predicting stock prices of companies listed on the National Stock Exchange (NSE). The proposed hybrid model leverages CNN’s capability for spatial feature extraction and LSTM’s proficiency in modelling temporal dependencies, effectively addressing the complex and volatile nature of stock price movements. Using a comprehensive dataset of historical stock prices and trading volumes from various sectors, the hybrid model achieved superior performance with a 15% improvement in RMSE compared to standalone CNN and LSTM models. Results demonstrate its robustness, particularly in volatile market conditions, showcasing its potential for accurate and reliable predictions. This study contributes a novel hybrid approach that integrates spatial and temporal learning to enhance stock market prediction, offering valuable insights for investors and financial analysts while providing a scalable framework for broader financial applications.

Machine learning models for predicting hospital admission in pediatric emergency departments: A systematic review.

Abstract: This research is driven by the challenges faced by Universitas Lancang Kuning (UNILAK) in attracting applicants amidst intense competition, especially after the government's policy opened independent pathways to State Universities (PTN) from 2022-2023, which impacted private university applicant numbers. To address this and support strategic planning, this study aims to predict the trend of prospective students applying to all study programs at UNILAK for the period 2025-2027. Two time series models were employed: ARIMA (AutoRegressive Integrated Moving Average) and LSTM (Long Short-Term Memory). Applicant data from 2019 to 2024 was used to build the model. The Augmented Dickey-Fuller (ADF) test confirmed the data's stationarity with a p-value of 0.0. ACF and PACF analyses determined the ARIMA parameters as p=1, d=1, q=1. The LSTM model was trained to capture more complex data patterns. ARIMA predictions for 2025, 2026, and 2027 are 3298.66, 3362.33, and 3371.30, respectively. LSTM predictions for the same years are 3335.64, 3476.52, and 3518.42. Evaluation using Root Mean Squared Error (RMSE) showed ARIMA (RMSE=588.72) to be more accurate than LSTM (RMSE=653.96). Nevertheless, LSTM provided a more optimistic prediction. This study concludes that ARIMA is better suited for short-term planning, while LSTM can be used for more ambitious long-term strategies. Keywords: arima; LSTM; applicants; prediction; university Abstrak: Penelitian ini didorong oleh tantangan Universitas Lancang Kuning (UNILAK) dalam menarik pendaftar di tengah persaingan ketat, khususnya setelah kebijakan pemerintah membuka jalur mandiri ke Perguruan Tinggi Negeri (PTN) sejak 2022-2023, yang menyebabkan penurunan jumlah pendaftar di universitas swasta. Untuk mendukung perencanaan strategis, studi ini bertujuan memprediksi tren jumlah calon mahasiswa yang mendaftar ke seluruh program studi di UNILAK untuk periode 2025-2027.Dua model deret waktu digunakan: ARIMA (AutoRegressive Integrated Moving Average) dan LSTM (Long Short-Term Memory). Data jumlah pendaftar dari 2019 hingga 2024 digunakan untuk membangun model. Uji Augmented Dickey-Fuller (ADF) menunjukkan data stasioner dengan p-value 0,0. Analisis ACF dan PACF menentukan parameter ARIMA sebagai p=1, d=1, q=1. Model LSTM dilatih untuk menangkap pola data yang lebih kompleks.Prediksi ARIMA untuk 2025, 2026, dan 2027 adalah 3298.66, 3362.33, dan 3371.30. Prediksi LSTM untuk tahun yang sama adalah 3335.64, 3476.52, dan 3518.42. Evaluasi menggunakan Root Mean Squared Error (RMSE) menunjukkan ARIMA (RMSE=588.72) lebih akurat daripada LSTM (RMSE=653.96). Meskipun demikian, LSTM memberikan prediksi yang lebih optimis. Studi ini menyimpulkan ARIMA lebih cocok untuk perencanaan jangka pendek, sementara LSTM dapat digunakan untuk strategi jangka panjang yang ambisius. Kata kunci: arima; LSTM; pendaftar; prediksi; universitas

Many countries around the globe, including Tanzania, are investing in renewable energy, partly because of the need to increase electric power generation and meet the ever-increasing electricity demands, and partly because of the need to combat global climate change caused by carbon emissions from fossil fuel-based electricity production. Kishapu Solar Project, located in Kishapu District, Shinyanga Region, Tanzania, is one of the renewable energy projects with a planned generation of 150 MW of electric power once completed, using solar photovoltaic technology. Accurate forecasting of solar radiation is important in managing and maintaining solar energy plants. Inaccurate forecasting of solar radiation can lead to poor harvest of solar energy and poor management of solar energy plants, which directly impacts electricity contribution to the national electricity grid. Long Short-Term Memory (LSTM) and its variant, PeepHole LSTM, are examples of Deep Learning models that can be used to predict solar radiation. The purpose of PeepHole LSTM is to introduce “peephole” connections in the LSTM model, which allow LSTM gates to directly inspect memory cell state and hence have richer information to process. However, there is still a research gap on the impact of “peephole” connections on the prediction performance of the LSTM model, especially in the context of forecasting solar radiation in Kishapu District, where the Kishapu Solar Project is located. The purpose of this study is to fill this research gap by developing both LSTM and PeepHole LSTM models and comparing their performances in forecasting solar radiation in Kishapu District. The 14-year Kishapu District daily solar radiation (Downward Shortwave Radiation) dataset was first pre-processed by scaling (normalizing) it using Standard Scaler and then split into training set (the first 70%), validation set (the next 15%) and test set (the last 15%) before being utilized to train and test the developed LSTM and PeepHole LSTM models. Both LSTM and PeepHole LSTM models used the previous 5 days' solar radiation values as inputs to predict the output (next day (6th day) solar radiation value). The trained and better-performing model between the two was finally saved in .h5 format and integrated with a Gradio based Web Application to provide a user interface for officials in Kishapu District and at Kishapu Solar Project to predict daily solar radiation. The findings from performance evaluation revealed that the LSTM model outperformed the PeepHole LSTM model in predicting solar radiation by achieving test Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) scores of 26.22 W/m2 and 36.65 W/m2, respectively, compared to MAE and RMSE scores of 30.84 W/m2 and 41.88 W/m2, respectively, achieved by the PeepHole LSTM model, concluding that “peephole” connections reduce the performance of the LSTM model rather than increasing it when used to forecast solar radiation in Kishapu District. This study’s recommendation to Artificial Intelligence (AI) software developers and researchers is to use the LSTM rather than the PeepHole LSTM Deep Learning model when forecasting solar radiation in Kishapu District and in locations with similar climatic conditions

The best practice of watershed management is through the understanding of the hydrological processes. As a matter of fact, hydrological processes are highly associated with stochastic, non-linear, and non-stationary phenomena. Hydrological processes simulation and modeling are challenging issues in the domains of hydrology, climate and environment. Hence, the development of machine learning (ML) models for solving those complex hydrological problems took essential place over the past couple decades. It can be observed, hydrological data availability has increased remarkably, and thus computational resources has led to a resurgence in ML models’ development. It has been witnessed huge efforts on the hydrological processes modeling using the facility of ML models and several review researches have been conducted. Literature studies approved the capacity of ML models in the field of hydrology over the classical “traditional models” based on their forecastability, flexibility, precision, generalization, and modeling execution convergence speed. However, although several potential merits were observed in ML model’s development, several limitations are allied such as the interpretability of those black-box models, the practicality of the ML models in watershed management, and difficulty to explain the physical hydrological processes. In this survey, an exhibition for all the published review articles on the development of ML models for hydrological processes and recognize all the research gaps and potential research direction. The ultimate aim of the current survey is to establish a new milestone for the interested hydrology, environment and climate researchers on the applications of ML models.

The long short-term memory (LSTM) model is widely used in multiple areas, mainly for speech recognition, natural language processing and activity recognition. In the last few years, we started to see many variants of LSTM for recurrent neural networks since its inception in 1997. However, there weren’t many studies that have addressed the LSTM’s gating mechanism. In this paper, we propose a novel LSTM framework where we modify the architecture of the LSTM unit by adding a new layer that we call the “outlier gate”. The latter controls the flow of information that goes into the LSTM cell. This added signal allows us to avoid both the carry-over effect that the outliers have on the forecasted point and a bias in the estimates of our LSTM model – caused by unusual or non-repetitive events. The proposed architecture led us to an end-to-end trainable model that we applied in this paper to a financial time-series forecasting problem. Our results demonstrate that the new proposed LSTM architecture achieves better performance than the state-of-the-art original LSTM model.

With the increase of temporal frequency of satellite images, time series analysis based on Artificial Neural Network has become a tendency to detect land cover changes in images. We briefly introduces a case study that uses Long-Short Term Memory (LSTM), a specific Recurrent Neural Network (RNN) for time series modelling and forecasting, and Support Vector Machine (SVM) to detect landslides triggered by Ms.6.5 earthquake in Ludian, China in 2014. The study uses 72 available images with 16/30m spatial resolution from Landsat-7/8, GaoFen-1 and HJ-1A/1B satellites. Firstly, per-pixel LSTM models are trained by Normalized Difference Vegetation Index (NDVI) images before the earthquake. Secondly, the trained LSTM models are used to predict NDVI images after the earthquake. Then, anomalies or changes are detected by comparing predicted and observed NDVI images. Finally, anomalies related to landslides are separated from other changes with a SVM model which was trained by multi-spectral images in the study area. Experiment demonstrates that the recall rate and precision rate of landslides detection are 82.09% and 76.21%, respectively. The study shows a potential that the combination of LSTM and SVM models can be used to detect landslides in Landsat-like time series images with 30m resolution.

Introduction: Achievement of specified qualitative indicators in machine learning solutions depends not only on the efficiency of algorithms, but also on data properties. One of the lines for the development of classification and regression models is the specification of local properties of data. Purpose: To improve the qualitative predictors when solving classification and regression problems based on the adaptive selection of various machine learning models on separate local segments of data sample. Results: We propose a method that uses a combination of different models and machine learning algorithms on subsamples in regression and classification problems. The method is based on the calculation of qualitative predictors and the selection of the best models on the local segments of data sample. The finding of transformations of data and time series allows to create sample sets, with the data having different properties (for example, variance, sampling fraction, data range, etc.). We consider the data segmentation based on the change point detection algorithm in time series trends and on analytical information. On the example of the real dataset, we show the experimental values of the loss function for the proposed method with different classifiers on separate segments and on the whole sample. Practical relevance: The results can be used in classification and regression problems for the development of machine learning models and methods. The proposed method allows to improve classification and regression qualitative predictors by assigning models that have the best performance on separate segments.

This paper investigates the importance of explainability versus accuracy of machine learning (ML) models. We propose that greater task uncertainty makes people want to interact more with the ML model, which increases the importance of explainability relative to accuracy. We focus on the use of ML models for product cost estimation during new product development. The paper provides mixed-methods evidence on the trade-off between explainability and accuracy of ML models. Specifically, we find support for an inverse relationship between explainability and accuracy from the perspective of cost experts. We also find that the accurate but complex and less explainable ML model of gradient boosted regression (GBR) was preferred in only a few situations; mostly, the more basic, better explainable models of multiple linear regression (MLR) and case-based reasoning (CBR) were preferred, although these were less accurate. This suggests that lack of explainability can indeed be a major limitation for the application of ML models. Furthermore, we investigate specific characteristics that could increase task uncertainty and the importance of explainability in our context: project unpredictability, product cost granularity, predecessor product availability, target cost gap, and product development phase.

Short-Term forecasting of floating photovoltaic power generation using machine learning models

The sea surface temperature (SST) is an environmental indicator closely related to climate, weather, and atmospheric events worldwide. Its forecasting is essential for supporting the decision of governments and environmental organizations. Literature has shown that single machine learning (ML) models are generally more accurate than traditional statistical models for SST time series modeling. However, the parameters tuning of these ML models is a challenging task, mainly when complex phenomena, such as SST forecasting, are addressed. Issues related to misspecification, overfitting, or underfitting of the ML models can lead to underperforming forecasts. This work proposes using hybrid systems (HS) that combine (ML) models using residual forecasting as an alternative to enhance the performance of SST forecasting. In this context, two types of combinations are evaluated using two ML models: support vector regression (SVR) and long short-term memory (LSTM). The experimental evaluation was performed on three datasets from different regions of the Atlantic Ocean using three well-known measures: mean square error (MSE), mean absolute percentage error (MAPE), and mean absolute error (MAE). The best HS based on SVR improved the MSE value for each analyzed series by 82.26%, 98.93%, and 65.03% compared to its respective single model. The HS employing the LSTM improved 92.15%, 98.69%, and 32.41% concerning the single LSTM model. Compared to literature approaches, at least one version of HS attained higher accuracy than statistical and ML models in all study cases. In particular, the nonlinear combination of the ML models obtained the best performance among the proposed HS versions.