Robot Arm Movement Control by Model-based Reinforcement Learning using Machine Learning Regression Techniques and Particle Swarm Optimization

Deep reinforcement learning is the technology of artificial neural networks in the field of decision-making and control. The traditional model-free reinforcement learning algorithm requires a large amount of environment interactive data to iterate the algorithm. This model’s performance also suffers due to low utilization of training data, while the model-based reinforcement learning (MBRL) algorithm improves the efficiency of the data, MBRL locks into low prediction accuracy. Although MBRL can utilize the additional data generated by the dynamic model, a system dynamics model with low prediction accuracy will provide low-quality data and affect the algorithm’s final result. In this paper, based on the A3C (Asynchronous Advantage Actor-Critic) algorithm, an improved model-based deep reinforcement learning algorithm using a learning degree network (MBRL-LDN) is presented. By comparing the differences between the predicted states outputted by the proposed multidynamic model and the original predicted states, the learning degree of the system dynamics model is calculated. The learning degree represents the quality of the data generated by the dynamic model and is used to decide whether to continue to interact with the dynamic model during a particular episode. Thus, low-quality data will be discarded. The superiority of the proposed method is verified by conducting extensive contrast experiments.

Hydro-power production capacity prediction based on machine learning regression techniques

In this work, four mainstream machine learning (ML) techniques are used to evaluate the bulk porosity and static elastic modulus of weathered Yungang Grottoes sandstone. Datasets are gathered from the experiments, which includes 432 groups effective experimental data including 8 inputs features. bulk porosity and static elastic modulus were considered as outputs to determine the weathering degrees of Yungang Grottoes sandstone. The 4 performance criteria were used to evaluate the ML models. Results demonstrate that the Artificial Neural Network (ANN) is the best-fitted models for estimating the bulk porosity and static elastic modulus compared to Multiple Linear Regression (MLR), Support Vector Regression (SVR), Gaussian Process Regression (GPR). The accuracy of the trained model for static elastic modulus is slightly higher than that of bulk porosity. The GPR and ANN model can accurately predict the bulk porosity and static elastic modulus in training stages. The ANN with multi-hidden layers developed is competent with high degree of precision and generalization ability for bulk porosity and static elastic modulus compared to other selected regression-based ML models (MLR, SVR, and GPR). The coefficient of determinations of ANN in the range of (0.9537–0.9641) during the testing stages is more stable and higher than that of (0.8883–0.9453) other built ML models. The prediction efficiency of pretrained ANN model was well adjusted for the actual and forecast datasets at the training and testing stages, and the error range was no more than 0.7% and 0.15 GPa at both stages of prediction for bulk porosity and static elastic modulus respectively. And the ANN based static elastic modulus prediction model’s error proportions significantly decreased and were confined to a modest range between + 10% and − 10%. The proposed surrogate models are valid for the bulk porosity ranging from 7 to 14% and the static elastic modulus ranging from 0.7 to 1.4 Gpa, which can be utilized for the accurate and fast prediction of the weathering degrees of Yungang Grottoes sandstone.

SummaryIn this work, we investigate the efficient estimation of the optimal design variables that maximize net present value (NPV) for the life-cycle production optimization during a single-well carbon dioxide (CO2) huff-n-puff (HnP) process in unconventional oil reservoirs. A synthetic unconventional reservoir model based on Bakken Formation oil composition is used. The model accounts for the natural fracture and geomechanical effects. Both the deterministic (based on a single reservoir model) and robust (based on an ensemble of reservoir models) production optimization strategies are considered. The injection rate of CO2, the production bottomhole pressure (BHP), the duration of injection and the production periods in each cycle of the HnP process, and the cycle lengths for a predetermined life-cycle time can be included in the set of optimum design (or well control) variables. During optimization, the NPV is calculated by a machine learning (ML) proxy model trained to accurately approximate the NPV that would be calculated from a reservoir simulator run. Similar to the ML algorithms, we use both least-squares (LS) support vector regression (SVR) and Gaussian process regression (GPR). Given a set of forward simulation runs with a commercial compositional simulator that simulates the miscible CO2 HnP process, a proxy is built based on the ML method chosen. Having the proxy model, we use it in an iterative-sampling-refinement optimization algorithm directly to optimize the design variables. As an optimization tool, the sequential quadratic programming (SQP) method is used inside this iterative-sampling-refinement optimization algorithm. Computational efficiencies of the ML proxy-based optimization methods are compared with those of the conventional stochastic simplex approximate gradient (StoSAG)-based methods. Our results show that the LS-SVR- and GPR-based proxy models are accurate and useful in approximating NPV in the optimization of the CO2 HnP process. The results also indicate that both the GPR and LS-SVR methods exhibit very similar convergence rates, but GPR requires 10 times more computational time than LS-SVR. However, GPR provides flexibility over LS-SVR to access uncertainty in our NPV predictions because it considers the covariance information of the GPR model. Both ML-based methods prove to be quite efficient in production optimization, saving significant computational times (at least 4 times more efficient) over a stochastic gradient computed from a high-fidelity compositional simulator directly in a gradient ascent algorithm. To our knowledge, this is the first study presenting a comprehensive review and comparison of two different ML-proxy-based optimization methods with traditional StoSAG-based optimization methods for the production optimization problem of a miscible CO2 HnP.

Leveraging hybrid machine learning and data fusion for accurate mapping of malaria cases using meteorological variables in western India

This paper employs three different kernel-based models--support vector regression (SVR), relevance vector machine (RVM) and Gaussian process regression (GPR)--for the prediction of cement compressive strength. The input variables for the model are taken as C3S (%), SO3 (%), Alkali (%) and Blaine (cm2/g), while the output is 28-day cement compressive strength (N/mm2) of the cement. The hyperparameters of the SVR are obtained using two different metaheuristic optimization algorithms--particle swarm optimization (PSO) and symbiotic organism search (SOS). Trial-and-error-based approach is used for arriving at the hyperparameters of RVM and GPR. The compressive strength predicted using different kernel-based models is also compared with that obtained from ANN and fuzzy logic models reported in the literature. The performance of the different kernel-based models is benchmarked using six different error indices and residual analysis. The performance of the kernel-based models is found to be at par with ANN. The better generalization capability and excellent empirical performance of the kernel-based models overcome the disadvantages associated with ANN and provide a good tool for the prediction of the cement compressive strength.

A machine learning based approach for predicting Pool boiling heat transfer coefficient of CNT + GO nanoparticle coated surfaces

In response to the challenge of limited model availability for predicting the lifespan of super-high arch dams, a hybrid model named EMD-PSO-GPR (EPR) is proposed in this study. The EPR model leverages Empirical Mode Decomposition (EMD), Gaussian Process Regression (GPR), and Particle Swarm Optimization (PSO) to provide an effective solution for super-high arch dam stress prediction. This research focuses on three strategically selected measurement points within the dam, characterized by complex stress conditions. The predicted results from the EPR are compared with those from GPR, Long Short-Term Memory (LSTM), and Support Vector Regression (SVR), using actual stress data measured at research points within a super-high arch dam in Southwest China. The findings reveal that the proposed EPR model attains a maximum mean absolute error (MAE) of 0.02916 and a maximum root mean square error (RMSE) of 0.03055, surpassing the compared models. As a result, the EPR model introduces an innovative computational framework for stress prediction in super-high arch dams, excelling in handling stress data characterized by high vibration frequencies and providing more accurate predictions.

A stock price index measures the change in several share prices, which can describe the market and assist investors in deciding on a specific investment. Thus, foreseeing the stock price index benefits investors in creating a better investment strategy. However, forecasting the stock price index can be challenging due to its non-linearity, non-stationary and high uncertainty. Gaussian process regression (GPR) is an attractive and powerful approach for prediction, especially when the data fluctuates over time with fewer restrictions. Besides, the GPR gains advantages over other forecasting techniques as it can offer predictions with uncertainty to provide margin errors. In this study, we evaluate the use of GPR to predict the stock price of Thailand (SET). The SET data are divided into 2 datasets; the data in the year 2015 - 2020 and the data in the year 2020 due to the massive change during the COVID-19 pandemic. The prediction results from the GPR are then compared to the machine learning approaches, artificial neural network (ANN) and recurrent neural network (RNN) using evaluation scores; the root mean square error (RMSE), the mean absolute error (MAE), the mean absolute percentage error (MAPE) and the Nash-Sutcliffe efficiency (NSE). The results indicate that the GPR is superior to the ANN and RNN for both datasets as it provides a high prediction accuracy. Moreover, the results suggest that the GPR is less sensitive to the number of input lags in the model. Therefore, the GPR is more favorable for the prediction of SET than the ANN and RNN.
 HIGHLIGHTS
 
 The Gaussian process regression (GPR) was applied to predict the stock price index of Thailand (SET)
 The predictive performance of the GPR was compared to artificial neural networks (ANNs) and recurrent neural networks (RNNs)
 The results indicate that GPR outperformed the other methods as it provided a high prediction accuracy along with prediction intervals
 
 GRAPHICAL ABSTRACT

With the increasing global warming and enormous pollution, it is very obvious to generate power from renewable energy sources. In fact, photo voltaic (PV) power is a main renewable energy source for sustainable power generation. But PV power generation is uncertain and intermittent in nature. Stable and reliable power supply may not be feasible with PV power. Power supply reliability is important as much as its availability. To handle the inconsistency of PV power generation, power sector is highly dependent on forecasting methods and techniques. In this paper the feed forward neural network (FFNN) of artificial neural networks(ANN) with optimization technique particle swarm optimization (PSO) and support vector regression (SVR) are used in short term PV power generation forecasting and their performance is compared with the error calculation in terms of the mean absolute error (MAE) and root mean squared error (RMSE). The PV power and meteorological data of solar irradiation and temperature of Kolkata region of India was used for forecasting. Similar days of the year sunny days, cloudy days and rainy days are clustered using k-means clustering technique. Thereafter, a feed forward neural network is implemented to forecast a day head PV power and week ahead photo voltaic power. PSO is used to optimize the weights of the neural network. The performance of the proposed forecasting method is compared to a data mining approach, support vector regression (SVR).In this paper, the significance of proper selection of input parameters in PV power forecasting is emphasized. With the application of K-means clustering, the accuracy of ANN-PSO approach is improved significantly in a day ahead PV power forecasting and a week ahead PV power forecasting with a good margin. Even the accuracy of ANN-PSO approach after K-means clustering is better than SVR model in a week ahead forecasting

The dynamic modulus (|E*|) of hot-mix asphalt (HMA) is a crucial mechanistic characteristic essential in defining the strain response of asphalt concrete (AC) mixtures under varying loading rates and temperatures. This paper aims to conduct a comprehensive investigation of classical machine learning (ML) and deep learning (DL) algorithms as applied to the prediction of |E*| and compare their performance with renowned |E*| regression models (Witczak NCHRP 1-37A, Witczak NCHRP 1-40D, and Hirsch). Eight state-of-the-art ML and DL algorithms are attempted with diverse structures, including multiple linear regression (MLR), decision trees (DT), support vector regression (SVR), ensemble trees (ET), Gaussian process regression (GPR), artificial neural networks (ANN), recurrent neural networks (RNN), and convolutional neural networks (CNN). A comprehensive database was assembled, incorporating 50 AC mixtures, of which 25 were from the Kingdom of Saudi Arabia and 25 were from the state of Idaho, USA. This database encompasses an extensive dataset of 3,720 |E*| measurements, associated with thirteen input features representing the proposed AC mixtures’ aggregate gradations, binder characteristics, and volumetric properties. This pioneering study surpasses existing research by examining various algorithms to predict |E*| on the same dataset, applying them with different structures and individual optimization to achieve optimal performance. The developed models are evaluated based on multi-stage assessment criteria, including the accuracy and complexity performance measures and rationality based on a sensitivity analysis. The multi-stage comparative analysis results reveal that the bagging ETs, GPR with exponential kernel, and DT record the highest prediction accuracy; however, only the bagging ETs yield the highest accuracy, lowest training and testing complexity, and rational trends throughout the sensitivity analysis. The research outcome has the potential to provide pavement engineers with advanced tools for predicting |E*| and, therefore, optimizing pavement designs and rehabilitations. Doi: 10.28991/CEJ-2025-011-01-06 Full Text: PDF

In the hot strip rolling (HSR) process, accurate prediction of bending force can improve the control accuracy of the strip crown and flatness, and further improve the strip shape quality. In this paper, six machine learning models, including Artificial Neural Network (ANN), Support Vector Machine (SVR), Classification and Regression Tree (CART), Bagging Regression Tree (BRT), Least Absolute Shrinkage and Selection operator (LASSO), and Gaussian Process Regression (GPR), were applied to predict the bending force in the HSR process. A comparative experiment was carried out based on a real-life dataset, and the prediction performance of the six models was analyzed from prediction accuracy, stability, and computational cost. The prediction performance of the six models was assessed using three evaluation metrics of root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R2). The results show that the GPR model is considered as the optimal model for bending force prediction with the best prediction accuracy, better stability, and acceptable computational cost. The prediction accuracy and stability of CART and ANN are slightly lower than that of GPR. Although BRT also shows a good combination of prediction accuracy and computational cost, the stability of BRT is the worst in the six models. SVM not only has poor prediction accuracy, but also has the highest computational cost while LASSO showed the worst prediction accuracy.

Comprehensive comparison of various machine learning algorithms for short-term ozone concentration prediction

River stream estimation is a subject matter that needs constant research and development since it is all-important in the management of water resources, meeting the water demand, irrigation and agricultural activities, and providing distant signal in unwanted situations such as floods. Unfortunately, a universal technique has been impossible to talk about yet although many techniques have been used for estimation and modeling. This has made it necessary to develop different techniques and / or to make comparisons between techniques and to determine the most accurate method for the parameters used. In this context, in this study, the distribution graphs of the flow data fourteen stations located in the Euphrates-Tigris Basin for the years 1981-2010 were created and their conformity to the normal distribution was investigated Evaluation has been made with Adaptive Neural Fuzzy Logic System (ANFIS), Support Vector Regression (SVR) techniques and the newly introduced Gaussian Process Regression (GPR), Extreme Learning Machine (ELM) and Emotional Neural Networks (ENN) artificial intelligence techniques. Root Mean Square Error (RMSE), Mean Square Error (MSE), Mean Absolute Error (MAE) and correlation coefficient (r), which are frequently used, were used to determine the model performance. In addition, it is aimed to find out which technique gives better results with rank analysis. Although all models work well, the sequence with regards to the comparison outcomes of the techniques obtained from rank analysis was observed to be ELM, GPR, ENN, SVR, ANFIS respectively. In order to better interpret the results, Taylor diagrams and time series graphics of the model outputs were created.

A novel machine learning-based model predictive control framework for improving the energy efficiency of air-conditioning systems