The Training of Pi-Sigma Artificial Neural Networks with Differential Evolution Algorithm for Forecasting

Introduction Atmospheric corrosion is the main cause of metallic heritage degradation when they are exposed to relative humidity, temperature cycle and air contaminants. Corrosivity maps are one of the preventive strategies for atmospheric corrosion mitigation, but they do not provide enough information about corrosion mechanism, besides that long experimentation periods are required [1]. Moreover, electrochemical techniques are alternatives to speed up results, but their relationship with constantly changing environmental variables is difficult to control.Currently, Artificial Intelligence has been introduced in corrosion engineering studies, because of its capability to make predictions based on big data. Also, experimental data of Electrochemical Impedance Spectroscopy had been used in Artificial Neural Network training to obtain Nyquist Diagrams [2]. The objective of this work is to obtain computational models of Artificial Neural Networks to predict atmospheric corrosion in Bronze with nanostructured patina with SiO2, in marine environments. Method Environmental parameters were taken from database previously reported [3,4], and from Civil Protection Secretariat of Veracruz. Whilst electrochemical evaluation consists in Polarization Resistance and Electrochemical Impedance Spectroscopy; they were carried out in a Gamry Interface 1000 Potentiostat with a Saturated Calomel Reference Electrode (SCE) and a graphite bar as counter electrode in an agar gelled cell. Meanwhile, working electrode was bronze with nanostructured patina with SiO2. Two patina were prepared, the first with CuSO4 solution (0.015 mol L-1) and the second applying a CuNO3 solution (20% wt).The 13 variables used in the database for Artificial Neural Networks (ANN) training were: exposure time, chloride concentration, sulfur compounds concentration, relative humidity, precipitation level, windspeed, temperature, nanocoating presence, corrosion potential, corrosion rate, frequency, real and imaginary component of impedance. The values of each variable were treated with measures of central tendency and dispersion, as well as correlation matrix and histograms, which allowed to find a relationship between variables, and choosing the inputs for ANN. The prediction models consisted in 5, 9, 10 and 12 neurons in the input layer (R5E, R9E, R10E and R12E, respectively), a hidden layer, and Zima as only neuron in the output layer. The number of neurons in hidden layer varied from 1 to 8 until the highest coefficient of determination was achieved. The computational model was performed using the toolbox in MATLAB with a feedforward ANN. It is worth mention that the hidden and output layers used a hyperbolic tangent and a linear transfer function, respectively, and a Levenberg-Marquardt algorithm as training function was used in the models. Results and Conclusions Figure 1 presents the coefficient of determination (R2) and root mean squared error (RMSE) with different ANN architectures. In the four models, the highest value was reached with 8 neurons in the hidden layer. Then, the linear regressions for R5E, R9E, R10E and R12E models and 8 neurons in the input layer were compared in Figure 2. In that sense, the Nyquist Diagrams in Figure 3 were simulated with the R9E and R12E models. Also, they were compared with experimental Nyquist Diagrams for Bronze with patina of CuSO4 and CuNO3, with and without SiO2 nanoparticles, at 56 days of exposure in marine atmospheres. The simulations demonstrated how an ANN model with 12 neurons in the input layer, 8 neurons in the hidden layer and 1 neuron in the output layer can be used as a corrosion prediction model of bronze with nanostructured patina with SiO2.

This article presents the use of artificial neural networks for the evaluation of integrals with finite number of pole singularities while formulating the integral equation for the electric surface current density. A feed-forward single-layer back-propagated artificial neural network (ANN) model has been trained to approximate the discontinuous integrand function. Generation of a soft continuous function obtained from the ANN model and closed-loop expressions for the evaluation of the integrals are presented. The proposed technique is applied to compute the input impedance of microstrip antenna and results have been compared with IE3D. Integration has been performed using n-point Gaussian quadrature rule for evaluating the reaction matrix.

Developing automatic history matching (AHM) methods to replace the traditional manual history matching (MHM) approach in adjusting the permeability distribution of the reservoir simulation model has been studied by many authors. Because permeability values need to be evaluated at hundreds of thousands of grid cells in a typical reservoir simulation model, it is necessary to apply a reparameterization technique to allow the optimization algorithms to be implemented with fewer variables. In basic reparameterization techniques including zonation and pilot point methods, the calibrations are usually based solely on the production data with no systematic link to the geological and geophysical data, and therefore, the obtained permeability distribution may be not geologically consistent. Several other reparameterization techniques have attempted to preserve geological consistency by incorporating 4D seismic data; however, these techniques cannot be applied to our fractured basement reservoirs (FBRs) as they do not have 4D seismic data. Taking into account these challenges, in this study, an AHM methodology and workflow have been developed using a new reparameterization technique. This approach attempts to minimize the potential for geological nonconsistency of the calibrated results by linking the permeability to geophysical data. The proposed methodology can be applied to fields with only traditional geophysical data (3D seismic and conventional well logs). In the proposed workflow, the spatial distributions of seismic attributes and geomechanical properties were calculated and estimated from 3D seismic data and well logs, respectively. After that, a feed-forward artificial neural network (ANN) model trained by the back-propagation algorithm of the relationship between initial permeability with seismic attributes and geomechanical properties of their grid cell values is developed. Then, the calibration of the permeability distribution is performed by adjustment of the ANN model. Modification of the ANN model is performed using the simultaneous perturbation stochastic approximation (SPSA) algorithm to calibrate transmission coefficients in the ANN model to minimize the discrepancy between the simulated results and observed data. The developed methodology is applied to calibrate the permeability distribution of a simulation model of Bach Ho FBR in Vietnam. The effectiveness of the methodology is evident by comparing the historical matches with an available manually history-matched simulation model. The application shows that the proposed methodology could be considered as a suitable practical approach for adjusting the permeability distribution for FBR reservoir simulation models.

<abstract><p>The training of artificial neural networks (ANNs) with rectified linear unit (ReLU) activation via gradient descent (GD) type optimization schemes is nowadays a common industrially relevant procedure. GD type optimization schemes can be regarded as temporal discretization methods for the gradient flow (GF) differential equations associated to the considered optimization problem and, in view of this, it seems to be a natural direction of research to <italic>first aim to develop a mathematical convergence theory for time-continuous GF differential equations</italic> and, thereafter, to aim to extend such a time-continuous convergence theory to implementable time-discrete GD type optimization methods. In this article we establish two basic results for GF differential equations in the training of fully-connected feedforward ANNs with one hidden layer and ReLU activation. In the first main result of this article we establish in the training of such ANNs under the assumption that the probability distribution of the input data of the considered supervised learning problem is absolutely continuous with a bounded density function that every GF differential equation admits for every initial value a solution which is also unique among a suitable class of solutions. In the second main result of this article we prove in the training of such ANNs under the assumption that the target function and the density function of the probability distribution of the input data are piecewise polynomial that every non-divergent GF trajectory converges with an appropriate rate of convergence to a critical point and that the risk of the non-divergent GF trajectory converges with rate 1 to the risk of the critical point. We establish this result by proving that the considered risk function is <italic>semialgebraic</italic> and, consequently, satisfies the <italic>Kurdyka-Łojasiewicz inequality</italic>, which allows us to show convergence of every non-divergent GF trajectory.</p></abstract>

This paper presents a novel technique for the supervised training of feed-forward artificial neural networks (ANN) using the Harmony Search (HS) algorithm. HS is a stochastic meta-heuristic that is inspired from the improvisation process of musicians. Unlike Backpropagation, HS is non-trajectory driven. By modifying an existing improved version of HS & adopting a suitable ANN data representation, we propose a training technique where two of HS probabilistic parameters are determined dynamically based on the best-to-worst (BtW) harmony ratio in the current harmony memory instead of the improvisation count. This would be more suitable for ANN training since parameters and termination would depend on the quality of the attained solution. We have empirically tested and verified our technique by training an ANN with a benchmarking problem. In terms of overall training time and recognition, our results have revealed that our method is superior to both the original improved HS and standard Backpropagation.

Natural compounds found in mangosteen peel such as squalene and α-cubebene were proven to exhibit antimicrobial, antibacterial and anticancer activity for cancer disease treatment. Supercritical Carbon Dioxide (SC-CO2) extraction process was conducted at constant flowrate of 24 mL/min within 40 minutes and by varying temperature and pressure from 50 to 80°C and from 34.5 to 55.1 MPa, respectively. A multi-layer feedforward back-propagation Artificial Neural Network (ANN) model was developed for solubility prediction of mangosteen peel oil in SC-CO2, where input variables were temperature and pressure. An optimal ANN model consisted of one hidden layer and five neurons was obtained with minimum value of Mean Square Error (MSE) i.e. 0.011 for 48 experimental data point used. The analysis showed that the ANN prediction model have a good correlation with the experimental data in which the values of correlation coefficient (R-value) for training, validating and testing obtained are 0.933, 0.982 and 0.927 proceedings.

Thermal and electrical contact resistances (TCR and ECR) of thermoelectric generator (TEG) exert essential impacts on its performance. In this study, through a series of experiments these two important properties have been estimated in a wide range of thermal and mechanical conditions, and with different interfacial materials. The magnitudes of the overall TCR were found in the range of (1.12–2.00) × 10–3 m2⋅K/W with air, (0.82–1.81) × 10–3 m2⋅K/W with graphene sheet, and (3.61–8.37) × 10–4 m2⋅K/W with thermal grease as interfacial materials when the heat-source temperature varied from 348.15 K to 598.15 K and the imposed pressure load from 266 kPa to 1266 kPa. Importantly, the detailed TCRs at different locations across the TEG system were also analyzed. The dominant components, which occupy more than 80 % of the overall TCR, have been identified at the interfaces of the thermoelectric module contacting the heat source and heat sink. In our experiment, the corresponding ECRs under the same working conditions were (1.03–1.52) × 10–9 Ω⋅m2, (0.56–9.60) × 10–10 Ω⋅m2, and (1.05–6.23) × 10–10 Ω⋅m2, respectively. Moreover, it is revealed that the TEG system delivered better performance at relatively low TCR and ECR when it operated at a high heat-source temperature, a large pressure load and using thermal grease as interfacial material. In addition to these experimental findings, a novel fully-connected feed-forward artificial neural network (ANN) model was also proposed to predict the overall TCR. It is shown that such an ANN model, as a promising approach, can achieve a cost-effective TCR prediction in good accuracy, with the mean square error and correlation coefficient being 2.36 × 10–9 and 99.4 %, respectively. These numerical and experimental results in this study will be of particular value for future TEG design and optimization.

Chromium contamination in water has become a major concern worldwide due to its adverse health effects. Chitosan oligosaccharide-coated iron oxide nanoparticles (CSO-INPs) were used in the present study for the magnetic separation of chromium from a chromium spiked water samples. Transmission electron microscope–energy-dispersive X-ray spectroscopy elemental mapping and scanning electron microscope–energy-dispersive X-ray spectroscopy elemental analysis were carried out to confirm the successful removal of the contaminant (total Cr) from the spiked water samples. A feedforward artificial neural network (ANN) model has been developed to predict the optimum efficiency of chromium ions removal from aqueous solution by CSO-INPs. Both the batch experiments and the ANN have been applied to assess the impact of various factors such as pH, nanoparticles dose, temperature, and time influencing the Cr removal efficiency of CSO-INPs. Removal efficiency has been found to be higher at low pH, with higher dose of nanoparticles at higher temperature. The ANN simulation further gives us a set of desired conditions (pH 3, CSO-INPs dose 0.7 mg/ml, temperature 28–30 °C, time 60 min) to achieve an optimum Cr removal efficiency of CSO-INPs.

Artificial neural networks (ANNs) are used to solve many problems, such as modeling, identification, prediction, and classification. The success of ANN is directly related to the training process. Meta-heuristic algorithms are used extensively for ANN training. Within the scope of this study, a feed-forward artificial neural network (FFNN) is trained using the marine predators algorithm (MPA), one of the current meta-heuristic algorithms. Namely, this study is aimed to evaluate the performance of MPA in ANN training in detail. Identification/modeling of nonlinear systems is chosen as the problem. Six nonlinear systems are used in the applications. Some of them are static, and some are dynamic. Mean squared error (MSE) is utilized as the error metric. Effective training and testing results were obtained using MPA. The best mean error values obtained for six nonlinear systems are 2.3 × 10−4, 1.8 × 10−3, 1.0 × 10−4, 1.0 × 10−4, 1.2 × 10−5, and 2.5 × 10−4. The performance of MPA is compared with 16 meta-heuristic algorithms. The results have shown that the performance of MPA is better than other algorithms in ANN training for the identification of nonlinear systems.

The purpose of this study is to develop a feed-forward neural network (FFNN) model with back-propagation learning algorithm to predict the dissolved oxygen from water temperature and 5 days-biological oxygen demand in the Tigris River, Baghdad-Iraq. The Artificial Neural Networks model was implemented utilizing measured data that were gathered from laboratories of water treatment plant, Baghdad-Iraq, during the year 2008. The correlation analysis between dissolved oxygen and dependent parameters were utilized in selecting the major inputs from water quality parameters for commencing of ANN models. The performance of ANN models were tested utilizing the coefficient of correlation (R), the efficiency coefficient of Nash-Sutcliffe (NS), mean square error (MSE) and mean absolute errors (MAE). The outputs revealed that the feed-forward neural networks using back-propagation learning algorithm which was prepared by temperature and biological oxygen demand offered a relatively high correlation coefficient of 0.885, and efficiency coefficient of 0.782, meanwhile a reasonably low mean square errors of 1.133, and mean absolute errors of 0.369 values for whole array period. The results of the present study demonstrate that the artificial neural networks using FFNN model is capable to forecast the dissolved oxygen values with acceptable accuracy. This is suggesting that the artificial neural network is a useful tool for Tigris River management in Baghdad-Iraq.

Multivariate regression (MVR) and different artificial neural network (ANN) models developed for optical transparency of conductive polymer nanocomposite films

Robust training of median dendritic artificial neural networks for time series forecasting

Comparative study on constitutive relationship of as-cast Ti60 titanium alloy during hot deformation based on Arrhenius-type and artificial neural network models

Water trap seal loss can lead to the ingress of foul odours from the sewer network carrying harmful pathogens which can affect the health of building occupants. This loss is due to excessive negative or positive pressures within the system as a consequence of rapidly changing flow due to the unsteady nature of air and water flow discharges from appliances. This pressure regime within the system is often represented by a pressure profile along the height of the main vertical stack in the Building Drainage System (BDS). Experimental data from peer-reviewed literature and data from a unique 34-storey drainage test rig have been used as pressure profile data (Test data) for an Artificial Neural Network (ANN) model. Discharge input height (storey number) and discharge flow rate are considered to be the two independent input parameters and the pressure along the vertical stack is considered to be the output. In this work, both a Feed Forward and Back Propagation (FFBP) ANN model and a Radial Basis Function (RBF) ANN model have been used to train the algorithm. The work has confirmed the applicability of the FFBP-ANN model for steady two-phase fluid flow phenomena in BDS and allows for the prediction of pressures in a system for which no pressure data exists, by the prediction of modelled weights, based only on its physical and flow characteristics. Practical Application: Of great concern to designers of building drainage systems (BDS) is the control of pressure fluctuations within the system to prevent water trap seal loss. Prediction of a pressure profile based on the characteristics of the building – e.g. height, location of water discharge, discharge flow rate and ventilation type, would therefore increase confidence in designs, particularly in tall buildings. The work presented here addresses, for the first time, the applications of two ANN models for predicting the pressure profile in the BDS vertical stack of multi-storey buildings.

Artificial neural network-based prediction of hydrogen content of coal in power station boilers