Neural versus difference equation modeling for 2D pattern recognition problems

Computer-aided Prediction of the ZrO2 Nanoparticles' Effects on Tensile Strength and Percentage of Water Absorption of Concrete Specimens

A multi-layer feed forward neural network model for accurate prediction of flue gas sulfuric acid dew points in process industries

Background: Data mining (DM) is an approach used in extracting valuable information from environmental processes. This research depicts a DM approach used in extracting some information from influent and effluent wastewater characteristic data of a waste stabilization pond (WSP) in Birjand, a city in Eastern Iran. Methods: Multiple regression (MR) and neural network (NN) models were examined using influent characteristics (pH, Biochemical oxygen demand [BOD 5 ], temperature, chemical oxygen demand [COD], total suspended solids [TSS], total dissolved solid [TDS], electrical conductivity [EC] and turbidity) as the regression input vectors. Models were adjusted to input attributes, effluent BOD 5 (BODout) and COD (CODout). The models performances were estimated by 10-fold external cross-validation. An internal 5-fold cross-validation was also used for the training data set in NN model. The models were compared using regression error characteristic (REC) plot and other statistical measures such as relative absolute error (RAE). Sensitivity analysis was also applied to extract useful knowledge from NN model. Results: NN models (with RAE = 78.71 ± 1.16 for BODout and 83.67 ± 1.35 for CODout) and MR models (with RAE = 84.40% ± 1.07 for BODout and 88.07 ± 0.80 for CODout) indicate different performances and the former was better (P < 0.05) for the prediction of both effluent BOD 5 and COD parameters. For the prediction of CODout the NN model with hidden layer size (H) = 4 and decay factor = 0.75 ± 0.03 presented the best predictive results. For BODout the H and decay factor were found to be 4 and 0.73 ± 0.03, respectively. TDS was found as the most descriptive influent wastewater characteristics for the prediction of the WSP performance. The REC plots confirmed the NN model performance superiority for both BOD and COD effluent prediction. Conclusion: Modeling the performance of WSP systems using NN models along with sensitivity analysis can offer better understanding on exploring the most significant parameters for the prediction of system performance. The findings of this study could build the foundation for prospective work on the characterization of WSP operations and optimization of their performances with a view to conducting statistical approaches.

The single-layer perceptron, introduced by Rosenblatt in 1958, is one of the earliest and simplest neural network models. However, it is incapable of classifying linearly inseparable patterns. A new era of neural network research started in 1986, when the backpropagation (BP) algorithm was rediscovered for training the multilayer perceptron (MLP) model. An MLP with a large number of hidden nodes can function as a universal approximator. To date, the MLP model is the most fundamental and important neural network model. It is also the most investigated neural network model. Even in this AI or deep learning era, the MLP is still among the few most investigated and used neural network models. Numerous new results have been obtained in the past three decades. This survey paper gives a comprehensive and state-of-the-art introduction to the perceptron model, with emphasis on learning, generalization, model selection and fault tolerance. The role of the perceptron model in the deep learning era is also described. This paper provides a concluding survey of perceptron learning, and it covers all the major achievements in the past seven decades. It also serves a tutorial for perceptron learning.

A review on neural network techniques for the prediction of road traffic accident severity

Handwriting recognition remains a challenge in the machine vision field, especially in optical character recognition (OCR). The OCR has various applications such as the detection of handwritten Farsi digits and the diagnosis of biomedical science. In expanding and improving quality of the subject, this research focus on the recognition of Farsi Handwriting Digits and illustration applications in biomedical science. The detection of handwritten Farsi digits is being widely used in most contexts involving the collection of generic digital numerical information, such as reading checks or digits of postcodes. Selecting an appropriate classifier has become an issue highlighted in the recognition of handwritten digits. The paper aims at identifying handwritten Farsi digits written with different handwritten styles. Digits are classified using several traditional methods, including K-nearest neighbor, artificial neural network (ANN), and support vector machine (SVM) classifiers. New features of digits, namely, geometric and correlation-based features, have demonstrated to achieve better recognition performance. A noble class of methods, known as deep neural networks (DNNs), is also used to identify handwritten digits through machine vision. Here, two types of introduce its expansion form, a convolutional neural network (CNN) and an auto-encoder, are implemented. Moreover, by using a new combination of CNN layers one can obtain improved results in classifying Farsi digits. The performances of the DNN-based and traditional classifiers are compared to investigate the improvements in accuracy and calculation time. The SVM shows the best results among the traditional classifiers, whereas the CNN achieves the best results among the investigated techniques. The ANN offers better execution time than the SVM, but its accuracy is lower. The best accuracy among the traditional classifiers based on all investigated features is 99.3% accuracy obtained by the SVM, and the CNN achieves the best overall accuracy of 99.45%.

1. In this study, neural network (NN) and response surface (RS) models were developed to investigate the response [average daily gain (ADG) and feed efficiency (FE)] of young broiler chickens to dietary protein and lysine. For this purpose, data on their responses to dietary protein and lysine were extracted from the literature and separate NN and RS models were constructed. 2. Comparison between the NN and RS models revealed higher accuracy of prediction with the NN models compared to the RS models. In terms of R 2 values, the NN models developed for both ADG (R 2 = 0.923) and FE (R 2 = 0.904) were far superior to the RS models (R 2 for ADG = 0.511; R 2 for FE = 0.67). This suggests that the NN models can serve as an alternative option to conventional regression approaches including use of RS models. 3. Optimisation of the NN models developed for response to protein and lysine showed that diets containing 220.7 (g/kg of diet) protein and 12.85 (g/kg of diet) lysine maximise ADG, whereas maximum FE is achieved with diets containing 241.3 and 13.12 (g/kg) protein and lysine, respectively. Based on the optimisation results, optimal dietary protein and lysine concentrations for maximum FE in broiler chickens during the starting period are higher than for ADG.

In the present paper, two models based on artificial neural networks (ANN) and genetic programming (GEP) for predicting flexural strength and percentage of water absorption of concretes containing Cr2O3 nanoparticles have been developed at different ages of curing. For purpose of building these models, training and testing using experimental results for 144 specimens produced with 16 different mixture proportions were conducted. The data used in the multilayer feed forward neural networks models and input variables of genetic programming models are arranged in a format of eight input parameters that cover the cement content (C), nanoparticle content (N), aggregate type (AG), water content (W), the amount of superplasticizer (S), the type of curing medium (CM), Age of curing (AC) and number of testing try (NT). According to these input parameters, in the neural networks and genetic programming models the flexural strength and percentage of water absorption values of concretes containing Cr2O3 nanoparticles were predicted. The training and testing results in the neural network and genetic programming models have shown that every two models have strong potential for predicting the flexural strength and percentage of water absorption values of concretes containing Cr2O3 nanoparticles. Although neural network have predicted better results, genetic programming is able to predict reasonable values with a simpler method rather than neural network.

Estimation of reservoir properties using pre-stack seismic inversion and neural network in mature oil field, Upper Assam basin, India

A multilayer feedforward neural network with a modification to the standard backpropagation training of neural nets is considered. Information on the gradients of target outputs with respect to network inputs is used in this modification. This modified neural network is then applied for approximating design requirements of aerospace composite components, such as an aircraft engine guide vane and a satellite reflector assembly. This neural network functional approximator is then used with an optimization algorithm for determining the optimal parameters for component design requirements. The modified neural network provides for faster convergence of learning process and requires a smaller number of hidden layer nodes for convergence to similar training error levels. HE application of neural networks as universal approximators has been the focus of tremendous amount of activity in the past few years.16 Neural networks have been used for approximating and modeling both memoryless systems and dynamical systems with memory. This paper does not go into the various neural net architectures and training schemes, and the interested reader could refer to standard texts on this subject matter.1'2 This paper considers static neural networks (systems without memory) formed by interconnections without loops, called feedforward networks.2 In addition, this work considers a backpropagation learning scheme with adaptation of learning rate. There have been many additions to, or adaptations of, the basic backpropagation scheme, such as adaptive learning rate, learning with momentum, and the like,1'2 for improving convergence of the learning scheme, and these are not addressed in this paper. This paper explores a modification of the backpropagation learning scheme using information on the gradient of the target outputs with respect to the inputs of the network. The performance of the modified learning scheme is compared with that of the standard backpropagation learning scheme.2 A feedforward network with a single hidden layer trained with this modified learning scheme is applied in structural optimization problems. Figure 1 shows a feedforward net with a single hidden layer. The restriction to a single hidden layer is not a limitation of the modified learning scheme, and the new training scheme can be used on networks with multiple hidden layers also. The use of neural networks for structural response approximation has been reported by several authors.46 The primary reason for investigating such properties is to assess the applicability of trained neural networks for quick and accurate approximation of response functions during structural design optimization. Structural optimization, utilizing either gradient-based numerical search or genetic search algorithms, requires repeated function (response) evaluations during the search process. A single finite element analysis of large-scale structures requires a significant amount of computational time and resource and hence the need for robust approxima

Introduction Artificial neural networks are predictive algorithms with higher discriminative ability than models derived from conventional logistic regression. Aim To compare the discriminative ability of an algorithm developed by an artificial neural network and the GRACE score for the prediction of mortality in acute coronary syndromes (ACS). Material/Methods :A database with consecutively included patients of acute coronary syndromes from a Community Hospital was analyzed. 57 variables were analyzed (50 cofactors and 7 covariates): demographic data, cardiovascular risk factors, cardiovascular and non-cardiovascular history, pre-admission medication, pre-admission procedures, vital signs at admission, admission Killip class , electrocardiographic patterns, peak levels of serum troponin T, serum creatinine, blood glucose, white blood cell count at admission, left ventricular function determined by echocardiography, invasive procedures and clinical events during hospitalization. Neural network models were trained with different methods (multilayer perceptron or radial basis), neural architectures and activation functions to select the best performing algorithm. The sample was subdivided into a 70% neural network training subpopulation and another 30% for validation. The areas under the ROC curve of the neural network and the GRACE score were compared using De Long's test. Calibration or goodness-of-fit analysis was performed by Hosmer Lemeshow test. Sensitivity analysis was performed to identify the variables with the highest independent normalized importance for the model. The SPSS 23.0 Statistics program was used for statistical analysis and neural network modeling. The end point of the analysis was hospital mortality. Results A total of 1818 ACS patients were analyzed. The best performing neural network model was the multilayer perceptron, with an architecture composed of a hidden layer with 4 neurons, tangent hyperbolic activation function of the in the hidden layer neurons and softmax of those in the output layer. The area under the ROC curve of the resulting algorithm for death prediction was 0.98 (95% CI 0.97-0.99). The GRACE score had an area under the ROC curve of 0.88 (95% CI 0.77-0.90). The comparison between both areas by De Long test showed significant differences: p&amp;lt;0.001. Calibration or goodness of fit showed better correlation between observed and calculated events (lower dispersion or residual values) with the neural networks chi-square 1.9 p= 0.78 with respect to the GRACE chi-square score of 4.9, p 0.06. The variables with the highest independent normalized importance in the artificial neural network model were serum creatinine (100%) and troponin T elevation (82%), Conclusions According to the present analysis, both the discriminatory capacity of hospital mortality as well as the calibration or goodness of fit of the artificial neural networks are significantly superior to the GRACE score.

Bubble pressures and liquid densities for the propylene (1) + 2-propanol (2) mixture have been determined at (300, 325, and 350) K and x1 close to 0.20, 0.52, and 0.65 using a vibrating tube densimeter. Liquid densities have been measured from bubble pressures up to 10 MPa and correlated with a multilayer feedforward neural network. The composition dependence of the excess molar volume has been evaluated in the temperature and pressure range of the experimental data. Bubble pressures have been compared with a multilayer feedforward neural network function regressed on the available literature data showing a good consistency between the two. The multilayer feedforward neural network correlations for liquid density and bubble pressure have been used to generate the saturated liquid density surface in the whole temperature range and for compositions including the pure components. Excess molar volumes and bubble pressure predictions obtained by the neural network models have been compared with those from a Peng–Robinson equation of state with Wong−Sandler mixing rules to show the consistent behavior in the regions where data are lacking.

Simulation tools have been widely used to complement experimentation for suspension design in the automotive industry not only for reducing the development time, but also to allow the optimization of the vehicle performance. Both a test method and a simulation tool are presented for the analysis of Noise-Vibration-Harshness (NVH) performances of road vehicles suspension systems. A single suspension (corner) has been positioned on a rotating drum (2.6 m diameter) installed in the Laboratory for the Safety of Transport of the Politecnico di Milano. The suspension system is excited as the wheel passes over different cleats fixed to the working surface of the drum. The forces and the moments acting at the suspension-chassis joints are measured up to 250 Hz by means of five six-axis load cells. A mathematical representation that can accurately reflect tyre dynamic behaviour while passing over different cleats is fundamental for evaluating the suspension system quality (NVH) and for developing new suspension design and control strategies. Since the phenomenon is highly non-linear, it is rather difficult to predict the actual performance by using a physical model. However universal "black-box" models can be successfully used in the identification and control of non-linear systems. The paper deals with the simulation of the tyre/suspension dynamics by using Recurrent Neural Networks (RNN). RNN are derived from the Multi Layer Feed-forward Neural Networks (MLFNN), by adding feedback connections between output and input layers. The Neural Network (NN) has been trained with the experimental data obtained in the laboratory. The results obtained from the NN demonstrate very good agreement with the experimental results over a wide range of operation conditions. The NN model can be effectively applied as a part of vehicle system model to accurately predict elastic bushings and tyre dynamics behaviour.

Loss landscape is a useful tool for characterizing and comparing neural network models. The main challenge for analysis of loss landscape for the deep neural networks is that they are generally highly nonconvex in very high-dimensional space. In this paper, we develop the “roughness” concept for understanding such landscapes in high dimensions and apply this technique to study two neural network models arising from solving differential equations. Our main innovation is the proposal of a well-defined and easy-to-compute roughness index (RI) which is based on the mean and variance of the (normalized) total variation for one-dimensional functions projected on randomly sampled directions. A large RI at the local minimizer indicates an oscillatory landscape profile and indicates a severe challenge for the first-order optimization method. Particularly, we observe the increasing-then-decreasing pattern for RI along the gradient descent path in most models. We apply our method to two types of loss functions used to solve partial differential equations (PDEs) when the solution of PDE is parametrized by neural networks. Our empirical results on these PDE problems reveal important and consistent observations that the landscapes from the deep Galerkin method around its local minimizers are less rough than the deep Ritz method.

This study aims to establish the fundamental concepts of multiple linear regression models involving qualitative predictor variables and to validate their associations using a Multilayer Feed Forward (MLFF) neural network. A simple guide is introduced to separate qualitative variables based on the number of classes, ensuring they meet the assumptions of multiple linear regression. The approach provides a basic and practical template for integrating qualitative predictors into applied linear models. Validation is carried out using the sum of square error and relative error obtained from the MLFF neural network. The low error values produced by the MLFF model highlight the effectiveness and superiority of the proposed methodology.