Diagnosis of Malignancy in Thyroid Tumors by Multi-Layer Perceptron Neural Networks With Different Batch Learning Algorithms

Introduction Atmospheric corrosion is the main cause of metallic heritage degradation when they are exposed to relative humidity, temperature cycle and air contaminants. Corrosivity maps are one of the preventive strategies for atmospheric corrosion mitigation, but they do not provide enough information about corrosion mechanism, besides that long experimentation periods are required [1]. Moreover, electrochemical techniques are alternatives to speed up results, but their relationship with constantly changing environmental variables is difficult to control.Currently, Artificial Intelligence has been introduced in corrosion engineering studies, because of its capability to make predictions based on big data. Also, experimental data of Electrochemical Impedance Spectroscopy had been used in Artificial Neural Network training to obtain Nyquist Diagrams [2]. The objective of this work is to obtain computational models of Artificial Neural Networks to predict atmospheric corrosion in Bronze with nanostructured patina with SiO2, in marine environments. Method Environmental parameters were taken from database previously reported [3,4], and from Civil Protection Secretariat of Veracruz. Whilst electrochemical evaluation consists in Polarization Resistance and Electrochemical Impedance Spectroscopy; they were carried out in a Gamry Interface 1000 Potentiostat with a Saturated Calomel Reference Electrode (SCE) and a graphite bar as counter electrode in an agar gelled cell. Meanwhile, working electrode was bronze with nanostructured patina with SiO2. Two patina were prepared, the first with CuSO4 solution (0.015 mol L-1) and the second applying a CuNO3 solution (20% wt).The 13 variables used in the database for Artificial Neural Networks (ANN) training were: exposure time, chloride concentration, sulfur compounds concentration, relative humidity, precipitation level, windspeed, temperature, nanocoating presence, corrosion potential, corrosion rate, frequency, real and imaginary component of impedance. The values of each variable were treated with measures of central tendency and dispersion, as well as correlation matrix and histograms, which allowed to find a relationship between variables, and choosing the inputs for ANN. The prediction models consisted in 5, 9, 10 and 12 neurons in the input layer (R5E, R9E, R10E and R12E, respectively), a hidden layer, and Zima as only neuron in the output layer. The number of neurons in hidden layer varied from 1 to 8 until the highest coefficient of determination was achieved. The computational model was performed using the toolbox in MATLAB with a feedforward ANN. It is worth mention that the hidden and output layers used a hyperbolic tangent and a linear transfer function, respectively, and a Levenberg-Marquardt algorithm as training function was used in the models. Results and Conclusions Figure 1 presents the coefficient of determination (R2) and root mean squared error (RMSE) with different ANN architectures. In the four models, the highest value was reached with 8 neurons in the hidden layer. Then, the linear regressions for R5E, R9E, R10E and R12E models and 8 neurons in the input layer were compared in Figure 2. In that sense, the Nyquist Diagrams in Figure 3 were simulated with the R9E and R12E models. Also, they were compared with experimental Nyquist Diagrams for Bronze with patina of CuSO4 and CuNO3, with and without SiO2 nanoparticles, at 56 days of exposure in marine atmospheres. The simulations demonstrated how an ANN model with 12 neurons in the input layer, 8 neurons in the hidden layer and 1 neuron in the output layer can be used as a corrosion prediction model of bronze with nanostructured patina with SiO2.

A suitable framework for the development of artificial neural networks is important because it decides the level of accuracy, which can be reached for a certain dataset and increases the certainty about the reached classification results. In this paper, we conduct a comparative study for the performance of four frameworks, Keras with TensorFlow, Pytorch, TensorFlow, and Cognitive Toolkit (CNTK), for the elaboration of neural networks. The number of neurons in the hidden layer of the neural networks is varied from 8 to 64 to understand its effect on the performance metrics of the frameworks. A test dataset is synthesized using an analytical model and real measured impedance spectra by an eddy current sensor coil on EUR 2 and TRY 1 coins. The dataset has been extended by using a novel method based on interpolation technique to create datasets with different difficulty levels to replicate the scenario with a good imitation of EUR 2 coins and to investigate the limit of the prediction accuracy. It was observed that the compared frameworks have high accuracy performance for a lower level of difficulty in the dataset. As the difficulty in the dataset is raised, there was a drop in the accuracy of CNTK and Keras with TensorFlow depending upon the number of neurons in the hidden layers. It was observed that CNTK has the overall worst accuracy performance with an increase in the difficulty level of the datasets. Therefore, the major comparison was confined to Pytorch and TensorFlow. It was observed for Pytorch and TensorFlow with 32 and 64 neurons in hidden layers that there is a minor drop in the accuracy with an increase in the difficulty level of the dataset and was above 90% until both the coins were 80% closer to each other in terms of electrical and magnetic properties. However, Pytorch with 32 neurons in the hidden layer has a reduction in model size by 70% and 16.3% and predicts the class, 73.6% and 15.6% faster in comparison to TensorFlow and Pytorch with 64 neurons.

We applied multilayer perceptron (MLP) and radial basis function (RBF) neural networks using data from two water quality monitoring stations at the Karaj Dam in Iran. Input data were calcium ions (Ca2+), magnesium ions (Mg2+), sodium ions (Na+), chloride ions (Cl−), sulfate ( SO 42−), and pH, and the output data were total dissolved solids (TDS). An MLP with one hidden layer containing eight neurons was selected for the upstream water quality station using normalized input data. We developed a second MLP neural network for the downstream station with one hidden layer containing 10 neurons in the hidden layer using normalized input data. Considering applying normalized input data and one hidden layer, the coefficient of determination (R2) and index of agreement (IA) between the observed and the predicted data for the upstream and downstream monitoring stations using the MLP neural networks were 0.985, 0.84, 0.99, and 0.92, respectively. The RBF neural network with 100 neurons in its hidden layer reached the minimum errors between the observed and the predicted results in upstream and downstream stations. The R2 between observed and predicted data for upstream and downstream monitoring stations for the RBF was 0.999 and 0.998, respectively. Data normalization improved the performance of the MLP neural networks. Sensitivity analysis indicated that magnesium is the most effective water quality parameter for predicting TDS, and sulfate is the second most effective water quality parameter affecting TDS prediction at the Karaj Dam.

RNNLM (Recurrent Neural Network Language Model) can save the historical information of the training dataset by the last hidden layer and can also as input for training. It has become an interesting topic in the field of Natural Language Processing research. However, the immense training time overhead is a big problem. The large output layer, hidden layer, last hidden layer and the connections among them will generate enormous matrix in training. It is the main facts to influence the efficiency and scalability. At the same time, output layer class and small hidden layer should decrease the accuracy of RNNLM. In general, the lack of parallel for artificial neuron is main reason for these. We change the structure of RNNLM and design the new large-scale RNNLM by the center of distributed artificial neurons in hidden layer to stimulate the parallel characteristic of biological neuron system. Meanwhile, we change training method, and present the coordination strategy for distributed neuron. At last, the prototype of new large-scale RNNLM system is implemented based on Spark. The testing and analysis results show that the training time overhead is far less than the growth rate of the distributed neuron in hidden layer and size of training dataset. These results show our large-scale RNNLM system has efficiency and scalability advantage.

We applied multilayer perceptron (MLP) and radial basis function (RBF) neural network in upstream and downstream water quality stations of the Karaj Reservoir in Iran. For both neural networks, inputs were pH, turbidity, temperature, chlorophyll-a, biochemical oxygen demand (BOD) and nitrate, and the output was dissolved oxygen (DO). We used an MLP neural network with two hidden layers, for upstream station 15 and 33 neurons in the first and second layers respectively, and for the downstream station, 16 and 21 neurons in the first and second hidden layer were used which had minimum amount of errors. For learning process 6-fold cross validation were applied to avoid over fitting. The best results acquired from RBF model, in which the mean bias error (MBE) and root mean squared error (RMSE) were 0.063 and 0.10 for the upstream station. The MBE and RSME were 0.0126 and 0.099 for the downstream station. The coefficient of determination (<TEX>$R^2$</TEX>) between the observed data and the predicted data for upstream and downstream stations in the MLP was 0.801 and 0.904, respectively, and in the RBF network were 0.962 and 0.97, respectively. The MLP neural network had acceptable results; however, the results of RBF network were more accurate. A sensitivity analysis for the MLP neural network indicated that temperature was the first parameter, pH the second and nitrate was the last factor affecting the prediction of DO concentrations. The results proved the workability and accuracy of the RBF model in the prediction of the DO.

This paper proposes the adjusted Levenberg-Marquardt(LM) technique to improve the performance of speech recognition. Number of difference neuron in hidden layer has been trialed in verities input speech signal which generated from 40 speakers: 20 males and 20 females. The results show that the system provides difference values of the accuracy percentage depending on the number of neuron in hidden layer. The highest accuracy rate for trained and untrained speech data of female approved at 96.67% and 98.33% for stress consonant respectively and also provided 86.92% and 90.00% for voiced consonant of male.

Linear combination rule in genetic algorithm for optimization of finite impulse response neural network to predict natural chaotic time series

Automated urban traffic monitoring systems are widely used to solve various tasks in intelligent transport systems of different regions. They include video enforcement, video surveillance, traffic management system, etc. Effective traffic management and rapid response to traffic incidents require continuous monitoring and analysis of information from these complexes, as well as time series forecasting for further anomaly detection in traffic flow. To increase the forecasting quality, data fusion from different sources is needed. It will reduce the forecasting error, related to possible incorrect values and data gaps. We implemented the approach for short-term and middle-term forecasting of traffic flow (5, 10, 15 min) based on data fusion from video enforcement and video surveillance systems. We made forecasting using different recurrent neural network architectures: LSTM, GRU, and bidirectional LSTM with one and two layers. We investigated the forecasting quality of bidirectional LSTM with 64 and 128 neurons in hidden layers. The input window size (1, 4, 12, 24, 48) was investigated. The RMSE value was used as a forecasting error. We got minimum RMSE = 0.032405 for basic LSTM with 64 neurons in the hidden layer and window size = 24.

This paper proposes a novel artificial neural network called sparse-Bayesian--based fast learning network (SBFLN). In SBFLN, sparse Bayesian regression is used to train the fast learning network (FLN), which is an improved extreme learning machine (ELM). The training process of SBFLN is to randomly generate the input weights and the hidden layer biases, and then find the probability distribution of other weights by the sparse Bayesian approach. SBFLN calculates the predicted output through Bayes estimator, so it can provide a natural marginal possibility for classification problems and can solve the overfitting problem caused by the least-squares estimation in FLN. In addition, the sparse Bayesian approach can automatically trim most redundant neurons in hidden layer, which makes the network more compact and accurate. To verify the effectiveness of the improvements in this paper, the results of SBFLN are evaluated in 15 benchmark classification problems. The experimental results show that SBFLN is not sensitive to the number of neurons in the hidden layer, and the performance of SBFLN is competitive or superior to some other state-of-the-art algorithms.

Mobile Ad Hoc network is a self-organizing infrastructure less network where the nodes are responsible for routing the packets. MANET has dynamic topology which allows nodes to join and leave the network at any time. This paper proposes a channel adaptive routing protocol which extends AOMDV routing protocol to accommodate channel fading. The proposed Channel Aware AOMDV protocol uses channel average nonfading duration to select stable links for route discovery. The main objective of this paper is to demonstrate approximation capability of artificial neural network. Feed forward network architecture with back propagation algorithm is used. Depending upon the size of database number of hidden layers and number of neurons in hidden layer are decided. Number of neurons in hidden layer is varied and effect on percentage deviation in the output parameter is observed. Depending upon nature of input and output parameters synthesis and analysis artificial neural network (ANN) models are trained.

Experiments were conducted to see the effects of a set of factors on the Resilient backpropagation (Rprop) artificial neural network classification of an Indian urban environment using IRS‐IC satellite data. Factors investigated were sample size, number of neurons in hidden layers and number of epochs. The effect of including texture information in the form of neighbourhood information and grey level co‐occurance matrix (GLCM) features in the classification process has been explored. Statistically similar overall classification accuracy is achieved for Rprop and Gaussian maximum likelihood classification (GMLC). Investigations have revealed that a large sample size gave higher test accuracy; variation in number of neurons in hidden layer did not affect the overall classification accuracy significantly; lesser number of epochs resulted in higher overall test accuracy. Incorporation of texture information by both approaches improved classification accuracy in a statistically significant manner.

This paper introduces a method for estimating the maximum depth (sub-millimeter) of minor cracks on the surface of aluminum plates used in the aeronautical industry. A set of C-scan eddy current (EC) images, including real and imaginary parts of the impedance, is analyzed to extract suitable features after reducing noise effects, such as background noises and edge noises. Based on the obtained features, e.g. maximum impedance, the background feature, background noises, type of sensors, a Multilayer Perceptron (MLP) Neural Network is built to estimate the maximum depth of the cracks. The network is optimized based on loss functions, such as mean absolute error and mean squared error. An optimal network structure with five neurons in the first hidden layer and eight neurons in the second hidden layer is chosen. The obtained result indicated that the relative error of estimations is lower than 10% for almost all experimental tested samples.

Solid phase microextraction/gas chromatography/mass spectrometry integrated with chemometrics for detection of Salmonella typhimurium contamination in a packaged fresh vegetable

Research into the problem of classification of radar returns from the ionosphere has been taken up as a challenging task for the neural networks (NNs). It appears from the literature review that for the Multi layer Perceptron (MLP) NN trained with backpropagation, reported average classification accuracy was about 96% on the test instances. This paper investigates and designs an optimal classifier using a radial basis function (RBF) NN. Authors compare the performance of two NN configurations, namely a well-known MLP NN model and the proposed RBF NN model on the radar dataset collected from the published studies. It is shown that the proposed RBF NN, consistently, has 100% accuracy on bad instances and 99.1935% accuracy on good instances. The results show that the proposed RBF NN classifier clearly outperforms the MLP NN one in various performance measures such as MSE, NMSE, correlation coefficients, area under the ROC curve and classification accuracy on the testing datasets even after attempting different data partitions.

The research area of handwriting recognition attracted number of researchers because of the challenges exist in it. The main difficulties in this area of research are: variations in handwriting styles of different writers, complex behavior of different languages used for writing etc. This paper focuses on Offline handwriting recognition process for an Indian language “Punjabi”. The script used to write Punjabi in India is Gurmukhi script. This paper presents a system which recognizes the handwritten words of Gurmukhi script. The technique used for recognition purpose in this work is based on Multi-Layered Perceptron (MLP) neural network. This paper presents a three layered architecture of neural network consisting Input layer, Hidden layer or training layer and Output layer. The segmentation phase in this work is implemented, using knowledge based approach. In segmentation phase, the success rate of 94.87 % is achieved to extract characters from Gurmukhi words and the overall recognition rate is observed as 82.06% for complete Gurmukhi words.