Adjusted Levenberg - Marquardt technique for improvement speech recognition system

Introduction Atmospheric corrosion is the main cause of metallic heritage degradation when they are exposed to relative humidity, temperature cycle and air contaminants. Corrosivity maps are one of the preventive strategies for atmospheric corrosion mitigation, but they do not provide enough information about corrosion mechanism, besides that long experimentation periods are required [1]. Moreover, electrochemical techniques are alternatives to speed up results, but their relationship with constantly changing environmental variables is difficult to control.Currently, Artificial Intelligence has been introduced in corrosion engineering studies, because of its capability to make predictions based on big data. Also, experimental data of Electrochemical Impedance Spectroscopy had been used in Artificial Neural Network training to obtain Nyquist Diagrams [2]. The objective of this work is to obtain computational models of Artificial Neural Networks to predict atmospheric corrosion in Bronze with nanostructured patina with SiO2, in marine environments. Method Environmental parameters were taken from database previously reported [3,4], and from Civil Protection Secretariat of Veracruz. Whilst electrochemical evaluation consists in Polarization Resistance and Electrochemical Impedance Spectroscopy; they were carried out in a Gamry Interface 1000 Potentiostat with a Saturated Calomel Reference Electrode (SCE) and a graphite bar as counter electrode in an agar gelled cell. Meanwhile, working electrode was bronze with nanostructured patina with SiO2. Two patina were prepared, the first with CuSO4 solution (0.015 mol L-1) and the second applying a CuNO3 solution (20% wt).The 13 variables used in the database for Artificial Neural Networks (ANN) training were: exposure time, chloride concentration, sulfur compounds concentration, relative humidity, precipitation level, windspeed, temperature, nanocoating presence, corrosion potential, corrosion rate, frequency, real and imaginary component of impedance. The values of each variable were treated with measures of central tendency and dispersion, as well as correlation matrix and histograms, which allowed to find a relationship between variables, and choosing the inputs for ANN. The prediction models consisted in 5, 9, 10 and 12 neurons in the input layer (R5E, R9E, R10E and R12E, respectively), a hidden layer, and Zima as only neuron in the output layer. The number of neurons in hidden layer varied from 1 to 8 until the highest coefficient of determination was achieved. The computational model was performed using the toolbox in MATLAB with a feedforward ANN. It is worth mention that the hidden and output layers used a hyperbolic tangent and a linear transfer function, respectively, and a Levenberg-Marquardt algorithm as training function was used in the models. Results and Conclusions Figure 1 presents the coefficient of determination (R2) and root mean squared error (RMSE) with different ANN architectures. In the four models, the highest value was reached with 8 neurons in the hidden layer. Then, the linear regressions for R5E, R9E, R10E and R12E models and 8 neurons in the input layer were compared in Figure 2. In that sense, the Nyquist Diagrams in Figure 3 were simulated with the R9E and R12E models. Also, they were compared with experimental Nyquist Diagrams for Bronze with patina of CuSO4 and CuNO3, with and without SiO2 nanoparticles, at 56 days of exposure in marine atmospheres. The simulations demonstrated how an ANN model with 12 neurons in the input layer, 8 neurons in the hidden layer and 1 neuron in the output layer can be used as a corrosion prediction model of bronze with nanostructured patina with SiO2.

A suitable framework for the development of artificial neural networks is important because it decides the level of accuracy, which can be reached for a certain dataset and increases the certainty about the reached classification results. In this paper, we conduct a comparative study for the performance of four frameworks, Keras with TensorFlow, Pytorch, TensorFlow, and Cognitive Toolkit (CNTK), for the elaboration of neural networks. The number of neurons in the hidden layer of the neural networks is varied from 8 to 64 to understand its effect on the performance metrics of the frameworks. A test dataset is synthesized using an analytical model and real measured impedance spectra by an eddy current sensor coil on EUR 2 and TRY 1 coins. The dataset has been extended by using a novel method based on interpolation technique to create datasets with different difficulty levels to replicate the scenario with a good imitation of EUR 2 coins and to investigate the limit of the prediction accuracy. It was observed that the compared frameworks have high accuracy performance for a lower level of difficulty in the dataset. As the difficulty in the dataset is raised, there was a drop in the accuracy of CNTK and Keras with TensorFlow depending upon the number of neurons in the hidden layers. It was observed that CNTK has the overall worst accuracy performance with an increase in the difficulty level of the datasets. Therefore, the major comparison was confined to Pytorch and TensorFlow. It was observed for Pytorch and TensorFlow with 32 and 64 neurons in hidden layers that there is a minor drop in the accuracy with an increase in the difficulty level of the dataset and was above 90% until both the coins were 80% closer to each other in terms of electrical and magnetic properties. However, Pytorch with 32 neurons in the hidden layer has a reduction in model size by 70% and 16.3% and predicts the class, 73.6% and 15.6% faster in comparison to TensorFlow and Pytorch with 64 neurons.

Mobile Ad Hoc network is a self-organizing infrastructure less network where the nodes are responsible for routing the packets. MANET has dynamic topology which allows nodes to join and leave the network at any time. This paper proposes a channel adaptive routing protocol which extends AOMDV routing protocol to accommodate channel fading. The proposed Channel Aware AOMDV protocol uses channel average nonfading duration to select stable links for route discovery. The main objective of this paper is to demonstrate approximation capability of artificial neural network. Feed forward network architecture with back propagation algorithm is used. Depending upon the size of database number of hidden layers and number of neurons in hidden layer are decided. Number of neurons in hidden layer is varied and effect on percentage deviation in the output parameter is observed. Depending upon nature of input and output parameters synthesis and analysis artificial neural network (ANN) models are trained.

Experiments were conducted to see the effects of a set of factors on the Resilient backpropagation (Rprop) artificial neural network classification of an Indian urban environment using IRS‐IC satellite data. Factors investigated were sample size, number of neurons in hidden layers and number of epochs. The effect of including texture information in the form of neighbourhood information and grey level co‐occurance matrix (GLCM) features in the classification process has been explored. Statistically similar overall classification accuracy is achieved for Rprop and Gaussian maximum likelihood classification (GMLC). Investigations have revealed that a large sample size gave higher test accuracy; variation in number of neurons in hidden layer did not affect the overall classification accuracy significantly; lesser number of epochs resulted in higher overall test accuracy. Incorporation of texture information by both approaches improved classification accuracy in a statistically significant manner.

To diagnose the malignancy in thyroid tumor, neural network approach is applied and the performances of thirteen batch learning algorithms are investigated on accuracy of the prediction. Therefore, a back propagation feed forward neural networks (BP FNNs) is designed and three different numbers of neuron in hidden layer are compared (5, 10 and 20 neurons). The pathology result after the surgery and clinical findings before surgery of the patients are used as the target outputs and the inputs, respectively. The best algorithm(s) is/are chosen based on mean or maximum accuracy values in the prediction and also area under Receiver Operating Characteristic Curve (ROC curve). The results show superiority of the network with 5 neurons in the hidden layer. In addition, the better performances are occurred for Polak-Ribiere conjugate gradient, BFGS quasi-newton and one step secant algorithms according to their accuracy percentage in prediction (83%) and for Scaled Conjugate Gradient and BFGS quasi-Newton based on their area under the ROC curve (0.905).

Wind speed forecasting for longer time horizons is important for efficient unit commitment and maintenance scheduling in wind power industry. Highly stochastic nature of the wind calls for accurate prediction techniques. This paper analyses Artificial Neural Network-based techniques for long-term wind speed prediction using average monthly weather data from 65 Indian cities. The data collected is divided into train and test data by 2:1 ratio and train data is used to train the neural network. The trained system is then tested with the remaining data. A total of 19 inputs like location coordinates, temperature, pressure, relative humidity, solar radiation, etc., are given to the neural network. Neural network with different number of neurons in hidden layer is used and each model is trained for considerable number of times to confirm the dependability.

An optimized neural network modeling approach with genetic algorithm for short-term load forecasting based on only multiple delayed historical power load data is proposed. Genetic algorithm is used to globally optimize the number of delayed input neurons and the number of neurons in the hidden layer of the neural network architecture. Modification of Levenberg-Marquardt algorithm with Bayesian regularization is used to improve the generalization ability of the neural network. The performance of our proposed approach has been compared using actual power load data sets. Numerical results show that our proposed power load forecasting approach is comparable to the existing approaches that use multiple input variables such as power load data, day type load patterns and weather conditions

Multilayer artificial neural networks (ANN) are often used for the solution of classification problems or for the time series forecasting. An appropriate number of learning and testing patterns must be available for the ANN training. Each training pattern is composed of n input parameters and m output parameters. Number m is usually given by the problem formulation, but the number n may be often selected from a greater set of input parameters. An optimal selection of input parameters is a very important task especially in a situation when the number of usable input parameters is great and the analytical relations between the input and output parameters are not known. The number of neurons in all ANN layers must be generally kept as small as possible because of the optimal generalisation ability. In this paper we present a possible way for the selection of significant input parameters (the so called markers), which are the most important ones from the point of view of influence on the output parameters. These parameters are later used for the training of ANN. A statistical approach is usually used for this reason [5]. After some experience in the ANN application we recognised that the approach based on mathematical logic, i. e. the GUHA method (General Unary Hypotheses Automaton) is also suitable for the determination of markers. Besides the minimisation of the number of elements in the input layer of ANN, also the number of neurons in hidden layers must be optimised. For this reason standard methods of pruning can be used, described e. g. in [1]. We have used this method in the following applications: - Optimisation of the intervals between the major overhaul of plane engines by the analysis of tribodiagnostic data. Only selected types of chemical pollution in oil can be taken into account. - Prediction of bleeding of patients with chronic lymphoblastic leukemia. Only a part of parameters about the patient is important from this point of view (see [2]). - Optimisation of the quality and reliability prediction of artificial resin production in chemical factory. Only a part of the production parameters (times of production phases, temperatures, percentage of components etc.) have straight influence on the product. - Optimisation of the prosody control in the text-to-speech synthesis. This application is described in the paper.

An important benchmark for closed-loop flow control is the suppression of the von Karman vortex street in the wake of a circular cylinder at a Reynolds number of 100. A low-dimensional Proper Orthogonal Decomposition (POD) is applied to the flow field and a four sensor configuration is placed based on the intensity of the resulting spatial Eigenfunctions. This effort focuses on the performance of a non-linear Artificial Neural Network Estimator (ANNE) used for real-time mapping of flow field sensors to POD states. This investigation examines the sensitivity of ANNE by varying the number of neurons in the hidden layer. The aim is to determine if there is an optimum number of neurons in the hidden layer. Three input data sets were studied, the first had no noise on the training and testing data, the second had 10% noise applied to the training and testing data, and the third had 25% noise applied to the training and testing data sets. In all cases 50 randomly selected training data sets were applied to the back-propagating network. A separate data set was used for testing the feed-forward network. During the validation process, network outputs were compared to desired outputs and the Root Mean Squared (RMS) error was calculated for each of the four output nodes. Results show that in general, the more nodes in the hidden layer the better the performance of the network. However, there appeared to be a consistently reduced error when eight (8) nodes were used in the hidden layer. Results from ANNE when compared to the state-of-the-art Linear Stochastic Estimator (LSE) shows significant benefits.

Object recognition and classification is an important step for many machine vision applications. Feature detectors and descriptors have been prominently used to address the variability of objects in the position, rotation, scale, color, light, and so on. Various types of automatic recognition problems generally need different features and learning programs. A supervised learning technique in the form of neural network feed-forward with back-propagation is used to deal with the identification of objects in the given image. Necessary image processing techniques include the enhancement of the quality of image, segmentation process for finding region of interest, and the extraction of image features from images that required before implementation of recognition algorithm. This chapter presents an application of artificial neural network to recognize mechanical tools irrespective of their position, size, and orientation. The results of neural network classifier compared for different number of training and testing data as well the number of neurons in hidden layer with adaptive learning rate.

In this study, the deep multi-layered group method of data handling (GMDH)-type neural network algorithm using revised heuristic self-organization method is proposed and applied to medical image diagnosis of liver cancer. The deep GMDH-type neural network can automatically organize the deep neural network architecture which has many hidden layers. The structural parameters such as the number of hidden layers, the number of neurons in hidden layers and useful input variables are automatically selected to minimize prediction error criterion defined as Akaike’s information criterion (AIC) or prediction sum of squares (PSS). The architecture of the deep neural network is automatically organized using the revised heuristic self-organization method which is a type of the evolutionary computation. This new neural network algorithm is applied to the medical image diagnosis of the liver cancer and the recognition results are compared with the conventional 3-layered sigmoid function neural network.

Methane number (MN) is a critical gas quality parameter for gaseous-fueled engines. It is a measure of knock resistance for gaseous fuels, as is the octane number for gasoline. Therefore, a priori knowledge of the MN of gaseous fuel is important to avoid any structural damage to the engine due to knock. In the present study, a model was developed to predict the MN of gaseous fuels using an artificial neural network (ANN). The model utilized measured MNs of 1202 different gaseous fuel compositions, out of which 90% of the data (randomly) was used to train the ANN model using the Levenberg–Marquardt algorithm. In order to obtain the best performance, the number of neurons in the hidden layer and the transfer function of the hidden and output layers were changed. The ANN model incorporating hyperbolic tangent sigmoid function in the hidden layer with 53 neurons, and linear function in the output layer, showed the best performance – with mean square error and correlation coefficient of 0.055 and 1, respectively. The MNs of the remaining 10% of the data were determined using the ANN model, and were compared with those determined by the AVL (Anstalt für Verbrennungskraftmaschinen List) model. The model was able to predict MN accurately (R = 0.999).

The performance of neural network as aclassifier depends on several factors such as initializationof weights, its architecture, between class imbalance in thedataset, activation function etc. Though a three layeredneural network is able to approximate any non-linearfunction, number of neurons in the hidden layer plays asignificant roll in the performance of the classifier. In thisstudy, the importance of the number of hidden layerneurons of neural network is analyzed for the classificationof ECG signals. Five different arrhythmias and thenormal beat are classified for different number of hiddenlayer neurons to examine the performance of theclassifier. In this study we get the best number as 35. Afterthe training of the neural network with the optimizednumber of neurons in the hidden layer, we have tested theperformance with three different datasets. The averagesensitivity, specificity and accuracy achieved is 94.91%,99.69% and 99.46% respectively.

A chatbot is an application of artificial intelligence in natural language processing and speech recognition. It is a computer program that imitates humans in making conversations with other people. Chatbots that specialize in a single topic, such as agriculture, are known as domain-specific chatbots. In this paper, we present a dataset for farmer intents. Intent identification is the first step in building a chatbot. The dataset includes five intents (pest or disease identification, irrigation, fertilization, weed identification, and plantation date). The length of the dataset is 720 records. We applied a Multi-Layers Perceptron (MLP) for intent classification. We tried different numbers of neurons per hidden layer and compared between increasing the number of neurons with the fixed number of epochs. The result shows that as the number of neurons in the hidden layers increases, the introduced MLP achieves high accuracy in a small number of epochs. MLP achieves 97% accuracy on the introduced dataset when the number of neurons in each hidden layer is 256 and the number of epochs is 10.

This paper proposes a novel artificial neural network called sparse-Bayesian--based fast learning network (SBFLN). In SBFLN, sparse Bayesian regression is used to train the fast learning network (FLN), which is an improved extreme learning machine (ELM). The training process of SBFLN is to randomly generate the input weights and the hidden layer biases, and then find the probability distribution of other weights by the sparse Bayesian approach. SBFLN calculates the predicted output through Bayes estimator, so it can provide a natural marginal possibility for classification problems and can solve the overfitting problem caused by the least-squares estimation in FLN. In addition, the sparse Bayesian approach can automatically trim most redundant neurons in hidden layer, which makes the network more compact and accurate. To verify the effectiveness of the improvements in this paper, the results of SBFLN are evaluated in 15 benchmark classification problems. The experimental results show that SBFLN is not sensitive to the number of neurons in the hidden layer, and the performance of SBFLN is competitive or superior to some other state-of-the-art algorithms.