Sparse Bayesian approach to fast learning network for multiclassification

This paper proposes a novel artificial neural network called Parallel Layer Perceptron Fast Learning Network (PLP-FLN). In PLP-FLN, a parallel single hidden layer feed-forward neural network is added on the basis of Fast Learning Network (FLN) which is an improved Extreme Learning Machine (ELM). Input weights and hidden layer biases are randomly generated. The weights connect the output nodes and the input nodes, and the weights connect the output nodes and the hidden nodes are analytically determined based on least squares methods. In order to test the PLP-FLN validity, this paper compared it with ELM, FLN, Kernel ELM and Incremental ELM through 12 regression applications and 7 classification problems. By comparing the experimental results, it shows that the PLP-FLN with much more compact networks have demonstrated better approximations, classification performances and generalization ability.

Introduction Atmospheric corrosion is the main cause of metallic heritage degradation when they are exposed to relative humidity, temperature cycle and air contaminants. Corrosivity maps are one of the preventive strategies for atmospheric corrosion mitigation, but they do not provide enough information about corrosion mechanism, besides that long experimentation periods are required [1]. Moreover, electrochemical techniques are alternatives to speed up results, but their relationship with constantly changing environmental variables is difficult to control.Currently, Artificial Intelligence has been introduced in corrosion engineering studies, because of its capability to make predictions based on big data. Also, experimental data of Electrochemical Impedance Spectroscopy had been used in Artificial Neural Network training to obtain Nyquist Diagrams [2]. The objective of this work is to obtain computational models of Artificial Neural Networks to predict atmospheric corrosion in Bronze with nanostructured patina with SiO2, in marine environments. Method Environmental parameters were taken from database previously reported [3,4], and from Civil Protection Secretariat of Veracruz. Whilst electrochemical evaluation consists in Polarization Resistance and Electrochemical Impedance Spectroscopy; they were carried out in a Gamry Interface 1000 Potentiostat with a Saturated Calomel Reference Electrode (SCE) and a graphite bar as counter electrode in an agar gelled cell. Meanwhile, working electrode was bronze with nanostructured patina with SiO2. Two patina were prepared, the first with CuSO4 solution (0.015 mol L-1) and the second applying a CuNO3 solution (20% wt).The 13 variables used in the database for Artificial Neural Networks (ANN) training were: exposure time, chloride concentration, sulfur compounds concentration, relative humidity, precipitation level, windspeed, temperature, nanocoating presence, corrosion potential, corrosion rate, frequency, real and imaginary component of impedance. The values of each variable were treated with measures of central tendency and dispersion, as well as correlation matrix and histograms, which allowed to find a relationship between variables, and choosing the inputs for ANN. The prediction models consisted in 5, 9, 10 and 12 neurons in the input layer (R5E, R9E, R10E and R12E, respectively), a hidden layer, and Zima as only neuron in the output layer. The number of neurons in hidden layer varied from 1 to 8 until the highest coefficient of determination was achieved. The computational model was performed using the toolbox in MATLAB with a feedforward ANN. It is worth mention that the hidden and output layers used a hyperbolic tangent and a linear transfer function, respectively, and a Levenberg-Marquardt algorithm as training function was used in the models. Results and Conclusions Figure 1 presents the coefficient of determination (R2) and root mean squared error (RMSE) with different ANN architectures. In the four models, the highest value was reached with 8 neurons in the hidden layer. Then, the linear regressions for R5E, R9E, R10E and R12E models and 8 neurons in the input layer were compared in Figure 2. In that sense, the Nyquist Diagrams in Figure 3 were simulated with the R9E and R12E models. Also, they were compared with experimental Nyquist Diagrams for Bronze with patina of CuSO4 and CuNO3, with and without SiO2 nanoparticles, at 56 days of exposure in marine atmospheres. The simulations demonstrated how an ANN model with 12 neurons in the input layer, 8 neurons in the hidden layer and 1 neuron in the output layer can be used as a corrosion prediction model of bronze with nanostructured patina with SiO2.

A suitable framework for the development of artificial neural networks is important because it decides the level of accuracy, which can be reached for a certain dataset and increases the certainty about the reached classification results. In this paper, we conduct a comparative study for the performance of four frameworks, Keras with TensorFlow, Pytorch, TensorFlow, and Cognitive Toolkit (CNTK), for the elaboration of neural networks. The number of neurons in the hidden layer of the neural networks is varied from 8 to 64 to understand its effect on the performance metrics of the frameworks. A test dataset is synthesized using an analytical model and real measured impedance spectra by an eddy current sensor coil on EUR 2 and TRY 1 coins. The dataset has been extended by using a novel method based on interpolation technique to create datasets with different difficulty levels to replicate the scenario with a good imitation of EUR 2 coins and to investigate the limit of the prediction accuracy. It was observed that the compared frameworks have high accuracy performance for a lower level of difficulty in the dataset. As the difficulty in the dataset is raised, there was a drop in the accuracy of CNTK and Keras with TensorFlow depending upon the number of neurons in the hidden layers. It was observed that CNTK has the overall worst accuracy performance with an increase in the difficulty level of the datasets. Therefore, the major comparison was confined to Pytorch and TensorFlow. It was observed for Pytorch and TensorFlow with 32 and 64 neurons in hidden layers that there is a minor drop in the accuracy with an increase in the difficulty level of the dataset and was above 90% until both the coins were 80% closer to each other in terms of electrical and magnetic properties. However, Pytorch with 32 neurons in the hidden layer has a reduction in model size by 70% and 16.3% and predicts the class, 73.6% and 15.6% faster in comparison to TensorFlow and Pytorch with 64 neurons.

Credit scoring is widely employed method for measuring the risk accompanying with a credit applicant. Credit risk estimation focuses to estimate the risk in areas such as creditworthiness, default risk analysis, fraudulent transactions, though credit scoring has the potential to boost the economy for that industry. So, AI and ML can be leveraged to bring a significant impact to harvest the applicants’ historical data. Extreme Learning Machine (ELM) is a classification tool with single-hidden layer feedforward neural networks, and it analytically determines the output. Moreover, ELM performance depends firstly on number of neurons in different layers and secondly on activation function, weights and biases assigned to those neurons. So, this study presents experimental result analysis of ELMs with various set of neurons in hidden layer and various activation functions, as an appropriate activation function may improve the performance of ELMs.

Mobile Ad Hoc network is a self-organizing infrastructure less network where the nodes are responsible for routing the packets. MANET has dynamic topology which allows nodes to join and leave the network at any time. This paper proposes a channel adaptive routing protocol which extends AOMDV routing protocol to accommodate channel fading. The proposed Channel Aware AOMDV protocol uses channel average nonfading duration to select stable links for route discovery. The main objective of this paper is to demonstrate approximation capability of artificial neural network. Feed forward network architecture with back propagation algorithm is used. Depending upon the size of database number of hidden layers and number of neurons in hidden layer are decided. Number of neurons in hidden layer is varied and effect on percentage deviation in the output parameter is observed. Depending upon nature of input and output parameters synthesis and analysis artificial neural network (ANN) models are trained.

Experiments were conducted to see the effects of a set of factors on the Resilient backpropagation (Rprop) artificial neural network classification of an Indian urban environment using IRS‐IC satellite data. Factors investigated were sample size, number of neurons in hidden layers and number of epochs. The effect of including texture information in the form of neighbourhood information and grey level co‐occurance matrix (GLCM) features in the classification process has been explored. Statistically similar overall classification accuracy is achieved for Rprop and Gaussian maximum likelihood classification (GMLC). Investigations have revealed that a large sample size gave higher test accuracy; variation in number of neurons in hidden layer did not affect the overall classification accuracy significantly; lesser number of epochs resulted in higher overall test accuracy. Incorporation of texture information by both approaches improved classification accuracy in a statistically significant manner.

Multilayer artificial neural networks (ANN) are often used for the solution of classification problems or for the time series forecasting. An appropriate number of learning and testing patterns must be available for the ANN training. Each training pattern is composed of n input parameters and m output parameters. Number m is usually given by the problem formulation, but the number n may be often selected from a greater set of input parameters. An optimal selection of input parameters is a very important task especially in a situation when the number of usable input parameters is great and the analytical relations between the input and output parameters are not known. The number of neurons in all ANN layers must be generally kept as small as possible because of the optimal generalisation ability. In this paper we present a possible way for the selection of significant input parameters (the so called markers), which are the most important ones from the point of view of influence on the output parameters. These parameters are later used for the training of ANN. A statistical approach is usually used for this reason [5]. After some experience in the ANN application we recognised that the approach based on mathematical logic, i. e. the GUHA method (General Unary Hypotheses Automaton) is also suitable for the determination of markers. Besides the minimisation of the number of elements in the input layer of ANN, also the number of neurons in hidden layers must be optimised. For this reason standard methods of pruning can be used, described e. g. in [1]. We have used this method in the following applications: - Optimisation of the intervals between the major overhaul of plane engines by the analysis of tribodiagnostic data. Only selected types of chemical pollution in oil can be taken into account. - Prediction of bleeding of patients with chronic lymphoblastic leukemia. Only a part of parameters about the patient is important from this point of view (see [2]). - Optimisation of the quality and reliability prediction of artificial resin production in chemical factory. Only a part of the production parameters (times of production phases, temperatures, percentage of components etc.) have straight influence on the product. - Optimisation of the prosody control in the text-to-speech synthesis. This application is described in the paper.

Food recognition is an important pattern recognition problem. It involves identification various food types for certain applications such as food industry, calories counting or restaurant automation. Feature extraction is an essential stage in food recognition. It involves extracting various mathematical or handcrafted features for classification. The numerous types of extracted features create an issue of redundancy and over fitting. This article proposes feature weighting based on the concept of entropy for food recognition. The feature weighting indicates to selecting the top 25% of features after sorting them according to their importance. We evaluate Local Binary Pattern, Gabor features, Histogram of Gradient for features weighting based on entropy. The evaluation is done based on two types of neural networks, namely, extreme learning machine ELM and fast learning machine fast learning network for whole features extraction and for only 25% selection of most important features. The model has shown a superiority of ELM and fast learning network when top 25% of features were selected for both ELM and fast learning network with more superiority of fast learning network.

Research community has recently put more attention to the Extreme Learning Machines (ELMs) algorithm in Neural Network (NN) area. The ELMs are much faster than the traditional gradient-descent-based learning algorithms due to its analytical determination of output weights with the random choice of input weights and hidden layer bias. However, since the input weights and bias are randomly assigned and not adjusted, the ELMs model shows an instability if we repeat the experiments many times. Such instability makes the ELMs less reliable than other computational intelligence models. In our investigation, we try to solve this problem by using the Random Production in the first layer of the ELMs. Thus, we can reduce the chance of using random weight assignment in ELMs by removing the bias in the hidden layer. Experiment son different data sets demonstrate that the proposed model has higher stability and reliability than the classical ELMs.

In this study, the deep multi-layered group method of data handling (GMDH)-type neural network algorithm using revised heuristic self-organization method is proposed and applied to medical image diagnosis of liver cancer. The deep GMDH-type neural network can automatically organize the deep neural network architecture which has many hidden layers. The structural parameters such as the number of hidden layers, the number of neurons in hidden layers and useful input variables are automatically selected to minimize prediction error criterion defined as Akaike’s information criterion (AIC) or prediction sum of squares (PSS). The architecture of the deep neural network is automatically organized using the revised heuristic self-organization method which is a type of the evolutionary computation. This new neural network algorithm is applied to the medical image diagnosis of the liver cancer and the recognition results are compared with the conventional 3-layered sigmoid function neural network.

RNNLM (Recurrent Neural Network Language Model) can save the historical information of the training dataset by the last hidden layer and can also as input for training. It has become an interesting topic in the field of Natural Language Processing research. However, the immense training time overhead is a big problem. The large output layer, hidden layer, last hidden layer and the connections among them will generate enormous matrix in training. It is the main facts to influence the efficiency and scalability. At the same time, output layer class and small hidden layer should decrease the accuracy of RNNLM. In general, the lack of parallel for artificial neuron is main reason for these. We change the structure of RNNLM and design the new large-scale RNNLM by the center of distributed artificial neurons in hidden layer to stimulate the parallel characteristic of biological neuron system. Meanwhile, we change training method, and present the coordination strategy for distributed neuron. At last, the prototype of new large-scale RNNLM system is implemented based on Spark. The testing and analysis results show that the training time overhead is far less than the growth rate of the distributed neuron in hidden layer and size of training dataset. These results show our large-scale RNNLM system has efficiency and scalability advantage.

This paper proposes the adjusted Levenberg-Marquardt(LM) technique to improve the performance of speech recognition. Number of difference neuron in hidden layer has been trialed in verities input speech signal which generated from 40 speakers: 20 males and 20 females. The results show that the system provides difference values of the accuracy percentage depending on the number of neuron in hidden layer. The highest accuracy rate for trained and untrained speech data of female approved at 96.67% and 98.33% for stress consonant respectively and also provided 86.92% and 90.00% for voiced consonant of male.

An important benchmark for closed-loop flow control is the suppression of the von Karman vortex street in the wake of a circular cylinder at a Reynolds number of 100. A low-dimensional Proper Orthogonal Decomposition (POD) is applied to the flow field and a four sensor configuration is placed based on the intensity of the resulting spatial Eigenfunctions. This effort focuses on the performance of a non-linear Artificial Neural Network Estimator (ANNE) used for real-time mapping of flow field sensors to POD states. This investigation examines the sensitivity of ANNE by varying the number of neurons in the hidden layer. The aim is to determine if there is an optimum number of neurons in the hidden layer. Three input data sets were studied, the first had no noise on the training and testing data, the second had 10% noise applied to the training and testing data, and the third had 25% noise applied to the training and testing data sets. In all cases 50 randomly selected training data sets were applied to the back-propagating network. A separate data set was used for testing the feed-forward network. During the validation process, network outputs were compared to desired outputs and the Root Mean Squared (RMS) error was calculated for each of the four output nodes. Results show that in general, the more nodes in the hidden layer the better the performance of the network. However, there appeared to be a consistently reduced error when eight (8) nodes were used in the hidden layer. Results from ANNE when compared to the state-of-the-art Linear Stochastic Estimator (LSE) shows significant benefits.

In this work, a new adaptive extreme learning machine (ELM) neural network-based fuzzy controller is designed and simulated for implementing speed regulation in a permanent magnet synchronous motor. ELM is a neural model wherein the number of hidden neurons to be placed in the hidden layer is tuned during the process of neural network training itself. A new adaptive ELM model is developed for placing the number of hidden neurons in the hidden layer, and this new adaptive ELM is tuned with artificial bee colony (ABC) algorithm for optimizing its weight parameters and also the number of hidden neurons. Fuzzy proportional–integral (PI) controller is developed in this work in order to eliminate the steady-state error. The new adaptive ELM neural model optimized with ABC algorithm is applied to tune the input parameters of the fuzzy PI controller and also on optimizing the rules and fuzzy membership functions. The optimized adaptive ELM neural network-based fuzzy PI controller is utilized to investigate the speed regulation of permanent magnet synchronous motor (PMSM) in this work. The developed new PI controller with the PMSM is tested for its performance characteristics and to prove its validity is compared with the traditional controller and other heuristic controllers proposed in earlier literature works.

Chinese text classification based on BP neural network is an important research topic in information retrieval. In this paper, by analyzing the method of the classification based on BP, we propose that used VC-dimension to certain the number of neurons in hidden layers, describe and Analysis of the algorithm. Then, evaluating results by precision rate, recall rate and F1. Experimental results show that the proposed method has high accuracy and efficiency.