Bio-inspired and gradient-based algorithms to train MLPs: The influence of diversity

In this paper, nonlinear functions generated by randomly initialized multilayer perceptrons (MLPs) and simultaneous recurrent neural networks (SRNs) are learned by MLPs and SRNs. Training SRNs is a challenging task and a new learning algorithm - DEPSO is introduced. DEPSO is a standard particle swarm optimization (PSO) algorithm with the addition of a differential evolution step to aid in swarm convergence. The results from DEPSO are compared with the standard backpropagation (BP) and PSO algorithms. It is further verified that functions generated by SRNs are harder to learn than those generated by MLPs but DEPSO provides better learning capabilities for the functions generated by MLPs and SRNs as compared to BP and PSO. These three algorithms are also trained on several benchmark functions to confirm results.

Intrusion detection system (IDS) is one of the most effective defensive systems which is on high demand due to the increasing number of devices connected to the internet these days. IDS monitors the network packets in order to detect malicious activities. These packets usually have a lot of features most of which are irrelevant and repetitive and can curtail the performance of the IDS. Therefore, it is important to use feature selection techniques to select an optimum subset of features. in this paper, a novel IDS is proposed which employs the butterfly optimization algorithm (BOA), a recent meta-heuristic, to perform feature selection. A multi-layer perceptron (MLP) classifier is used to evaluate the capability of the selected features to predict attacks. In order to improve the MLP classifier, in addition to the gradient descent (GD) training method, two other meta-heuristic methods, particle swarm optimization (PSO) and genetic algorithm (GA) have been used to optimize the classification structure. The NSL-KDD dataset is used in experiments to prove the efficiency of the proposed IDS in detecting four attack types namely: DoS, Probe, R2L and U2R. After repeated tests of the proposed method on four types of attack, the results are reported in terms of classification accuracy. A comparison of classical and intelligent methods in classifier training is made. Comparison with other similar works prove the superiority of the proposed method in reducing the number of features as well as increasing classification accuracy.

Neuro-fuzzy systems have been proved effective in training classifiers, especially when it comes to noisy, inaccurate or incomplete datasets. For this reason, and due to their simple comprehensible nature, these systems have become popular in designing classifiers. One of the major challenges in designing a neuro-fuzzy classifier is achieving the optimum system parameters such as the type and position of the membership function as well as its training method. These factors could affect the function of the classifier significantly. In this paper, a novel method based on evolutionary algorithms such as inclined planes optimization algorithm (IPO), particle swarm optimizer (PSO) and genetic algorithm (GA) is introduced to design a neuro-fuzzy classifier in such a way that the accuracy is increased and the error rate is minimized. To prove the efficiency of the proposed method, several experiments are conducted on well-known datasets with different number of classes and different feature vector lengths. Results indicate that the proposed evolutionary-based neuro-fuzzy classifier is superior to a normal neuro-fuzzy classifier in terms of accuracy. In addition, experiments showed that the proposed method is able to properly classify the data with a relatively high stability.

We applied multilayer perceptron (MLP) and radial basis function (RBF) neural networks using data from two water quality monitoring stations at the Karaj Dam in Iran. Input data were calcium ions (Ca2+), magnesium ions (Mg2+), sodium ions (Na+), chloride ions (Cl−), sulfate ( SO 42−), and pH, and the output data were total dissolved solids (TDS). An MLP with one hidden layer containing eight neurons was selected for the upstream water quality station using normalized input data. We developed a second MLP neural network for the downstream station with one hidden layer containing 10 neurons in the hidden layer using normalized input data. Considering applying normalized input data and one hidden layer, the coefficient of determination (R2) and index of agreement (IA) between the observed and the predicted data for the upstream and downstream monitoring stations using the MLP neural networks were 0.985, 0.84, 0.99, and 0.92, respectively. The RBF neural network with 100 neurons in its hidden layer reached the minimum errors between the observed and the predicted results in upstream and downstream stations. The R2 between observed and predicted data for upstream and downstream monitoring stations for the RBF was 0.999 and 0.998, respectively. Data normalization improved the performance of the MLP neural networks. Sensitivity analysis indicated that magnesium is the most effective water quality parameter for predicting TDS, and sulfate is the second most effective water quality parameter affecting TDS prediction at the Karaj Dam.

A technique is outlined for improving the learning rate of a multilayer perceptron (MLP) network. Each network node is assigned its own training rate parameter which is adapted using fuzzy logic as part of the error backpropagation process. This involves the development of target values for hidden layer node output. These values are based on the current network weight state and are therefore different for each epoch. Using two test vector distributions it is demonstrated that this approach can reduce MLP convergence time and is compared to three other training methods: standard backpropagation, fuzzy adapted global training rate parameter, and the delta-bar-delta learning rule.

Artificial neural networks (ANN) and evolutionary algorithms are two relatively young research areas that were subject to a steadily growing interest during the past years. This paper examines the use of different evolutionary algorithms, imperialist competitive algorithm (ICA), genetic algorithm (GA), ICA-GA and recursive ICA-GA (R-ICA-GA) to train a classification problem on a multi layer perceptron (MLP) neural network. All of named evolutionary training algorithms are compared together in this paper. The first goal of the paper is to apply new evolutionary optimization algorithms ICA-GA and R-ICA-GA for training the ANN and the second goal of the paper is to compare different evolutionary algorithms. It is shown that the ICA-GA has the best performance, in number of epochs, compared to the other algorithms. For this purpose, learning algorithms are applied on six known datasets (WINE, PIMA, WDBC, IRIS, SONAR and GLASS) which are used for classification problems.

Breast cancer is the most common kind of cancer, which is the cause of death among the women worldwide. There is evidence that shows that the early detection and treatment can increase the survival rate of patients who suffered this disease. Therefore, this paper proposes an automatic breast cancer diagnosis technique using a genetic algorithm for simultaneous feature selection and parameter optimization of an Multi Layer Perceptron (MLP) neural network. The aim of this paper is to propose a hybrid classification algorithm based on Multi-stage Weights Adjustment in the MLP (MWAMLP) neural network in two parts to improve the breast cancer diagnosis. In the first part, the three classifiers are trained simultaneously on the learning dataset. The output of the first part classifier together with the learning dataset is placed in a new dataset. This dataset uses a hybrid classifier method to model the mapping between the outputs of each ordinary classifier of the first part with real output labels. The proposed algorithm is implemented with three different variations of the backpropagation (BP) technique, namely the Levenberg–Marquardt, resilient BP and gradient descent with momentum for fine tuning of the weight of MLP neural network and their performances are compared. Interestingly, one of the proposed algorithms titled MWAMLP-RP produces the best and on average, 99.35% and 98.74% correct classification, respectively, on the Wisconsin Breast Cancer Database dataset, which is comparable with the obtained results from the methods titled GP-DLNN, GAANN and CAFS and other works found in the literature.

Combining cluster analysis with classifier ensembles to predict financial distress

In the last decades, the use of ensembles of classifiers in different domains of application has received significant attention of the Machine Learning community. In a typical architecture of ensemble, a new input pattern is presented to all K components. The individual classifiers provide their output and send them to a combination method, which is responsible for providing the final output. Ensemble of classifiers can be applied to any classification problem, single-label or multi- label problems. Multi-label (ML) classification has been received much attention from various research domains, such as text categorization, bioinformatics, computer vision, among others. In this paper, we investigate the use of ensemble of classifiers when applied to multi-label problems, focusing on the use of diversity measures in these systems when applied to multi-label problems. In order to do this, the present work proposes an adapted approach for two well-known diversity measures, good and bad, for ensembles of multi-label classifiers. In order to assess the feasibility of the proposed diversity measures in ensemble of multi-label classifiers, an empirical analysis is conducted, with eight multi-label classification problems. Generally, our finds indicate that there is a strong correlation between performance of the ML ensembles with the proposed diversity measures, showing that these diversity measures can be used as an important tool in the design of efficient ensembles of multi-label classifiers.

Hydraulic calculations of water distribution systems are currently performed using computer programs. In addition to the basic calculation procedure, modules responsible for evaluating the obtained calculation results are introduced more and more often into the programs. This article presents the results of research on artificial neural networks with a radial base function (RBF) and a multilayer perceptron (MLP), aimed at determining whether they can be used to model the relationship between the variables describing the computational section of the water distribution system and the diameter of the water pipe. The classification capabilities of the RBF and MLP networks were analyzed according to the number of neurons in the hidden layer of the network. A comparative analysis of RBF networks with multilayer perceptron (MLP) networks was performed. The results showed that the MLP networks have much better classification properties and are better suited for the task of assessing the selected diameters of the water pipes.

Bidikmisi scholarships are tuition assistance for prospective students who are economically disadvantaged but have the good academic potential to study at State Universities in Indonesia . With the amount Bidikmisi applicants, there is a need to use a system that can speed up the selection process and avoid human errors. The purpose of this study is to develop a classification system for Bidikmisi scholarship applicants at Universities in Indonesia. This study uses a multilayer perceptron (MLP) neural network to build a classification model. Because MLP has some hidden layers, MLP has advantages to predict or analyze complex problems. The methodology in this study is data collection, data preprocessing, data encoding, data modeling, and evaluation. The results of the classification of bidikmisi scholarship applicants are divided into two classes, namely accepted and rejected. The multilayer perceptron method generates an accuracy value of 72.81%. The Multilayer perceptron is compared with other classification methods, namely Naïve Bayes and K-Nearest Neighbors (KNN). The results of the classification using the Naïve Bayes method obtained an accuracy value of 79.47%, while the accuracy value using KNN was 78.33%. Evaluation using MLP resulted in precision, recall, and F1-measure values of 68.36%, 72.81%, and 66.33%. the results of the evaluation using Naïve Bayes were 75.58%, 79.47%, and 71.24%, while the KNN values were 71.90%, 78.33%, and 72.53%. In order to the conclusion in this research is the MLP method can generate good performance for the classification of bidikmisi scholarship applicants. However, Naïve Bayes’ performance is better than MLP for the classification of bidikmisi scholarship applicants.

Complexity Analysis of Multilayer Perceptron Neural Network Embedded into a Wireless Sensor Network

For evaluating the accuracy and capability of MLP (Multi Layer Perceptron) and RBF (Radial Basis Function) classifier algorithms; projection profile features for the character images are extracted and are merged with the binarization features after preprocessing every character image, and the combined features thus obtained are used to train both the classifiers (MLP and RBF Network) selected for performing character recognition task. Simulation studies are examined extensively and the proposed hybrid features are found to deliver the recognition accuracy of 93.46% when used with RBF Network as a classifier which is better than the 88.76% accuracy obtained when used with MLP Network as a classifier.

Estimating impervious surfaces from medium spatial resolution imagery using the self-organizing map and multi-layer perceptron neural networks

Neural networks are a soft computing technique with wide application in signal processing as well as system and process modeling. In the present study, multilayer perceptron (MLP) neural networks were employed to process multidimensional signals generated in metal machining operations (including three-dimensional cutting force signals and three-dimensional cutting vibration signals) and to establish a model for predicting the machined surface roughness. This paper describes in detail our methods of multidimensional signal processing and modeling with MLP neural networks. The MLP neural network model developed in the present study fills an important research gap by taking into account the critical effect of tool-edge radius in machining. As compared to regression models, the MLP neural network model developed in the present study has significantly higher accuracy in predicting the machined surface roughness.