Robust Extreme Learning Machine Using New Activation and Loss Functions Based on M-Estimation for Regression and Classification

A fast training algorithm for extreme learning machine based on matrix decomposition

An improved classification approach is proposed to solve the hot research problem of some complex multiclassification samples based on extreme learning machine (ELM). ELM was proposed based on the single‐hidden layer feed‐forward neural network (SLFNN). ELM is characterized by the easier parameter selection rules, the faster converge speed, the less human intervention, and so on. In order to further improve the classification precision of ELM, an improved generation method of the network structure of ELM is developed by dynamically adjusting the number of hidden nodes. The number change of the hidden nodes can serve as the computational updated step length of the ELM algorithm. In this paper, the improved algorithm can be called the variable step incremental extreme learning machine (VSI‐ELM). In order to verify the effect of the hidden layer nodes on the performance of ELM, an open‐source machine learning database (University of California, Irvine (UCI)) is provided by the performance test data sets. The regression and classification experiments are used to study the performance of the VSI‐ELM model, respectively. The experimental results show that the VSI‐ELM algorithm is valid. The classification of different degrees of broken wires is now still a problem in the nondestructive testing of hoisting wire rope. The magnetic flux leakage (MFL) method of wire rope is an efficient nondestructive method which plays an important role in safety evaluation. Identifying the proposed VSI‐ELM model is effective and reliable for actually applying data, and it is used to identify the classification problem of different types of samples from MFL signals. The final experimental results show that the VSI‐ELM algorithm is of faster classification speed and higher classification accuracy of different broken wires.

Extreme Learning Machine (ELM) is a learning algorithm proposed recently to train single hidden layer feed forward networks (SLFN). It has many attractive properties that include better generalization performance and very fast learning. ELM starts by assigning random values to the input weights and hidden biases and then in one step it determines the output weights using Moore-Penrose generalized inverse. Despite the aforementioned advantages, ELM performance might be affected by the random initialization of weights and biases or by the large generated network which might contain unnecessary number of neurons. In order to increase the generalization performance and to produce more compact networks, a hybrid model that combines ELM with competitive swarm optimizer (CSO) is proposed in this paper. The proposed model (CSONN-ELM) optimizes the weights and biases and dynamically determines the most appropriate number of neurons. To evaluate the effectiveness of the CSONN-ELM, it is experimented using 23 benchmark datasets, and compared to a set of static rules extracted from literature that are used to determine the number of neurons of SLFN. Moreover, it is compared to two dynamic methods that are used to enhance the performance of ELM, that are Optimally pruned ELM (OP-ELM) and metaheuristic based ELMs (Particle Swarm Optimization-ELM and Differential Evolution-ELM). The obtained results show that the proposed method enhances the generalization performance of ELM and overcomes the static and dynamic methods.

Credit scoring is widely employed method for measuring the risk accompanying with a credit applicant. Credit risk estimation focuses to estimate the risk in areas such as creditworthiness, default risk analysis, fraudulent transactions, though credit scoring has the potential to boost the economy for that industry. So, AI and ML can be leveraged to bring a significant impact to harvest the applicants’ historical data. Extreme Learning Machine (ELM) is a classification tool with single-hidden layer feedforward neural networks, and it analytically determines the output. Moreover, ELM performance depends firstly on number of neurons in different layers and secondly on activation function, weights and biases assigned to those neurons. So, this study presents experimental result analysis of ELMs with various set of neurons in hidden layer and various activation functions, as an appropriate activation function may improve the performance of ELMs.

Robustness of Extreme Learning Machine in the prediction of hydrological flow series

This paper develops a state-of-charge (SOC) estimation model for a lithium-ion battery using an improved extreme learning machine (ELM) algorithm. ELM is suitable for an SOC estimation since the ELM algorithm has fast estimation speed, good generalization performance, and high accuracy. However, the performance of ELM is highly dependent on training accuracy and the number of neurons in a hidden layer. Hence, a gravitational search algorithm (GSA) is applied to improve the ELM computational intelligence by searching for the optimal value hidden layer neurons. The optimal ELM-based GSA model does not require internal battery knowledge and mathematical model for an SOC estimation. The model robustness is validated at different temperatures using different electric vehicle drive cycles. The performance of the ELM-GSA model is verified with two popular neural network methods: back-propagation neural network (BPNN) and radial basis function neural network (RBFNN). The results are evaluated using different error rates and computation costs. The results demonstrate that the ELM-based GSA model offers a higher accuracy and lower SOC error rate than those of BPNN-based GSA and RBFNN-based GSA models. Furthermore, a detailed comparative study between the proposed model and existing SOC strategies is conducted, which also demonstrates the superiority of the proposed model.

Dynamic adjustment of hidden layer structure for convex incremental extreme learning machine

A data-attribute-space-oriented double parallel (DASODP) structure for enhancing extreme learning machine: Applications to regression datasets

Extreme learning machine (ELM) is an easy-to use and effective learning algorithm of single-hidden layer feed-forward neural networks (SLFNs). The classical learning algorithm in neural network, e. g. backpropagation, requires setting several user-defined parameters and may get into local minimum. However, ELM only requires setting the number of hidden neurons and the activation function. It does not require adjusting the input weights and hidden layer biases during the implementation of the algorithm, and it produces only one optimal solution. Therefore, ELM has the advantages of fast learning speed and good generalization performance. In this paper, ELM is introduced in predicting reservoir permeability. By comparing to SVM, we analyze its feasibility and advantages in reservoir permeability prediction. The experimental results show that ELM has similar accuracy compared to SVR, but it has obvious advantages in parameter selection and learning speed.

Temperature extraction for Brillouin optical fiber sensing system based on extreme learning machine

Deep Learning methods have proven to be very successful in classifying large data sets of high feature dimensionality. However, their success usually implies very long training times. In this paper we examine learning methods combining the Random Neural Network, a biologically inspired neural network and the Extreme Learning Machine that achieve state of the art classification performance while requiring much shorter training time. The Random Neural Network is a integrate and fire computational model of a neural network whose mathematical structure permits the efficient analysis of large ensembles of neurons. An activation function is derived from the RNN and used in an Extreme Learning Machine. We compare the performance of this combination against the ELM with various activation functions, we reduce the input dimensionality via PCA and compare its performance vs. autoencoder based versions of the RNN-ELM.

Extreme learning machine (ELM) is a fast and quick learning algorithm with better generalization performance. However, the randomness of input weight and hidden layer bias may affect the overall performance of ELM. This paper proposed a new approach to determine the optimized values of input weight and hidden layer bias for ELM using whale optimization algorithm (WOA), which we call WOA-ELM. An online gearbox vibration signals is used in this study. Empirical mode decomposition (EMD) and complementary mode decomposition (CEEMD) are used to decompose the signals into sub-signals known as intrinsic mode functions (IMFs). Then, statistical features are extracted from selected IMFs. WOA-ELM is used for classification of healthy and faulty condition of gearbox. The result shows that WOA-ELM provide better classification result as compared with conventional ELM. Therefore, this study provide a new diagnosis approach for gearbox application.

The learning speed of the feed forward neural network takes a lot of time to be trained which is a major drawback in their applications since the past decades. The key reasons behind may be due to the slow gradient-based learning algorithms which are extensively used to train the neural networks or due to the parameters in the networks which are tuned iteratively using some learning algorithms. Thus, in order to eradicate the above pitfalls, a new learning algorithm was proposed known as Extreme Learning Machines (ELM). This algorithm tries to compute Hidden-layer-output matrix that is made of randomly assigned input layer and hidden layer weights and randomly assigned biases. Unlike the other feedforward networks, ELM has the access of the whole training dataset before going into the computation part. Here, we have devised a new two-layer-feedforward network (TFFN) for ELM in a new manner with randomly assigning the weights and biases in both the hidden layers, which then calculates the output-hidden layer weights using the Moore-Penrose generalized inverse. TFFN doesn't restricts the algorithm to fix the number of hidden neurons that the algorithm should have. Rather it searches the space which gives an optimized result in the neurons combination in both the hidden layers. This algorithm provides a good generalization capability than the parent Extreme Learning Machines at an extremely fast learning speed. Here, we have experimented the algorithm on various types of datasets and various popular algorithm to find the performances and report a comparison.

Robust regularized extreme learning machine for regression using iteratively reweighted least squares

Extreme learning machine (ELM) is prominent for its rapid training speed and good generalization performance. However, its performance severely deteriorates in the presence of outliers, and its network structure often lacks sparsity, leading to model overfitting. Although existing robust ELM variants have made progress, they often struggle with asymmetrically distributed outliers and high outlier ratios ( > 20%), and frequently fail to achieve satisfactory sparsity due to the use of L2-type regularization. To address these limitations, this paper proposes a novel ELM model termed APCELM, which integrates the pinball loss function and the capped L1 norm regularization. The asymmetric pinball loss enhances robustness against outliers by assigning different penalties to positive and negative errors, while the capped L1 norm promotes sparsity by effectively pruning redundant hidden neurons. The optimization problem of APCELM is efficiently solved using the alternating direction method of multipliers (ADMM), which decomposes the problem into tractable sub-problems with guaranteed convergence. Extensive experiments on both artificial and benchmark datasets demonstrate that APCELM significantly outperforms state-of-the-art methods in terms of robustness, sparsity, and stability, especially under high outlier ratios (up to 40%).