A Gradient-Based ELM Algorithm in Regressing Multi-variable Functions

Extreme learning machine (ELM) is a well-known algorithm for single layer feedforward neural networks (SLFNs) and their learning speed is faster than traditional gradient-based neural networks. However, many of the tasks that ELM focuses on are single-label, where an instance of the input set is associated with one label. This paper proposes a new method for training ELM that will be capable of multi-label classification using the Canonical Correlation Analysis (CCA). The new method is named CCA-ELM. There are 4 steps in the training process: the first step is to compute any correlations between the input features and the set of labels using CCA, the second step maps the input space and label space to the new space, the third step uses ELM to classify and the last step is to map to the original input space. The experimental results show that CCA-ELM can improve ELM for classification on multi-label learning and its recognition performances are better than the other comparative algorithms that use the same standard CCA.

Ensemble dropout extreme learning machine via fuzzy integral for data classification

Extreme learning machine (ELM) is an easy-to use and effective learning algorithm of single-hidden layer feed-forward neural networks (SLFNs). The classical learning algorithm in neural network, e. g. backpropagation, requires setting several user-defined parameters and may get into local minimum. However, ELM only requires setting the number of hidden neurons and the activation function. It does not require adjusting the input weights and hidden layer biases during the implementation of the algorithm, and it produces only one optimal solution. Therefore, ELM has the advantages of fast learning speed and good generalization performance. In this paper, ELM is introduced in predicting reservoir permeability. By comparing to SVM, we analyze its feasibility and advantages in reservoir permeability prediction. The experimental results show that ELM has similar accuracy compared to SVR, but it has obvious advantages in parameter selection and learning speed.

Among the different computational intelligence techniques avalaible for hyperspectral data classification, support vector machines (SVMs) have played a dominant role. Recently, a new learning algorithm for single-layer feedforward neural networks called the extreme learning machine (ELM) was proposed. This technique is competitive with SVMs in terms of accuracy, learning speed, and computational scalability. In this article, we propose and evaluate the use of ELM for land-cover classification from hyperspectral images. In addition, we consider two ELM-based techniques integrating spectral and spatial information of the image. The first is a scheme that uses a majority vote approach in order to combine the results of a pixel-wise spectral classification by ELM and a segmentation map obtained by a watershed algorithm. The second introduces spatial information from a small spatial neighbourhood after the classification by ELM. We show the usefulness of spatial–spectral ELM-based classification techniques in hyperspectral imaging. The results are compared to those obtained by similar SVM-based techniques and show improved classification results and much lower execution time. These simple and computationally cheap methods can be combined with others traditionally applied to hyperspectral images.

Extreme learning machine (ELM) is a learning algorithm for single-hidden layer feedforward neural networks. In theory, this algorithm is able to provide good generalization capability at extremely fast learning speed. Comparative studies of benchmark function approximation problems revealed that ELM can learn thousands of times faster than conventional neural network (NN) and can produce good generalization performance in most cases. Unfortunately, the research on damage localization using ELM is limited in the literature. In this chapter, the ELM is extended to the domain of damage localization of plate structures. Its effectiveness in comparison with typical neural networks such as back-propagation neural network (BPNN) and least squares support vector machine (LSSVM) is illustrated through experimental studies. Comparative investigations in terms of learning time and localization accuracy are carried out in detail. It is shown that ELM paves a new way in the domain of plate structure health monitoring. Both advantages and disadvantages of using ELM are discussed.

Extreme Learning Machine (ELM) is an extension of feed forward Neural Network model with extremely learning capacity and good generalization capabilities. Generally, the classification performance of ELM highly focused on three parameters such as the input weight matrix, the value of bias and the number of hidden neurons presented. ELM randomly chooses the input weights and bias values to minimize the training time. The arbitrary selection of biases and the input weight produce an unforeseen result which causes training error and also produces lesser prediction accuracy. For improving efficiency of ELM training, Ant Colony Optimization with continuous domain algorithm (ACO R ) used to find the optimum input weight and hidden bias values for ELM. With the unequal distribution of classes in imbalanced data sets, ELM algorithms tussle to find good accuracy. ELM algorithm doesn't acquire the mandatory information about the minority class to make an accurate classification. To deal the issues of ELM, in this paper the hybrid algorithms Weighted ELM and Weighted ELM with ACO R are proposed. The main objective of weighted ELM is that the related weight value is computed and assigned for each training sample to increase the classification rate. ACO R method is also integrated with the weighted ELM to find the optimum input weight and bias to maximize the accuracy of classification task. The comparative analysis has been accomplished over Hepatitis dataset. Further, the experimental results clearly revealed that Weighted ELM with ACO R is superior to other proposed methods.

This paper proposes a novel artificial neural network called Parallel Layer Perceptron Fast Learning Network (PLP-FLN). In PLP-FLN, a parallel single hidden layer feed-forward neural network is added on the basis of Fast Learning Network (FLN) which is an improved Extreme Learning Machine (ELM). Input weights and hidden layer biases are randomly generated. The weights connect the output nodes and the input nodes, and the weights connect the output nodes and the hidden nodes are analytically determined based on least squares methods. In order to test the PLP-FLN validity, this paper compared it with ELM, FLN, Kernel ELM and Incremental ELM through 12 regression applications and 7 classification problems. By comparing the experimental results, it shows that the PLP-FLN with much more compact networks have demonstrated better approximations, classification performances and generalization ability.

This chapter approaches the topic of Extreme Learning Machines (ELM); this technique is implemented in a single layer feedforward neural network and provides to the neural network with a fast performance since it is not necessary to adjust weights and hidden layer biases. ELM is implemented and combined with three metaheuristic algorithms. The selected algorithms belong to the family of Swarm intelligence such as Particle Swarm Optimization (PSO), Artificial Bee Colony (ABC) and Grey Wolf Optimization (GWO); these algorithms are applied to optimize the hidden biases and input weights and are implemented for improving the performance along with techniques of Extreme learning machines in data classification. Also, the performance and behavior of each algorithm can be observed. The data employed in this work consist of information about Australian credit, diabetes detection and hearth disease. The investigation result show that GWO obtained better results in the first dataset and PSO are better in the second and third datasets.KeywordsExtreme learning machinesMetaheuristicPSOGWOABCFitness function

Research community has recently put more attention to the Extreme Learning Machines (ELMs) algorithm in Neural Network (NN) area. The ELMs are much faster than the traditional gradient-descent-based learning algorithms due to its analytical determination of output weights with the random choice of input weights and hidden layer bias. However, since the input weights and bias are randomly assigned and not adjusted, the ELMs model shows an instability if we repeat the experiments many times. Such instability makes the ELMs less reliable than other computational intelligence models. In our investigation, we try to solve this problem by using the Random Production in the first layer of the ELMs. Thus, we can reduce the chance of using random weight assignment in ELMs by removing the bias in the hidden layer. Experiment son different data sets demonstrate that the proposed model has higher stability and reliability than the classical ELMs.

This paper presents variable monsoon trends over Kerala in winter, pre-monsoon, summer monsoon and post-monsoon periods. The decreasing trends in seasonal rainfall and post-monsoon rainfall and increasing occurrence of the deficit rainfall years indicates the probable intensification of water scarcity in Kerala. Artificial intelligent methods like Single layer Feed-Forward Neural Network (SLFN) and Extreme Learning Machine (ELM) were used to predict the summer monsoon rainfall as heavy rainfall occurs during this period in the state. The proposed SLFN network prediction model provides 6.3977% of mean absolute error whereas ELM provides 3.8729% of error. ELM is better as compared to SLFN and can be used as a prediction tool for analyzing complex monsoon phenomena.

The Extreme Learning Machine (ELM) is a fast and straightforward learning algorithm designed for single hidden layer feed-forward neural networks (SLFNs). The ELM randomly assigns input weights and calculates the output weights analytically using the Moore-Penrose generalized inverse (MP) (also called Pseudo inverse). However, the MP is not preferable for ill-posed systems as it is sensitive to perturbation and may produce inaccurate solutions. Furthermore, it is computationally expensive as it involves matrix inversion. Regularized ELM (RELM) was proposed to address the limitations of the MP. In RELM, the regularization parameter is the primary factor that controls the classifier's performance. Though the RELM is stable and produces accurate results, finding a n a ppropriate regularization parameter value is tedious and increases the computation. This work proposes a variant of ELM named LSQR-ELM algorithm, which balances the ELM and RELM by maintaining stability and accuracy and avoiding the regularization entirely. The LSQR-ELM is based on the Least Squares QR (LSQR) iterative technique. Since the LSQR finds p a rtial Q R rather than full QR, it reduces unnecessary computation. To verify the proposed algorithm's ability, comparisons are made with several existing methods including ELM, Minimum Residual Method (MT-ELM), SVD-based SG-ELM, Optimally Pruned ELM (OP-ELM), and K-nearest neighbor (KNN). Extensive experimen-tation on various datasets demonstrates that the LSQR-ELM algorithm outperforms ELM, SG-ELM, OP-ELM, and KNN on most datasets. Moreover, it exhibits comparable performance to MT-ELM while requiring significantly f e wer computations.

The theory and implementation of extreme learning machine (ELM) prove that it is a simple, efficient, and accurate machine learning method. Compared with other single hidden layer feedforward neural network algorithms, ELM is characterized by simpler parameter selection rules, faster convergence speed, and less human intervention. The multiple hidden layer regularized extreme learning machine (MRELM) inherits these advantages of ELM and has higher prediction accuracy. In the MRELM model, the number of hidden layers is randomly initiated and fixed, and there is no iterative tuning process. However, the optimal number of hidden layers is the key factor to determine the generalization ability of MRELM. Given this situation, it is obviously unreasonable to determine this number by trial and random initialization. In this paper, an incremental MRELM training algorithm (FC-IMRELM) based on forced positive-definite Cholesky factorization is put forward to solve the network structure design problem of MRELM. First, an MRELM-based prediction model with one hidden layer is constructed, and then a new hidden layer is added to the prediction model in each training step until the generalization performance of the prediction model reaches its peak value. Thus, the optimal network structure of the prediction model is determined. In the training procedure, forced positive-definite Cholesky factorization is used to calculate the output weights of MRELM, which avoids the calculation of the inverse matrix and Moore-Penrose generalized inverse of matrix involved in the training process of hidden layer parameters. Therefore, FC-IMRELM prediction model can effectively reduce the computational cost brought by the process of increasing the number of hidden layers. Experiments on classification and regression problems indicate that the algorithm can be effectively used to determine the optimal network structure of MRELM, and the prediction model training by the algorithm has excellent performance in prediction accuracy and computational cost.

A seemingly unrelated regression (SUR) refers to several individual equations among which there is not an explicit connection such as one equation’s observation is another equation’s response, but there exists an implicit relation represented by correlated disturbances of response variables. In this paper, SUR is applied to extreme learning machine (ELM) which is a single hidden layer feed-forward neural network where input weights and hidden layer biases are randomly assigned but the weight parameters between hidden and output layers are least-square solutions of a regression equation. A correlation-based extreme learning machine is built using the auxiliary sample which is related to the main sample which we focus on. Considering the weights between hidden and output layers in ELM as a random vector, we derive an explicit representation for the vector’s covariance matrix. The proof of theorems and simulation process indicate that the stronger correlation between main sample and auxiliary sample is, the higher generalization ability is.

A fast and robust algorithm for training feedforward neural networks (FNNs) by using a variable forgetting factor and U-D factorization-based recursive prediction error (RPE) method is proposed. In comparison with the backpropagation (BP) and RPE based learning algorithms, the proposed algorithm, called UD-RPE, can provide much more accurate learning results in fewer iterations with fewer hidden nodes and improve convergence rate and numerical stability (robustness). In addition, it is less sensitive to start-up parameters, such as initial weights and initial covariance matrix, and the randomness in the observed data. It also has good generalization ability and needs less learning time. Simulation results of nonlinear dynamic system modeling and identification show that the algorithm proposed here is an effective and efficient learning algorithm for FNNs.

Coral reefs are one of the most prominent marine ecosystems on the Earth; they are threatened due to various factors, including growing anthropogenic impacts and the effects of global change. Hence, the automatic classification of coral species is significant for tracking and detecting threatened and susceptible coral species. This paper proposes a new feature descriptor known as fractal adaptive weight synthesized-local directional pattern (FAWS-LDP) method to classify coral images using enhanced tree seed algorithm with extreme learning machine (ETSA-ELM) technique. The proposed feature descriptor (FAWS-LDP) inherits the advantage of both fractal pixel intensity information and local directional characteristics by indexing both feature vector values. Finally, the extracted features are imported to the extreme learning machine (ELM) network for classification. The ELM classifier is a single hidden layer feed-forward neural network with a faster learning speed and produces good generalization performance. The random selection of input weight and biases of the ELM classifier produces non-optimal or unnecessary input biases and weights to the network. Hence, to fine-tune the parameters of the ELM classifier, an enhanced tree seed algorithm (ETSA) is proposed. The proposed ETSA is a new learning technique to overcome the drawbacks like local optima and a lower coverage rate. The classification performance of ELM employing the ETSA optimizer is compared to the original TSA and other well-known metaheuristic algorithm (MHA) trainers, such as genetic algorithm (GA), particle swarm optimization (PSO), and artificial bee colony (ABC), utilizing model performance measures like specificity, sensitivity, and classification accuracy. The evaluation of coral classification datasets reveals that the proposed ETSA-ELM produces consistently superior performance to existing methods. Finally, the proposed feature descriptor technique is statistically analyzed using a non-parametric Friedman test to demonstrate the efficiency.