Fast Learning Network with Parallel Layer Perceptrons

This paper presents an evaluation of the Extreme Learning Machine (ELM), Kernel Extreme Learning Machine (KELM), and the Kernel Extreme Learning Machine optimized by the Whale Optimization Algorithm (WOA-KELM) in the context of fruit image classification. Fruits play a crucial role in everyday life, and the application of machine learning for fruit classification can significantly improve efficiency. The ELM, a single hidden layer feedforward neural network, is the foundation for this comparison. The KELM, an evolution of the ELM, incorporates a kernel function to map input data into a high-dimensional feature space, thereby enhancing performance. The development of WOA-KELM involves optimizing the regularization parameter in KELM and the width parameter of the radial basis function through the Whale Optimization Algorithm, further enhancing its capabilities. In classifying twelve types of fruit images, the ELM achieved a classification accuracy of 77%. In contrast, the WOA-KELM, as proposed in this paper, significantly outperforms with a remarkable classification accuracy of 92%.

A seemingly unrelated regression (SUR) refers to several individual equations among which there is not an explicit connection such as one equation’s observation is another equation’s response, but there exists an implicit relation represented by correlated disturbances of response variables. In this paper, SUR is applied to extreme learning machine (ELM) which is a single hidden layer feed-forward neural network where input weights and hidden layer biases are randomly assigned but the weight parameters between hidden and output layers are least-square solutions of a regression equation. A correlation-based extreme learning machine is built using the auxiliary sample which is related to the main sample which we focus on. Considering the weights between hidden and output layers in ELM as a random vector, we derive an explicit representation for the vector’s covariance matrix. The proof of theorems and simulation process indicate that the stronger correlation between main sample and auxiliary sample is, the higher generalization ability is.

This paper proposes a novel artificial neural network called sparse-Bayesian--based fast learning network (SBFLN). In SBFLN, sparse Bayesian regression is used to train the fast learning network (FLN), which is an improved extreme learning machine (ELM). The training process of SBFLN is to randomly generate the input weights and the hidden layer biases, and then find the probability distribution of other weights by the sparse Bayesian approach. SBFLN calculates the predicted output through Bayes estimator, so it can provide a natural marginal possibility for classification problems and can solve the overfitting problem caused by the least-squares estimation in FLN. In addition, the sparse Bayesian approach can automatically trim most redundant neurons in hidden layer, which makes the network more compact and accurate. To verify the effectiveness of the improvements in this paper, the results of SBFLN are evaluated in 15 benchmark classification problems. The experimental results show that SBFLN is not sensitive to the number of neurons in the hidden layer, and the performance of SBFLN is competitive or superior to some other state-of-the-art algorithms.

The Extreme Learning Machine (ELM) is a fast and straightforward learning algorithm designed for single hidden layer feed-forward neural networks (SLFNs). The ELM randomly assigns input weights and calculates the output weights analytically using the Moore-Penrose generalized inverse (MP) (also called Pseudo inverse). However, the MP is not preferable for ill-posed systems as it is sensitive to perturbation and may produce inaccurate solutions. Furthermore, it is computationally expensive as it involves matrix inversion. Regularized ELM (RELM) was proposed to address the limitations of the MP. In RELM, the regularization parameter is the primary factor that controls the classifier's performance. Though the RELM is stable and produces accurate results, finding a n a ppropriate regularization parameter value is tedious and increases the computation. This work proposes a variant of ELM named LSQR-ELM algorithm, which balances the ELM and RELM by maintaining stability and accuracy and avoiding the regularization entirely. The LSQR-ELM is based on the Least Squares QR (LSQR) iterative technique. Since the LSQR finds p a rtial Q R rather than full QR, it reduces unnecessary computation. To verify the proposed algorithm's ability, comparisons are made with several existing methods including ELM, Minimum Residual Method (MT-ELM), SVD-based SG-ELM, Optimally Pruned ELM (OP-ELM), and K-nearest neighbor (KNN). Extensive experimen-tation on various datasets demonstrates that the LSQR-ELM algorithm outperforms ELM, SG-ELM, OP-ELM, and KNN on most datasets. Moreover, it exhibits comparable performance to MT-ELM while requiring significantly f e wer computations.

It is well-known that solar power depends on many complex parameters such as humidity, radiation, temperature, dust and wind speed. In order to cope with these complex structures and to provide an accurate forecast, the development of reliable and effective forecasting methodology is very significant. In this study, an Artificial Neural Network (ANN)-based system has been developed to forecast very short-term (2 to 4-h) power output of a grid tied Photovoltaic Power Plant (PVPP). An algorithm called Extreme Learning Machine (ELM) has gained increasing interest through its extremely fast learning and good generalization capability. The Kernel Extreme Learning Machine (KELM) which is the improved version of the ELM is proposed to develop a very short-term PVPP power forecast system. The most important feature of KELM is that it has less adjustable parameters and better generalization ability when compared with classical ELM. Experimental studies have been carried out on a grid tied PVPP that has 1 (MW) installed power capacity. The inputs of the KELM-based forecast system are selected as solar power, humidity, radiation and temperature. All data are divided into four parts to analyze the effect of seasons on the performance of the proposed forecast system. The comparison studies are carried out to clearly observe the forecast ability and performance of the KELM. From the experimental studies of the winter season which has not symmetric diurnal mean power curve, Correlation Coefficients (R) values of KELM are calculated as 0.896 for 2-h ahead, 0.865 for 3-h ahead, 0.833 for 4-h ahead. Those of ELM and (Levenberg Marquardt) LM are 0.866–0.812 for 2-h ahead, 0.831–0.669 for 3-h ahead, 0.799–0.502 for 4-h ahead, respectively. Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) values of KELM for winter season are 12.08%-6.14% for 2-h ahead, 14.06%-7.43% for 3-h ahead, 15.33%-7.74% for 4-h ahead, respectively. While these values of ELM are obtained as 13.82%-7.05% for 2-h ahead, 15.52%-8.77% for 3-h ahead, 17.63%-9.33% for 4-h ahead. The RMSE and MAE values of LM are 17.90%-9.14% for 2-h ahead, 20.04%-9.85% for 3-h ahead, 21.22%-10.68% for 4-h ahead. According to the obtained results, it is clearly seen that KELM provides a more powerful and reliable forecasting performance.

Shortwave radiation density flux (SRDF) modeling can be key in estimating actual evapotranspiration in plants. SRDF is the result of the specific and scattered reflection of shortwave radiation by the underlying surface. SRDF can have profound effects on some plant biophysical processes such as photosynthesis and land surface energy budgets. Since it is the main energy source for most atmospheric phenomena, SRDF is also widely used in numerical weather forecasting. In the current study, an improved version of the extreme learning machine was developed for SRDF forecasting using the historical value of this variable. To do that, the SRDF through 1981–2019 was extracted by developing JavaScript-based coding in the Google Earth Engine. The most important lags were found using the auto-correlation function and defined fifteen input combinations to model SRDF using the improved extreme learning machine (IELM). The performance of the developed model is evaluated based on the correlation coefficient (R), root mean square error (RMSE), mean absolute percentage error (MAPE), and Nash–Sutcliffe efficiency (NSE). The shortwave radiation was developed for two time ahead forecasting (R = 0.986, RMSE = 21.11, MAPE = 8.68%, NSE = 0.97). Additionally, the estimation uncertainty of the developed improved extreme learning machine is quantified and compared with classical ELM and found to be the least with a value of ±3.64 compared to ±6.9 for the classical extreme learning machine. IELM not only overcomes the limitation of the classical extreme learning machine in random adjusting of bias of hidden neurons and input weights but also provides a simple matrix-based method for practical tasks so that there is no need to have any knowledge of the improved extreme learning machine to use it.

Coral reefs are one of the most prominent marine ecosystems on the Earth; they are threatened due to various factors, including growing anthropogenic impacts and the effects of global change. Hence, the automatic classification of coral species is significant for tracking and detecting threatened and susceptible coral species. This paper proposes a new feature descriptor known as fractal adaptive weight synthesized-local directional pattern (FAWS-LDP) method to classify coral images using enhanced tree seed algorithm with extreme learning machine (ETSA-ELM) technique. The proposed feature descriptor (FAWS-LDP) inherits the advantage of both fractal pixel intensity information and local directional characteristics by indexing both feature vector values. Finally, the extracted features are imported to the extreme learning machine (ELM) network for classification. The ELM classifier is a single hidden layer feed-forward neural network with a faster learning speed and produces good generalization performance. The random selection of input weight and biases of the ELM classifier produces non-optimal or unnecessary input biases and weights to the network. Hence, to fine-tune the parameters of the ELM classifier, an enhanced tree seed algorithm (ETSA) is proposed. The proposed ETSA is a new learning technique to overcome the drawbacks like local optima and a lower coverage rate. The classification performance of ELM employing the ETSA optimizer is compared to the original TSA and other well-known metaheuristic algorithm (MHA) trainers, such as genetic algorithm (GA), particle swarm optimization (PSO), and artificial bee colony (ABC), utilizing model performance measures like specificity, sensitivity, and classification accuracy. The evaluation of coral classification datasets reveals that the proposed ETSA-ELM produces consistently superior performance to existing methods. Finally, the proposed feature descriptor technique is statistically analyzed using a non-parametric Friedman test to demonstrate the efficiency.

This chapter approaches the topic of Extreme Learning Machines (ELM); this technique is implemented in a single layer feedforward neural network and provides to the neural network with a fast performance since it is not necessary to adjust weights and hidden layer biases. ELM is implemented and combined with three metaheuristic algorithms. The selected algorithms belong to the family of Swarm intelligence such as Particle Swarm Optimization (PSO), Artificial Bee Colony (ABC) and Grey Wolf Optimization (GWO); these algorithms are applied to optimize the hidden biases and input weights and are implemented for improving the performance along with techniques of Extreme learning machines in data classification. Also, the performance and behavior of each algorithm can be observed. The data employed in this work consist of information about Australian credit, diabetes detection and hearth disease. The investigation result show that GWO obtained better results in the first dataset and PSO are better in the second and third datasets.KeywordsExtreme learning machinesMetaheuristicPSOGWOABCFitness function

In this paper, we present a robust semi-blind video watermarking scheme in lifting wavelet transform (LWT) domain using Extreme Learning Machine (ELM). In this scheme, first the static video summary is generated using extraction of color features from video frames. Second, the frames comprised of video summary are watermarked in LWT domain. To develop a robust and real time watermarking scheme, a fast Single hidden Layer Feedforward Neural Network (SLFN) known as ELM is used for watermark embedding and extraction. To evaluate the performance of the present scheme, several signal processing attacks are applied to each watermarked frame. Experimental evidence shows that the proposed scheme is robust against selected attacks. Due to fastprocessing of frames, the proposed scheme is also found to be suitable for real time watermarking of video.

Extreme Learning Machine (ELM) is a single hidden layer feed-forward neural network learning method, which has a high generalization performance as well as faster. In this paper, odour data is discriminated based on the sensor response curve by using ELM, and the main objective is to investigate the optimum number of nodes in the hidden layer of ELM for olfactory detection. The relationship between the number of nodes in the hidden layer and the number of attributes or classes of dataset is queried to achieve the goal. Three odour datasets taken from different sources in literature and two transfer functions for the ELM are used to verify the results of the study. The backpropagation (BP) algorithm is also used for training an artificial neural network for comparison purposes. The analysis is performed for the three datasets by using ELM and BP and obtained results present that the time consumption of ELM is too small to be compared with BP even though the number of nodes is high and better accuracy rates are obtained by ELM.

Stream-flow forecasting using extreme learning machines: A case study in a semi-arid region in Iraq

Extreme learning machine is a novel single hidden layer feedforward neural networks with a strong abilities, for example simple net structure, fast learning speed, good generalization and so on. Aimed at the system with a delay unit, A new control strategy for internal model control is proposed to set up an inverse model of the minimal phase subsystem by using extreme learning machine with in-out system datum. Moreover, the relative stable error for internal model control system with a delay unit is presented to evaluate the system performance. The features for the internal model control system based on extreme learning machine are compared with that based on neural network. The experimental results indicate that the internal model control system based on extreme learning machine has small stable error and strong robustness.

Research community has recently put more attention to the Extreme Learning Machines (ELMs) algorithm in Neural Network (NN) area. The ELMs are much faster than the traditional gradient-descent-based learning algorithms due to its analytical determination of output weights with the random choice of input weights and hidden layer bias. However, since the input weights and bias are randomly assigned and not adjusted, the ELMs model shows an instability if we repeat the experiments many times. Such instability makes the ELMs less reliable than other computational intelligence models. In our investigation, we try to solve this problem by using the Random Production in the first layer of the ELMs. Thus, we can reduce the chance of using random weight assignment in ELMs by removing the bias in the hidden layer. Experiment son different data sets demonstrate that the proposed model has higher stability and reliability than the classical ELMs.

Extreme learning machine (ELM) is an emergent method for train- ing single hidden layer feedforward neural networks (SLFNs) with extremely fast training speed, easy implementation and good generalization performance. This work presents effective ensemble procedures for combining ELMs by exploiting di- versity. A large number of ELMs are initially trained in three different scenarios: the original feature input space, the obtained feature subset by forward selection and different random subsets of features. The best combination of ELMs is con- structed according to an exact ranking of the trained models and the useless net- works are discarded. The experimental results on several regression problems show that robust ensemble approaches that exploit diversity can effectively improve the performance compared with the standard ELM algorithm and other recent ELM extensions.

The two-dimensional morphology of the cladding layer has an important influence on the quality of the cladding layer and the crack tendency. Using the powerful nonlinear processing ability of the single hidden layer feedforward neural network, a prediction model between the cladding technological parameters and the two-dimensional morphology of the cladding layer is established. Taking the cladding parameters as the input and the two-dimensional morphology of the cladding as the output, the experimental data is used to train the network to achieve a high-level mapping of the input and output. On this basis, the algorithm of extreme learning machine is used to optimize the single hidden layer feedforward neural network to overcome the problems of slow convergence speed, more network training parameters and easy local convergence in back-propagation algorithm. The results show that the relationship between the cladding process parameters and the two-dimensional morphology of the cladding layer can be roughly reflected by the back-propagation algorithm. However, the prediction results are not stable and the error rate is between 10% and 40%. The neural network optimized by the extreme learning machine is utilized to get a better prediction result. The error rate is 10–20%.