GPU‐accelerated and mixed norm regularized online extreme learning machine

The smart grid is vulnerable to network attacks, thus requiring a high detection rate and fast detection speed for intrusion detection systems. With a fast training speed and a strong model generalization ability, the extreme learning machine (ELM) perfectly meets the needs of intrusion detection of the smart grid. In this paper, the ELM is applied to the field of smart grid intrusion detection. Aiming at the problem that the randomness of input weights and hidden layer bias in the ELM cannot guarantee the optimal performance of the ELM intrusion detection model, a genetic algorithm (GA)-ELM algorithm based on a genetic algorithm (GA) is proposed. GA is used to optimize the input weight and hidden layer bias of the ELM. Firstly, the input weight and hidden layer bias of the ELM are mapped to the chromosome vector of a GA, and the test error of the ELM model is set as the fitness function of the GA. Then, the parameters of the ELM intrusion detection model are optimized by genetic operation; the input weight and bias, corresponding to the minimum test error, are selected to improve the performance of the ELM model. Compared with the ELM and online sequential extreme learning machine (OS-ELM), the GA-ELM effectively improves the accuracy, detection rate and precision of intrusion detection and reduces the false positive rate and missing report rate.

Rolling element bearings constitute the key parts on rotating machinery and their fault diagnosis are of great importance. In this paper, a new intelligent fault diagnosis scheme based on Wavelet Packet Transform (WPT) and Extreme Learning Machine (ELM) is proposed. 16-dimensional wavelet packet node energies were extracted from the original datasets as the feature vector input to the classifiers. A novel classifier, ELM, and its variants, error-minimized ELM (EM-ELM) and online sequential ELM (OS-ELM), were introduced in this study to diagnose the fault on bearings. ELM has been proved to be extremely fast and can provide good generalization performance on many pattern recognition cases. However, preliminary ELM is a batch learning algorithm with a fixed network structure. EM-ELM and OS-ELM are the extends of the preliminary ELM to allow network to grow in the learning process and to learn data sequentially. ELM, EM-ELM and OS-ELM classifiers were evaluated on 13 fault datasets and the empirical results showed that they are really fast and perform well on all the 13 datasets.

In recent years, as a result of climate change as well as rainfall reduction in arid and semi‐arid regions, modelling qualitative and quantitative parameters belonging to aquifers has become crucially important. In Iran, as aquifers are treated as the most commonly used drinking water resources, modelling their qualitative and quantitative parameters is enormously important. In this paper, for the first time, values of salinity, total dissolved solids (TDS), groundwater level (GWL) and electrical conductivity (EC) of the Arak Plain, located in Markazi Province, Iran, are simulated by means of four modern artificial intelligence models including extreme learning machine (ELM), wavelet extreme learning machine (WELM), online sequential extreme learning machine (OSELM) and wavelet online sequential extreme learning machine (WOSELM) as well as the MODFLOW software for a 15‐year period monthly. To develop the hybrid artificial intelligence models, the wavelet is employed. First, the effective lags in estimating the qualitative and quantitative parameters of the groundwater are identified using the autocorrelation function (ACF) and the partial autocorrelation function (PACF) analysis. After that, four different models are developed by the selected input combinations and also the ACF and the PACF in the form of different lags for each of ELM, WAELM, OSELM and WOSELM methods. Then, the superior models in simulating the groundwater qualitative and qualitative parameters are detected by conducting a sensitivity analysis. To forecast the electrical conductivity (EC) by the best WOSELM model, the values of the Nash–Sutcliffe efficiency coefficient (NSC), Mean Absolute Error (MAE) and the scatter index (SI) are obtained to be 0.991, 18.005 and 4.28E‐03, respectively. In addition, the most effective lags in estimating these parameters are introduced. Subsequently, the results found by the MODFLOW model are compared with those of the artificial intelligence models and it is concluded that the latter are more accurate. For instance, the scatter index and Nash–Sutcliffe efficiency coefficient values calculated by WOSELM for TDS, respectively, are 5.34E‐03 and 0.991. Finally, an uncertainty analysis is conducted to evaluate the performance of different numerical models. For example, MODFLOW has an underestimated performance in simulating the salinity parameter.

Online-sequential extreme learning machine (OS-ELM) shows a good solution to online learning using extreme learning machine approach for single-hidden-layer feedforward network. However, the algorithm tends to be data-dependent, i.e. the bias values need to be adjusted depending on each particular problem. In this paper, we propose an enhancement to OS-ELM, which is referred to as robust OS-ELM (ROS-ELM). ROS-ELM has a systematic method to select the bias that allows the bias to be selected following the input weights. Hence, the proposed algorithm works well for every benchmark dataset. ROS-ELM has all the pros of OS-ELM, i.e. the capable of learning one-by-one, chunk-by-chunk with fixed or varying chunk size. Moreover, the performance of the algorithm is higher than OS-ELM and it produces a better generalization performance with benchmark datasets.

This article presents hourly load forecasting by using an incremental learning model called Online Sequential Extreme Learning Machine (OS-ELM), which can learn and adapt automatically according to new arrival input. However, the use of OS-ELM requires a sufficient amount of initial training sample data, which makes OS-ELM inoperable if sufficiently accurate sample data cannot be obtained. To solve this problem, a synthesis of the initial training sample is proposed. The synthesis of the initial sample is achieved by taking the first data received at the start of working and adding random noises to that data to create new and sufficient samples. Then the synthesis samples are used to initial train the OS-ELM. This proposed method is compared with Fully Online Extreme Learning Machine (FOS-ELM), which is an incremental learning model that also does not require the initial training samples. Both the proposed method and FOS-ELM are used for hourly load forecasting from the Hourly Energy Consumption dataset. Experiments have shown that the proposed method with a wide range of noise levels, can forecast hourly load more accurately than the FOS-ELM.

Comparison of extreme learning machine models for gasoline octane number forecasting by near-infrared spectra analysis

Online sequential extreme learning machine with forgetting mechanism

In this paper, we develop an online sequential learning algorithm for single hidden layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes in a unified framework. The algorithm is referred to as online sequential extreme learning machine (OS-ELM) and can learn data one-by-one or chunk-by-chunk (a block of data) with fixed or varying chunk size. The activation functions for additive nodes in OS-ELM can be any bounded nonconstant piecewise continuous functions and the activation functions for RBF nodes can be any integrable piecewise continuous functions. In OS-ELM, the parameters of hidden nodes (the input weights and biases of additive nodes or the centers and impact factors of RBF nodes) are randomly selected and the output weights are analytically determined based on the sequentially arriving data. The algorithm uses the ideas of ELM of Huang et al. developed for batch learning which has been shown to be extremely fast with generalization performance better than other batch training methods. Apart from selecting the number of hidden nodes, no other control parameters have to be manually chosen. Detailed performance comparison of OS-ELM is done with other popular sequential learning algorithms on benchmark problems drawn from the regression, classification and time series prediction areas. The results show that the OS-ELM is faster than the other sequential algorithms and produces better generalization performance.

This article discusses an important problem in network management and security—accurate and efficient classification of internet traffic. The need for real-time processing, as well as the complexity and everchanging nature of the network traffic, calls for both high accuracy and computational efficiency. To meet these demands, this study proposes a new approach that incorporates the Online Sequential Extreme Learning Machine (OSELM) and Wavelet Transform, which has been optimized by the Artificial Bee Cloning Algorithm. The proposed OSELM-Wavelet method uses Daubechies 4 wavelet transform to capture relevant frequency-domain features and, at the same time, preserve raw time-domain signals. This method improves the feature set by capturing time and frequency domain features simultaneously, boosting the input data by 800 features. Classification is carried out via the OSELM framework which supports efficient online training and inference and is thus suitable for real-time applications. The Artificial Bee Clony Algorithm is used to optimize the OSELM model parameters and improve the classification accuracy. This algorithm is inspired by the intelligent foraging behavior of bees and can effectively explore and exploit the parameter space for optimal solutions. The experiments bee algorithm-optimized OSELM-Wavelet conducted for internet traffic classification tasks demonstrated high accuracy and robustness.It outperforms conventional statistical and machine learning methods, particularly in situations demanding rapid adaptation and online learning. To summarize, the article combines OSELM-Wavelet with Artificial Bee Cloning Algorithm for Internet traffic classification, presenting a new solution. It aids in precisely and swiftly classifying traffic with minimal computation, thereby improving network security and management.

In this era of big data, more and more models need to be trained to mine useful knowledge from large scale data. It has become a challenging problem to train multiple models accurately and efficiently so as to make full use of limited computing resources. As one of ELM variants, online sequential extreme learning machine (OS-ELM) provides a method to learn from incremental data. MapReduce, which provides a simple, scalable and fault-tolerant framework, can be utilized for large scale learning. In this paper, we propose an efficient parallel method for batched online sequential extreme learning machine (BPOS-ELM) training using MapReduce. Map execution time is estimated with historical statistics, where regression method and inverse distance weighted interpolation method are used. Reduce execution time is estimated based on complexity analysis and regression method. Based on the estimations, BPOS-ELM generates a Map execution plan and a Reduce execution plan. Finally, BPOS-ELM launches one MapReduce job to train multiple OS-ELM models according to the generated execution plan, and collects execution information to further improve estimation accuracy. Our proposal is evaluated with real and synthetic data. The experimental results show that the accuracy of BPOS-ELM is at the same level as those of OS-ELM and parallel OS-ELM (POS-ELM) with higher training efficiencies.

Computational complexity and sample selection are two main factors that limited the performance of online sequential extreme learning machines (OS-ELMs). This paper proposes a new model that introduces the concept of hybrid kernel and sample selection method based on an online learning model using a membership function. In other words, an online sequential extreme learning machine based on a hybrid kernel function (HKOS-ELM) is presented. The algorithm only calculates the kernel function to determine the final output function, mostly solving the computational complexity of the algorithm. The hybrid kernel function proposed in this paper has the advantages of strong learning ability and good generalization performance of single kernel function. Based on the classification essence of the OS-ELM classification, the membership function is introduced into the sample selection to remove the noise point and the outlier point. The experimental results showed that the HKOS-ELM algorithm adding the membership degree with mixed kernel functions preserves the advantages of kernel functions and online learning and improves the classification performance of the system.

Online sequential extreme learning machine with kernels for nonstationary time series prediction

Letter to the editor

This paper develops a batch-to-batch iterative learning control (ILC) strategy based on online sequential extreme learning machine (OS-ELM) for batch optimal control. On the basis of extreme learning machine (ELM), a data-based nonlinear model is first adopted to capture the batch process characteristics aiming to obtain superior predictive accuracy. Subsequently, due to the model−plant mismatch in real batch processes, an ILC algorithm with adjusting input trajectory by means of error feedback is employed focusing on the improvement of the final product quality. In order to cope with the problems of the unknown disturbances and process variations from batch to batch, when a batch run is completed, OS-ELM is utilized to update the model weights so as to guarantee the precision of the model for optimal control, which corresponds to a nonlinear updating procedure. The feasibility and effectiveness of the proposed method are demonstrated via the application to a simulated bulk polymerization of the styrene batch process, and the simulation results show superior performance.

Hot metal silicon content is an important indicator of measuring the smooth operation of the blast furnace. However, the hot metal silicon content cannot be directly detected online. Hence, this paper establishes a prediction model of the hot metal silicon content based on multi-layer online sequential extreme learning machine (ML-OSELM). ML-OSELM is an online sequential version of multi-layer extreme learning machine (ML-ELM), and it can learn data one-by-one or chunk-by-chunk. It uses online sequential extreme learning machine auto-encoder (OS-ELM-AE) to perform layer-by-layer unsupervised sequential learning. The prediction model of the hot metal silicon content based on ML-OSELM is verified by using the production data of the blast furnace. Simulation results demonstrate that this prediction model can accurately and quickly predict the hot metal silicon content. Moreover, this prediction model can provide efficient decision for subsequent operation of the blast furnace.