Meta-cognitive Recurrent Recursive Kernel OS-ELM for concept drift handling

Online sequential extreme learning machine with forgetting mechanism

Concept drift (CD) in data streaming scenarios such as networking intrusion detection systems (IDS) refers to the change in the statistical distribution of the data over time. There are five principal variants related to CD: incremental, gradual, recurrent, sudden, and blip. Genetic programming combiner (GPC) classification is an effective core candidate for data stream classification for IDS. However, its basic structure relies on the usage of traditional static machine learning models that receive onetime training, limiting its ability to handle CD. To address this issue, we propose an extended variant of the GPC using three main components. First, we replace existing classifiers with alternatives: online sequential extreme learning machine (OSELM), feature adaptive OSELM (FA-OSELM), and knowledge preservation OSELM (KP-OSELM). Second, we add two new components to the GPC, specifically, a data balancing and a classifier update. Third, the coordination between the sub-models produces three novel variants of the GPC: GPC-KOS for KA-OSELM; GPC-FOS for FA-OSELM; and GPC-OS for OSELM. This article presents the first data stream-based classification framework that provides novel strategies for handling CD variants. The experimental results demonstrate that both GPC-KOS and GPC-FOS outperform the traditional GPC and other state-of-the-art methods, and the transfer learning and memory features contribute to the effective handling of most types of CD. Moreover, the application of our incremental variants on real-world datasets (KDD Cup '99, CICIDS-2017, CSE-CIC-IDS-2018, and ISCX '12) demonstrate improved performance (GPC-FOS in connection with CSE-CIC-IDS-2018 and CICIDS-2017; GPC-KOS in connection with ISCX2012 and KDD Cup '99), with maximum accuracy rates of 100% and 98% by GPC-KOS and GPC-FOS, respectively. Additionally, our GPC variants do not show superior performance in handling blip drift.

Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift

In this paper, we develop an online sequential learning algorithm for single hidden layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes in a unified framework. The algorithm is referred to as online sequential extreme learning machine (OS-ELM) and can learn data one-by-one or chunk-by-chunk (a block of data) with fixed or varying chunk size. The activation functions for additive nodes in OS-ELM can be any bounded nonconstant piecewise continuous functions and the activation functions for RBF nodes can be any integrable piecewise continuous functions. In OS-ELM, the parameters of hidden nodes (the input weights and biases of additive nodes or the centers and impact factors of RBF nodes) are randomly selected and the output weights are analytically determined based on the sequentially arriving data. The algorithm uses the ideas of ELM of Huang et al. developed for batch learning which has been shown to be extremely fast with generalization performance better than other batch training methods. Apart from selecting the number of hidden nodes, no other control parameters have to be manually chosen. Detailed performance comparison of OS-ELM is done with other popular sequential learning algorithms on benchmark problems drawn from the regression, classification and time series prediction areas. The results show that the OS-ELM is faster than the other sequential algorithms and produces better generalization performance.

In this era of big data, more and more models need to be trained to mine useful knowledge from large scale data. It has become a challenging problem to train multiple models accurately and efficiently so as to make full use of limited computing resources. As one of ELM variants, online sequential extreme learning machine (OS-ELM) provides a method to learn from incremental data. MapReduce, which provides a simple, scalable and fault-tolerant framework, can be utilized for large scale learning. In this paper, we propose an efficient parallel method for batched online sequential extreme learning machine (BPOS-ELM) training using MapReduce. Map execution time is estimated with historical statistics, where regression method and inverse distance weighted interpolation method are used. Reduce execution time is estimated based on complexity analysis and regression method. Based on the estimations, BPOS-ELM generates a Map execution plan and a Reduce execution plan. Finally, BPOS-ELM launches one MapReduce job to train multiple OS-ELM models according to the generated execution plan, and collects execution information to further improve estimation accuracy. Our proposal is evaluated with real and synthetic data. The experimental results show that the accuracy of BPOS-ELM is at the same level as those of OS-ELM and parallel OS-ELM (POS-ELM) with higher training efficiencies.

Because it is difficulty to classify level of fabric wrinkle, this paper proposes a fabric winkle level classification model via online sequential extreme learning machine based on improved sine cosine algorithm (SCA). The SCA has excellent global optimization ability, can explore different search spaces, and effectively avoid falling into local optimum. Because the initial population of SCA will have an impact on its optimization speed and quality, the SCA is initialized by differential evolution (DE) to avoid local optimization, and then the output weight and hidden layer bias are optimized; that is, the improved SCA is used to select the optimal parameters of the online sequential extreme learning machine (OSELM) to improve the generalization performance of the algorithm. To verify the performance of the proposed model DE-SCA-OSELM, it will be compared with other algorithms using a fabric wrinkles dataset collected under standard conditions. The experimental results indicate that the proposed model can effectively find the optimal parameter value of OSELM. The average classification accuracy increased by 6.95%, 3.62%, 6.67%, and 3.34%, respectively, compared with the partial algorithms OSELM, SCAELM, RVFL and PSOSVM, which meets expectations.

An evolutionary online sequential extreme learning machine for maximum power point tracking and control in multi-photovoltaic microgrid system

Air-ratio is an important engine parameter that relates closely to engine emissions, power, and brake-specific fuel consumption. Model predictive controller (MPC) is a well-known technique for air-ratio control. This paper utilizes an advanced modelling technique, called online sequential extreme learning machine (OSELM), to develop an online sequential extreme learning machine MPC (OEMPC) for air-ratio regulation according to various engine loads. The proposed OEMPC was implemented on a real engine to verify its effectiveness. Its control performance is also compared with the latest MPC for engine air-ratio control, namely diagonal recurrent neural network MPC, and conventional proportional---integral---derivative (PID) controller. Experimental results show the superiority of the proposed OEMPC over the other two controllers, which can more effectively regulate the air-ratio to specific target values under external disturbance. Therefore, the proposed OEMPC is a promising scheme to replace conventional PID controller for engine air-ratio control.

A novel compound fault-tolerant method based on online sequential extreme learning machine with cycle reservoir for turbofan engine direct thrust control

Online sequential extreme learning machine with kernels for nonstationary time series prediction

Under Gaussian assumption, online sequential extreme learning machine (OS-ELM) can achieve optimal performance. However, OS-ELM is based on the mean square error (MSE) criterion which is not a good choice for non-Gaussian signals. In this paper, a novel OS-ELM algorithm based on the generalized maximum correntropy criterion (OS-ELM-GMCC) is derived. Since the maximum correntropy criterion (MCC) can be generalized by the generalized maximum correntropy criterion (GMCC), the GMCC-based adaptive filtering algorithms with an appropriate shape parameter achieve better filtering performance than the MCC-based ones in the presence of non-Gaussian noise. As the important parameters, the number of hidden node, the shape parameter and scale parameter are discussed by simulations. Simulations in the context of the two examples including the system identification and Mackey-Glass (MG) chaotic time series demonstrate the superiority of OS-ELM-GMCC over OS-ELM and OS-ELM-MC.

This article discusses an important problem in network management and security—accurate and efficient classification of internet traffic. The need for real-time processing, as well as the complexity and everchanging nature of the network traffic, calls for both high accuracy and computational efficiency. To meet these demands, this study proposes a new approach that incorporates the Online Sequential Extreme Learning Machine (OSELM) and Wavelet Transform, which has been optimized by the Artificial Bee Cloning Algorithm. The proposed OSELM-Wavelet method uses Daubechies 4 wavelet transform to capture relevant frequency-domain features and, at the same time, preserve raw time-domain signals. This method improves the feature set by capturing time and frequency domain features simultaneously, boosting the input data by 800 features. Classification is carried out via the OSELM framework which supports efficient online training and inference and is thus suitable for real-time applications. The Artificial Bee Clony Algorithm is used to optimize the OSELM model parameters and improve the classification accuracy. This algorithm is inspired by the intelligent foraging behavior of bees and can effectively explore and exploit the parameter space for optimal solutions. The experiments bee algorithm-optimized OSELM-Wavelet conducted for internet traffic classification tasks demonstrated high accuracy and robustness.It outperforms conventional statistical and machine learning methods, particularly in situations demanding rapid adaptation and online learning. To summarize, the article combines OSELM-Wavelet with Artificial Bee Cloning Algorithm for Internet traffic classification, presenting a new solution. It aids in precisely and swiftly classifying traffic with minimal computation, thereby improving network security and management.

Letter to the editor

Computational complexity and sample selection are two main factors that limited the performance of online sequential extreme learning machines (OS-ELMs). This paper proposes a new model that introduces the concept of hybrid kernel and sample selection method based on an online learning model using a membership function. In other words, an online sequential extreme learning machine based on a hybrid kernel function (HKOS-ELM) is presented. The algorithm only calculates the kernel function to determine the final output function, mostly solving the computational complexity of the algorithm. The hybrid kernel function proposed in this paper has the advantages of strong learning ability and good generalization performance of single kernel function. Based on the classification essence of the OS-ELM classification, the membership function is introduced into the sample selection to remove the noise point and the outlier point. The experimental results showed that the HKOS-ELM algorithm adding the membership degree with mixed kernel functions preserves the advantages of kernel functions and online learning and improves the classification performance of the system.

At present, machine learning techniques is frequently used to extraction and/or prediction some information from enormous collected data. Traditional machine learning is performed batch learning like a popular one, Artificial Neural Network (ANN). That means it can’t integrate new information into already trained model but it is retrained from scratch. Online Sequential Extreme Learning Machine (OS-ELM) is a one of online incremental machine learning techniques that can learn and update model from new receive data without to retrain the model. This paper shows the result of comparison experiment between ANN and OS-ELM and found that, the OS-ELM has acceptable performance in situation of few initial data for training and/or statistic properties of data is changing while it working.