A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks

Online sequential extreme learning machine with forgetting mechanism

Online sequential extreme learning machine with kernels for nonstationary time series prediction

Under Gaussian assumption, online sequential extreme learning machine (OS-ELM) can achieve optimal performance. However, OS-ELM is based on the mean square error (MSE) criterion which is not a good choice for non-Gaussian signals. In this paper, a novel OS-ELM algorithm based on the generalized maximum correntropy criterion (OS-ELM-GMCC) is derived. Since the maximum correntropy criterion (MCC) can be generalized by the generalized maximum correntropy criterion (GMCC), the GMCC-based adaptive filtering algorithms with an appropriate shape parameter achieve better filtering performance than the MCC-based ones in the presence of non-Gaussian noise. As the important parameters, the number of hidden node, the shape parameter and scale parameter are discussed by simulations. Simulations in the context of the two examples including the system identification and Mackey-Glass (MG) chaotic time series demonstrate the superiority of OS-ELM-GMCC over OS-ELM and OS-ELM-MC.

Letter to the editor

Because it is difficulty to classify level of fabric wrinkle, this paper proposes a fabric winkle level classification model via online sequential extreme learning machine based on improved sine cosine algorithm (SCA). The SCA has excellent global optimization ability, can explore different search spaces, and effectively avoid falling into local optimum. Because the initial population of SCA will have an impact on its optimization speed and quality, the SCA is initialized by differential evolution (DE) to avoid local optimization, and then the output weight and hidden layer bias are optimized; that is, the improved SCA is used to select the optimal parameters of the online sequential extreme learning machine (OSELM) to improve the generalization performance of the algorithm. To verify the performance of the proposed model DE-SCA-OSELM, it will be compared with other algorithms using a fabric wrinkles dataset collected under standard conditions. The experimental results indicate that the proposed model can effectively find the optimal parameter value of OSELM. The average classification accuracy increased by 6.95%, 3.62%, 6.67%, and 3.34%, respectively, compared with the partial algorithms OSELM, SCAELM, RVFL and PSOSVM, which meets expectations.

In this era of big data, more and more models need to be trained to mine useful knowledge from large scale data. It has become a challenging problem to train multiple models accurately and efficiently so as to make full use of limited computing resources. As one of ELM variants, online sequential extreme learning machine (OS-ELM) provides a method to learn from incremental data. MapReduce, which provides a simple, scalable and fault-tolerant framework, can be utilized for large scale learning. In this paper, we propose an efficient parallel method for batched online sequential extreme learning machine (BPOS-ELM) training using MapReduce. Map execution time is estimated with historical statistics, where regression method and inverse distance weighted interpolation method are used. Reduce execution time is estimated based on complexity analysis and regression method. Based on the estimations, BPOS-ELM generates a Map execution plan and a Reduce execution plan. Finally, BPOS-ELM launches one MapReduce job to train multiple OS-ELM models according to the generated execution plan, and collects execution information to further improve estimation accuracy. Our proposal is evaluated with real and synthetic data. The experimental results show that the accuracy of BPOS-ELM is at the same level as those of OS-ELM and parallel OS-ELM (POS-ELM) with higher training efficiencies.

Multi-sensor real-time monitoring of dam behavior using self-adaptive online sequential learning

An evolutionary online sequential extreme learning machine for maximum power point tracking and control in multi-photovoltaic microgrid system

Air-ratio is an important engine parameter that relates closely to engine emissions, power, and brake-specific fuel consumption. Model predictive controller (MPC) is a well-known technique for air-ratio control. This paper utilizes an advanced modelling technique, called online sequential extreme learning machine (OSELM), to develop an online sequential extreme learning machine MPC (OEMPC) for air-ratio regulation according to various engine loads. The proposed OEMPC was implemented on a real engine to verify its effectiveness. Its control performance is also compared with the latest MPC for engine air-ratio control, namely diagonal recurrent neural network MPC, and conventional proportional---integral---derivative (PID) controller. Experimental results show the superiority of the proposed OEMPC over the other two controllers, which can more effectively regulate the air-ratio to specific target values under external disturbance. Therefore, the proposed OEMPC is a promising scheme to replace conventional PID controller for engine air-ratio control.

Meta-cognitive Recurrent Recursive Kernel OS-ELM for concept drift handling

This article discusses an important problem in network management and security—accurate and efficient classification of internet traffic. The need for real-time processing, as well as the complexity and everchanging nature of the network traffic, calls for both high accuracy and computational efficiency. To meet these demands, this study proposes a new approach that incorporates the Online Sequential Extreme Learning Machine (OSELM) and Wavelet Transform, which has been optimized by the Artificial Bee Cloning Algorithm. The proposed OSELM-Wavelet method uses Daubechies 4 wavelet transform to capture relevant frequency-domain features and, at the same time, preserve raw time-domain signals. This method improves the feature set by capturing time and frequency domain features simultaneously, boosting the input data by 800 features. Classification is carried out via the OSELM framework which supports efficient online training and inference and is thus suitable for real-time applications. The Artificial Bee Clony Algorithm is used to optimize the OSELM model parameters and improve the classification accuracy. This algorithm is inspired by the intelligent foraging behavior of bees and can effectively explore and exploit the parameter space for optimal solutions. The experiments bee algorithm-optimized OSELM-Wavelet conducted for internet traffic classification tasks demonstrated high accuracy and robustness.It outperforms conventional statistical and machine learning methods, particularly in situations demanding rapid adaptation and online learning. To summarize, the article combines OSELM-Wavelet with Artificial Bee Cloning Algorithm for Internet traffic classification, presenting a new solution. It aids in precisely and swiftly classifying traffic with minimal computation, thereby improving network security and management.

At present, machine learning techniques is frequently used to extraction and/or prediction some information from enormous collected data. Traditional machine learning is performed batch learning like a popular one, Artificial Neural Network (ANN). That means it can’t integrate new information into already trained model but it is retrained from scratch. Online Sequential Extreme Learning Machine (OS-ELM) is a one of online incremental machine learning techniques that can learn and update model from new receive data without to retrain the model. This paper shows the result of comparison experiment between ANN and OS-ELM and found that, the OS-ELM has acceptable performance in situation of few initial data for training and/or statistic properties of data is changing while it working.

This study proposes a solution for the issue of the low classification accuracy of clothing images. Using Fashion-MNIST as the clothing image dataset, we propose a clothing image classification technology based on an online sequential extreme learning machine (OSELM) optimised by the dragonfly algorithm (DA). First, we transform the Fashion-MNIST dataset into a data set that we extract from the corresponding grey image. Then, considering that the input weight and hidden layer bias of an OSELM are generated randomly, a DA is proposed to optimise the input weight and hidden layer bias of the OSELM to reduce the influence of random generation on the classification results. Finally, the optimised OSELM is applied to the clothing image classification. Compared to the other seven types of classification algorithms, the proposed clothing image classification model with the DA-optimised OSELM reached 93.98% accuracy when it contained 350 hidden nodes. Its performance was superior to other algorithms that were configured with the same number of hidden nodes. From a stability analysis of the box-plot, it was found that there were no outliers exhibited by the DA-OSELM model, whereas some other models had outliers or had lower stability compared to the model proposed, thereby validating the efficacy of the solution proposed.

Online-sequential extreme learning machine (OS-ELM) shows a good solution to online learning using extreme learning machine approach for single-hidden-layer feedforward network. However, the algorithm tends to be data-dependent, i.e. the bias values need to be adjusted depending on each particular problem. In this paper, we propose an enhancement to OS-ELM, which is referred to as robust OS-ELM (ROS-ELM). ROS-ELM has a systematic method to select the bias that allows the bias to be selected following the input weights. Hence, the proposed algorithm works well for every benchmark dataset. ROS-ELM has all the pros of OS-ELM, i.e. the capable of learning one-by-one, chunk-by-chunk with fixed or varying chunk size. Moreover, the performance of the algorithm is higher than OS-ELM and it produces a better generalization performance with benchmark datasets.

Extreme machine learning and its variants have shown good generalization performance and high leaning speed in many applications through its fast convergence. Despite the parallel and distributed ELM on MapReduce framework able to handle very large scale dataset for bigdata applications, the process of coping up with the rapidly updating data is a challenging one. Among the unified algorithms, the ELM with kernel uses kernels instead of random feature mappings. After, analyzing the property of ELM, it is observed that its most expensive computational part is matrix multiplication and matrix inversion. With the exponentially increasing volume of data, the matrix operations cannot be directly implemented on MapReduce. This paper proposes a novel approach, online sequential ELM with kernel (OS-ELM-Ker) based on sparsification criteria. Consequently, the efficient learning of frequently updated massive dataset is obtained. Based on the extensive experiments on synthetic dataset, it is observed that the proposed OS-ELM-Ker is highly efficient in classifying massive rapidly updated training dataset in terms of generalizations error and training time.