Deep and wide feature based extreme learning machine for image classification

This paper presents a fast algorithmic method to train convolutional neural network (CNN) classifiers through extreme learning which has been verified on popular datasets on classification and pedestrian detection. CNN has been one of the best classifiers for images and object recognition. However, the Backpropagation (BP) algorithm, mostly used for training CNN, suffers from slow learning, local minimum, and poor generalization. To solve these problems, a novel architecture called CNN-ELM has been proposed here. Its core architecture is based on a local image (local receptive field) version of the ELM (Extreme Learning Machine) adopting random feature learning. Using MATLAB 2015a, classification experiments using the raw image data as input, show a comparable or mostly better classification performance compared to the BP trained CNN, with its training speed up to 200 times faster on MNIST, NORB, and CIFAR-10 datasets. Pedestrian detection experiments using INRIA datasets also exhibits much faster training than the BP trained CNN without sacrificing detection performance.

The accurate separation of ECG signals has become crucial to identify heart diseases. Machine learning methods are widely used to separate ECG signals. The aim of this study was to obtain optimal number of hidden neurons of the Extreme Learning Machine (ELM) using the differential evolution algorithm (DEA) and increase the accuracy rate of ECG classification. In this study, a public database on PhysioNet was used for ECG signal classification. A deep feature method using convolutional neural network was used to extract the major features of the ECG samples. Then, a conventional ELM was applied to the ECG signals. Subsequently, the ECG signals with deep properties were shared with the MATLAB classifier toolbox (k-NN, SVM, Decision Trees). In addition, the ECG signals in the dataset were tested using the Genetic Algorithm Wavelet Kernel-ELM (GAWK-ELM). Finally, the DEA-ELM was improved for the determination of the number of hidden neurons. This study optimized the hidden neuron numbers of traditional ELM with DEA using deep learning capabilities in the feature extraction. The aim of was to maximize the best cost of the DEA and achieve the optimal number of hidden neurons in ELM. Accuracy (Acc), sensitivity (Se), specificity (Spe) and F-measure were used as the performance metrics for the classifier performances. The classification results were 80.60%, 81.50%, and 83.12% with SVM, ELM and DEA-ELM, respectively. Thus, the best classification scores were accomplished with an accuracy of 83.12% with the algorithm supported by the DEA.

With growth of data sets, the efficiency of Extreme Learning Machine (ELM) model combined with accustomed hardware implementation such as Field-programmable gate array (FPGA) became attractive for many real-time learning tasks. In order to reduce resource occupation in eventual trained model on FPGA, it is more efficient to store fixed-point data rather than double-floating data in the on-chip RAMs. This paper conducts the fixed-point evaluation of ELM for classification. We converted the ELM algorithm into a fixed-point version by changing the operation type, approximating the complex function and blocking the large-scale matrixes, according to the architecture ELM would be implemented on FPGA. The performance of classification with single bit-width and mixed bit-width were evaluated respectively. Experimental results show that the fixed-point representation used on ELM does work for some application, while the performance could be better if we adopt mixed bit-width.

It is well-known that solar power depends on many complex parameters such as humidity, radiation, temperature, dust and wind speed. In order to cope with these complex structures and to provide an accurate forecast, the development of reliable and effective forecasting methodology is very significant. In this study, an Artificial Neural Network (ANN)-based system has been developed to forecast very short-term (2 to 4-h) power output of a grid tied Photovoltaic Power Plant (PVPP). An algorithm called Extreme Learning Machine (ELM) has gained increasing interest through its extremely fast learning and good generalization capability. The Kernel Extreme Learning Machine (KELM) which is the improved version of the ELM is proposed to develop a very short-term PVPP power forecast system. The most important feature of KELM is that it has less adjustable parameters and better generalization ability when compared with classical ELM. Experimental studies have been carried out on a grid tied PVPP that has 1 (MW) installed power capacity. The inputs of the KELM-based forecast system are selected as solar power, humidity, radiation and temperature. All data are divided into four parts to analyze the effect of seasons on the performance of the proposed forecast system. The comparison studies are carried out to clearly observe the forecast ability and performance of the KELM. From the experimental studies of the winter season which has not symmetric diurnal mean power curve, Correlation Coefficients (R) values of KELM are calculated as 0.896 for 2-h ahead, 0.865 for 3-h ahead, 0.833 for 4-h ahead. Those of ELM and (Levenberg Marquardt) LM are 0.866–0.812 for 2-h ahead, 0.831–0.669 for 3-h ahead, 0.799–0.502 for 4-h ahead, respectively. Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) values of KELM for winter season are 12.08%-6.14% for 2-h ahead, 14.06%-7.43% for 3-h ahead, 15.33%-7.74% for 4-h ahead, respectively. While these values of ELM are obtained as 13.82%-7.05% for 2-h ahead, 15.52%-8.77% for 3-h ahead, 17.63%-9.33% for 4-h ahead. The RMSE and MAE values of LM are 17.90%-9.14% for 2-h ahead, 20.04%-9.85% for 3-h ahead, 21.22%-10.68% for 4-h ahead. According to the obtained results, it is clearly seen that KELM provides a more powerful and reliable forecasting performance.

A key challenge in radar micro-Doppler classification is the difficulty in obtaining a large amount of training data due to costs in time and human resources. Small training datasets limit the depth of deep neural networks (DNNs), and, hence, attainable classification accuracy. In this work, a novel method for diversifying Kinect-based motion capture (MOCAP) simulations of human micro-Doppler to span a wider range of potential observations, e.g. speed, body size, and style, is proposed. By applying three transformations, a small set of MOCAP measurements is expanded to generate a large training dataset for network initialization of a 30-layer deep residual neural network. Results show that the proposed training methodology and residual DNN yield improved bottleneck feature performance and the highest overall classification accuracy among other DNN architectures, including transfer learning from the 1.5 million sample ImageNet database.

One of the challenging problems in understanding high-resolution remote sensing images is aerial scene classification. A well-designed feature representation method and classifier can improve classification accuracy. In this paper, we construct a new two-stream deep architecture for aerial scene classification. First, we use two pretrained convolutional neural networks (CNNs) as feature extractor to learn deep features from the original aerial image and the processed aerial image through saliency detection, respectively. Second, two feature fusion strategies are adopted to fuse the two different types of deep convolutional features extracted by the original RGB stream and the saliency stream. Finally, we use the extreme learning machine (ELM) classifier for final classification with the fused features. The effectiveness of the proposed architecture is tested on four challenging datasets: UC-Merced dataset with 21 scene categories, WHU-RS dataset with 19 scene categories, AID dataset with 30 scene categories, and NWPU-RESISC45 dataset with 45 challenging scene categories. The experimental results demonstrate that our architecture gets a significant classification accuracy improvement over all state-of-the-art references.

This paper considers the accident images and develops a deep learning method for feature extraction together with a mixture of experts for classification. For the first task, the outputs of the last max-pooling layer of a Convolution Neural Network (CNN) are used to extract the hidden features automatically. For the second task, a mixture of advanced variations of Extreme Learning Machine (ELM) including basic ELM, constraint ELM (CELM), On-Line Sequential ELM (OSELM) and Kernel ELM (KELM), is developed. This ensemble classifier combines the advantages of different ELMs using a gating network and its accuracy is very high while the processing time is close to real-time. To show the efficiency, the different combinations of the traditional feature extraction and feature selection methods and the various classifiers are examined on two kinds of benchmarks including accident images’ data set and some general data sets. It is shown that the proposed system detects the accidents with 99.31% precision, recall and F-measure. Besides, the precisions of accident-severity classification and involved-vehicle classification are 90.27% and 92.73%, respectively. This system is suitable for on-line processing on the accident images that will be captured by Unmanned Aerial Vehicles (UAV) or other surveillance systems.

Rolling element bearings constitute the key parts on rotating machinery and their fault diagnosis are of great importance. In this paper, a new intelligent fault diagnosis scheme based on Wavelet Packet Transform (WPT) and Extreme Learning Machine (ELM) is proposed. 16-dimensional wavelet packet node energies were extracted from the original datasets as the feature vector input to the classifiers. A novel classifier, ELM, and its variants, error-minimized ELM (EM-ELM) and online sequential ELM (OS-ELM), were introduced in this study to diagnose the fault on bearings. ELM has been proved to be extremely fast and can provide good generalization performance on many pattern recognition cases. However, preliminary ELM is a batch learning algorithm with a fixed network structure. EM-ELM and OS-ELM are the extends of the preliminary ELM to allow network to grow in the learning process and to learn data sequentially. ELM, EM-ELM and OS-ELM classifiers were evaluated on 13 fault datasets and the empirical results showed that they are really fast and perform well on all the 13 datasets.

In this work, we address some issues in the classification of varying length patterns of speech represented as sets of continuous-valued feature vectors using kernel methods. Kernels designed for varying length patterns are called as dynamic kernels. We propose two dynamic kernels namely segment-level pyramid match kernel (SLPMK) and segment-level probabilistic sequence kernel (SLPSK) for classification of long duration speech, represented as varying length sets of feature vectors using extreme learning machine (ELM). SLPMK and SLPSK are designed by partitioning the speech signal into increasingly finer segments and matching the corresponding segments. SLPSK is built upon a set of Gaussian basis functions, where half of the basis functions contain class-specific information while the other half implicates the common characteristics of all the speech utterances of all classes. The computational complexity of SVM training algorithms is usually intensive, which is at least quadratic with respect to the number of training examples. It is difficult to deal with the immense amount of data using traditional SVMs. For reducing the training time of classifier we propose to use a simple algorithm namely ELM. ELM refers to a wider type of generalized single hidden layer feedforward networks (SLFNs) whose hidden layer need not be tuned. In our work, we proposed to explore kernel based ELM to exploit dynamic kernels. We study the performance of the ELM-based classifiers using the proposed SLPSK and SLPMK for speech emotion recognition and speaker identification tasks and compare with other kernels for varying length patterns. Experimental studies showed that proposed ELM-based approach offer a 10–12% of relative improvement over baseline approach, and a 3–9% relative improvement over ELMs/SVMs using other state-of-the-art dynamic kernels.

The Extreme Learning Machine (ELM), recently proposed by Huang et al. [6], is a single-hidden-layered neural network architecture which has been successfully applied to nonlinear regression and classification tasks [5]. A crucial step in the design of the ELM is the computation of the output weight matrix, a step usually performed by means of the ordinary least-squares (OLS) method - a.k.a. Moore-Penrose generalized inverse technique. However, it is well-known that the OLS method produces predictive models which are highly sensitive to outliers in the data. In this paper, we develop an extension of ELM which is robust to outliers caused by labelling errors. To deal with this problem, we suggest the use of M-estimators, a parameter estimation framework widely used in robust regression, to compute the output weight matrix, instead of using the standard OLS solution. The proposed model is robust to label noise not only near the class boundaries, but also far from the class boundaries which can result from mistakes in labelling or gross errors in measuring the input features. We show the usefulness of the proposed classification approach through simulation results using synthetic and real-world data.

Data-driven rolling-bearing fault diagnosis methods are mostly based on deep-learning models, and their multilayer nonlinear mapping capability can improve the accuracy of intelligent fault diagnosis. However, problems such as gradient disappearance occur as the number of network layers increases. Moreover, directly taking the raw vibration signals of rolling bearings as the network input results in incomplete feature extraction. In order to efficiently represent the state characteristics of vibration signals in image form and improve the feature learning capability of the network, this paper proposes fault diagnosis model MTF-ResNet based on a Markov transition field and deep residual network. First, the data of raw vibration signals are augmented by using a sliding window. Then, vibration signal samples are converted into two-dimensional images by MTF, which retains the time dependence and frequency structure of time-series signals, and a deep residual neural network is established to perform feature extraction, and identify the severity and location of the bearing faults through image classification. Lastly, experiments were conducted on a bearing dataset to verify the effectiveness and superiority of the MTF-ResNet model. Features learned by the model are visualized by t-SNE, and experimental results indicate that MTF-ResNet showed better average accuracy compared with several widely used diagnostic methods.

Shortwave radiation density flux (SRDF) modeling can be key in estimating actual evapotranspiration in plants. SRDF is the result of the specific and scattered reflection of shortwave radiation by the underlying surface. SRDF can have profound effects on some plant biophysical processes such as photosynthesis and land surface energy budgets. Since it is the main energy source for most atmospheric phenomena, SRDF is also widely used in numerical weather forecasting. In the current study, an improved version of the extreme learning machine was developed for SRDF forecasting using the historical value of this variable. To do that, the SRDF through 1981–2019 was extracted by developing JavaScript-based coding in the Google Earth Engine. The most important lags were found using the auto-correlation function and defined fifteen input combinations to model SRDF using the improved extreme learning machine (IELM). The performance of the developed model is evaluated based on the correlation coefficient (R), root mean square error (RMSE), mean absolute percentage error (MAPE), and Nash–Sutcliffe efficiency (NSE). The shortwave radiation was developed for two time ahead forecasting (R = 0.986, RMSE = 21.11, MAPE = 8.68%, NSE = 0.97). Additionally, the estimation uncertainty of the developed improved extreme learning machine is quantified and compared with classical ELM and found to be the least with a value of ±3.64 compared to ±6.9 for the classical extreme learning machine. IELM not only overcomes the limitation of the classical extreme learning machine in random adjusting of bias of hidden neurons and input weights but also provides a simple matrix-based method for practical tasks so that there is no need to have any knowledge of the improved extreme learning machine to use it.

Extreme Learning Machine (ELM) is an excellent candidate for its exemplary performance as a generalized Single Layer Feed-forward Network (SLFN). In this regard, ELM has attracted widespread attention for tackling multiclass classification and regression problems with relative ease. Due to the random allocation of weights to the input and hidden layer biases, the running time spans of ELM are observed to be in seconds and agree with real-time events whilst giving a competitive performance. In this work, the suitability of ELM and its Online Sequential variant for recognition of unknown face samples is investigated under uncontrolled environments, including varying pose, variance, and illumination. The presented face recognition approach is evaluated sing four datasets from YALE, CMU, BIOID, and LFW. To track the face features of a sample face in its spatial domain with varying windows corresponding to different datasets, the Viola-Jones object detection framework is used. The face features are then extracted and evaluated using a Histogram of Oriented Gradients, which forms the dataset to be fed to ELM and OS-ELM classifiers. The running time for training the proposed face recognition approach is found to span within milliseconds to seconds with the computed time complexity of , justifying the suitability of ELM and OS-ELM classifiers for real-time face recognition applications with a high recognition rate outcome.

Brain tumors are formed due to the abnormal growth of the cells that get multiplied and become in uncontrollable perspective. Tumors can damage brain cells by pressing down the skull, which consociate begins in, which negatively affects human health. In advanced stages, a brain tumor is a more dangerous infection that cannot be relieved. Brain tumor detection and early prevention are necessary in today's world. An extreme learning machine (ELM) is a widely adopted algorithm in machine learning. It is proposed to use classification models in brain tumor imaging. This classification is based on the techniques implemented: Convolutional Neural Networks(CNN) and Generative Adversarial Networks(GAN). CNN efficiently solves the convex optimization problem and is faster, requiring less human effort. The algorithmic architecture of GAN uses two neural networks, pocking one against the other. These networks are implemented in various fields to classify brain Tumor images. The present study mainly aims to introduce a new proposed classification system of preschool children brain imaging with Hybrid Convolutional Neural Networks and with the techniques of GAN. The proposed technique is compared with the existing hybrid-CNN and hybrid-GAN techniques. The outcomes are encouraging because the loss is deduced, and the accuracy facet increases. The proposed system achieved a training accuracy of 97.8% and a validation accuracy of 89%. The outcomes of the studies show that ELM in the platform of GAN for preschool children brain imaging classification has achieved higher predictive performance than the traditional classification mechanisms in increasingly complex scenarios. Time elapsed for training brain images samples finds inference value for training samples and time elapsed value increased by 28.9855%. Approximation ratio for cost based on probability, finding Approximation ratio for low probability range is increased by 88.1%. The combination of CNN, GAN and hybrid-CNN, hybrid-GAN, and hybrid CNN + GAN, compared with the proposed hybrid system, increased Detection latency for low range learning rate by 3.31%.

IntelELM: A Python Framework for Intelligent Metaheuristic-based Extreme Learning Machine