ELM-MC: multi-label classification framework based on extreme learning machine

In previous work, we proposed a neural network approach to estimate the blocking probability of optical networks with fixed routing. The neural network was implemented by the extreme learning machine (ELM) framework, in which the training inputs were the optical network parameters, and the output was the overall blocking probability. The numerical results showed that the neural-network-based estimation was accurate and thousands of times faster than a computer simulation. In this paper, we apply the neural network approach to optical circuit switching (OCS) networks with fixed-alternate routing and improve the training method by using an enhancement of ELM framework. Unlike the previous ELM framework, the enhancement of ELM framework provides a random-search-based selection phase for the hidden nodes during the training step. As a result, similar performance can be achieved using fewer hidden nodes than the previous ELM framework. The numerical results show that the new enhancement of ELM training algorithm provides more accurate blocking probability estimates while reducing the required number of hidden nodes by a third compared with the previous ELM training algorithm. Furthermore, for some light traffic loading situations, our new training algorithm is hundreds of times more accurate than the existing well-known analytical approximation method.

Multi-site structural damage identification using a multi-label classification scheme of machine learning

In the paper, the multi-class pattern classification using extreme learning machine (ELM) is studied. The study is based on either a series of ELM binary classifiers or a single ELM classifier. When using binary ELM classifiers, the multi-class problem is decomposed into two-class problem using the one-against-all (OAA) and one-against-one (OAO) schemes, which are named as ELM-OAA and ELM-OAO respectively for brevity. In a single ELM classifier, the multi-class problem is implemented with an architecture of multi-output nodes which is equal to the number of pattern classes. Their performance is evaluated using some multi-class benchmark problems and simulation results show that ELM-OAA and ELM-OAO requires fewer hidden nodes than the single ELM classifier. In addition ELM-OAO usually has similar or less computation burden than the single ELM classifier when the pattern class labels is not larger than 10.

Multi-label classification (MLC) deals with the task of assigning an instance to all its relevant classes. This task becomes challenging in the presence of the label dependencies. The MLC methods that assume label independence do not use the dependencies among labels. We present a two-stage framework which improves the performance of MLC by using label dependencies. In the first stage, a standard MLC method is used to get the confidence scores for different labels. A conditional random field (CRF) is used in the second stage that improves the performance of the first-stage MLC by using the label dependencies among labels. An optimization-based framework is used to learn the structure and parameters of the CRF. Experiments show that the proposed model performs better than the state-of-the-art methods for MLC.

Extreme learning machine (ELM) is a well-known algorithm for single layer feedforward neural networks (SLFNs) and their learning speed is faster than traditional gradient-based neural networks. However, many of the tasks that ELM focuses on are single-label, where an instance of the input set is associated with one label. This paper proposes a new method for training ELM that will be capable of multi-label classification using the Canonical Correlation Analysis (CCA). The new method is named CCA-ELM. There are 4 steps in the training process: the first step is to compute any correlations between the input features and the set of labels using CCA, the second step maps the input space and label space to the new space, the third step uses ELM to classify and the last step is to map to the original input space. The experimental results show that CCA-ELM can improve ELM for classification on multi-label learning and its recognition performances are better than the other comparative algorithms that use the same standard CCA.

In this paper, a progressive learning algorithm for multi-label classification to learn new labels while retaining the knowledge of previous labels is designed. New output neurons corresponding to new labels are added and the neural network connections and parameters are automatically restructured as if the label has been introduced from the beginning. This work is the first of the kind in multi-label classifier for class-incremental learning. It is useful for real-world applications such as robotics where streaming data are available and the number of labels is often unknown. Based on the Extreme Learning Machine framework, a novel universal classifier with plug and play capabilities for progressive multi-label classification is developed. Experimental results on various benchmark synthetic and real datasets validate the efficiency and effectiveness of our proposed algorithm.

Large-scale multi-label classification using unknown streaming images

Machine learning involves the task of training systems to be able to make decisions without being explicitly programmed. Important among machine learning tasks is classification involving the process of training machines to make predictions from predefined labels. Classification is broadly categorized into three distinct groups: single-label (SL), multi-class, and multi-label (ML) classification. This research work presents an application of a multi-label classification (MLC) technique in automating Quranic verses labeling. MLC has been gaining attention in recent years. This is due to the increasing amount of works based on real-world classification problems of multi-label data. In traditional classification problems, patterns are associated with a single-label from a set of disjoint labels. However, in MLC, an instance of data is associated with a set of labels. In this paper, three standard <em>MLC</em> methods: <span>binary relevance (BR), classifier chain (CC), and label powerset (LP) algorithms are implemented with four baseline classifiers: support vector machine (SVM), naïve Bayes (NB), k-nearest neighbors (k-NN), and J48. The research methodology adopts the multi-label problem transformation (PT) approach. The results are validated using six conventional performance metrics. These include: hamming loss, accuracy, one error, micro-F1, macro-F1, and avg. precision. From the results, the classifiers effectively achieved above 70% accuracy mark. Overall, SVM achieved the best results with CC and LP algorithms.</span>

We consider the problem of intrusion detection in a computer network, and investigate the use of extreme learning machines (ELMs) to classify and detect the intrusions. With increasing connectivity between networks, the risk of information systems to external attacks or intrusions has increased tremendously. Machine learning methods like support vector machines (SVMs) and neural networks have been widely used for intrusion detection. These methods generally suffer from long training times, require parameter tuning, or do not perform well in multi-class classification. We propose a basic ELM method based on random features, and a kernel based ELM method for classification. We compare our methods with commonly used SVM techniques in both binary and multi-class classifications. Simulation results show that the proposed basic ELM approach outperforms SVM in training and testing speed, while the proposed kernel based ELM achieves higher detection accuracy than SVM in multi-class classification case.

Ensemble of kernel extreme learning machine based elimination optimization for multi-label classification

Multi-label classification is a vital problem, as it has numerous applications in computer vision, such as automatic image annotation. The label set for each instance is always assumed to be in the original whole form. However, missing labels often occur because manual labelling is a time-consuming and label-intensive work in the case of large amount of data. The incompleteness of labels can certainly increase the difficulty of training the multi-label model. In this paper, a novel multi-label classification method is proposed that can learn the inductive classifier while explicitly dealing with missing labels. An individual sparsity inducing l1-norm is employed to capture the sparse label interdependencies. A group sparsity inducing l2,1-norm is utilized to select the discriminative input features. The semantic label hierarchy is included to diversify the label dependency. Meanwhile, the consistency between the predicted labels and the original labels as well as the regularization of smoothness on the predicted labels are also enforced to improve the classification performance. Furthermore, an efficient method based on the alternating direction method of multipliers is designed to facilitate classifier and label correlation learning process. Experiments on two widely used large-scale image datasets demonstrate that the efficacy of the proposed method on multi-label classification when only a limited number of labels are given for each training sample.

Complex power quality disturbances (CPQDs) classification can be regarded as a typical application of multi-label (ML) learning. In this study, we propose a new recognition method for CPQDs based on S-transform (ST) and a hybrid kernel function-based extreme learning machine (ELM) for ML learning (HKEML). The signal processing techniques S-transform is utilized to extract the distinctive features of the CPQDs. A novel ML classifier called HKEML is constructed by combining hybrid kernel function-based multiclass ELM and a thresholding learning method-based kernel ELM. Finally, a test study was conducted using Matlab synthetic signals and real signals sampled from a three-phase standard source under different noise conditions. Compared with several recent state-of-the-art ML learning algorithms, HKEML achieved better classification performance but with greatly superior computational speed.

Extreme learning machine (ELM) was extended from the generalized single hidden layer feedforward networks where the input weights of the hidden layer nodes can be assigned randomly. It has been widely used for its much faster learning speed and less manual works. Considering the field of multi-label text classification, in this paper, we propose an ELM based algorithm combined with \(L_{21}\)-norm minimization of the output weights matrix called \(L_{21}\)-norm Minimization ELM, which not only fully inherits the merits of ELM but also facilitates group sparsity and reduces complexity of the learning model. Extensive experiments on several benchmark data sets show a more desirable performance compared with other common multi-label classification algorithms.

Gas turbine combustor anomaly detection plays a critical role in reducing operation and maintenance costs in power plant operations. One-class classification, to learn a model that properly describes the normal samples, is one of many anomaly detection approaches that have shown promising performance in real-world applications. Extreme learning machines (ELMs), a recent developed machine learning technique, have outperformed SVMs and other machine learning methods for many machine learning problems. ELMs have been originally used for binary or multiclass classification and regression as well. Using ELM for one-class classification has emerged very recently, but has not been widely explored for real world applications. In this paper, we adopt ELMs to a new application domain - industrial machine condition monitoring. More specifically, we apply one-class ELMs for more accurate anomaly detection of gas turbine combustors. We evaluate different one-class ELMs and compare them against other one-class classifiers. Using a real-world gas turbine combustor anomaly detection as the case study, we demonstrate that one-class ELMs can be more effective than other one-class classification algorithms in early detecting gas turbine combustor anomalies.

Multi label classification is concerned with learning from a set of instances that are associated with a set of labels, that is, an instance could be associated with multiple labels at the same time. This task occurs frequently in application areas like text categorization, multimedia classification, bioinformatics, protein function classification and semantic scene classification. Current multi-label classification methods could be divided into two categories. The first is called problem transformation methods, which transform multi-label classification problem into single label classification problem, and then apply any single label classifier to solve the problem. The second category is called algorithm adaptation methods, which adapt an existing single label classification algorithm to handle multi-label data. In this paper, we propose a multi-label classification approach based on correlations among labels that use both problem transformation methods and algorithm adaptation methods. The approach begins with transforming multi-label dataset into a single label dataset using least frequent label criteria, and then applies the PART algorithm on the transformed dataset. The output of the approach is multi-labels rules. The approach also tries to get benefit from positive correlations among labels using predictive Apriori algorithm. The proposed approach has been evaluated using two multi-label datasets named (Emotions and Yeast) and three evaluation measures (Accuracy, Hamming Loss, and Harmonic Mean). The experiments showed that the proposed approach has a fair accuracy in comparison to other related methods.