Regularised extreme learning machine with misclassification cost and rejection cost for gene expression data classification.

Real-time transient stability status prediction (RTSSP) is very important to maintain the safety and stability of electrical power systems, where any unstable contingency will be likely to cause large-scale blackout. Most of machine learning methods used for RTSSP attempt to attain a low classification error, which implies that the misclassification costs of different categories are the same. However, misclassifying an unstable case as stable one usually leads to much higher costs than misclassifying a stable case as unstable one. In this paper, a new RTSSP method based on cost-sensitive extreme learning machine (CELM) is proposed, which recognizes the RTSSP as a cost-sensitive classification problem. The CELM is constructed pursuing the minimum misclassification costs, and its detailed implementation procedures for RSSTP are also researched in this work. The proposed method is implemented on the New England 39-bus electrical power system. Compared with three cost-blind methods (ELM, SVM and DT) and two cost-sensitive methods (cost-sensitive DT, cost-sensitive SVM), the simulation results have proved that the lower total misclassification costs and false dismissal rate with low computational complexity can be achieved by the proposed method, which meets the demands for the computation speed and the reliability of RTSSP.

Recently, multilayer extreme learning machine with subnetwork nodes and cost sensitive learning was applied to representation learning and classification. However, the existing ML-ELM methods suffer from several drawbacks: 1) manual tuning for the number of hidden nodes in each layer will influent the training speed of the model, the best optimal determine approach for the number of hidden nodes is still for probing; 2) random projection of the input weight and bias in each layer still suboptimal and the network structure hard to reach best generalization performance. Inspired by the Subnetwork nodes and cost sensitive methods, using multiple hidden output matrix, a novel ELM method which called Multilayer Incremental Hybrid Cost Sensitive Extreme Learning Machine with Multiple Hidden Output Matrix and Subnetwork Hidden Nodes is proposed in this paper. Whose contributions are 1) a hybrid hidden nodes optimization methods is given which based on ant clone method and multiple grey wolf optimization method; 2) a multiple hidden layer output matrices are being using through weighted calculation of different output matrices which can optimize the network architecture effectively; 3) a novel ELM structure is proposed by merging the merits of Cost Sensitive Extreme Learning Machine and Subnetwork Hidden Nodes, while the regularized parameter $C$ is not sensitive to the generalization performance. Extensive experiments results show that the proposed method achieves better performance than existing state-of-art ELM methods.

ELM (Extreme learning machine) has drawn great attention due its high training speed and outstanding generalization performance. To solve the problem that the long training time of kernel ELM auto-encoder and the difficult setting of the weight of kernel function in the existing multi-kernel models, a multi-kernel cost-sensitive ELM method based on expectation kernel auto-encoder is proposed. Firstly, from the view of similarity, the reduced kernel auto-encoder is defined by randomly selecting the reference points from the input data; then, the reduced expectation kernel auto-encoder is designed according to the expectation kernel ELM, and the combination of random mapping and similarity mapping is realized. On this basis, two multi-kernel ELM models are designed, and the output of the classifier is converted into posterior probability. Finally, the cost-sensitive decision is realized based on the minimum risk criterion. The experimental results on the public and realistic datasets verify the effectiveness of the method.

In such real data mining applications as medical diagnosis, fraud detection and fault classification, and so on, the two problems that the error cost is expensive and the reject cost is class-dependent are often encountered. In order to overcome those problems, firstly, the general mathematical description of the Binary Classification Problem with Error Cost and Class-dependent Reject Cost (BCP-EC <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> RC) is proposed. Secondly, as one of implementation methods of BCP-EC <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> RC, the new algorithm, named as Cost-sensitive Support Vector Machines with the Error Cost and the Class-dependent Reject Cost (CSVM-EC <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> RC), is presented. The CSVM-EC <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> RC algorithm involves two stages: estimating the classification reliability based on trained SVM classifier, and determining the optimal reject rate of positive class and negative class by minimizing the average cost based on the given error cost and class-dependent reject cost. The experiment studies based on a benchmark data set illustrate that the proposed algorithm is effective.

In such real data mining applications as medical diagnosis, fraud detection and fault classification, and so on, the two problems that the error cost is expensive and the reject cost is class-dependent are often encountered. In order to overcome those problems, firstly, the general mathematical description of the Binary Classification Problem with Error Cost and Class-dependent Reject Cost (BCP-EC2RC) is proposed. Secondly, as one of implementation methods of BCP-EC2RC, the new algorithm, named as Cost-sensitive Support Vector Machines with the Error Cost and the Class-dependent Reject Cost (CSVM-EC2RC), is presented. The CSVM-EC2RC algorithm involves two stages: estimating the classification reliability based on trained SVM classifier, and determining the optimal reject rate of positive class and negative class by minimizing the average cost based on the given error cost and class-dependent reject cost. The experiment studies based on a benchmark data set illustrate that the proposed algorithm is effective.

Abstract: Decision tree induction has been widely studied and applied. In safety applications, such as determining whether a chemical process is safe or whether a person has a medical condition, the cost of misclassification in one of the classes is significantly higher than in the other class. Several authors have tackled this problem by developing cost-sensitive decision tree learning algorithms or have suggested ways of changing the distribution of training examples to bias the decision tree learning process so as to take account of costs. A prerequisite for applying such algorithms is the availability of costs of misclassification. Although this may be possible for some applications, obtaining reasonable estimates of costs of misclassification is not easy in the area of safety. This paper presents a new algorithm for applications where the cost of misclassifications cannot be quantified, although the cost of misclassification in one class is known to be significantly higher than in another class. The algorithm utilizes linear discriminant analysis to identify oblique relationships between continuous attributes and then carries out an appropriate modification to ensure that the resulting tree errs on the side of safety. The algorithm is evaluated with respect to one of the best known cost-sensitive algorithms (ICET), a well-known oblique decision tree algorithm (OC1) and an algorithm that utilizes robust linear programming.

The study was intended to eliminate the noise in three-dimensional transvaginal ultrasound (3D-TVS) images and improve the diagnostic accuracy in intrauterine adhesion (IUA). The extreme learning machine (ELM) algorithm was introduced first for statement. One hundred and thirty cases of suspected IUA patients were taken as the research subjects. The denoising effects of ELM algorithm were evaluated in terms of mean square errors (MSE), peak signal-to-noise ratio (PSNR), and running time, and its diagnostic efficiency of IUA was identified from precise, specificity, and sensitivity. Furthermore, the support vector machine (SVM) algorithm was introduced for comparison. It was found that the MSE and PSNR of the ELM algorithm were 0.0021 and 64.5, respectively, and its average operation time was 11.22 ± 0.89s, that the MSE values of SVM algorithm and ELM algorithm were 0.0045 and 0.0021 and the PSNR values were 52.3 and 64.5, respectively, and that the average running time of SVM algorithm was 16.35 ± 1.33s, and the average running time of ELM algorithm was 11.22 ± 0.89s, superior to the SVM algorithm in denoising effects. Moreover, the ELM algorithm showed excellent diagnostic efficiency for patients with various degrees of IUA. In conclusion, ELM can effectively eliminate noise in 3D-TVS images and demonstrates excellent diagnostic efficiency on IUA, which is worthy of clinical application.

The nodules and the multiple times larger non-nodules of the regions of interested(ROIs) in lung areas are achieved, that would lead to a serious imbalance on the sample data. Many scholars have proposed some algorithms to solve this problem. In this paper, in order to guarantee that there is no correlation among the extracted characteristics, the PCA method is adopted to optimize and reduce dimensions, and then the modified support vector machine(SVM) classifiers using the sequential minimal optimization(SMO) algorithm and the grid research method are proposed to improve the computing efficiency. Furthermore, the abundant lung CT images from the hospital partnership could confirm the experimental results. We compare the classification performance between the ensemble SVM(ESVM) classifier and the cost-sensitive SVM(CSVM) classifier to deal with this problem. Experimental results show the performance of the CSVM classifier based on grid search is satisfactory than the ESVM.

BackgroundEffective and accurate diagnosis of attention-deficit/hyperactivity disorder (ADHD) is currently of significant interest. ADHD has been associated with multiple cortical features from structural MRI data. However, most existing learning algorithms for ADHD identification contain obvious defects, such as time-consuming training, parameters selection, etc. The aims of this study were as follows: (1) Propose an ADHD classification model using the extreme learning machine (ELM) algorithm for automatic, efficient and objective clinical ADHD diagnosis. (2) Assess the computational efficiency and the effect of sample size on both ELM and support vector machine (SVM) methods and analyze which brain segments are involved in ADHD.MethodsHigh-resolution three-dimensional MR images were acquired from 55 ADHD subjects and 55 healthy controls. Multiple brain measures (cortical thickness, etc.) were calculated using a fully automated procedure in the FreeSurfer software package. In total, 340 cortical features were automatically extracted from 68 brain segments with 5 basic cortical features. F-score and SFS methods were adopted to select the optimal features for ADHD classification. Both ELM and SVM were evaluated for classification accuracy using leave-one-out cross-validation.ResultsWe achieved ADHD prediction accuracies of 90.18% for ELM using eleven combined features, 84.73% for SVM-Linear and 86.55% for SVM-RBF. Our results show that ELM has better computational efficiency and is more robust as sample size changes than is SVM for ADHD classification. The most pronounced differences between ADHD and healthy subjects were observed in the frontal lobe, temporal lobe, occipital lobe and insular.ConclusionOur ELM-based algorithm for ADHD diagnosis performs considerably better than the traditional SVM algorithm. This result suggests that ELM may be used for the clinical diagnosis of ADHD and the investigation of different brain diseases.

BackgroundHeart disease is the primary cause of morbidity and mortality in the world. It includes numerous problems and symptoms. The diagnosis of heart disease is difficult because there are too many factors to analyze. What’s more, the misclassification cost could be very high.MethodsA cost-sensitive ensemble method was proposed to improve the efficiency of diagnosis and reduce the misclassification cost. The proposed method contains five heterogeneous classifiers: random forest, logistic regression, support vector machine, extreme learning machine and k-nearest neighbor. T-test was used to investigate if the performance of the ensemble was better than individual classifiers and the contribution of Relief algorithm.ResultsThe best performance was achieved by the proposed method according to ten-fold cross validation. The statistical tests demonstrated that the performance of the proposed ensemble was significantly superior to individual classifiers, and the efficiency of classification was distinctively improved by Relief algorithm.ConclusionsThe proposed ensemble gained significantly better results compared with individual classifiers and previous studies, which implies that it can be used as a promising alternative tool in medical decision making for heart disease diagnosis.

Various myoelectric prostheses controlled by electromyography (EMG) signals have been developed. However, there have been few studies that provide fast and accurate methods to predict handgrip force from EMG signals. Rapid and precise handgrip force prediction is required, especially for the real-time control system of myoelectric prostheses. In this study, extreme learning machine (ELM) is applied to predict handgrip force from surface EMG signals of forearm muscles. Furthermore, ELM is compared with support vector machine (SVM) and multiple nonlinear regression (MNLR). The below 10 % of the surface EMG and handgrip force signals were cut away, and then the root mean square feature extracted from the modified surface EMG signals was taken as input vector for these three kinds of predicting mechanisms. For the testing dataset, ELM achieved a slightly larger root mean squared error than SVM did and a smaller one than MNLR did. Meanwhile, all three methods showed high correlation coefficients. For the total processing time, ELM and MNLR consumed much less time than SVM did. Experimental results demonstrate that ELM possesses a relatively good accuracy and little consumed time, although SVM is effective for handgrip force estimation in terms of accuracy. Overall, ELM has a promising potential for predicting handgrip force rapidly and precisely.

Robustness of Extreme Learning Machine in the prediction of hydrological flow series

Cost-sensitive matrixized classification learning with information entropy

Paraffin odor intensity is an important quality indicator when a paraffin inspection is performed. Currently, paraffin odor level assessment is mainly dependent on an artificial sensory evaluation. In this paper, we developed a paraffin odor analysis system to classify and grade four kinds of paraffin samples. The original feature set was optimized using Principal Component Analysis (PCA) and Partial Least Squares (PLS). Support Vector Machine (SVM), Random Forest (RF), and Extreme Learning Machine (ELM) were applied to three different feature data sets for classification and level assessment of paraffin. For classification, the model based on SVM, with an accuracy rate of 100%, was superior to that based on RF, with an accuracy rate of 98.33–100%, and ELM, with an accuracy rate of 98.01–100%. For level assessment, the R2 related to the training set was above 0.97 and the R2 related to the test set was above 0.87. Through comprehensive comparison, the generalization of the model based on ELM was superior to those based on SVM and RF. The scoring errors for the three models were 0.0016–0.3494, lower than the error of 0.5–1.0 measured by industry standard experts, meaning these methods have a higher prediction accuracy for scoring paraffin level.

&amp;#160;Forecasting stream flow accurately is essential for managing water resources, preventing flooding, and designing the environment for intricate watersheds. This study conducts a comprehensive assessment of streamflow simulation models&amp;#8212;Multilayer Perceptron (MLP), Extreme Learning Machine (ELM), Support Vector Machine (SVM), and the Soil and Water Assessment Tool (SWAT) hydrological model&amp;#8212;covering the period from 1985 to 2018 in the Astore Basin, Pakistan. The GMRC-WAPDA and SWHP-WAPDA provided daily streamflow data from the Doyian gauging station in the Astor River Basin, as well as meteorological data collected from two automatic weather stations (AWS) located at Rama, Rattu and Astor. A total number of four soil classes (lithosols, calcaric, gleysols and fluvisol) was observed in the basin. The primary objective was to comprehensively assess the predictive performances of these models across distinct time segments and gauge their reliability in simulating streamflow dynamics. The study commenced with examining the SWAT model's performance, utilizing the NSE, PBIAS, R2, and RMSE metrics during calibration (1985&amp;#8211;2000) and validation (2001&amp;#8211;2009) periods. While the SWAT model effectively estimated streamflow, it exhibited limitations in accurately predicting peak and low-flow conditions. Subsequently, the machine learning models (MLP, ELM, and SVM) were scrutinized concerning their performance metrics&amp;#8212;R2, NSE, PBIAS, and RMSE&amp;#8212;across training (1985&amp;#8211;1995), validation (1996&amp;#8211;2005), and testing (2005&amp;#8211;2009) datasets. ELM displayed superior performance during the training phase, boasting a remarkable R2 of 0.94, followed by SVM and MLP. MLP showcased consistent strength in validation, maintaining an R2 of 0.73, while SVM followed with an R2 of 0.71. Despite their merits, none of the models precisely replicated observed streamflow patterns, as evidenced by the discrepancies between the observed flow and the SWAT model's simulations. This emphasizes the necessity for ongoing refinement and validation to enhance predictive accuracy and ensure closer alignment with real-world hydrological dynamics. This extensive comparative analysis offers critical insights into the nuances of MLP, ELM, SVM, and SWAT model performances, highlighting their varied strengths and limitations across distinct temporal segments. It underscores the importance of continual refinement and validation to improve predictive capabilities, which is essential for accurate streamflow simulations and effective water resource management in the Astor Basin and similar hydrological contexts.