Improved Sparse Multi-Class SVM and Its Application for Gene Selection in Cancer Classification

As two major parts for tackling high-dimensional cancer microarray gene data sets, feature selection and classification have attracted an increasing interest in academia and medical community. Since cancer gene expression data sets have small samples, high dimensionality, and class imbalance problems, extracting useful gene information and effective classification becomes more challenging. In this paper, we propose a novel feature selection algorithm called ISVM-RFE(FPD) for classification, which fully utilizes classification performance of each feature subset. Compared to the existing algorithms, ISVM-RFE(FPD) takes into account not only the intrinsic characteristic of the data, but also both linear and nonlinear correlation among features. The experimental results demonstrate that ISVM-RFE(FPD) outperforms the existing SVM-based feature selection algorithms in terms of recall rate of positive samples (rr p ) and G-mean (G).

Support vector machine (SVM), which is based on statistical learning theory (SLT), has shown much better performance than most other existing machine learning methods, which are based on the traditional statistics. The original SVM was developed to solve the dichotomy classification problem. Various approaches have been presented to solve multi-class problems. Using multi-class SVM classifier we have obtained high class rate of 95.4% in remote sensing image classification. However for the class number of remote sensing image is much great, manually obtaining of training samples is a much time-consuming work. Hence, we present a multi-class SVM based semi-supervised approach. We choose the initial cluster centers manually first, then label the samples as the training ones automatically with fuzzy C-means clustering algorithm. It is believed that this method upgrades the classification efficiency greatly with practicable class rate.

The setting of parameters in the support vector machines (SVMs) is very important with regard to its accuracy and efficiency. In this paper, we employ the firefly algorithm to train all parameters of the SVM simultaneously, including the penalty parameter, smoothness parameter, and Lagrangian multiplier. The proposed method is called the firefly-based SVM (firefly-SVM). This tool is not considered the feature selection, because the SVM, together with feature selection, is not suitable for the application in a multiclass classification, especially for the one-against-all multiclass SVM. In experiments, binary and multiclass classifications are explored. In the experiments on binary classification, ten of the benchmark data sets of the University of California, Irvine (UCI), machine learning repository are used; additionally the firefly-SVM is applied to the multiclass diagnosis of ultrasonic supraspinatus images. The classification performance of firefly-SVM is also compared to the original LIBSVM method associated with the grid search method and the particle swarm optimization based SVM (PSO-SVM). The experimental results advocate the use of firefly-SVM to classify pattern classifications for maximum accuracy.

The solution of binary classification problems using the Support Vector Machine (SVM) method has been well developed. Though multi-class classification is typically solved by combining several binary classifiers, recently, several multi-class methods that consider all classes at once have been proposed. However, these methods require resolving a much larger optimization problem and are applicable to small datasets. Three methods based on binary classifications: one-against-all (OAA), one-against-one (OAO), and directed acyclic graph (DAG), and two approaches for multi-class problem by solving one single optimization problem, are implemented to predict protein secondary structure. Our experiments indicate that multi-class SVM methods are more suitable for protein secondary structure (PSS) prediction than the other methods, including binary SVMs, because their capacity to solve an optimization problem in one step. Furthermore, in this paper, we argue that it is feasible to extend the prediction accuracy by adding a second-stage multi-class SVM to capture the contextual information among secondary structural elements and thereby further improving the accuracies. We demonstrate that two-stage SVMs perform better than single-stage SVM techniques for PSS prediction using two datasets and report a maximum accuracy of 79.5%.

High-dimensional pseudo-logistic regression and classification with applications to gene expression data

Cancer remains a leading cause of death worldwide; the World Health Organization (WHO) reports that there have been nearly 10 million cancer-related deaths in recent years, with breast cancer affecting over 2.1 million women annually on a global scale, posing significant challenges for early detection and diagnosis. Gene selection, using DNA microarray data, is crucial for reducing the presence of less informative genes and ensuring the selection of genes relevant to disease diagnosis. Cancer classification involves identifying the type of cancer and determining the extent of tumor growth and spread. This research focuses on improving gene selection for cancer classification using the XGBoost classifier, an efficient open-source implementation of the gradient-boosted trees algorithm. The primary goal is to enhance the performance of gene selection, enabling timely and appropriate treatments for cancer patients, as early detection is vital for ensuring a full recovery. Additionally, this research aims to reduce the time and expense associated with gene selection for cancer classification while increasing classification accuracy. The proposed method achieved an accuracy of approximately 93%, with precision, recall, and F1-score values of 93%, 87%, and 90%, respectively. The study highlights the potential of the XGBoost classifier in optimizing gene selection and improving diagnostic processes. Future work will focus on enhancing the accuracy of gene selection for cancer classification and reducing the number of irrelevant genes before proceeding to subsequent processes. This approach holds promises for streamlining the diagnostic process, improving patient outcomes, and offering significant benefits in timely cancer treatment.

Feature selection is a commonly addressed problem in classification. In gene expression-based cancer classification, a large number of genes in conjunction with a small number of samples makes the gene selection problem more important but also more challenging. Support vector machine as a popular classification algorithm, has been successfully used in SVM-RFE method for gene selection. This paper proposes a variant of SVM-RFE to do gene selection for cancer classification with expression data. Multiple support vector machine classifiers from a leave-one-out procedure are used to compute the feature ranking scores. The numerical experiments also show the good and stable performance of the proposed method.

In recent years, gene selection for cancer classification based on the expression of a small number of gene biomarkers has been the subject of much research in genetics and molecular biology. The successful identification of gene biomarkers will help in the classification of different types of cancer and improve the prediction accuracy. Recently, regularized logistic regression using the L 1 regularization has been successfully applied in high-dimensional cancer classification to tackle both the estimation of gene coefficients and the simultaneous performance of gene selection. However, the L 1 has a biased gene selection and dose not have the oracle property. To address these problems, we investigate L 1 / 2 regularized logistic regression for gene selection in cancer classification. Experimental results on three DNA microarray datasets demonstrate that our proposed method outperforms other commonly used sparse methods ( L 1 and L E N ) in terms of classification performance.

Fault diagnosis and severity analysis of rolling bearings using vibration image texture enhancement and multiclass support vector machines

Recently, focus on maintenance strategies has been shifted towards prognostic health management (PHM) and a number of state of the art algorithms based on data-driven prognostics have been developed to predict the health states of degrading components based on sensory data. Amongst these algorithms, Multiclass Support Vector Machines (MC-SVM) has gained popularity due to its relatively high classification accuracy, ability to classify multiple patterns and capability to handle noisy / incomplete data. However, its application is limited by the difficulty in determining the required kernel function and penalty parameters. To address this problem, this paper proposes a hybrid differential evolution – particle swarm optimization (DE-PSO) algorithm to optimize the MC-SVM kernel function and penalty parameters. The differential algorithm (DE) obtains the search limit for the SVM parameters, while the particle swarm optimization algorithm (PSO) determines the global optimum parameters for a given training data set. Since degrading machinery components display several degradation stages in their lifetime, the MC-SVM trained with optimum parameters are used to estimate the health states of a degrading machinery component, from which the remaining useful life (RUL) is predicted. This method improves the classification accuracy of MC-SVM in predicting the health states of a machinery component and consequently increases the accuracy of RUL predictions. The feasibility of the method is validated using bearing prognostic run-to-failure data obtained from NASA public data repository. A comparative study between MC-SVM with parameters obtained using simple grid search with n-fold cross validation and MC- SVM with DE-PSO based on prognostic performance metrics reveals that the proposed method has better performance, with all the cases considered falling within a 10 % error margin. The method also outperforms other soft computing methods proposed in literature.

To reduce the occurrence of traffic accidents caused by distraction. a detection method based on histogram of oriented gradient (HOG) and support vector machine (SVM) is proposed for driver’s distraction behavior in this paper. Interest region of driver was detected first from video image, also the image was enhanced, denoised and normalized. Then the histogram of oriented gradient is used to extract the feature of the image. Meanwhile, the cross-validation method is used to optimize parameters in SVM. Finally, the effectiveness of the method is verified by compared with classical SVM algorithm and Local Binary Pattern algorithm (LBP) based on SVM algorithms. The results show that, the proposed method can obtain better classification accuracy.

Educational data mining provides the process of applying different data mining tools and techniques to analyze and visualize the data of an institution and can be used to discover a unique pattern of students' academic performance. Secondary schools are increasing rapidly from past years and progress of the institution can be measured based on student's success and failure rates. Failure rates can be measured in terms of a core subject such as mathematics which has been considered in this proposed system. Real data was collected using school reports and questionnaire method by the Portugal school which has been used for the project. The students will be classified according to the grades assigned for the range of marks scored by them. This involves classifying the students into five levels of grading system starting from grade ‘A’ to grade ‘F’ where grade ‘A’ represents the student getting the highest marks, grade ‘B’ being the second highest, grade ‘C’ being third, grade ‘D’ being the fourth and the grade ‘F’ which implies that the student has failed. To carry out this type of classification many machine learning algorithms can be used to implement it. Comparison was made between the algorithms like Multiclass Support Vector Machine and Neural Networks using the Weka tool. Based on the analysis carried out Multiclass Support Vector Machine showed prominent accuracy. The Multiclass Support Vector Machine is implemented on the basis of one-to-rest strategy use of class labels which is primarily an extension of linear Support Vector Machine. In order to get appropriate results in terms of the accuracy of the model, parameters like ‘C’ and ‘Gamma’ is tuned while implementing Multiclass Support Vector Machine. The result gives a good predictive accuracy based on the grades that are provided by the school. The accuracy of prediction made by Support Vector Machine Classifier is determined by using K-fold cross validation, according to which the accuracy is 89%.

Support Vector Machines for pattern recognition are addressed to binary classification problems. The problem of multi-class classification is typically solved by the combination of 2-class decision functions using voting scheme methods or decison trees. We present a new multi-class classification SVM for the separable case, called K-SVCR. Learning machines operating in a kernel-induced feature space are constructed assigning output +1 or -1 if training patterns belongs to the classes to be separated, and assigning output 0 if patterns have a different label to the formers. This formulation of multi-class classification problem ever assigns a meaningful answer to every input and its architecture is more fault-tolerant than standard methods one.

<p style='text-indent:20px;'>Variable selection and parameter estimation are fundamental and important problems in high dimensional data analysis. In this paper, we employ the hard thresholding regularization method [<xref ref-type="bibr" rid="b1">1</xref>] to handle these issues under the framework of high-dimensional and sparse linear regression model. Theoretically, we establish a sharp non-asymptotic estimation error for the global solution and further show that the support of the global solution coincides with the target support with high probability. Motivated by the KKT condition, we propose a primal dual active set algorithm (PDAS) to solve the minimization problem, and show that the proposed PDAS algorithm is essentially a generalized Newton method, which guarantees that the proposed PDAS algorithm will converge fast if a good initial value is provided. Furthermore, we propose a sequential version of the PDAS algorithm (SPDAS) with a warm-start strategy to choose the initial value adaptively. The most significant advantage of the proposed procedure is its fast calculation speed. Extensive numerical studies demonstrate that the proposed method performs well on variable selection and estimation accuracy. It has favorable exhibition over the existing methods in terms of computational speed. As an illustration, we apply the proposed method to a breast cancer gene expression data set.</p>

The classification of high dimensional data is a challenging problem due to the presence of redundant and irrelevant features in a higher amount. These unwanted features degrade accuracy and increase the computational complexity of machine learning algorithms. In this paper, we propose a hybrid method that integrates the complementary strengths of feature selection and feature weighting approaches for improving the classification of high dimensional data on the Nearest Neighbor classifier. Specifically, we suggest four strategies that combine filter and wrapper methods of feature selection and feature weighting. Experiments are performed on 12 high dimensional datasets and outcomes are supported by Friedman as well as Holm statistical tests for validation. Extended Adjusted Ratio of Ratios is used to recognize the best method considering accuracy, feature selection, and runtime. The results show that two proposed strategies outperform other well-known methods in accuracy and features reduction. The hybrid feature selection-feature weighting wrapper method is the best among all in accuracy while the hybrid feature selection filter-feature weighting wrapper method is the most suitable for reducing features and runtime. Thus, the promising outcomes validate the importance of hybridizing feature selection and feature weighting while dealing with high dimensional data.