The construction of support vector machine classifier using the firefly algorithm.

The solution of binary classification problems using the Support Vector Machine (SVM) method has been well developed. Though multi-class classification is typically solved by combining several binary classifiers, recently, several multi-class methods that consider all classes at once have been proposed. However, these methods require resolving a much larger optimization problem and are applicable to small datasets. Three methods based on binary classifications: one-against-all (OAA), one-against-one (OAO), and directed acyclic graph (DAG), and two approaches for multi-class problem by solving one single optimization problem, are implemented to predict protein secondary structure. Our experiments indicate that multi-class SVM methods are more suitable for protein secondary structure (PSS) prediction than the other methods, including binary SVMs, because their capacity to solve an optimization problem in one step. Furthermore, in this paper, we argue that it is feasible to extend the prediction accuracy by adding a second-stage multi-class SVM to capture the contextual information among secondary structural elements and thereby further improving the accuracies. We demonstrate that two-stage SVMs perform better than single-stage SVM techniques for PSS prediction using two datasets and report a maximum accuracy of 79.5%.

흡수, 분포, 대사, 배설 특성 및 독성을 예측하기 위한 효과적인 툴을 개발하는 것은 신약개발의 초기단계에서 NCE(new chemical entity)에 대한 가장 중요한 업무 중의 하나이다. 최근에 이런 시도중의 하나로서 ADME/T(absorption, distribution, metabolism, excretion, toxicity)관련 성질들의 예측에 support vector machine(SVM)을 이용하고 있다. 그리고 SVM은 ADME/T 성질들을 정확하게 예측하는데 많이 사용 되고 있다. 그러나 SVM 모델링에 두 가지 문제가 있다. 특성 선택(feature selection) 과 매개변수 설정(parameter setting)은 여전히 해결해야 할 과제이다. 이 두 가지 문제들은 SVM 분류의 효율성과 정확도에 결정적인 영향을 끼친다. 특히 특성 선택과 최적화된 SVM 변수의 설정은 서로 영향을 주기 때문에 동시에 다루어져야 한다. 여기서 우리는 genetic algorithm(GA) – 특성 선택에 사용 – 과 grid search(GS) method– 변수최적화에 사용 – 두 가지를 통합하는 효과적인 해결책을 제시하였다. ADME/T관련 성질 중 하나인 심장부정맥을 야기시키는 hERG 이온채널 저해제 분류 모델이 여기서 제안된 GA-GS-SVM을 위해 할당되고 테스트 되었다. 1891개의 화합물을 가지는 트레이닝 셋으로 단일 모델 3개, 앙상블 모델 3개, 총 6개의 모델을 만들었고 175개의 외부 데이터를 테스트 셋으로 사용하여 검증하였다. 데이터의 불균형 문제를 해결하기 위하여 GA-GS-SVM 단일 모델에 의한 예측 정확도와 GA-GS-SVM 앙상블 모델 예측 정확도를 비교하였으며, 앙상블모델을 사용하여 예측의 정확도를 높일 수 있었다. Developing effective tools for predicting absorption, distribution, metabolism, excretion properties and toxicity (ADME/T) of new chemical entities in the early stage of drug design is one of the most important tasks in drug discovery and development today. As one of these attempts, support vector machines (SVM) has recently been exploited for the prediction of ADME/T related properties. However, two problems in SVM modeling, i.e. feature selection and parameters setting, are still far from solved. The two problems have been shown to be crucial to the efficiency and accuracy of SVM classification. In particular, the feature selection and optimal SVM parameters setting influence each other, which indicates that they should be dealt with simultaneously. In this account, we present an integrated practical solution, in which genetic-based algorithm (GA) is used for feature selection and grid search (GS) method for parameters optimization. hERG ion-channel inhibitor classification models of ADME/T related properties has been built for assessing and testing the proposed GA-GS-SVM. We generated 6 different models that are 3 different single models and 3 different ensemble models using training set - 1891 compounds and validated with external test set - 175 compounds. We compared single model with ensemble model to solve data imbalance problems. It was able to improve accuracy of prediction to use ensemble model.

The modern automotive system, based on the convergence of information and communication technologies, is equipped with various functions to ensure vehicle safety and convenience of the driver. A driver-identification technology is an effective method to perform vehicle-theft detection. It can also provide customized driver-personalization services, such as healthcare or insurance. In this article, we propose and evaluate a driver-identification method based on wavelet transform by performing driving-pattern analysis for each driver. We compare the performances of three different machine-learning algorithms, namely Support Vector Machine (SVM), Random Forest, and XGBoost for performing driver identification. The proposed method is applicable to both binary and multiclass classifications for the driving data of five drivers. In the case of motorway, the XGBoost classifier identifies each driver and delivers an accuracy of up to 96.18% in binary classification and an accuracy of 91.6% in multiclass classification. Moreover, in the case of an urban road, the SVM classifier achieves an accuracy of up to 95.07% in binary classification and accuracy of 89.06% in multiclass classification. The proposed method provides a context for a better understanding of the association between driver behavior, which is an in-vehicle event, and mechanical reactions. Our results shall help researchers to broaden the understanding of driver identification using in-vehicle data.

BackgroundTo harness the full potential of social media for epidemiological surveillance of drug abuse trends, the field needs a greater level of automation in processing and analyzing social media content.ObjectivesThe objective of the study is to describe the development of supervised machine-learning techniques for the eDrugTrends platform to automatically classify tweets by type/source of communication (personal, official/media, retail) and sentiment (positive, negative, neutral) expressed in cannabis- and synthetic cannabinoid–related tweets.MethodsTweets were collected using Twitter streaming Application Programming Interface and filtered through the eDrugTrends platform using keywords related to cannabis, marijuana edibles, marijuana concentrates, and synthetic cannabinoids. After creating coding rules and assessing intercoder reliability, a manually labeled data set (N=4000) was developed by coding several batches of randomly selected subsets of tweets extracted from the pool of 15,623,869 collected by eDrugTrends (May-November 2015). Out of 4000 tweets, 25% (1000/4000) were used to build source classifiers and 75% (3000/4000) were used for sentiment classifiers. Logistic Regression (LR), Naive Bayes (NB), and Support Vector Machines (SVM) were used to train the classifiers. Source classification (n=1000) tested Approach 1 that used short URLs, and Approach 2 where URLs were expanded and included into the bag-of-words analysis. For sentiment classification, Approach 1 used all tweets, regardless of their source/type (n=3000), while Approach 2 applied sentiment classification to personal communication tweets only (2633/3000, 88%). Multiclass and binary classification tasks were examined, and machine-learning sentiment classifier performance was compared with Valence Aware Dictionary for sEntiment Reasoning (VADER), a lexicon and rule-based method. The performance of each classifier was assessed using 5-fold cross validation that calculated average F-scores. One-tailed t test was used to determine if differences in F-scores were statistically significant.ResultsIn multiclass source classification, the use of expanded URLs did not contribute to significant improvement in classifier performance (0.7972 vs 0.8102 for SVM, P=.19). In binary classification, the identification of all source categories improved significantly when unshortened URLs were used, with personal communication tweets benefiting the most (0.8736 vs 0.8200, P<.001). In multiclass sentiment classification Approach 1, SVM (0.6723) performed similarly to NB (0.6683) and LR (0.6703). In Approach 2, SVM (0.7062) did not differ from NB (0.6980, P=.13) or LR (F=0.6931, P=.05), but it was over 40% more accurate than VADER (F=0.5030, P<.001). In multiclass task, improvements in sentiment classification (Approach 2 vs Approach 1) did not reach statistical significance (eg, SVM: 0.7062 vs 0.6723, P=.052). In binary sentiment classification (positive vs negative), Approach 2 (focus on personal communication tweets only) improved classification results, compared with Approach 1, for LR (0.8752 vs 0.8516, P=.04) and SVM (0.8800 vs 0.8557, P=.045).ConclusionsThe study provides an example of the use of supervised machine learning methods to categorize cannabis- and synthetic cannabinoid–related tweets with fairly high accuracy. Use of these content analysis tools along with geographic identification capabilities developed by the eDrugTrends platform will provide powerful methods for tracking regional changes in user opinions related to cannabis and synthetic cannabinoids use over time and across different regions.

Machine learning involves the task of training systems to be able to make decisions without being explicitly programmed. Important among machine learning tasks is classification involving the process of training machines to make predictions from predefined labels. Classification is broadly categorized into three distinct groups: single-label (SL), multi-class, and multi-label (ML) classification. This research work presents an application of a multi-label classification (MLC) technique in automating Quranic verses labeling. MLC has been gaining attention in recent years. This is due to the increasing amount of works based on real-world classification problems of multi-label data. In traditional classification problems, patterns are associated with a single-label from a set of disjoint labels. However, in MLC, an instance of data is associated with a set of labels. In this paper, three standard &lt;em&gt;MLC&lt;/em&gt; methods: &lt;span&gt;binary relevance (BR), classifier chain (CC), and label powerset (LP) algorithms are implemented with four baseline classifiers: support vector machine (SVM), naïve Bayes (NB), k-nearest neighbors (k-NN), and J48. The research methodology adopts the multi-label problem transformation (PT) approach. The results are validated using six conventional performance metrics. These include: hamming loss, accuracy, one error, micro-F1, macro-F1, and avg. precision. From the results, the classifiers effectively achieved above 70% accuracy mark. Overall, SVM achieved the best results with CC and LP algorithms.&lt;/span&gt;

Automatic recognition of skin symptom plays an importance role in the skin diagnosis and treatment. Feature selection is to increase the classification performance of skin symptom. In this paper, the effects of feature selection on the classification of 4-class skin symptoms (chloasma, blackhead, freckled and comedone) are analyzed. Support vector machine (SVM) is employed to construct classifier, the parameters setting is very crucial to the learning results and generalization ability of SVM, and many irrelevant and redundant features degrade the performance of classification, furthermore, the choice of SVM parameters is influenced by the feature subset taken into account and vice versa. So a joint optimization method for selecting features and SVM parameters is proposed. Genetic algorithms, which have a lot of improvements about fitness, crossover and mutation in comparison with simple genetic algorithms, is to achieve a balance between the classification accuracy and the size of the feature subsets selected and have a fast convergence rate. Furthermore, ‘one-against-one’ approach is used to solve multi-class classification problems and a comparison of the model performances is made among the results of the joint optimization and the single optimization of features or SVM parameters in the experiments. The simulation result shows that our approach can find subsets of features for 4-class skin symptoms classification with higher average cross validation accuracy and a higher rate of convergence compared with single optimization of features or SVM parameters.

Support Vector Machine (SVM) is mainly used to classify the data into two categories. To solve the multi-category problems using SVM, researchers used two approaches. The first approach based on solving multiple SVM binary classifiers, whereas another approach based on solving a single optimization problem. In this paper, we have used the first approach and proposed an Efficient Multiclass Support Vector Machine (ESVM) algorithm using a skewed binary tree. To construct the skewed binary tree, no extra efforts are required as compared to the binary tree approach. The algorithm is tested on the benchmark data sets, and the results are compared with both the multiclass approaches of SVM. The ESVM’s results are compared with five techniques of solving multiple binary SVM classifiers and four techniques of solving a single optimization problem. The comparative experiments prove the efficiency of the ESVM in terms of its accuracy as compared to other contemporary algorithms. Further, ESVM is successfully applied for classification of the email dataset into positive, negative and neutral sentiments.

An effective bearing fault diagnosis model based on machine learning is proposed in this study. The model can separate into three stages: feature extraction, feature selection, and classification. In the stage of feature extraction, multiresolution analysis (MRA) and fast Fourier transform (FFT) are applied to extract the features from the raw signal measured from the rotating machine. In the second stage, a powerful feature selection method is proposed and utilized in the stage of feature selection. The new feature selection method is based on grey wolf optimization (GWO) and heap-based optimizer (HBO) with new strategies combined. Finally, support vector machine (SVM) and linear discriminant analysis (LDA) are used as the classifier independently. To verify the capability of the proposed model, four different datasets are applied to test the model in this study, respectively University of California Irvine (UCI) benchmark dataset, bearing dataset, Case Western Reserve University (CWRU) benchmark dataset, and Machinery Failure Prevention Technology (MFPT) benchmark dataset. The proposed method is compared with the existing methods and can certify the robustness with the experiment results.

To reduce the occurrence of traffic accidents caused by distraction. a detection method based on histogram of oriented gradient (HOG) and support vector machine (SVM) is proposed for driver’s distraction behavior in this paper. Interest region of driver was detected first from video image, also the image was enhanced, denoised and normalized. Then the histogram of oriented gradient is used to extract the feature of the image. Meanwhile, the cross-validation method is used to optimize parameters in SVM. Finally, the effectiveness of the method is verified by compared with classical SVM algorithm and Local Binary Pattern algorithm (LBP) based on SVM algorithms. The results show that, the proposed method can obtain better classification accuracy.

A novel memetic algorithm for discovering knowledge in binary and multi class predictions based on support vector machine

Breast cancer is widely prevalent in women. To diagnose it, pathologists evaluate breast histopathology images under a microscope in various magnifications. However, the number of pathologists per population is low in many countries, and a mistake is probable. Nowadays, deep learning has become a popular artificial intelligence research trends. Even though deep neural networks have obtained promising results in image processing, the need for extensive training data avoids its use in medical image processing. After the evaluation of various state-of-the-art deep learning methods and algorithms in medical data processing, this study proposes an effective deep transfer learning-based model, relying on pre-trained DCNN using an extensive data of ImageNet dataset which improves state-of the-art systems in both binary and multi-class classification. The weights of the pre-trained DesneNet121 on the Imagenet are transferred as initial weights first, and then the model is fine-tuned with a deep classifier along with data augmentation to distinguish various malignant and benign samples in the two categories of binary and multi-class classification. In the multi-class classification, the proposed model obtained up to 97% image-level accuracy. In binary classification, the model obtained up to 100% image-level accuracy. The achieved-results are outperforming previous studies accuracies in multiple performance metrics in breast cancer CAD systems. Moreover, the proposed method is both flexible and scalable, meaning it can be easily expanded to cover the detection of other types of diseases in the future and be integrated with more CNNs to increase its generalization capabilities.

Determining environmental chemical carcinogenicity is urgently needed as humans are increasingly exposed to these chemicals. In this study, we developed a hybrid neural network (HNN) method called HNN-Cancer to predict potential carcinogens of real-life chemicals. The HNN-Cancer included a new SMILES feature representation method by modifying our previous 3D array representation of 1D SMILES simulated by the convolutional neural network (CNN). We developed binary classification, multiclass classification, and regression models based on diverse non-congeneric chemicals. Along with the HNN-Cancer model, we developed models based on the random forest (RF), bootstrap aggregating (Bagging), and adaptive boosting (AdaBoost) methods for binary and multiclass classification. We developed regression models using HNN-Cancer, RF, support vector regressor (SVR), gradient boosting (GB), kernel ridge (KR), decision tree with AdaBoost (DT), KNeighbors (KN), and a consensus method. The performance of the models for all classifications was assessed using various statistical metrics. The accuracy of the HNN-Cancer, RF, and Bagging models were 74%, and their AUC was ~0.81 for binary classification models developed with 7994 chemicals. The sensitivity was 79.5% and the specificity was 67.3% for the HNN-Cancer, which outperforms the other methods. In the case of multiclass classification models with 1618 chemicals, we obtained the optimal accuracy of 70% with an AUC 0.7 for HNN-Cancer, RF, Bagging, and AdaBoost, respectively. In the case of regression models, the correlation coefficient (R) was around 0.62 for HNN-Cancer and RF higher than the SVM, GB, KR, DTBoost, and NN machine learning methods. Overall, the HNN-Cancer performed better for the majority of the known carcinogen experimental datasets. Further, the predictive performance of HNN-Cancer on diverse chemicals is comparable to the literature-reported models that included similar and less diverse molecules. Our HNN-Cancer could be used in identifying potentially carcinogenic chemicals for a wide variety of chemical classes.

Support Vector Machine (SVM) is originally a binary large-margin classifier emerged from the concept of structural risk minimization. Multiple solutions such as one-versus-one and one-versus-all have been proposed for creating multi-class SVM using elementary binary SVMs. Also multiple solutions have been proposed for SVM model selection, adjusting margin-parameter C and the Gaussian kernel variance. Here, an improved classifier named SVM-SVM is proposed for multi-class problems which increases accuracy and decreases dependency to margin-parameter selection. SVM-SVM adopts two K-class one-vs-one SVMs in a cascaded two-layer structure. In the first layer, input features are fed to one-vs-one SVM with non-linear kernels. We introduce this layer as a large-margin non-linear feature transform that maps input feature space to a discriminative K*(K-1)/2 dimensional space. To assess our hierarchical classifier, some datasets from the UCI repository are evaluated. Standard one-vs-one SVM and one-vs-one fuzzy SVM are used as reference classifiers in experiments. Results show significant improvements of our proposed method in terms of test accuracy and robustness to the model (margin and kernel) parameters in comparison with the reference classifiers. Our observations suggest that a multi-layer (deep) SVM structures can gain the same benefits as is seen in the deep neural nets (DNNs).

BackgroundMicroarray techniques provide promising tools for cancer diagnosis using gene expression profiles. However, molecular diagnosis based on high-throughput platforms presents great challenges due to the overwhelming number of variables versus the small sample size and the complex nature of multi-type tumors. Support vector machines (SVMs) have shown superior performance in cancer classification due to their ability to handle high dimensional low sample size data. The multi-class SVM algorithm of Crammer and Singer provides a natural framework for multi-class learning. Despite its effective performance, the procedure utilizes all variables without selection. In this paper, we propose to improve the procedure by imposing shrinkage penalties in learning to enforce solution sparsity.ResultsThe original multi-class SVM of Crammer and Singer is effective for multi-class classification but does not conduct variable selection. We improved the method by introducing soft-thresholding type penalties to incorporate variable selection into multi-class classification for high dimensional data. The new methods were applied to simulated data and two cancer gene expression data sets. The results demonstrate that the new methods can select a small number of genes for building accurate multi-class classification rules. Furthermore, the important genes selected by the methods overlap significantly, suggesting general agreement among different variable selection schemes.ConclusionsHigh accuracy and sparsity make the new methods attractive for cancer diagnostics with gene expression data and defining targets of therapeutic intervention.Availability: The source MATLAB code are available from http://math.arizona.edu/~hzhang/software.html.

Accurate intrusion detection is necessary to preserve network security. However, developing efficient intrusion detection system is a complex problem due to the nonlinear nature of the intrusion attempts, the unpredictable behaviour of network traffic, and the large number features in the problem space. Hence, selecting the most effective and discriminating feature is highly important. Additionally, eliminating irrelevant features can improve the detection accuracy as well as reduce the learning time of machine learning algorithms. However, feature reduction is an NPhard problem. Therefore, several metaheuristics have been employed to determine the most effective feature subset within reasonable time. In this paper, two intrusion detection models are built based on a modified version of the firefly algorithm to achieve the feature selection task. The first and, the second models have been used for binary and multiclass classification, respectively. The modified firefly algorithm employed a mutation operation to avoid trapping into local optima through enhancing the exploration capabilities of the original firefly. The significance of the selected features is evaluated using a Naïve Bayes classifier over a benchmark standard dataset, which contains different types of attacks. The obtained results revealed the superiority of the modified firefly algorithm against the original firefly algorithm in terms of the classification accuracy and the number of selected features under different scenarios. Additionally, the results assured the superiority of the proposed intrusion detection system against other recently proposed systems in both binary classification and multi-classification scenarios. The proposed system has 96.51% and 96.942% detection accuracy in binary classification and multi-classification, respectively. Moreover, the proposed system reduced the number of attributes from 41 to 9 for binary classification and to 10 for multi-classification.