Joint Optimization of Feature Selection and Parameters for Multi-class SVM in Skin Symptomatic Recognition

흡수, 분포, 대사, 배설 특성 및 독성을 예측하기 위한 효과적인 툴을 개발하는 것은 신약개발의 초기단계에서 NCE(new chemical entity)에 대한 가장 중요한 업무 중의 하나이다. 최근에 이런 시도중의 하나로서 ADME/T(absorption, distribution, metabolism, excretion, toxicity)관련 성질들의 예측에 support vector machine(SVM)을 이용하고 있다. 그리고 SVM은 ADME/T 성질들을 정확하게 예측하는데 많이 사용 되고 있다. 그러나 SVM 모델링에 두 가지 문제가 있다. 특성 선택(feature selection) 과 매개변수 설정(parameter setting)은 여전히 해결해야 할 과제이다. 이 두 가지 문제들은 SVM 분류의 효율성과 정확도에 결정적인 영향을 끼친다. 특히 특성 선택과 최적화된 SVM 변수의 설정은 서로 영향을 주기 때문에 동시에 다루어져야 한다. 여기서 우리는 genetic algorithm(GA) – 특성 선택에 사용 – 과 grid search(GS) method– 변수최적화에 사용 – 두 가지를 통합하는 효과적인 해결책을 제시하였다. ADME/T관련 성질 중 하나인 심장부정맥을 야기시키는 hERG 이온채널 저해제 분류 모델이 여기서 제안된 GA-GS-SVM을 위해 할당되고 테스트 되었다. 1891개의 화합물을 가지는 트레이닝 셋으로 단일 모델 3개, 앙상블 모델 3개, 총 6개의 모델을 만들었고 175개의 외부 데이터를 테스트 셋으로 사용하여 검증하였다. 데이터의 불균형 문제를 해결하기 위하여 GA-GS-SVM 단일 모델에 의한 예측 정확도와 GA-GS-SVM 앙상블 모델 예측 정확도를 비교하였으며, 앙상블모델을 사용하여 예측의 정확도를 높일 수 있었다. Developing effective tools for predicting absorption, distribution, metabolism, excretion properties and toxicity (ADME/T) of new chemical entities in the early stage of drug design is one of the most important tasks in drug discovery and development today. As one of these attempts, support vector machines (SVM) has recently been exploited for the prediction of ADME/T related properties. However, two problems in SVM modeling, i.e. feature selection and parameters setting, are still far from solved. The two problems have been shown to be crucial to the efficiency and accuracy of SVM classification. In particular, the feature selection and optimal SVM parameters setting influence each other, which indicates that they should be dealt with simultaneously. In this account, we present an integrated practical solution, in which genetic-based algorithm (GA) is used for feature selection and grid search (GS) method for parameters optimization. hERG ion-channel inhibitor classification models of ADME/T related properties has been built for assessing and testing the proposed GA-GS-SVM. We generated 6 different models that are 3 different single models and 3 different ensemble models using training set - 1891 compounds and validated with external test set - 175 compounds. We compared single model with ensemble model to solve data imbalance problems. It was able to improve accuracy of prediction to use ensemble model.

This research work presents a method of smokestack emission predicting model based on Latin hypercube sampling (LHS) and support vector machines (SVM). To solve the problem of predicting the concentration of smokestack emission, the Latin hypercube sampling method is adopted in a circular area above the smokestack to choose sample points, while the support vector machines regression analysis is carried out on the small data set and is evaluated by the mean square error. At the same time, an efficient parameter optimization method is proposed to reduce the mean square error and improve the prediction precision. With LIBSVM toolbox of MATLAB achieving the SVM regression analysis, SVM regression analysis and parameter optimization method were verified by a mathematical way. It turns out that the optimal parameters C and σ is respectively 2.8482 and 16, and the mean square error is 0.023011 which is acceptable. This study provides a reference sampling method and prediction model with parameters optimization for the detection of smokestack emission concentration.

In order to obtain the higher classification accuracy in specific categories for the different feature subset, a hierarchical classification algorithm based on Feature Selection is proposed, and is used for synthetic aperture radar (SAR) image classification, and feature selection is achieved by Genetic algorithm. The algorithm has two main characteristics: one is hierarchical classification which consists of many two-class classifier, and the two-class classifier is trained by the optimal feature subset which is selected according to different categories; the second is the classifier of support vector machine SVM (Support Vector Machine); the two is Genetic algorithm which can search out the optimal feature subset and parameters of support vector machine that is most suitable for the category, by unified coding the feature set and the parameters of SVM to constitute the Chromosome. The experiment on the first polarimetric SAR data show that the algorithm can obtain higher classification accuracy rate.

Three dimension finite element model (3D-FEM) of the spherical motor is presented in this paper. By analysis of one pair of stator-rotor poles, nonlinear regression model of one pair of stator-rotor poles based on the support vector machine (SVM) is introduced. Parameters optimization of one pair of stator-rotor poles for the maximal torque is also introduced which is based on chaos and uses the SVM regression model. Simulation results prove that using the SVM nonparametric modeling and parameter optimization based on chaos provide a novel way for finding the optimal structural parameters of the spherical motor.

Background Machine learning (ML) algorithm support vector machine (SVM) performed better than Thrombolysis in Myocardial Infarction (TIMI) score for ASIAN STEMI patients. However, Deep Learning (DL) effectiveness in the multiethnic ASIAN population has yet to be determined. DL has automatic learning of the feature from a given dataset without the need to conduct feature selection. However, the selected features by the algorithm is black box. Identifying features associated with mortality is essential to recognize characteristics of patients with high risk for better patient management. Purpose To develop a DL algorithm for in-hospital mortality in multiethnic STEMI patients using predictors identified from the SVM algorithm. To investigate DL performance constructed using predictors from SVM feature extraction and expert-recommended predictors. Methods We constructed four algorithms; a) DL and SVM algorithms with predictors identified from the SVM variable importance b) DL and SVM using predictors based on expert recommendation. We used registry data from the National Cardiovascular Disease Database of 11397 patient's. Fifty parameters including demographics, cardiovascular risk, medications and clinical variables were considered. The Area under the curve (AUC) is the performance evaluation metric. Algorithms were validated against the TIMI and tested using the same validation data. SVM variable importance with backward elimination was used to select and rank important variables. Results DL algorithms outperform SVM and TIMI on the validation dataset; i) DL with SVM selected predictors (15 predictors, AUC = 0.97), ii) DL with expert-recommended predictors (16 predictors, AUC = 0.96), iii) SVM with selected predictors (15 predictors, AUC = 0.92), iv) SVM with expert-recommended predictors (AUC = 0.89) and TIMI (AUC = 0.82). Common predictors across SVM feature selection, expert-recommendation and TIMI are: age, heart rate, Killip class, fasting blood glucose, systolic blood pressure, comorbid diseases and ST-elevation. SVM feature selection also identified diuretics, PCI and pharmacotherapy drugs as predictors that improve mortality prediction in STEMI patients. Our findings suggest that the TIMI score underestimates patients risk of mortality. DL algorithm using selected predictors classified 35% of nonsurvival patients as high risk (risk probabilities >50%) compared to only 12.7% nonsurvival patients by TIMI (score >5) (Figure below). Conclusions In the ASIAN population, patients with STEMI can be better classified using the DL algorithm compared to the ML and TIMI score. Combining ML feature selection with DL allows the identification of distinct factors in a unique ASIAN population for better mortality prediction than relying solely on an expert recommendation as it is a very subjective approach. Continuous validation on population-specific algorithms using DL and ML is needed before implementing in a real clinical setting. Funding Acknowledgement Type of funding sources: Public grant(s) – National budget only. Main funding source(s): Technology Development Fund 1 TIMI performance on validation setDL performance on validation set

Support vector machines are powerful and often used technique of supervised learning applied to classification. Quality of the constructed classifier can be improved by appropriate selection of the learning parameters. These parameters are often tuned using grid search with relatively large step. This optimization process can be done computationally more efficiently and more precisely using stochastic search metaheuristics. In this paper we propose adjusted bat algorithm for support vector machines parameter optimization and show that compared to the grid search it leads to a better classifier. We tested our approach on standard set of benchmark data sets from UCI machine learning repository. Additionally, proposed algorithm was compared to other approaches from the literature where our algorithm obtained more accurate classifiers, with larger percent of correct classifications, especially in the more realistic cases where separate test sets were used instead of cross validation only.

BackgroundMachine learning (ML) algorithm support vector machine (SVM) performed better than Thrombolysis in Myocardial Infarction (TIMI) score for ASIAN STEMI patients. However, Deep Learning (DL) effectiveness in the multiethnic ASIAN population has yet to be determined. DL has automatic learning of the feature from a given dataset without the need to conduct feature selection. However, the selected features by the algorithm is black box. Identifying features associated with mortality is essential to recognize characteristics of patients with high risk for better patient management.PurposeTo develop a DL algorithm for in-hospital mortality in multiethnic STEMI patients using predictors identified from the SVM algorithm. To investigate DL performance constructed using predictors from SVM feature extraction and expert-recommended predictors.MethodsWe constructed four algorithms; a) DL and SVM algorithms with predictors identified from the SVM variable importance b) DL and SVM using predictors based on expert recommendation. We used registry data from the National Cardiovascular Disease Database of 11397 patient's. Fifty parameters including demographics, cardiovascular risk, medications and clinical variables were considered. The Area under the curve (AUC) is the performance evaluation metric. Algorithms were validated against the TIMI and tested using the same validation data. SVM variable importance with backward elimination was used to select and rank important variables.ResultsDL algorithms outperform SVM and TIMI on the validation dataset; i) DL with SVM selected predictors (15 predictors, AUC = 0.97), ii) DL with expert-recommended predictors (16 predictors, AUC = 0.96), iii) SVM with selected predictors (15 predictors, AUC = 0.92), iv) SVM with expert-recommended predictors (AUC = 0.89) and TIMI (AUC = 0.82). Common predictors across SVM feature selection, expert-recommendation and TIMI are: age, heart rate, Killip class, fasting blood glucose, systolic blood pressure, comorbid diseases and ST-elevation. SVM feature selection also identified diuretics, PCI and pharmacotherapy drugs as predictors that improve mortality prediction in STEMI patients. Our findings suggest that the TIMI score underestimates patients risk of mortality. DL algorithm using selected predictors classified 35% of nonsurvival patients as high risk (risk probabilities >50%) compared to only 12.7% nonsurvival patients by TIMI (score >5) (Figure below).ConclusionsIn the ASIAN population, patients with STEMI can be better classified using the DL algorithm compared to the ML and TIMI score. Combining ML feature selection with DL allows the identification of distinct factors in a unique ASIAN population for better mortality prediction than relying solely on an expert recommendation as it is a very subjective approach. Continuous validation on population-specific algorithms using DL and ML is needed before implementing in a real clinical setting.Funding AcknowledgementType of funding sources: Public grant(s) – National budget only. Main funding source(s): Technology Development Fund 1TIMI performance on validation setDL performance on validation set

Support vector machine (SVM) is of great interest to classify high dimensional data. Parameter setting in the SVM significantly influences the classification accuracy. An improved algorithm based on particle swarm optimization (PSO) (M-PSO-SVM) for SVM parameter optimization is proposed to improve the classifier classification accuracy. This method introduces mutation operation, which allows particles to reinitialize after each iteration and update to avoid falling into local optimal solution and premature convergence. The improved PSO-based SVM classifier has been validated on a real hyperspectral dataset. Classification results show that the proposed method outperforms classifiers as well as SVM and PSO-based SVM.

The changes in China’s stock market are inseparable from the country’s economic development and macroeconomic regulation and control and have far-reaching significance in promoting China’s national economic growth. Compared with the Western developed capital market, China’s current stock market’s main smart investment strategy still has certain defects. Based on the SVM model, this paper establishes a predictive model that combines kernel parameters and parameter optimization to model. The mesh search method, genetic algorithm, and particle swarm optimization algorithm are used to optimize the parameters of the SVM under various kernel functions such as radial basis kernel function. The algorithm and particle swarm optimization algorithm optimize the parameters of the SVM to strengthen the applicability of the model in practice. The empirical results show that under the three-parameter optimization algorithms, the prediction results are higher than the random prediction accuracy, which indicates that it is effective to optimize the model by adjusting the parameters of the SVM. Among them, the SVM using the genetic algorithm parameter optimization under the radial basis kernel function shows the better prediction effect, which is the closest to the real value in the stock market forecast. The particle swarm algorithm supports the vector machine to predict the effect is slightly lower than the grid. Search method. In addition, through comparison experiments, the guess accuracy of BP neural network is worse than that of the support vector machine model before the adjustment. Finally, this paper uses the well-trained model to plan the stock smart investment plan.

Disease diagnosis is conducted with a machine learning method. We have proposed a novel machine learning method that hybridizes support vector machine (SVM), particle swarm optimization (PSO), and cuckoo search (CS). The new method consists of two stages: firstly, a CS based approach for parameter optimization of SVM is developed to find the better initial parameters of kernel function, and then PSO is applied to continue SVM training and find the best parameters of SVM. Experimental results indicate that the proposed CS-PSO-SVM model achieves better classification accuracy and F-measure than PSO-SVM and GA-SVM. Therefore, we can conclude that our proposed method is very efficient compared to the previously reported algorithms.

This paper proposes a hybrid Electroencephalogram (EEG) classification approach based on grey wolf optimizer (GWO) enhanced support vector machines (SVMs) called GWO-SVM approach for automatic seizure detection. In order to decompose EEG into five sub-band components, the discrete wavelet transform (DWT) was utilized to extracted features set. Then, this features are used to train the SVM with radial basis function (RBF) kernel function. Further, GWO was used for selecting the significant feature subset and the optimal parameters of SVM in order to obtain a successful EEG classification. The experimental results proved that the proposed GWO-SVM approach, able to detect epileptic and could thus further enhance the diagnosis of epilepsy with accuracy 100%. Furthermore, the proposed approach has been compared with genetic algorithm (GA) with support vector machines (GA-SVMs) and SVM using RBF kernel function. The computational results reveal that GWO-SVM approach achieved better classification accuracy outperforms both GA-SVM and typical SVMs.

Land use/cover (LU/LC) classification provides proxies of the natural and social processes related to urban development, providing stakeholders with crucial information. Remotely sensed images combined with supervised classification are common to define land use, but high-performance classifiers remain difficult to achieve, due to the presence of model hyperparameters. Conventional approaches rely on manual adjustment, which is time consuming and often unsatisfying. Therefore, the goal of this study has been to optimize the parameters of the support vector machine (SVM) algorithm for the generation of land use/cover maps from Sentinel-2 satellite imagery in selected humid and arid (three study sites each) climatic regions of Iran. For supervised SVM classification, we optimized two important parameters (gamma in kernel function and penalty parameter) of the LU/LC classification. Using the radial basis function (RBF) of the SVM classification method, we examined seven values for both parameters ranging from 0.001 to 1000. For both climate types, the penalty parameters (PP) showed a direct relationship with overall accuracy (OA). Statistical results confirmed that in humid study regions, LU/LC maps produced with a penalty parameter >100 were more accurate. However, for regions with arid climates, LU/LC maps with a penalty parameter >0.1 were more accurate. Mapping accuracy for both climate types was sensitive to the penalty parameter. In contrast, variations of the gamma values in the kernel function had no effect on the accuracy of the LU/LC maps in either of the climate zones. These new findings on SVM image classification are directly applicable to LU/LC for planning and environmental and natural resource management.

Hyperspectral remote sensing imagery, due to its rich source of spectral information, provides an efficient tool for land cover classifications in complex geographical areas. However, the high-dimensional space of this imagery poses two important challenges in the classification process: the Hughes phenomena and the existence of relevant and redundant features. The robustness of Support Vector Machines (SVM) in high-dimensional space makes them an efficient tool for classifying hyperspectral imagery. However, optimum SVM parameter determination and optimum feature selection are the two optimization issues that strongly effect SVM performance. Traditional optimization algorithms can discover optimum solutions in a limited search space with one local optimum. Nevertheless, in high-dimensional space traditional optimization algorithms usually get trapped in a local optimum, therefore it is necessary to apply meta-heuristic optimization algorithms to obtain near-global optimum solutions. This study evaluates the potential of Ant Colony Optimization (ACO) for determining SVM parameters and selecting features. Results obtained from AVIRIS and ROSIS hyperspectral datasets demonstrate the superior performance of SVM, achieved by simultaneously optimizing SVM parameters and subsets of the input feature. For comparison, the evaluation is also performed by applying it to other meta-heuristic optimization algorithms such as simulated annealing, tabu search, and genetic algorithm. The results demonstrate a better performance of the ACO-based algorithm in regards to improving the classification accuracy and decreasing the size of selected feature subsets.

To reduce the occurrence of traffic accidents caused by distraction. a detection method based on histogram of oriented gradient (HOG) and support vector machine (SVM) is proposed for driver’s distraction behavior in this paper. Interest region of driver was detected first from video image, also the image was enhanced, denoised and normalized. Then the histogram of oriented gradient is used to extract the feature of the image. Meanwhile, the cross-validation method is used to optimize parameters in SVM. Finally, the effectiveness of the method is verified by compared with classical SVM algorithm and Local Binary Pattern algorithm (LBP) based on SVM algorithms. The results show that, the proposed method can obtain better classification accuracy.

近年来，支持向量机(SVM)理论广泛应用于模式分类，然而影响其分类准确率的两个主要因素特征选择和参数优化又是相互影响和制约的。文章提出一种BA + SVM算法，利用蝙蝠算法(BA)来同步完成SVM的参数优化和输入数据的特征属性选择，提高了SVM的分类能力。设计的三种实验方式在10个测试数据集上实验结果表明，BA + SVM同步优化算法与单一进行参数优化或单一进行特征选择算法相比，具有输入特征少分类准确率高的优势。 Support vector machines (SVM), which is a popular method for pattern classification, has been recently adopted in range of problems. In training procedure of SVM, feature selection and parameter optimization are two main factors that impact on classification accuracy. In order to improve the classification accuracy by optimizing parameter and choosing feature subset for SVM, a new algorithm is proposed through combining Bat Algorithm (BA) with SVM, termed BA + SVM. For assessing the performance of BA + SVM, 10 public data-sets are employed to test the classification accuracy rate. Compared with grid algorithm, conventional parameter optimization method, our study concludes that BA + SVM has a higher classification accuracy with fewer input features for support vector classification.