Robust variable selection based on bagging classification tree for support vector machine in metabonomic data analysis

At some alpine treelines in the Rocky Mountains, whitebark pine (Pinus albicaulis)—a keystone species—plays a central role in tree island development through facilitation. Whitebark pine occurs both as a solitary tree and also as a component of tree islands, although relative importance of these two patterns varies geographically. We examine the utility of four predictive models to understand how the functional role of a keystone species varies spatially with biophysical conditions. We use a novel data set to predict whitebark pine's functional role, characterized by spatial association and relative position within a tree island at three North American Rocky Mountain treelines. For the study areas combined, and at a study area level, we compared prediction accuracy and variable importance among these modeling approaches: general linear models, classification and regression trees, random forests, and support vector machines. Results revealed that the keystone role of whitebark pine varied spatially. For the combined model, growing season temperature and slope curvature were the most important predictive variables for association and relative position, as revealed by overall agreement among the four models. Prediction accuracy and variable importance varied at the study area level, though, indicating that different conclusions could be drawn from each model, if examined independently. We advocate comparing results from different modeling approaches for complex, field-derived data sets because it might enable a better understanding of model and variable selection and appropriateness of input data resolution. Furthermore, comparative modeling enables assessment of the relative predictive and interpretive capacities of each modeling approach.

Classification Tree (CT) can establish explicit classification rules of Satellite Imagery (SI). However, the accuracy of explicit classification rules are poor. Back-Propagation Networks (BPN) and Support Vector Machine (SVM) both can establish a highly accurate model to predict the classification of SI but cannot generate the explicit rules. This study proposes a novel mining rule method named Evolutionary Classification Tree (ECT) which is composed of Particle Bee Algorithm (PBA) and Classification Tree (CT) that automatically produce self-organized rules to predict the classification of SI. In ECT, CT plays the architecture to represent explicit rules and PBA plays the optimization mechanism to optimize CT to fit the experimental data. 600 experimental data sets were used to compare accuracy and complexity of four model building techniques, CT, BPN, SVM and ECT. The results showed that ECT can produce rules which are more accurate than CT and SVM but less accurate than BPN models. However, BPN is black box models while ECT can produce explicit rules which is an important advantage to mining the explicit rules and knowledge in practical applications.

A hybrid support vector machine (SVM) and hidden Markov model (HMM) approach is proposed for designing continuous speech recognition systems. Using novel properties of SVMs and combining them with HMMs one can obtain models that map easily to hardware and lead to more modular and scalable design. The overall architecture of the proposed system is based on the MAP (maximum a posteriori) framework which offers a direct, feedforward recognition model. The SVMs generate smooth estimates of local transition probabilities in the HMM, conditioned on the acoustic inputs. The transition probabilities are then used to estimate the global posterior probabilities of HMM state sequences. A parallel architecture that implements a simple speech recognition model in real-time is presented.

The widely used support vector machine (SVM) method has shown to yield very good results in supervised classification problems. Other methods such as classification trees have become more popular among practitioners than SVM thanks to their interpretability, which is an important issue in data mining.In this work, we propose an SVM-based method that automatically detects the most important predictor variables and the role they play in the classifier. In particular, the proposed method is able to detect those values and intervals that are critical for the classification. The method involves the optimization of a linear programming problem in the spirit of the Lasso method with a large number of decision variables. The numerical experience reported shows that a rather direct use of the standard column generation strategy leads to a classification method that, in terms of classification ability, is competitive against the standard linear SVM and classification trees. Moreover, the proposed method is robust; i.e., it is stable in the presence of outliers and invariant to change of scale or measurement units of the predictor variables.When the complexity of the classifier is an important issue, a wrapper feature selection method is applied, yielding simpler but still competitive classifiers.

Interpretation of synthetic aperture radar (SAR) data processing is difficult because the geometry and spectral range of SAR are different from optical imagery. Consequently, SAR imaging can be a complementary data to multispectral (MS) optical remote sensing techniques because it does not depend on solar illumination and weather conditions. This study presents a multisensor fusion of SAR and MS data based on the use of classification and regression tree (CART) and support vector machine (SVM) through a decision fusion system. First, different feature extraction strategies were applied on SAR and MS data to produce more spectral and textural information. To overcome the redundancy and correlation between features, an intrinsic dimension estimation method based on noise-whitened Harsanyi, Farrand, and Chang determines the proper dimension of the features. Then, principal component analysis and independent component analysis were utilized on stacked feature space of two data. Afterward, SVM and CART classified each reduced feature space. Finally, a fusion strategy was utilized to fuse the classification results. To show the effectiveness of the proposed methodology, single classification on each data was compared to the obtained results. A coregistered Radarsat-2 and WorldView-2 data set from San Francisco, USA, was available to examine the effectiveness of the proposed method. The results show that combinations of SAR data with optical sensor based on the proposed methodology improve the classification results for most of the classes. The proposed fusion method provided approximately 93.24% and 95.44% for two different areas of the data.

Comparison of the performance of multiclass classifiers in chemical data: Addressing the problem of overfitting with the permutation test

Application of genetic algorithm and greedy stepwise to select input variables in classification tree models for the prediction of habitat requirements of Azolla filiculoides (Lam.) in Anzali wetland, Iran

Aim: The aim of the work is to evaluate the accuracy and precision in predicting heart disease using Support Vector Machine (SVM), Random forest (RF), Logistic Regression (LR), Decision Tree (DT) Classification algorithms. Materials and Methods: Classification algorithm is appealed on a heart dataset which consists of 180 records. A framework for heart disease prediction in the medical sector comparing Random forest, Logistic Regression, Decision Tree and SVM classifiers has been proposed and developed. The sample size was calculated as 55 in each group using G power 80%. Sample size was calculated using clincalc analysis, with alpha and beta values 0.05 and 0.5, 95% confidence, pretest power 80% and enrolment ratio 1. Results: The Novel Random Forest Algorithm (92.13%), Support Vector Machine (62.51%), Logistic Regression (84.89%), Decision Tree (86.25%) classifiers produce respectively. SVM, RF exists a statistically significant difference between the two groups (p=0.001,p=.004;p<0.05).LR, RF exists a statistically insignificant difference between the two groups (p=.103, P=.080;p>0.05) both with confidence interval 95%. Hence Random forest is better than SVM, RF, DT classifiers. Conclusion: The results show that the performance of RF is better when compared with SVM, LR and DT in terms of both precision and accuracy.

Water is essential for the life and nourishment of all living things. River water quality has deteriorated in recent years as a result of toxic wastes and contaminants. This rapid augmentation in water pollution is a major source of concern since it degrades water quality and renders it unsuitable for any usage. In India, Yamuna is considered one of the most important rivers in the northern region which plays a major role in development. In this work, we have focused on the Yamuna River water quality classification according to its usability as excellent, poor and unsuitable for drinking. To achieve this objective, we have presented a novel hybrid machine learning technique by taking the advantage of an ensemble learning mechanism. The proposed approach considers relief-based feature selection which helps to reduce the dimensionality, forward feature ranking which helps to consider the stage-wise features and a decision tree classifier. The performance of proposed water quality classification technique is compared with other classification techniques including Naïve Bayes, Support Vector Machine (SVM), bagged tree and boosted tree classifiers. The proposed approach achieves an overall accuracy of 99.65% which proves the robustness of this scheme in terms of predicting the water quality.

This research aims to detect subtle changes by combining binary change analysis, the Iteratively Reweighted Multivariate Alteration Detection (IRMAD), over dual polarimetric Advanced Land Observing Satellite (ALOS) backscatter with augmented data for post-classification change analysis. The accuracy of change detection was iteratively evaluated based on thresholds composed of mean and a range constant of standard deviation. Four datasets were examined for post-classification change analysis including the dual polarimetric backscatter as the benchmark and its augmented data with indices, entropy alpha decomposition and selected texture features. Variable importance was then evaluated to build a best subset model employing seven classifiers, including Bagged Classification and Regression Tree (CAB), Extreme Learning Machine Neural Network (ENN), Bagged Multivariate Adaptive Regression Spline (MAB), Regularised Random Forest (RFG), Original Random Forest (RFO), Support Vector Machine (SVM), and Extreme Gradient Boosting Tree (XGB). The best accuracy was 98.8%, which resulted from thresholding MAD variate-2 with constants at 1.7. The highest improvement of classification accuracy was obtained by amending the grey level co-occurrence matrix (GLCM) texture. The identification of variable importance (VI) confirmed that selected GLCM textures (mean and variance of HH or HV) were equally superior, while the contribution of index and decomposition were negligible. The best model produced similar classification accuracy at about 90% for both years 2007 and 2010. Tree-based algorithms including RFO, RFG and XGB were more robust than SVM and ENN. Subtle changes indicated by binary change analysis were somewhat hidden in post-classification analysis. Reclassification by combining all important variables and adding five classes to include subtle changes assisted by Google Earth yielded an accuracy of 82%.

Using information technology such as search engine for collecting and monitoring of network public opinion is a practical and effective method. This paper puts forward an improved algorithm of SVM classifier based on incremental learning online, and then implements a topic crawler system for network public opinion and to grab the public opinion.

Permeability is an important parameter connected with oil reservoir. In the last two decades, artificial intelligence models have been used. The current best prediction model in permeability prediction is extreme learning machine (ELM). It produces fairly good results but a clear explanation of the model is hard to come by because it is so complex. The aim of this research is to propose a way out of this complexity through the design of a hybrid intelligent model. The model combines classification and regression. In order to handle the high range of the permeability value, a classification tree is utilized. ELM is used as a final predictor. Results demonstrate that this proposed model performs better when compared with support vector machines (SVM) and ELM in term of correlation coefficient. Moreover, the classification tree model potentially leads to better communication among petroleum engineers and has wider implications for oil reservoir management efficiency.

This paper investigates the performance of three supervised machine learning algorithms K-Nearest Neighbor (KNN), Decision Tree (DT), and Support Vector Machine (SVM) for diabetes classification using the Pima Indians Diabetes Dataset. The study aims to provide a fair and consistent comparison by applying unified preprocessing procedures, including median imputation for clinically invalid values, feature standardization, and stratified 5-fold cross-validation. Model performance is evaluated using accuracy, precision, recall, and F1-score, with particular emphasis on recall for the diabetic class due to its clinical significance in reducing false negative diagnoses. Experimental results show that the Decision Tree model achieves the most balanced performance, with an average accuracy of 0.78 and an F1-score of 0.75, while maintaining higher recall for diabetic cases compared to KNN and SVM. Although SVM and KNN demonstrate acceptable overall accuracy, both models exhibit limitations in identifying minority-class instances. These findings highlight the importance of algorithm selection based not only on accuracy but also on clinical priorities such as interpretability and sensitivity to positive cases. The study contributes practical insights for the development of reliable machine learning–based decision support systems for early diabetes screening.

This paper presents a modification to the Optimal Hyper- plane synthesis to improve the linear separating properties of Support Vector Machines. The proposed modification is based on a modified op- timization criterion, which is extended to non-linearly separable train- ing data via the Soft Margin Optimal Hyperplane. Performance metrics demonstrate that the proposed modification maximizes the margin dur- ing hyperplane construction by increasing the average distance of both training patterns and support vectors, thereby improving classification accuracy.

The white-clawed crayfish’s habitat has been profoundly modified in Piedmont (NW Italy) due to environmental changes caused by human impact. Consequently, native populations have decreased markedly. In this research project, support vector machines were tested as possible tools for evaluating the ecological factors that determine the presence of white-clawed crayfish. A system of 175 sites was investigated, 98 of which recorded the presence of Austropotamobius pallipes. At each site 27 physical-chemical, environmental and climatic variables were measured according to their importance to A. pallipes. Various feature selection methods were employed. These yielded three subsets of variables that helped build three different types of models: (1) models with no variable selection; (2) models built by applying Goldberg’s genetic algorithm after variable selection; (3) models built by using a combination of four supervised-filter evaluators after variable selection. These different model types helped us realise how important it was to select the right features if we wanted to build support vector machines that perform as well as possible. In addition, support vector machines have a high potential for predicting indigenous crayfish occurrence, according to our findings. Therefore, they are valuable tools for freshwater management, tools that may prove to be much more promising than traditional and other machine-learning techniques.