Associating kNN and SVM for Higher Classification Accuracy

Depression or Major Depressive Disorder (MDD) is a psychiatric disorder. It is the major contributor to overall global burden of disease. Any deterioration in brain functioning is reflected in Electroencephalogram (EEG) signals. EEG signals are highly complex, nonstationary and nonlinear. So, it is very difficult to analyze visually and identify changes in the waveform in order to classify MDD and normal subjects. Thus, computerized quantitative EEG is used for the analysis of signals. Support Vector Machine (SVM) using band power feature reported an accuracy of 98.33% and using Kernel Eigen-Filter-Bank Common Spatial Patterns (KEFB-CSP) gave an accuracy of 91.67% [1, 2]. Logistic Regression (LR) using band power feature reported an accuracy of 98.33%, using 4 nonlinear features combination provided an accuracy of 90%, using wavelet transform provided an accuracy of 87.5% and using only alpha power gave an accuracy of 73.3% [1, 3, 4]. Naive Bayesian (NB) using band power feature provided an accuracy of 96.8% [1]. Artificial Neural Network using Relative Wavelet Energy (RWE) reported an accuracy of 98.11%, using power spectrum feature gave accuracy of 84% and using Lep–Ziv complexity accuracy of 60–80% was reported [5, 6, 7]. Linear Discriminant Analysis (LDA) reported an accuracy of 91.2% using SASI (Spectral Asymmetry Index) and DFA (Detrended Fluctuation Analysis) [8]. Decision Tree provided an accuracy of 80% using EEG band power as feature [9]. The study reveals that, in general, high classification accuracy is achieved by SVM, LR and ANN and highest classification accuracy of 98.33% is achieved by SVM. Highest accuracy is achieved by SVM because it is more robust and computationally more efficient due to maximal margin gap between separating hyper planes and kernel trick. The study gives some ideas which could be helpful for guiding and improving future researches. Since linear and nonlinear method for feature extraction are both equally efficient. So any of the linear/nonlinear feature can be used. For feature selection and reduction, Genetic Algorithm (GA), Principal Components Analysis (PCA) and Linear Discriminant Analysis (LDA) can be used. Since SVM, LR and ANN give high classification accuracy and any of them can be used or any hybrid technique like GA-SVM, GA-ANN can be used. This chapter compares various EEG signal analysis techniques, compares their accuracy and methodology used and finally recommends the most suitable technique based on the accuracy for detection of depression. The chapter also summarizes some of the key finding related to EEG features based on present state of art. These results could be helpful for guiding and improving the future research in this area.

ObjectiveWe implement 2 different multitask learning (MTL) techniques, hard parameter sharing and cross-stitch, to train a word-level convolutional neural network (CNN) specifically designed for automatic extraction of cancer data from unstructured text in pathology reports. We show the importance of learning related information extraction (IE) tasks leveraging shared representations across the tasks to achieve state-of-the-art performance in classification accuracy and computational efficiency.Materials and MethodsMultitask CNN (MTCNN) attempts to tackle document information extraction by learning to extract multiple key cancer characteristics simultaneously. We trained our MTCNN to perform 5 information extraction tasks: (1) primary cancer site (65 classes), (2) laterality (4 classes), (3) behavior (3 classes), (4) histological type (63 classes), and (5) histological grade (5 classes). We evaluated the performance on a corpus of 95 231 pathology documents (71 223 unique tumors) obtained from the Louisiana Tumor Registry. We compared the performance of the MTCNN models against single-task CNN models and 2 traditional machine learning approaches, namely support vector machine (SVM) and random forest classifier (RFC).ResultsMTCNNs offered superior performance across all 5 tasks in terms of classification accuracy as compared with the other machine learning models. Based on retrospective evaluation, the hard parameter sharing and cross-stitch MTCNN models correctly classified 59.04% and 57.93% of the pathology reports respectively across all 5 tasks. The baseline models achieved 53.68% (CNN), 46.37% (RFC), and 36.75% (SVM). Based on prospective evaluation, the percentages of correctly classified cases across the 5 tasks were 60.11% (hard parameter sharing), 58.13% (cross-stitch), 51.30% (single-task CNN), 42.07% (RFC), and 35.16% (SVM). Moreover, hard parameter sharing MTCNNs outperformed the other models in computational efficiency by using about the same number of trainable parameters as a single-task CNN.ConclusionsThe hard parameter sharing MTCNN offers superior classification accuracy for automated coding support of pathology documents across a wide range of cancers and multiple information extraction tasks while maintaining similar training and inference time as those of a single task–specific model.

A comprehensive analysis of an automated system for epileptic seizure detection is explained in this work. When a seizure occurs, it is quite difficult to differentiate the non-stationary patterns from the discharges occurring in a rhythmic manner. The proposed approach deals with it efficiently by clustering it initially for the sake of feature extraction by using six different techniques categorized under two different methods, e.g., bio-inspired clustering and learning-based clustering. Learning-based clustering includes K-means clusters and Fuzzy C-means (FCM) clusters, while bio-inspired clusters include Cuckoo search clusters, Dragonfly clusters, Firefly clusters, and Modified Firefly clusters. Clustered values were then classified with 10 suitable classifiers, and after the performance comparison analysis of the EEG time series, the results proved that this methodology flow achieved a good performance index and a high classification accuracy. A comparatively higher classification accuracy of 99.48% was achieved when Cuckoo search clusters were utilized with linear support vector machines (SVM) for epilepsy detection. A high classification accuracy of 98.96% was obtained when K-means clusters were classified with a naive Bayesian classifier (NBC) and Linear SVM, and similar results were obtained when FCM clusters were classified with Decision Trees yielding the same values. The comparatively lowest classification accuracy, at 75.5%, was obtained when Dragonfly clusters were classified with the K-nearest neighbor (KNN) classifier, and the second lowest classification accuracy of 75.75% was obtained when Firefly clusters were classified with NBC.

In recent decades, terahertz (THz) radiation has been widely applied in many chemical and biomedical areas. Due to its ability to resolve the absorption features of many compounds noninvasively, it is a promising technique for chemical recognition of substances such as drugs or explosives. A key challenge for THz technology is to be able to accurately classify spectral measurements acquired in unknown complicated environments, rather than those from ideal laboratory conditions. Support vector machine (SVM) and deep neural networks (DNNs) are powerful and widely adopted approaches for complex classification with a high accuracy. In this article, we explore and apply the SVM and DNN methods for classifying the frequency spectra of glucose and lactose. We measured 372 groups of independent signals under different conditions to provide a sufficient training set. The classification accuracies achieved were 99% for the SVM method and 89.6% for the DNN method. These high classification accuracies demonstrate great potential in chemical recognition.

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

Recently, a high dimensional classification framework has been proposed to introduce spatial and anatomical priors in support vector machine (SVM) optimization scheme for brain image analysis. However, classical SVM has to convert 3D discrete brain images naturally represented by higher-order tensors to one-dimensional vectors in order to meet the input requirements. This traditional method destroys the natural structure and correlation in the original data, and generates high dimensional vectors. In this manuscript, the method is improved by a modified support tensor machine (STM) algorithm to make full use of spatial prior and the inherent information of tensor. The new approach reduces memory requirements and computational complexity significantly, and it is comparably demonstrated by experimental results on classification of Alzheimer patients and elderly controls. Introduction Alzheimer disease (AD) is the most prevalent neurodegenerative dementia in worldwide, and thus prevention and early accurate diagnosis of AD is increasingly crucial. In the last few years, support vector machine (SVM) methods for AD subject classification have become an incredibly active research topic. In the related work, SVM approach, as same as feature selection and extraction, has been taking the specificity of neuroimaging data into account. As an example, a frame work is proposed in [1] to include spatial and anatomical priors into SVM by using regularization operators, and it indicates a flexible way to model various types of proximity. However, for the purpose of satisfying the input requirements of classical SVM, 3D discrete brain images naturally represented by higher-order tensors have to convert to vectors in advance. This conversion breaks the inherent structure and correlation in the original data, and easily produces high dimensional vectors and leads to a poor performance in time complexity and space complexity. In order to improve above method including spatial prior of 3D discrete brain images in SVM, a new method is proposed in this manuscript to maintain the natural structure in the original tensor data and include spatial prior of all frontal slices of the tensors in STM [2]. To illustrate the improvement, we apply the proposed method to classification of 299 subjects, 137 AD patients and 162 cognitively normal (CN) controls, as the same study population in [3]. It can be observed that the proposed method reduces size of memory requested and the time complexity significantly and produces better classification performances. Spatial prior in SVM SVM is a supervised learning algorithm introduced by Vapnik and performs classification by mapping the data into higher dimensional space [4]. Let (xs, ys), s∈1,...,S be a training set of instance-labeled pairs, xs∈Ω, ys∈{-1,1} and Ω is the input space, then SVM optimization problem can be written as:

The sorting process of glass recycling is crucial to determine the grade of glass cullet. Hence, this paper aims to develop an algorithm for the classification of contaminants in glass recycling, presenting a hybrid model integrating two remarkable classifiers: Convolutional Neural Network (CNN) and Support Vector Machines (SVM), which have been proven to achieve the state-of-the-art results in solving classification problem. The proposed approach utilizes pre-trained CNN model as automatic feature extractor and SVM as an output predictor. The hybrid model extracts feature of contaminants from raw images and predicts the corresponding class. A contaminant database consisting around 2000 images is collected with a set of constraints to best portray the realism in practical environment. The performance of three different pre-trained CNN models (i.e. AlexNet, VGG-16 and GoogLeNet) in the hybrid model is tested and evaluated using comparative analysis. Since there is no similar approach used in this research field of interest in the past, the performance of the hybrid models is compared to the fine-tuned CNN models in the aspect of classification accuracy. In this study, the integration of VGG-16 and SVM was improved by 0.17% with SVM as classifier and 0.50% with data augmentation, achieving the highest classification accuracy of 100% among the other models. Among the factors that cause the VGG-16+SVM model with data augmentation to outperform are the depth of network, input size, stride size, size of filters and total number of weights and MACs. The proposed method is more accurate and efficient as compared to current existing method, producing high grade glass cullet at low cost.

The problems of adulteration and mislabeling are very common in the food industry. Laser-induced breakdown spectroscopy (LIBS) coupled with chemometric methods has many intrinsic advantages on adulteration analysis of various materials. In this work, several chemometric algorithms, i.e., principal component analysis (PCA), decision tree (DT), random forest (RF), partial least squares discriminant analysis (PLS-DA), linear discriminant analysis (LDA), and support vector machine (SVM), were carried out assisted by LIBS technology to study the classification performances on rice geographic origins. A series of samples, including 20 kinds of rice samples from different geographic origins, was detected using LIBS with no pretreatment processes. For data analysis, PCA was employed to reduce the input variables, and to reduce the collinearity of LIBS spectral results as well. The results showed the classification accuracies of the mentioned chemometric algorithms of DT, RF, PLS-DA, LDA, and SVM with 89 input variables of 86.80%, 96.30%, 96.80%, 98.60%, and 99.20%, respectively. At the same time, the operation times of these algorithms were 3.81 s, 54.64 s, 3.63 s, 2.09 s, and 531.01 s, respectively. On the other hand, 30 principal components of input variables were also tested under the same conditions. The classification accuracies for the above algorithms were 81.60%, 98.00%, 95.70%, 98.40%, and 99.20%, respectively. The operation times were 2.01 s, 4.88 s, 3.67 s, 0.36 s, and 308.55 s, respectively. In addition, the five-fold cross-validation classification accuracies with 30 input variables for DT, RF, PLS-DA, LDA, and SVM were 83.75%, 97.95%, 94.75%, 98.35%, and 99.25%, respectively. As a result, LDA was demonstrated to be the best and most efficient tool for rice geographic origin classification assisted by LIBS with high accuracy and analytical speed, which has great potential for rapid identification of adulterated products in agriculture without use of any chemical reagent.

BackgroundInformation on reproductive success is crucial to understanding population dynamics but can be difficult to obtain, particularly for species that birth while denning. For grizzly (Ursus arctos) and polar bears (U. maritimus), den visits are impractical because of safety and logistical considerations. Reproduction is typically documented through direct observation, which can be difficult, costly, and often occurs long after den departure. Reproduction could be documented remotely, however, from post-denning movement data if discernable differences exist between females with and without cubs.MethodsWe trained support vector machines (SVMs) with eight variables derived from telemetry data of female grizzly (2000–2022) and polar bears (1985–2016) with or without cubs during seven periods with lengths ranging from 5 to 60 days starting at den departure. We assessed SVM classification accuracy by withholding two samples (one cub-present, one cub-absent), training SVMs with the remaining data, predicting classification of the withheld samples, and repeating this process for each sample combination. Additionally, we evaluated how classification accuracy for grizzly bears was influenced by sample size, length of the post-departure period, and frequency of standardized location estimates.ResultsAccuracy of predicting cub presence or absence was 87% for grizzly bears with only 5 days of post-departure data and increased to a maximum of 92% with 20 days of data. For polar bears, accuracy was 86% at 5 days post-departure and increased to a maximum of 93% at 50 days. Classification accuracy for grizzly bears increased from 76 to 90% when sample size increased from 10 to 30 bears while holding period length constant (30 days) but did not increase at larger sample sizes. When sample size was held constant, increasing the length of the post-departure period did not affect classification accuracy markedly.ConclusionPresence or absence of grizzly and polar bear cubs can be identified with high accuracy even when SVM models are trained with limited data. Detecting cub presence or absence remotely could improve estimates of reproductive success and litter survival, enhancing our understanding of factors affecting cub recruitment.

Satellite instruments monitor the Earth’s surface day and night, and, as a result, the size of Earth observation (EO) data is dramatically increasing. Machine Learning (ML) techniques are employed routinely to analyze and process these big EO data, and one well-known ML technique is a Support Vector Machine (SVM). An SVM poses a quadratic programming problem, and quantum computers including quantum annealers (QA) as well as gate-based quantum computers promise to solve an SVM more efficiently than a conventional computer; training the SVM by employing a quantum computer/conventional computer represents a quantum SVM (qSVM)/classical SVM (cSVM) application. However, quantum computers cannot tackle many practical EO problems by using a qSVM due to their very low number of input qubits. Hence, we assembled a coreset (“core of a dataset”) of given EO data for training a weighted SVM on a small quantum computer, a D-Wave quantum annealer with around 5000 input quantum bits. The coreset is a small, representative weighted subset of an original dataset, and its performance can be analyzed by using the proposed weighted SVM on a small quantum computer in contrast to the original dataset. As practical data, we use synthetic data, Iris data, a Hyperspectral Image (HSI) of Indian Pine, and a Polarimetric Synthetic Aperture Radar (PolSAR) image of San Francisco. We measured the closeness between an original dataset and its coreset by employing a Kullback–Leibler (KL) divergence test, and, in addition, we trained a weighted SVM on our coreset data by using both a D-Wave quantum annealer (D-Wave QA) and a conventional computer. Our findings show that the coreset approximates the original dataset with very small KL divergence (smaller is better), and the weighted qSVM even outperforms the weighted cSVM on the coresets for a few instances of our experiments. As a side result (or a by-product result), we also present our KL divergence findings for demonstrating the closeness between our original data (i.e., our synthetic data, Iris data, hyperspectral image, and PolSAR image) and the assembled coreset.

A novel hybrid undersampling method for mining unbalanced datasets in banking and insurance

Wajah adalah bagian tubuh manusia yang berfungsi sebagai pusat ekspresi, pengenalan dan juga komunikasi. Penciptaan teknik yang berguna untuk mengidentifikasi dan menganalisis ekspresi wajah sangat penting untuk penelitian ini. Support Vector Machine (SVM) merupakan teknik pembelajaran mesin yang digunakan untuk pengenalan pola klasifikasi ekspresi wajah Manusia. Support Vector Machine (SVM) adalah metode klasifikasi yang menggunakan konsep mencari hyperplane yang optimal dalam suatu ruang feature untuk memisahkan beberapa kelas. Dari keempat klasifikasi tersebut ternyata presentasi accuracy, precision dan recall tidak berbeda jauh untuk mendeteksi ekspresi wajah dari tiap kelas tersebut dengan menggunakan metode SVM. Berdasarkan uji ekstraksi hasil rata-rata accuracy tertinggi yaitu jenis ekspresi marah dengan accuracy paling tinggi. Tingkat rata-rata tertinggi accuracy yang kedua yaitu ekspresi senang. tingkat rata-rata tertinggi accuracy ketiga yaitu ekspresi netral, lalu tingkat rata-rata tertinggi accuracy keempat yaitu ekspresi senyum dan tingkat rata-rata terendah accuracy yaitu ekspresi sedih. The face is a part of the human body that functions as a center of expression, recognition and communication. The creation of useful techniques for identifying and analyzing facial expressions was crucial for this study. Support Vector Machine (SVM) is a machine learning technique used for pattern recognition classification of Human facial expressions. Support Vector Machine (SVM) is a classification method that uses the concept of finding the optimal hyperplane in a feature space to separate classes. From the four classifications, it turns out that the presentation of accuracy, precision and recall is not much different to detect facial expressions from each class using the SVM method. Based on the extraction test, the highest average accuracy result is the type of angry expression with the highest accuracy. The second highest average level of accuracy is the expression of pleasure. The third highest average level of accuracy is neutral expression, then the fourth highest average level of accuracy is smile expression and the lowest average level of accuracy is sad expression.

Objectives: To develop a machine learning-based model to select the most important features from a high-dimensional dataset to classify the patterns at high accuracy and reduce their dimensionality. Methods: The proposed feature selection method (FSIFC) forms and combines feature clusters incrementally and produces feature subsets each time. The method uses K-means clustering and Mutual Information (MI) to refine the feature selection process iteratively. Initially, two clusters of features are formed using K-means clustering (K=2) by taking features as the basis of clustering instead of taking the patterns (a traditional way). From these two clusters, the features with the highest MI value in each cluster are kept in a feature subset. Classification accuracies (CA) of the feature subset are calculated using three classifiers namely Support Vector Machines (SVM), Random Forest (RF), and k-nearest Neighbor (knn). The process is repeated by incrementing the value of K i.e. number of clusters; until a maximum user-defined value of K is reached. The best value of CA obtained from these trials is recorded and the corresponding feature set is finally accepted. Findings: The proposed method is demonstrated using ten datasets and the results are compared with the existing published results using three classifiers to determine the method's performance. The ten datasets are classified with average CAs of 92.72%, 93.13%, and 91.5%, using the SVM, RF, and K-NN classifiers respectively. The proposed method selects a maximum of thirty features from the datasets. In terms of selecting the most effective and the smallest feature sets, the proposed method outperforms eight other feature selection methods considering CAs. Novelty: The proposed model applies feature reduction using combined feature clustering and filter methods in an incremental way. This provides an improved selection of relevant features while removing those which are irrelevant at different trials. Keywords: Feature selection, High-dimensional datasets, K-means algorithm, Mutual information, Machine learning

Support vector machine (SVM) has attracted substantial interest in the community of machine learning. As the extension of SVM, nonparallel hyperplane SVM (NHSVM) classification algorithms have become current researching hot spots in machine learning during the last few years. For binary classification tasks, the idea of NHSVM algorithms is to find a hyperplane for each class, such that each hyperplane is proximal to the data points of one class and far from the data points of the other class. Compared with the classical SVM, NHSVM algorithms have lower computational complexity, work better on XOR problems and can get better generalization performance. This paper reviews three representative NHSVM algorithms, including generalized eigenvalue proximal SVM (GEPSVM), twin SVM (TWSVM) and projection twin SVM (PTSVM), and gives the research progress of them. The aim of this overview is to provide an insightful organization of current developments of NHSVM algorithms, identify their limitations and give suggestions for further research.

This work proposes a methodology for content-based image retrieval of glioblastoma multiforme (GBM) and non-GBM tumors. Regions containing GBM lesions from 40 patients and non-GBM lesions from 20 patients were manually segmented from MR imaging studies (T1 post-contrast and T2 weighted channels) to form the training set. In addition to the two acquired channels, a composite image was formed by an image fusion method. Data reduction techniques, principal component analysis (PCA) and linear discriminant analysis (LDA), were applied on the training sets (T1 post, T2, composite, and multi-channel combining the PCA features from T1 post and T2). The retrieval accuracy was evaluated using a 'leave-one-out' strategy with query images belonging to 'normal', 'GBM' and 'non-GBM' classes. Several combinations of the similarity metric and classifier were used: Euclidean similarity measures with k-means classifier for the PCA and LDA features and support vector machine (SVM) nonlinear classifier (radial basis function kernel) with the PCA derived features. The SVM classifier served as a comparison of nonlinear techniques vs. linear ones. Multi-channel PCA was 100% accurate in classifying a query image as either 'normal' or 'abnormal'. The highest accuracy in classification of tumor grade (GBM or other Grade 3) was 77% and was achieved by SVM coupled with the PCA features. The proposed algorithm intent is to be integrated into an automated decision support system for MR brain tumor studies.