M-estimator based robust kernels for support vector machines

In this paper, we propose M-estimator based robust kernels for support vector machine. The main motivation for our proposed kernels is that the sum of squared difference in the widely used Gaussian radial basis function kernels is not robust to outlier or noise. In addition, inspired by using a robust loss function in support vector machine regression to control training error and the idea of robust template matching with M-estimator, we apply M-estimator techniques to Gaussian radial basis functions and form a new class of robust kernels for support vector machines. We test our proposed kernels in several classification benchmark datasets and experimental results show that SVM with proposed kernels are better than SVM with Gaussian radial basis function kernels.

Active learning based on minimization of the expected path-length of random walks on the learned manifold structure

Accurate and reliable classification of depression is crucial for timely diagnosis, effective intervention, and improved patient outcomes. Traditional diagnostic methods for depression rely heavily on subjective assessments, which can introduce variability and inaccuracies. To overcome these limitations, advanced machine learning techniques, particularly Support Vector Machines (SVM), offer promising avenues for objective, precise classification. This research systematically investigates three newly proposed hybrid kernel strategies, Linear Combination, Multiplicative Combination, and Nested Combination, derived from Multiquadric (MQ-RBF) and Gaussian (GA-RBF) radial basis function kernels within the SVM framework. The empirical evaluations were conducted using two publicly available depression datasets encompassing diverse demographic, psychosocial, academic, and behavioral features. Experimental results clearly indicate the superior performance of the Nested Combination kernel, achieving classification accuracy of up to 88.05%, precision of 85.11%, recall of 94.12%, and an impressive ROC-AUC of 0.968. This kernel consistently outperformed baseline kernels and other hybrid forms across different training-test splits, demonstrating its robustness and adaptability. These promising outcomes underscore the practical significance and clinical potential of the Nested Combination kernel as a diagnostic support tool. However, future validation with larger and more diverse clinical datasets is recommended to further reinforce and generalize the kernel’s applicability in real-world depression classification scenarios.

In this paper, we develop an approach to exploiting kernel methods with manifold-valued data. In many computer vision problems, the data can be naturally represented as points on a Riemannian manifold. Due to the non-Euclidean geometry of Riemannian manifolds, usual Euclidean computer vision and machine learning algorithms yield inferior results on such data. In this paper, we define Gaussian radial basis function (RBF)-based positive definite kernels on manifolds that permit us to embed a given manifold with a corresponding metric in a high dimensional reproducing kernel Hilbert space. These kernels make it possible to utilize algorithms developed for linear spaces on nonlinear manifold-valued data. Since the Gaussian RBF defined with any given metric is not always positive definite, we present a unified framework for analyzing the positive definiteness of the Gaussian RBF on a generic metric space. We then use the proposed framework to identify positive definite kernels on two specific manifolds commonly encountered in computer vision: the Riemannian manifold of symmetric positive definite matrices and the Grassmann manifold, i.e., the Riemannian manifold of linear subspaces of a Euclidean space. We show that many popular algorithms designed for Euclidean spaces, such as support vector machines, discriminant analysis and principal component analysis can be generalized to Riemannian manifolds with the help of such positive definite Gaussian kernels.

The kernel function plays a basic role in support vector machines (SVM) algorithms. This paper presents an automatic method for selecting the parameters of the Gaussian radial basis function (GRBF) kernel which is one of the commonly used forms in SVM regression algorithms. The proposed method uses the expectation maximization (EM) algorithm for the automatic selection. The SVM regression algorithm is used to solve two major problems in remote sensing image segmentation: the density estimation problem and the Markov random field modeling problem. The density estimation is used for class conditional probabilities in Bayesian setups. The MRF is used for region modeling in boosting the Bayesian image segmentation. The proposed algorithms are integrated in a framework for remote sensing image segmentation. Experimental evaluation of the proposed algorithms using synthetic data set and hyperspectral data set illustrates the outstanding performance of the proposed algorithms.

Kernel methods have long played a central role in machine learning by enabling nonlinear modeling in high dimensional feature spaces. Among them, the Gaussian Radial Basis Function (RBF) kernel is widely used due to its smoothness, universality, and excellent performance in supervised learning. In contrast, the Jaccard/Tanimoto coefficient, originally defined over sets and binary vectors, has achieved notable success in chemoinformatics and information retrieval. Recent studies have shown that the Jaccard/Tanimoto coefficient can be extended to real-valued vectors and interpreted as a valid kernel. In this paper, we propose a natural generalization of the Tanimoto kernel by introducing a bandwidth-like parameter, leading to the Bandwidth-Adjustable Tanimoto (BAT) kernel. We focus on the shared structure of the BAT and the Gaussian RBF kernels in terms of their infinite polynomial expansions and investigate the universality of the BAT kernels, clarifying their relationship to the universal Taylor kernels. In particular, we show that by incorporating a bias term, the BAT kernel attains the universal approximation property comparable to that of the Gaussian kernel. Through experiments on classification, regression, and clustering tasks using benchmark datasets, we demonstrate that the BAT kernel performs comparably to the Gaussian kernel, while even the original Tanimoto kernel achieves competitive results. In particular, the BAT kernel attains performance on par with the Gaussian kernel in classification tasks, and it slightly surpasses the Gaussian kernel in regression and clustering tasks. A comparison of their functional forms, together with experiments on both regression and classification tasks involving nonlinear structures, further shows that the BAT kernel yields smoother predictive surfaces than the RBF kernel. Overall, our results indicate that the BAT kernel offers universality and practical performance, making it an alternative to the Gaussian kernel.

Support Vector Machines (SVMs) versus Multilayer Perception (MLP) in data classification

This work describes a Support Vector Machine (SVM) method used to analyze ECG signals for diagnosing cardiac arrhythmias effectively. The proposed method can accurately classify and differentiate normal (Normal) and abnormal heartbeats. Abnormal heartbeats include left bundle branch block (LBBB), right bundle branch block (RBBB), atrial premature contractions (APC) and premature ventricular contractions (PVC). This paper proposes a two stage, feature extraction and classification method for the detection of ECG beat types. Feature extraction module extracts higher order statistics of ECG signals in conjunction with three timing interval features. Then a number of support vector machine (SVM) classifiers with different value parameters are designed. These parameters are: Gaussian radial basis function (GRBF) kernel parameter and C penalty parameter of SVM classifier. We compared the classification ability of five different classes of ECG signals that were achieved over eight files from the MIT/BIH arrhythmia database.

In this paper, we evaluate the use of an electronic nose (EN) containing 13 conducting polymer gas sensors to discriminate between patterns of volatile organic compounds (VOCs) emitted by plants. The VOC patterns examined were produced by tomato, cucumber and pepper plants under both healthy and infected or infested conditions. Leaves from the plants were subjected to mechanical damage or pest and disease attacks (i.e. spider mites infested or mildew infected) and others were judged against undamaged healthy leaves. Support vector machines (SVMs) with linear, polynomial and Gaussian radial basis function (RBF) kernels were used to process and classify the raw data collected. The SVM illustrated an ability to discriminate between different VOC patterns and hence was able to classify correctly the infected leaves using the EN data. The results indicate that the array of 13 EN gas sensors can discriminate among VOC patterns from undamaged and artificially damaged leaves of the three plant species. This study demonstrates the potential application of such an EN technology coupled with suitable pattern recognition and signal processing methods to be used as a real time pest and disease detection system in the greenhouse environment.

An IKPCA-ELM-based intrusion detection method is developed to address the problem of the low accuracy and slow speed of intrusion detection caused by redundancies and high dimensions of data in the network. First, in order to reduce the effects of uneven sample distribution and sample attribute differences on the extraction of KPCA features, the sample attribute mean and mean square error are introduced into the Gaussian radial basis function and polynomial kernel function respectively, and the two improved kernel functions are combined to construct a hybrid kernel function. Second, an improved particle swarm optimization (IPSO) algorithm is proposed to determine the optimal hybrid kernel function for improved kernel principal component analysis (IKPCA). Finally, IKPCA is conducted to complete feature extraction, and an extreme learning machine (ELM) is applied to classify common attack type detection. The experimental results demonstrate the effectiveness of the constructed hybrid kernel function. Compared with other intrusion detection methods, IKPCA-ELM not only ensures high accuracy rates, but also reduces the detection time and false alarm rate, especially reducing the false alarm rate of small sample attacks.

Tumors are composed of an abnormal clone of cells which are capable of continuous growth. They are classified as benign and malignant based on their biological behaviour. Malignant ones are aggressively behaving and benign ones are non-aggressively behaving. Malignant tumors shows a number of variations in their nuclear morphological pattern which help to identify them. In this paper, we propose a method using Support Vector Machine learning for the classification of Benign and Malignant tissues based on the data obtained from the DNA ploidy analysis. We define the cancer detection process as a two-class classification using SVM, the two classes being Benign and Malignant cases. SVM is trained as a nonlinear classifier to automatically detect whether a given input data belongs to Benign tissue or a Malignant tissue. The images of aspiration tissue samples cannot be directly used for the extraction of feature values since they have uneven background and intensity variations between the nuclei and surrounding cytoplasm. Hence we apply some pre-processing on the input images before feature extraction using Intensity Thresholding and Active Contour Segmentation. Ten features were derived from the input images which helps the identification of Malignant and Benign tissues. Mean and Variance of the morphological features (area and perimeter) as well as the intensity based features (Total Optical Density, Hue and Saturation) are the selected inputs given to the SVM classfier. Datasets collected from a total of 50 input images (25 benign and 25 malignant) were used for training the classifier. The classifier performance was evaluated using a test dataset of 34 cases. 100% efficiency was obtained with a Gaussian Radial Basis Function (RBF) kernel with the available test data cases.

For hyperspectral image classification, feature extraction is a crucial pre-process for avoiding the Hughes phenomena. Some feature extraction methods such as linear discriminant analysis (LDA), nonparametric weighted feature extraction (NWFE), and their kernel versions, generalized discriminant analysis (GDA) and kernel nonparametric weighted feature extraction method (KNWFE) have been shown that they can improve the classification performance. However, for GDA and KNWFE, it is hard to find the suitable kernel parameters. Hence, although they have been published about 14 or 6 years, respectively, researchers rarely implement them for dealing with hyperspectral image classification problem. An automatic kernel parameter selection method (APS) was proposed to predetermine the appropriate radial basis function (RBF) kernel for support vector machine (SVM) and GDA. In this study, APS was applied to find the suitable RBF kernel function for KNWFE. From the experiment results on PAVIA data set, the classification performance of KNWFE still outperforms those of GDA [10] and SVM [10]. The most important of this research, the kernel parameters of GDA and KNWFE based on RBF kernel can be “automatically” determined and the researcher can implement them directly without tuning the kernel parameter.

Modelling heating and cooling energy demand for building stock using a hybrid approach

The stability of commodity prices for food is very influential on the economy of a region because stable prices have a direct impact on the level of people's purchasing power. The need to maintain the stability of food commodity prices is the background of this research and this can be anticipated by forecasting the Consumer Price Index (CPI). The CPI is an index number that calculates the average change in prices of goods and services consumed by households and society. The purpose of this study is to predict the CPI of the Foodstuff Group using the Support Vector Regression (SVR) method by utilizing Linear, Polynomial, Gaussian Radial Basis Function (RBF) and SPLine Kernel Functions. Selection of this SVR method, because SVR is able to map input vectors into higher dimensions and can produce a function with a trend of bumpy data following the data path formed, resulting in more accurate predictive values Research is limited to the city of Surabaya, the period of time the study begins January 1, 2016 until December 31, 2018. The data source used is the Surabaya Food Basic Price data as an input variable with 34 input attributes and the CPI data for Surabaya city Foodstuffs group as output variables. The results of this study are expected to be able to predict CPI with an error rate below 5%, which is indicated by MSE (Mean Square Error) < 0.05

Introduction:This paper proposes a reliable and efficient technique to recognize different epilepsy states, including healthy, interictal, and ictal states, using Electroencephalogram (EEG) signals.Methods:The proposed approach consists of pre-processing, feature extraction by higher order spectra, feature normalization, feature selection by genetic algorithm and ranking method, and classification by support vector machine with Gaussian and polynomial radial basis function kernels. The proposed approach is validated on a public benchmark dataset to compare it with previous studies.Results:The results indicate that the combined use of above elements can effectively decipher the cognitive process of epilepsy and seizure recognition. There are several bispectrum and bicoherence peaks at every bi-frequency plane, which reveal the location of the quadratic phase coupling. The proposed approach can reach, in almost all of the experiments, up to 100% performance in terms of sensitivity, specificity, and accuracy.Conclusion:Comparing between the obtained results and previous approaches approves the effectiveness of the proposed approach for seizure and epilepsy recognition.