Support Vector Machines (SVMs) versus Multilayer Perception (MLP) in data classification

In this paper, we propose M-estimator based robust kernels for support vector machine. The main motivation for our proposed kernels is that the sum of squared difference in the widely used Gaussian radial basis function kernels is not robust to outlier or noise. In addition, inspired by using a robust loss function in support vector machine regression to control training error and the idea of robust template matching with M-estimator, we apply M-estimator techniques to Gaussian radial basis functions and form a new class of robust kernels for support vector machines. We test our proposed kernels in several classification benchmark datasets and experimental results show that SVM with proposed kernels are better than SVM with Gaussian radial basis function kernels.

In this paper, we develop an approach to exploiting kernel methods with manifold-valued data. In many computer vision problems, the data can be naturally represented as points on a Riemannian manifold. Due to the non-Euclidean geometry of Riemannian manifolds, usual Euclidean computer vision and machine learning algorithms yield inferior results on such data. In this paper, we define Gaussian radial basis function (RBF)-based positive definite kernels on manifolds that permit us to embed a given manifold with a corresponding metric in a high dimensional reproducing kernel Hilbert space. These kernels make it possible to utilize algorithms developed for linear spaces on nonlinear manifold-valued data. Since the Gaussian RBF defined with any given metric is not always positive definite, we present a unified framework for analyzing the positive definiteness of the Gaussian RBF on a generic metric space. We then use the proposed framework to identify positive definite kernels on two specific manifolds commonly encountered in computer vision: the Riemannian manifold of symmetric positive definite matrices and the Grassmann manifold, i.e., the Riemannian manifold of linear subspaces of a Euclidean space. We show that many popular algorithms designed for Euclidean spaces, such as support vector machines, discriminant analysis and principal component analysis can be generalized to Riemannian manifolds with the help of such positive definite Gaussian kernels.

Kernel methods have long played a central role in machine learning by enabling nonlinear modeling in high dimensional feature spaces. Among them, the Gaussian Radial Basis Function (RBF) kernel is widely used due to its smoothness, universality, and excellent performance in supervised learning. In contrast, the Jaccard/Tanimoto coefficient, originally defined over sets and binary vectors, has achieved notable success in chemoinformatics and information retrieval. Recent studies have shown that the Jaccard/Tanimoto coefficient can be extended to real-valued vectors and interpreted as a valid kernel. In this paper, we propose a natural generalization of the Tanimoto kernel by introducing a bandwidth-like parameter, leading to the Bandwidth-Adjustable Tanimoto (BAT) kernel. We focus on the shared structure of the BAT and the Gaussian RBF kernels in terms of their infinite polynomial expansions and investigate the universality of the BAT kernels, clarifying their relationship to the universal Taylor kernels. In particular, we show that by incorporating a bias term, the BAT kernel attains the universal approximation property comparable to that of the Gaussian kernel. Through experiments on classification, regression, and clustering tasks using benchmark datasets, we demonstrate that the BAT kernel performs comparably to the Gaussian kernel, while even the original Tanimoto kernel achieves competitive results. In particular, the BAT kernel attains performance on par with the Gaussian kernel in classification tasks, and it slightly surpasses the Gaussian kernel in regression and clustering tasks. A comparison of their functional forms, together with experiments on both regression and classification tasks involving nonlinear structures, further shows that the BAT kernel yields smoother predictive surfaces than the RBF kernel. Overall, our results indicate that the BAT kernel offers universality and practical performance, making it an alternative to the Gaussian kernel.

Active learning based on minimization of the expected path-length of random walks on the learned manifold structure

In this paper, we evaluate the use of an electronic nose (EN) containing 13 conducting polymer gas sensors to discriminate between patterns of volatile organic compounds (VOCs) emitted by plants. The VOC patterns examined were produced by tomato, cucumber and pepper plants under both healthy and infected or infested conditions. Leaves from the plants were subjected to mechanical damage or pest and disease attacks (i.e. spider mites infested or mildew infected) and others were judged against undamaged healthy leaves. Support vector machines (SVMs) with linear, polynomial and Gaussian radial basis function (RBF) kernels were used to process and classify the raw data collected. The SVM illustrated an ability to discriminate between different VOC patterns and hence was able to classify correctly the infected leaves using the EN data. The results indicate that the array of 13 EN gas sensors can discriminate among VOC patterns from undamaged and artificially damaged leaves of the three plant species. This study demonstrates the potential application of such an EN technology coupled with suitable pattern recognition and signal processing methods to be used as a real time pest and disease detection system in the greenhouse environment.

In this paper, we propose M-estimator based robust kernels for support vector machine. The main motivation for our proposed kernels is that the sum of squared difference in the widely used Gaussian radial basis function kernels is not robust to outlier or noise. In addition, inspired by using a robust loss function in support vector machine regression to control training error and the idea of robust template matching with M-estimator, we apply M-estimator techniques to Gaussian radial basis functions and form a new class of robust kernels for support vector machines. We test our proposed kernels in several classification benchmark datasets and experimental results show that SVM with proposed kernels are better than SVM with Gaussian radial basis function kernels.

An IKPCA-ELM-based intrusion detection method is developed to address the problem of the low accuracy and slow speed of intrusion detection caused by redundancies and high dimensions of data in the network. First, in order to reduce the effects of uneven sample distribution and sample attribute differences on the extraction of KPCA features, the sample attribute mean and mean square error are introduced into the Gaussian radial basis function and polynomial kernel function respectively, and the two improved kernel functions are combined to construct a hybrid kernel function. Second, an improved particle swarm optimization (IPSO) algorithm is proposed to determine the optimal hybrid kernel function for improved kernel principal component analysis (IKPCA). Finally, IKPCA is conducted to complete feature extraction, and an extreme learning machine (ELM) is applied to classify common attack type detection. The experimental results demonstrate the effectiveness of the constructed hybrid kernel function. Compared with other intrusion detection methods, IKPCA-ELM not only ensures high accuracy rates, but also reduces the detection time and false alarm rate, especially reducing the false alarm rate of small sample attacks.

The kernel function plays a basic role in support vector machines (SVM) algorithms. This paper presents an automatic method for selecting the parameters of the Gaussian radial basis function (GRBF) kernel which is one of the commonly used forms in SVM regression algorithms. The proposed method uses the expectation maximization (EM) algorithm for the automatic selection. The SVM regression algorithm is used to solve two major problems in remote sensing image segmentation: the density estimation problem and the Markov random field modeling problem. The density estimation is used for class conditional probabilities in Bayesian setups. The MRF is used for region modeling in boosting the Bayesian image segmentation. The proposed algorithms are integrated in a framework for remote sensing image segmentation. Experimental evaluation of the proposed algorithms using synthetic data set and hyperspectral data set illustrates the outstanding performance of the proposed algorithms.

This work describes a Support Vector Machine (SVM) method used to analyze ECG signals for diagnosing cardiac arrhythmias effectively. The proposed method can accurately classify and differentiate normal (Normal) and abnormal heartbeats. Abnormal heartbeats include left bundle branch block (LBBB), right bundle branch block (RBBB), atrial premature contractions (APC) and premature ventricular contractions (PVC). This paper proposes a two stage, feature extraction and classification method for the detection of ECG beat types. Feature extraction module extracts higher order statistics of ECG signals in conjunction with three timing interval features. Then a number of support vector machine (SVM) classifiers with different value parameters are designed. These parameters are: Gaussian radial basis function (GRBF) kernel parameter and C penalty parameter of SVM classifier. We compared the classification ability of five different classes of ECG signals that were achieved over eight files from the MIT/BIH arrhythmia database.

The stability of commodity prices for food is very influential on the economy of a region because stable prices have a direct impact on the level of people's purchasing power. The need to maintain the stability of food commodity prices is the background of this research and this can be anticipated by forecasting the Consumer Price Index (CPI). The CPI is an index number that calculates the average change in prices of goods and services consumed by households and society. The purpose of this study is to predict the CPI of the Foodstuff Group using the Support Vector Regression (SVR) method by utilizing Linear, Polynomial, Gaussian Radial Basis Function (RBF) and SPLine Kernel Functions. Selection of this SVR method, because SVR is able to map input vectors into higher dimensions and can produce a function with a trend of bumpy data following the data path formed, resulting in more accurate predictive values Research is limited to the city of Surabaya, the period of time the study begins January 1, 2016 until December 31, 2018. The data source used is the Surabaya Food Basic Price data as an input variable with 34 input attributes and the CPI data for Surabaya city Foodstuffs group as output variables. The results of this study are expected to be able to predict CPI with an error rate below 5%, which is indicated by MSE (Mean Square Error) < 0.05

The problem of conditional simulation of non-Gaussian stochastic processes and fields has gained a significant interest recently because of its applications in many fields, such as wind engineering, ocean engineering, and soil engineering. In this paper, the support vector machines (SVM) approach is developed for the conditional simulation of non-Gaussian stochastic processes and fields. To show the advantages of the presented method, the conditional simulation of non-Gaussian fluctuating wind pressures is carried out by using SVM and artificial neural networks (ANN). SVM considers three kinds of kernel function, such as linear function, Gaussian radial basis function, and exponential radial basis function, whereas ANN employs back-propagation and generalized regression. In machine learning of these artificial intelligences, two ways (interpolation and extrapolation) are employed to train finite non-Gaussian samples. The feasibility and validity of these algorithms are evaluated through the correlation coefficients, root mean square errors, skewness errors, and kurtosis errors between simulated samples and target samples and probability density functions (PDF), power spectral density (PSD) functions, and autocorrelation functions (ACF) of the simulated non-Gaussian fluctuating wind pressures versus their corresponding targets. The results show that the accuracy and effectiveness of SVM with an appropriate kernel function are superior to the back-propagation neural network (BPNN) and generalized regression neural network (GRNN). Furthermore, the advantage of the presented SVM approach is very obvious when the trained non-Gaussian samples are few.

Tumors are composed of an abnormal clone of cells which are capable of continuous growth. They are classified as benign and malignant based on their biological behaviour. Malignant ones are aggressively behaving and benign ones are non-aggressively behaving. Malignant tumors shows a number of variations in their nuclear morphological pattern which help to identify them. In this paper, we propose a method using Support Vector Machine learning for the classification of Benign and Malignant tissues based on the data obtained from the DNA ploidy analysis. We define the cancer detection process as a two-class classification using SVM, the two classes being Benign and Malignant cases. SVM is trained as a nonlinear classifier to automatically detect whether a given input data belongs to Benign tissue or a Malignant tissue. The images of aspiration tissue samples cannot be directly used for the extraction of feature values since they have uneven background and intensity variations between the nuclei and surrounding cytoplasm. Hence we apply some pre-processing on the input images before feature extraction using Intensity Thresholding and Active Contour Segmentation. Ten features were derived from the input images which helps the identification of Malignant and Benign tissues. Mean and Variance of the morphological features (area and perimeter) as well as the intensity based features (Total Optical Density, Hue and Saturation) are the selected inputs given to the SVM classfier. Datasets collected from a total of 50 input images (25 benign and 25 malignant) were used for training the classifier. The classifier performance was evaluated using a test dataset of 34 cases. 100% efficiency was obtained with a Gaussian Radial Basis Function (RBF) kernel with the available test data cases.

Background/Objectives: Speech produced by individuals with hearing loss differs notably from that of normal-hearing (NH) individuals. Although cochlear implants (CIs) provide sufficient auditory input to support speech acquisition and control, there remains considerable variability in speech intelligibility among CI users. As a result, speech produced by CI talkers often exhibits distinct acoustic characteristics compared to that of NH individuals. Methods: Speech data were obtained from eight cochlear-implant (CI) and eight normal-hearing (NH) talkers, while electroencephalogram (EEG) responses were recorded from 11 NH listeners exposed to the same speech stimuli. Support Vector Machine (SVM) classifiers employing 3-fold cross-validation were evaluated using classification accuracy as the performance metric. This study evaluated the efficacy of Support Vector Machine (SVM) algorithms using four kernel functions (Linear, Polynomial, Gaussian, and Radial Basis Function) to classify speech produced by NH and CI talkers. Six acoustic features—Log Energy, Zero-Crossing Rate (ZCR), Pitch, Linear Predictive Coefficients (LPC), Mel-Frequency Cepstral Coefficients (MFCCs), and Perceptual Linear Predictive Cepstral Coefficients (PLP-CC)—were extracted. These same features were also extracted from electroencephalogram (EEG) recordings of NH listeners who were exposed to the speech stimuli. The EEG analysis leveraged the assumption of quasi-stationarity over short time windows. Results: Classification of speech signals using SVMs yielded the highest accuracies of 100% and 94% for the Energy and MFCC features, respectively, using Gaussian and RBF kernels. EEG responses to speech achieved classification accuracies exceeding 70% for ZCR and Pitch features using the same kernels. Other features such as LPC and PLP-CC yielded moderate to low classification performance. Conclusions: The results indicate that both speech-derived and EEG-derived features can effectively differentiate between CI and NH talkers. Among the tested kernels, Gaussian and RBF provided superior performance, particularly when using Energy and MFCC features. These findings support the application of SVMs for multimodal classification in hearing research, with potential applications in improving CI speech processing and auditory rehabilitation.

The small leak in the propulsion system pipeline of the sounding rocket is prone to occur in the connections because of the screw thread loosening. Due to economic and technical bottleneck, the traditional soap bubble method is widely used in practice to evaluate whether existing a leak or not by visually observing the bubble’s size and numbers. Thus doing so will result in the low inspection efficiency and high cost. Using acoustic emission (AE) techniques, this paper presents an experimental study on small leak detection on the screw thread connection in the propulsion system pipeline of sounding rocket. The time and frequency characteristics of the corresponding small leak AE signals are investigated. After characteristic indices extraction and selection, the multi-class support vector machines (MCSVM)-based leak rates recognition algorithm in One–vs–All (OVA) is proposed. It has been validated that, for the propulsion system pipeline of the sounding rocket, the dominant characteristic frequency band of the small leak AE signals induced by screw thread loosening concentrates on 35–45 kHz. The proposed optimal OVA SVM models can achieve good classification accuracy of >98% by using the characteristic index set {Envelope area, standard deviation (STD), root-mean-square (RMS), Energy, Average frequency} and Gaussian Radial Basis Function (RBF) kernel function. The drastic drops in the false alarm attribute to use the combination of time- and frequency-domain characteristic indices. Especially, once adding the “Envelope area” into the characteristic index set, the classification accuracies of the OVA SVM models are further improved significantly regardless of the effect of kernel functions.

Mixed kernel function support vector regression for global sensitivity analysis