Comparison of scaling behavior between fuzzy c-means based classifier with many parameters and LibSVM

Support vector machines (SVMs) has been accepted as a fashionable method in machine learning community. However, it cannot be easily scaled to handle large scale problems for its time and space complexity that is around quadratic with respect to the number of training samples. This paper proposes to combine distributed SVMs by simple Bayesian formalism rules (B-SVMs). B-SVMs randomly decomposes a large-scale task into many smaller and simpler sub-tasks in training phase and uses simple Bayesian formalism rules to make decision for final classification in test phase. B-SVMs was compared with single SVMs that is trained on entire training data set, parallel SVMs combined by majority voting (MV-SVMs), and one kind of fast modular SVMs (FM-SVMs). Experimental results on four problems show that B-SVMs can get higher accuracy than MV-SVMs and FM-SVMs does, the proposed algorithm can significantly reduce training and test time. More importantly, it produces test accuracy that is almost the same as single SVMs does.

Rough set theory (RST) can mine useful information from a large number of data and generate decision rules without prior knowledge. Support vector machines (SVMs) have good classification performances and good capabilities of fault-tolerance and generalization. To inherit the merits of both RST and SVMs, a hybrid classifier called rough set support vector machines (RS-SVMs) is proposed to recognize radar emitter signals in this paper. RST is used as preprocessing step to improve the performances of SVMs. A large number of experimental results show that RS-SVMs achieve lower recognition error rates than SVMs and RS-SVMs have stronger capabilities of classification and generalization than SVMs, especially when the number of training samples is small. RS-SVMs are superior to SVMs greatly.

Due to high dimensionality of face images and finite number of training samples, the linear subspace technique for face recognition pose challenges for better performance. In this paper, we compare the performance of linear subspace methods which involve the computation of scatter matrices for face recognition under illumination variation. The performance of these methods is evaluated in terms of classification accuracy, computational training and testing time. Extensive empirical experiments are performed to compare the performance using AR, Pie, Yale and YaleB face databases. In absence of sufficient number of training samples, classification accuracy of linear subspace methods deteriorate. Experimental results show that the performance of Dual LDA is best in terms of average classification accuracy. It is also observed that Fisherface takes minimum training time and both ERE and SVM takes minimum testing time. No linear subspace method outperforms others in terms of all performance measures.

Ordinal regression is one of the most important tasks of relation learning, and several techniques based on support vector machines (SVMs) have also been proposed for tackling it, but the scalability aspect of these approaches to handle large datasets still needs much of exploration. In this paper, we will extend the recent proposed algorithm Core Vector Machine (CVM) to the ordinal-class data, and propose a new algorithm named as Ordinal-Class Core Vector Machine (OCVM). Similar with CVM, its asymptotic time complexity is linear with the number of training samples, while the space complexity is independent with the number of training samples. We also give some analysis for OCVM, which mainly includes two parts, the first one shows that OCVM can guarantee that the biases are unique and properly ordered under some situation; the second one illustrates the approximate convergence of the solution from the viewpoints of objective function and KKT conditions. Experiments on several synthetic and real world datasets demonstrate that OCVM scales well with the size of the dataset and can achieve comparable generalization performance with existing SVM implementations.

Kernel machines have gained much popularity in applications of machine learning. Support vector machines (SVMs) are a subset of kernel machines and generalize well for classification, regression, and anomaly detection tasks. The training procedure for traditional SVMs involves solving a quadratic programming (QP) problem. The QP problem scales super linearly in computational effort with the number of training samples and is often used for the offline batch processing of data. Kernel machines operate by retaining a subset of observed data during training. The data vectors contained within this subset are referred to as support vectors (SVs). The work presented in this paper introduces a subset selection method for the use of kernel machines in online, changing environments. Our algorithm works by using a stochastic indexing technique when selecting a subset of SVs when computing the kernel expansion. The work described here is novel because it separates the selection of kernel basis functions from the training algorithm used. The subset selection algorithm presented here can be used in conjunction with any online training technique. It is important for online kernel machines to be computationally efficient due to the real-time requirements of online environments. Our algorithm is an important contribution because it scales linearly with the number of training samples and is compatible with current training techniques. Our algorithm outperforms standard techniques in terms of computational efficiency and provides increased recognition accuracy in our experiments. We provide results from experiments using both simulated and real-world data sets to verify our algorithm.

How Can a Massive Training Artificial Neural Network (MTANN) Be Trained With a Small Number of Cases in the Distinction Between Nodules and Vessels in Thoracic CT? 1

This paper deals with the problem of fault detection and diagnosis in sensors considering erratic, drift, hard-over, spike, and stuck faults. The data set containing samples of the above mentioned fault signals was acquired as follows: normal data signals were obtained from a temperature-to-voltage converter by using an Arduino Uno microcontroller board and MATLAB. Then, faults were simulated in normal data to get 100 samples of each fault, in which one sample is composed of 1000 data elements. A support vector machine (SVM) was used for data classification in a one-versus-rest manner. The statistical time-domain features, extracted from a sample, were used as a single observation for training and testing SVM. The number of features varied from 5 to 10 to examine the effect on accuracy of SVM. Three different kernel functions used to train SVM include linear, polynomial, and radial-basis function kernels. The fault occurrence event in fault samples was chosen randomly in some cases to replicate a practical scenario in industrial systems. The results show that an increase in the number of features from 5 to 10 hardly increases the total accuracy of the classifier. However, using ten features gives the highest accuracy for fault classification in an SVM. An increase in the number of training samples from 40 to 60 caused an overfitting problem. The k-fold cross-validation technique was adopted to overcome this issue. The increase in number of data elements per sample to 2500 increases the efficiency of the classifier. However, an increase in the number of training samples to 400 reduces the capability of SVM to classify stuck fault. The receiver operating characteristics curve comparison shows the efficiency of SVM over a neural network.

When dealing with large data sets, the traditional support vector machine (SVM) needs long training time which is aroused by the complexity of computation for kernel function. Moreover, if there are noises in a given training set, the classification accuracy rate of the traditional SVM is usually low. To overcome the shortcomings above, the algorithm of SVM based on half-suppressed fuzzy c-means clustering (HSFCM) is proposed. There are two phases in the proposed algorithm. First, the samples in each of the two classes are clustered by HSFCM. Second, the traditional SVM is trained only by the cluster centers obtained in the first phase. Experimental results show that the proposed method can reduce the number of training samples, enhance the training speed and classification accuracy rate of the traditional SVM effectively.

Brain-computer interaction (BCI) and physiological computing are terms that refer to using processed neural or physiological signals to influence human interaction with computers, environment, and each other. A major challenge in developing these systems arises from the large individual differences typically seen in the neural/physiological responses. As a result, many researchers use individually-trained recognition algorithms to process this data. In order to minimize time, cost, and barriers to use, there is a need to minimize the amount of individual training data required, or equivalently, to increase the recognition accuracy without increasing the number of user-specific training samples. One promising method for achieving this is collaborative filtering, which combines training data from the individual subject with additional training data from other, similar subjects. This paper describes a successful application of a collaborative filtering approach intended for a BCI system. This approach is based on transfer learning (TL), active class selection (ACS), and a mean squared difference user-similarity heuristic. The resulting BCI system uses neural and physiological signals for automatic task difficulty recognition. TL improves the learning performance by combining a small number of user-specific training samples with a large number of auxiliary training samples from other similar subjects. ACS optimally selects the classes to generate user-specific training samples. Experimental results on 18 subjects, using both nearest neighbors and support vector machine classifiers, demonstrate that the proposed approach can significantly reduce the number of user-specific training data samples. This collaborative filtering approach will also be generalizable to handling individual differences in many other applications that involve human neural or physiological data, such as affective computing.

Alzheimer’s disease is the most common form of dementia occurring in the elderly persons. Its early diagnosis may help in providing proper treatment. To date, there is no appropriate technique available to automatically classify it using MR brain images. In this work, first-and-second-order-statistics (FSOS) was employed for classification of Alzheimer’s from T2 trans-axial brain MR images. Although FSOS is a simple and well known feature extraction technique, it is not yet explored for Alzheimer’s classification. Performance of FSOS was compared with the state-of-the-art feature extraction techniques. Five commonly used classifiers were employed to build decision models. The performance of the models was evaluated in terms of sensitivity, specificity, accuracy, F-measure, training, and testing time. These models were built with varying number of training samples. Results showed that FSOS outperforms all the other existing feature extraction techniques in terms of all the considered performance measures. This was also validated by a statistical test. Interestingly, it was found that FSOS gives high performance irrespective of the choice of classifier and it works well even on small available number of samples, which is usually desired for all real time problems.Keyword: Discrete Wavelet Transform, Feature Extraction, First and Second Order Statistics, Gabor Transform, Magnetic Resonance Imaging, Slantlet Transform

Prediction of Alzheimer's diagnosis using semi-supervised distance metric learning with label propagation

Combining spectral and spatial information can significantly improve the classification performance of hyperspectral image (HSI). Currently, a lot of spectral–spatial HSI classification methods have been proposed. However, the task of HSI classification has remained challenging since the number of training samples is limited in real scenarios. In this article, we propose a novel HSI classification framework with single sample, in which the spectral self-similarity and spatial polygon structure information are fully combined to improve the classification performance. On the one hand, spectral self-similarity is used to expand training samples, which makes it possible to obtain sufficient samples with minimal cost. On the other hand, polygonal partition is introduced to acquire the geometrical structure of land covers in man-made environments. Specifically, the edge information of geometric objects is captured by polygonal partition, which can be utilized to constrain the spatial range of sample expansion and optimize the classification results. Experimental results on three real HSIs illustrate that the proposed method performs very well under small training sample size even when the number of samples is single per class.

Activity recognition is an increasingly important feature in mobile consumer devices that enables the design of context-aware applications and the customisation of user experiences. Recent deep learning-based recognisers demonstrate promising performances for hand activity recognition –e. g., washing hands– or human activity recognition –e. g., cycling– when exploiting accelerometer data from smartwatches or Magnetic Angular Rate and Gravity (MARG) data from body-worn sensors. However, the robustness and the generalisation capabilities of deep learning models can be an issue, since the consumers’ locomotion data is typically not available at training time. In this paper, we propose for the first time a pipeline of pre-trained deep feature extractors and online learning-capable shallow classifiers –such as passive-aggressive classifiers, random forests, and support vector classifiers– for the personalisation of mobile activity recognition systems. Our suggested solution could enable efficient on-device online learning for personalisation, thereby contributing to users’ privacy preservation. We report a series of experiments exploring the potential and the limitations of this approach to increase user-specific performance given only a limited number of training samples. Our results are encouraging and show (i) that a single additional data point per activity class (in a 25-class problem) from a new user can reduce recognition errors relatively by up to 20 %, and (ii) that with an increasing number of training samples our personalised models outperform comparable approaches in the literature, achieving accuracies of up to 91.4 %.

This paper addresses an important problem of training set selection for support vector machines (SVMs). It is a critical step in case of large and noisy data sets due to high time and memory complexity of the SVM training. There have been several methods proposed so far, in majority underpinned with the analysis of data geometry either in the input or kernel space. Here, we propose a new dynamically adaptive genetic algorithm (DAGA) to select valuable training sets. We demonstrate that not only can DAGA quickly select the training data, but in addition it dynamically determines the desired training set size without any prior information. We analyze the impact of the support vectors ratio, defined as the percentage of support vectors in the training set, on the DAGA performance. Also, we investigate and discuss the possibility of incorporating reduced SVMs into the proposed algorithm. Extensive experimental study shows that DAGA offers fast and effective training set optimization that is independent on the entire training set size.

The accurate recognition of speech is beneficial to the fields of machine translation and intelligent human–computer interaction. After briefly introducing speech recognition algorithms, this study proposed to recognize speech with a recurrent neural network (RNN) and adopted the connectionist temporal classification (CTC) algorithm to align input speech sequences and output text sequences forcibly. Simulation experiments compared the RNN-CTC algorithm with the Gaussian mixture model–hidden Markov model and convolutional neural network-CTC algorithms. The results demonstrated that the more training samples the speech recognition algorithm had, the higher the recognition accuracy of the trained algorithm was, but the training time consumption increased gradually; the more samples a trained speech recognition algorithm had to test, the lower the recognition accuracy and the longer the testing time. The proposed RNN-CTC speech recognition algorithm always had the highest accuracy and the lowest training and testing time among the three algorithms when the number of training and testing samples was the same.