Theory and Applications of Clifford Support Vector Machines

This paper introduces the Clifford support vector machines as a generalization of the real- and complex- valued support vector machines using the Clifford geometric algebra. In this framework we handle the design of kernels involving the Clifford or geometric product for linear and nonlinear classification. We present an interesting application to classify mechanical tools using the concept of Clifford moments which is invariant under affine transformations.

This paper explains how, following the representation of 3D crystallographic space groups in Clifford's geometric algebra, it is further possible to similarly represent the 162 so called subperiodic groups of crystallography in Clifford's geometric algebra. A new compact geometric algebra group representation symbol is constructed, which allows to read off the complete set of geometric algebra generators. For clarity moreover the chosen generators are stated explicitly. The group symbols are based on the representation of point groups in geometric algebra by versors (Clifford monomials, Lipschitz elements). Keywords: Subperiodic groups, Clifford's geometric algebra, versor representation, frieze groups, rod groups, layer groups .

Proximal support vector machine is a variation of standard support vector machine and can be trained extremely efficiently for binary classification. However in many application fields, multi-class classification and incremental learning must be supported. Incremental linear proximal support vector classifier for multi-class classification has been developed in recent years, but only its performance in one-against-all manner has been investigated, and the application of proximal support vector machine for nonlinear multi-class classification has not been studied. In order to apply proximal support vector machine to more fields, three multi-class classification policies (one-against-all, one-against-one, DAGSVM) applied to incremental linear proximal support vector classifier are compared and incremental nonlinear proximal support vector classifier for multi-class classification based on Gaussian kernel is investigated in the paper. The experiments indicate that one-against-all policy is best for incremental linear proximal support vector classifier according to the tradeoff between computing complexity and correctness, and the introduced incremental nonlinear proximal support vector classifier is effective in one-against-all manner when the reduce rate is below 0.6.

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

A boosted SVM classifier trained by incremental learning and decremental unlearning approach

Three-dimensional image modeling is essential in many scientific disciplines, including computer vision and precision agriculture. So far, various methods of creating three-dimensional (3D) models have been considered. However, the processing of transformation matrices of each input image data is not controlled. Site-specific crop mapping is essential because it helps farmers determine yield, biodiversity, energy, crop coverage, etc. Clifford Geometric Algebraic understanding of signaling and image processing has become increasingly important in recent years. Geometric Algebraic treats multi-dimensional signals in a holistic way to maintain relationship between side sizes and prevent loss of information. This article has used agricultural images acquired by unmanned aerial vehicles (UAVs) to construct three-dimensional models using Clifford geometric algebra. The qualitative and quantitative performance evaluation results show that Clifford geometric algebra can generate a three-dimensional geometric statistical model directly from drones' RGB images. Through peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and visual comparison, the proposed algorithm's performance is compared with latest algorithms. Experimental results show that proposed algorithm is better than other leading 3D modeling algorithms.

A Clifford support vector machine (CSVM) learns the decision surface from multi distinct classes of the multiple input points using the Clifford geometric algebra. In many applications, each multiple input point may not be fully assigned to one of these multi-classes. In this paper, we apply a fuzzy membership to each multiple input point and reformulate the CSVM for multiclass classification to make different input points have their own different contributions to the learning of decision surface. We call the proposed method Clifford fuzzy SVM.

Machine learning involves the task of training systems to be able to make decisions without being explicitly programmed. Important among machine learning tasks is classification involving the process of training machines to make predictions from predefined labels. Classification is broadly categorized into three distinct groups: single-label (SL), multi-class, and multi-label (ML) classification. This research work presents an application of a multi-label classification (MLC) technique in automating Quranic verses labeling. MLC has been gaining attention in recent years. This is due to the increasing amount of works based on real-world classification problems of multi-label data. In traditional classification problems, patterns are associated with a single-label from a set of disjoint labels. However, in MLC, an instance of data is associated with a set of labels. In this paper, three standard <em>MLC</em> methods: <span>binary relevance (BR), classifier chain (CC), and label powerset (LP) algorithms are implemented with four baseline classifiers: support vector machine (SVM), naïve Bayes (NB), k-nearest neighbors (k-NN), and J48. The research methodology adopts the multi-label problem transformation (PT) approach. The results are validated using six conventional performance metrics. These include: hamming loss, accuracy, one error, micro-F1, macro-F1, and avg. precision. From the results, the classifiers effectively achieved above 70% accuracy mark. Overall, SVM achieved the best results with CC and LP algorithms.</span>

Model comparison for automatic characterization and classification of average ERPs using visual oddball paradigm

In credit card portfolio management a major challenge is to classify and predict credit cardholders’ behaviors in a reliable precision because cardholders’ behaviors are rather dynamic in nature. Multiclass classification refers to classify data objects into more than two classes. Many real-life applications require multiclass classification. The purpose of this paper is to compare three multiclass classification approaches: decision tree, Multiple Criteria Mathematical Programming (MCMP), and Hierarchical Method for Support Vector Machines (SVM). While MCMP considers all classes at once, SVM was initially designed for binary classification. It is still an ongoing research issue to extend SVM from two-class classification to multiclass classification and many proposed approaches use hierarchical method. In this paper, we focus on one common hierarchical method – one-against-all classification. We compare the performance of See5, MCMP and SVM one-against-all approach using a real-life credit card dataset. Results show that MCMP achieves better overall accuracies than See5 and one-against-all SVM.

Electroencephalogram (EEG) comprises valuable details related to the different physiological state of the brain. In this paper, a framework is offered for detecting the epileptic seizures from EEG data recorded from normal subjects and epileptic patients. This framework is based on a discrete wavelet transform (DWT) analysis of EEG signals using linear and nonlinear classifiers. The performance of the different combinations of two-class epilepsy detection is studied using Support Vector Machine (SVM) and neural network analysis (NNA) classifiers for the derived statistical features from DWT. In this new approach first parse EEG signals to sub-bands in different categories with the help of discrete wavelet transform (DWT) and then we derive statistical features such as Mean, Median, Standard Deviation, Kurtosis, Entropy, Skewness for each sub-band. These features, extracted from details and approximation coefficients of DWT sub-bands, are used as input to Principal Component Analysis (PCA). The classification is based on reducing the feature dimension using PCA and deriving the Support Vector Machine (SVM) and neural network analysis (NNA). In classification of normal and epileptic, results obtained exhibited an accuracy of 100% by applying NNA and 99% by SVM it has been found that the computation time of NNA classifier is lesser than SVM to provide 100% accuracy.

PurposeThe widespread acceptance of various social platforms has increased the number of users posting about various services based on their experiences about the services. Finding out the intended ratings of social media (SM) posts is important for both organizations and prospective users since these posts can help in capturing the user’s perspectives. However, unlike merchant websites, the SM posts related to the service-experience cannot be rated unless explicitly mentioned in the comments. Additionally, predicting ratings can also help to build a database using recent comments for testing recommender algorithms in various scenarios.Design/methodology/approachIn this study, the authors have predicted the ratings of SM posts using linear (Naïve Bayes, max-entropy) and non-linear (k-nearest neighbor, k-NN) classifiers utilizing combinations of different features, sentiment scores and emotion scores.FindingsOverall, the results of this study reveal that the non-linear classifier (k-NN classifier) performed better than the linear classifiers (Naïve Bayes, Max-entropy classifier). Results also show an improvement of performance where the classifier was combined with sentiment and emotion scores. Introduction of the feature “factors of importance” or “the latent factors” also show an improvement of the classifier performance.Originality/valueThis study provides a new avenue of predicting ratings of SM feeds by the use of machine learning algorithms along with a combination of different features like emotional aspects and latent factors.

In a usual crystallization process, the researchers evaluate the protein crystallization growth states based on visual impressions and repeatedly assign scores throughout the growth process. Although the development of crystallization robotic systems has generally realised the automation of the setup and storage of crystallization samples, evaluation of crystallization states has not yet been completely automated. The method presented here attempts to categorize individual crystallization droplet images into five classes using multiple classifiers. In particular, linear and nonlinear classifiers are utilized. The algorithm is comprised of pre-processing, feature extraction from images using texture analysis and a categorization process using linear discriminant analysis (LDA) and support vector machine (SVM). The performance of this method has been evaluated by comparing the results obtained using the method with the results obtained by a human expert and the concordance rate was 84.4%.

Error Correcting Output Codes reveal an efficient strategy in dealing with multi-class classification problems. According to this technique, a multi-class problem is decomposed into several binary ones. On these created sub-problems we apply binary classifiers and then, by combining the acquired solutions, we are able to solve the initial multi-class problem. In this paper we consider the optimization of the Linear Discriminant Error Correcting Output Codes framework using Particle Swarm Optimization. In particular, we apply the Particle Swarm Optimization algorithm in order to optimally select the free parameters that control the split of the initial problem’s classes into sub-classes. Moreover, by using the Support Vector Machine as classifier we can additionally apply the Particle Swarm Optimization algorithm to tune its free parameters. Our experimental results show that by applying Particle Swarm Optimization on the Sub-class Linear Discriminant Error Correcting Output Codes framework we get a significant improvement in the classification performance.

In multiclass classification problems we face the challenge of having many binary classifiers. Consulting this large number of classifiers might be confusing and time consuming. In this paper, we propose a new framework for training and prediction in multiclass problems. In this framework, we perform traditional training. Next we map training examples to prediction models. Finally we produce the Example Classifier (EC). In prediction a new example is passed through the EC to determine the appropriate classifier which in turn makes the last prediction decision. We conduct experiments comparing our framework with one-VS-one and Directed Acyclic Graph (DAG) using Support Vector Machines. Additionally, we compare our model with well-known ensemble models, namely, AdaBoost and Bagging, Our results indicate that prediction accuracy is comparable to other methodologies with the advantage of consuming less prediction time.