Fast support vector machine training and classification on graphics processors

This paper presents an optimized support vector machine (SVM) training processor employing the alternative direction method of multipliers (ADMM) optimizer. Low-rank approximation is exploited to reduce the dimension of the kernel matrix by employing the Nyström method. Verified in four datasets, the proposed ADMM-based training processor with rank approximation reduces 32× of matrix dimension with only 2% drop in inference accuracy. Compared to the conventional sequential minimal optimization (SMO) algorithm, the ADMM-based training algorithm is able to achieve a 9.8×10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">7</sup> shorter latency for training 2048 samples. Hardware optimization techniques, including pre-computation and memory sharing, are proposed to reduce the computational complexity by 62% and the memory usage by 60%. As a proof of concept, an epileptic seizure detector is designed to demonstrate the effectiveness of the proposed optimization techniques. The chip achieves a 153,310× higher energy efficiency and a 364× higher throughput-to-area ratio for SVM training than a high-end CPU. This work provides a promising solution for edge devices which require low-power and real-time training.

Sequential minimal optimization (SMO) algorithm is one of the simplest decomposition methods for learning of support vector machines (SVMs). Keerthi and Gilbert have recently studied the convergence property of SMO algorithm and given a proof that SMO algorithm always stops within a finite number of iterations. In this letter, we point out the incompleteness of their proof and give a more rigorous proof.

An optimization model of the portfolio adjusting problem with fuzzy return and a SMO algorithm

In radiation transport calculations, the effects of material temperature on neutron/nucleus interactions must be taken into account through Doppler broadening adjustments to the microscopic cross section data. Historically, Monte Carlo transport simulations have accounted for this temperature dependence by interpolating among precalculated Doppler broadened cross sections at a variety of temperatures. More recently, there has been much interest in on-the-fly Doppler broadening methods, where reference data is broadened on-demand during particle transport to any temperature. Unfortunately, Doppler broadening operations are expensive on traditional central processing unit (CPU) architectures, making on-the-fly Doppler broadening unaffordable without approximations or complex data preprocessing. This work considers the use of graphics processing unit (GPU)s, which excel at parallel data processing, for on-the-fly Doppler broadening in continuous-energy Monte Carlo simulations. Two methods are considered for the broadening operations – a GPU implementation of the standard SIGMA1 algorithm and a novel vectorized algorithm that leverages the convolution properties of the broadening operation in an attempt to expose additional parallelism. Numerical results demonstrate that similar cross section lookup throughput is obtained for on-the-fly broadening on a GPU as cross section lookup throughput with precomputed data on a CPU, implying that offloading Doppler broadening operations to a GPU may enable on-the-fly temperature treatment of cross sections without a noticeable reduction in cross section processing performance in Monte Carlo transport codes.

It is known that parallel computing systems have some advantages to solve large scale and hard problems. However, traditional multi-processor parallel architectures are not widely used since they are hard to operate and they have high cost in setting-up .It has been an important point to Graphical Processing Units (GPU) were shown to be used in general-purpose applications because they naturally have hundreds of parallel cores. Studies related with the usage of GPUs in general-purpose are increasing day by day. In this study, Fast Fourier Transform (FFT), Inverse Fast Fourier Transform (IFFT) and Radon Transforms widely used in signal and image processing problems have been implemented on Quadro FX 3800 GPU which is one of the recent graphical cards of NVIDIA. From the results, applications on GPU provided gain in terms of time as compared to the applications implemented on traditional Central Processing Unit (CPU).

Abstract. Thanks to the nature of the graphics processing, the newly released products offer highly parallel processing units with high-memory bandwidth and computational power of more than teraflops per second. The modern GPUs are not only powerful graphic engines but also they are high level parallel programmable processors with very fast computing capabilities and high-memory bandwidth speed compared to central processing units (CPU). Data-parallel computations can be shortly described as mapping data elements to parallel processing threads. The rapid development of GPUs programmability and capabilities attracted the attentions of researchers dealing with complex problems which need high level calculations. This interest has revealed the concepts of “General Purpose Computation on Graphics Processing Units (GPGPU)” and “stream processing”. The graphic processors are powerful hardware which is really cheap and affordable. So the graphic processors became an alternative to computer processors. The graphic chips which were standard application hardware have been transformed into modern, powerful and programmable processors to meet the overall needs. Especially in recent years, the phenomenon of the usage of graphics processing units in general purpose computation has led the researchers and developers to this point. The biggest problem is that the graphics processing units use different programming models unlike current programming methods. Therefore, an efficient GPU programming requires re-coding of the current program algorithm by considering the limitations and the structure of the graphics hardware. Currently, multi-core processors can not be programmed by using traditional programming methods. Event procedure programming method can not be used for programming the multi-core processors. GPUs are especially effective in finding solution for repetition of the computing steps for many data elements when high accuracy is needed. Thus, it provides the computing process more quickly and accurately. Compared to the GPUs, CPUs which perform just one computing in a time according to the flow control are slower in performance. This structure can be evaluated for various applications of computer technology. In this study covers how general purpose parallel programming and computational power of the GPUs can be used in photogrammetric applications especially direct georeferencing. The direct georeferencing algorithm is coded by using GPGPU method and CUDA (Compute Unified Device Architecture) programming language. Results provided by this method were compared with the traditional CPU programming. In the other application the projective rectification is coded by using GPGPU method and CUDA programming language. Sample images of various sizes, as compared to the results of the program were evaluated. GPGPU method can be used especially in repetition of same computations on highly dense data, thus finding the solution quickly.

This paper proposes to use border training patterns in order to improve Support Vector Machine (SVM) classification accuracy of hyperspectral images. In the proposed approach, border training patterns which are close to the separating hyperplane, are obtained in two consecutive steps and considered as final training set. In the first step, clustering is performed to the full initial training data of each class. Then, cluster centers of each class are taken as the reduced size training data and forwarded to the second step. In the second step, this reduced size training data is used in the training of SVM and cluster centers which are obtained as support vectors at this step are regarded to be located close to the hyperplane border. Finally, cluster centers which are found as support vectors and original training samples contained in these clusters only are assigned as border training patterns. Experimental results are presented to show that the proposed approach improves SVM classification accuracy.

The demand for multitasking on graphics processing units (GPUs) is constantly increasing as they have become one of the default components on modern computer systems along with traditional processors (CPUs). Preemptive multitasking on CPUs has been primarily supported through context switching. However, the same preemption strategy incurs substantial overhead due to the large context in GPUs. The overhead comes in two dimensions: a preempting kernel suffers from a long preemption latency, and the system throughput is wasted during the switch. Without precise control over the large preemption overhead, multitasking on GPUs has little use for applications with strict latency requirements. In this paper, we propose Chimera, a collaborative preemption approach that can precisely control the overhead for multitasking on GPUs. Chimera first introduces streaming multiprocessor (SM) flushing, which can instantly preempt an SM by detecting and exploiting idempotent execution. Chimera utilizes flushing collaboratively with two previously proposed preemption techniques for GPUs, namely context switching and draining to minimize throughput overhead while achieving a required preemption latency. Evaluations show that Chimera violates the deadline for only 0.2% of preemption requests when a 15us preemption latency constraint is used. For multi-programmed workloads, Chimera can improve the average normalized turnaround time by 5.5x, and system throughput by 12.2%.

Graphics Processing Units (GPUs) have been used to run a range of cryptographic algorithms. The main reason to choose a GPU is to accelerate the encryption/decryption speed. Since GPUs are mainly used for graphics rendering, and only recently have they become a fully-programmable parallel computing device, there has been little attention paid to their vulnerability to side-channel attacks. In this paper we present a study of side-channel vulnerability on a state-of-the-art graphics processor. To the best of our knowledge, this is the first work that attempts to extract the secret key of a block cipher implemented to run on a GPU. We present a side-channel power analysis methodology to extract all of the last round key bytes of a CUDA AES (Advanced Encryption Standard) implementation run on an NVIDIA TESLA GPU. We describe how we capture power traces and evaluate the power consumption of a GPU. We then construct an appropriate power model for the GPU. We propose effective methods to sample and process the GPU power traces so that we can recover the secret key of AES. Our results show that parallel computing hardware systems such as a GPU are highly vulnerable targets to power-based side-channel attacks, and need to be hardened against side-channel threats.

Land degradation caused by soil erosion remains an important global issue due to its adverse consequences on food security and environment. Geospatial prediction of erosion through susceptibility analysis is very crucial to sustainable watershed management. Previous susceptibility studies devoid of some crucial conditioning factors (CFs) termed dynamic CFs whose impacts on the accuracy have not been investigated. Thus, this study evaluates erosion susceptibility under the influence of both non-redundant static and dynamic CFs using support vector machine (SVM), remote sensing and GIS. The CFs considered include drainage density, lineament density, length-slope and soil erodibility as non-redundant static factors, and land surface temperature, soil moisture index, vegetation index and rainfall erosivity as the dynamic factors. The study implements four kernel tricks of SVM with sequential minimal optimization algorithm as a classifier for soil erosion susceptibility modeling. Using area under the curve (AUC) and Cohen’s kappa index (k) as the validation criteria, the results showed that polynomial function had the highest performance followed by linear and radial basis function. However, sigmoid SVM underperformed having the lowest AUC and k values coupled with higher classification errors. The CFs’ weights were implemented for the development of soil erosion susceptibility map. The map would assist planners and decision makers in optimal land-use planning, prevention of soil erosion and its related hazards leading to sustainable watershed management.

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

The aim of the study is to conduct subjective tests on audio excerpt assignment to music genre and to carry out automatic classification of musical genres with the use of decision algorithms. First, the musicology background of classifying music into styles and genres is discussed. Then, an online survey is created to perform subjective tests with a group of listeners, whose task is assigning audio samples to selected music genres. Next, a set of music descriptors is proposed and all music excerpts are parametrized. For checking parameter redundancy the Principal Component Analysis (PCA) is performed. The created database containing feature vectors is then utilized for automatic music genre classification. Two classifiers, namely: Belief Networks and SMO (Sequential Minimal Optimization Algorithm) are employed for the purpose of music genre classification. The last step of this study is to compare the efficiency of the listeners classification with the automatic music genre classification system designed by the authors. Conducted tests show to what extent listeners’ assignment and the automatic classification results agree. It is also observed that very known performers are often rated without problems. Contrarily, songs of less known artists are more difficult to assign to the given genre.

Internet of Things (IoT) emerged as one of the leading technological advancements of our days. IoT generates enormous quantities of valuable data that need on time processing, resulting in reliable, and accurate decisions based on the Internet of Things vision. The quality of the generated data is inadequate, incomplete, uncertain, and produced from multiple sources. Although cloud servers can analyze and store enormous data, they need a lot of time to send full‐size data for storage and analysis as well as the high overhead they have that not satisfactory in many applications. This article provides a systematic way to review the IoT environment according to big data analytics together with limitations and challenges. Moreover, a cloud‐fog‐mist combination for handling IoT data concerning centralized and distributed data mining is explained. A proposed hybrid real‐time remote patient monitoring framework introduced that consists of the integration among the mist, fog, and cloud for healthcare treatment, which remote‐monitors patients continuously. In addition, Reduced‐Error Pruning tree (REPtree), MultiLayer Perceptron, naïve Bayes, and Sequential Minimal Optimization algorithms have applied to “Gas sensors for home activity monitoring” dataset to demonstrate the feasibility of traditional data mining algorithms to IoT data. The results showed that the REPtree algorithm achieved better accuracy against others with accuracy ranged between 90.66% and 93.6% according to the size of the data used in the study. Still, for the time metric, naive Bayes outperformed them with the lowest time between 1 and 18 seconds for building the model.

Combination of independent component analysis and support vector machines for intelligent faults diagnosis of induction motors

In this paper, the authors investigated the applicability of combining machine-learning-based models toward slope stability assessment. To do this, several well-known machine-learning-based methods, namely multiple linear regression (MLR), multi-layer perceptron (MLP), radial basis function regression (RBFR), improved support vector machine using sequential minimal optimization algorithm (SMO-SVM), lazy k-nearest neighbor (IBK), random forest (RF), and random tree (RT), were selected to evaluate the stability of a slope through estimating the factor of safety (FOS). In the following, a comparative classification was carried out based on the five stability categories. Based on the respective values of total scores (the summation of scores obtained for the training and testing stages) of 15, 35, 48, 15, 50, 60, and 57, acquired for MLR, MLP, RBFR, SMO-SVM, IBK, RF, and RT, respectively, it was concluded that RF outperformed other intelligent models. The results of statistical indexes also prove the excellent prediction from the optimized structure of the ANN and RF techniques.