Hardware-Based Support Vector Machine Classification in Logarithmic Number Systems

In the context of support vector machines (SVM), high dimensional input vectors often reduce the computational efficiency and significantly slow down the classification process. In this paper, we propose a strategy to rank individual components according to their influence on the class assignments. This ranking is used to select an appropriate subset of the features. It replaces the original feature set without significant loss in classification accuracy. Often, the generalization ability of the classifier even increases due to the implicit regularization achieved by feature pruning.

Deep Convolutional Neural Networks (CNNs) perform billions of operations for classifying a single input. To reduce these computations, this paper offers a solution that leverages a combination of runtime information and the algorithmic structure of CNNs. Specifically, in numerous modern CNNs, the outputs of compute-heavy convolution operations are fed to activation units that output zero if their input is negative. By exploiting this unique algorithmic property, we propose a predictive early activation technique, dubbed SnaPEA. This technique cuts the computation of convolution operations short if it determines that the output will be negative. SnaPEA can operate in two distinct modes, exact and predictive. In the exact mode, with no loss in classification accuracy, SnaPEA statically re-orders the weights based on their signs and periodically performs a single-bit sign check on the partial sum. Once the partial sum drops below zero, the rest of computations can simply be ignored, since the output value will be zero in any case. In the predictive mode, which trades the classification accuracy for larger savings, SnaPEA speculatively cuts the computation short even earlier than the exact mode. To control the accuracy, we develop a multi-variable optimization algorithm that thresholds the degree of speculation. As such, the proposed algorithm exposes a knob to gracefully navigate the trade-offs between the classification accuracy and computation reduction. Compared to a state-of-the-art CNN accelerator, SnaPEA in the exact mode, yields, on average, 28% speedup and 16% energy reduction in various modern CNNs without affecting their classification accuracy. With 3% loss in classification accuracy, on average, 67.8% of the convolutional layers can operate in the predictive mode. The average speedup and energy saving of these layers are 2.02x and 1.89x, respectively. The benefits grow to a maximum of 3.59x speedup and 3.14x energy reduction. Compared to static pruning approaches, which are complimentary to the dynamic approach of SnaPEA, our proposed technique offers up to 63% speedup and 49% energy reduction across the convolution layers with no loss in classification accuracy.

Using information technology such as search engine for collecting and monitoring of network public opinion is a practical and effective method. This paper puts forward an improved algorithm of SVM classifier based on incremental learning online, and then implements a topic crawler system for network public opinion and to grab the public opinion.

e14528 Background: Treatment monitoring in patients with High-Grade Glioma (HGG) and identification of disease progression, remains a major challenge in clinical neurooncology. We aimed to develop a support vector machine (SVM) classifier utilising combined longitudinal conventional and Dynamic Susceptibility Contrast (DSC) perfusion MRI to classify between Stable Disease (SD), Pseudoprogression (PsP) and Progressive Disease (PD) in glioma patients under surveillance. Methods: Conventional (269) and perfusion (62) MRI studies of HGG patients acquired between 2012 and 2018 were prospectively analysed. Study participants were separated into two groups: Group I with a single DSC time point (64 participants) and Group II with multiple DSC time points (19 participants). The SVM classifier was trained using all available MRI for each group. Classification accuracy was assessed for the use of features extracted from different feature dataset and time point combinations and compared to the experienced radiologists’ predictions. Results: The study included 64 participants (mean age: 48.5 ± 12.8 yrs [standard deviation], 24 female). SVM classification based on combined perfusion and structural features outperformed standalone datasets across all groups. For the clinically relevant classification step (SD/PSP vs PD), both feature combination as well as the addition of multiple DSC time points, improved classification performance (lowest median error rate: 0.016). The SVM algorithm outperformed radiologists in predicting lesion destiny in both groups. Optimal performance was observed in Group II, in which SVM sensitivity/specificity/accuracy was 100/91.67/94.7% for analysis based on the first time point and 85.71/100/ 94.7% based on multiple time points compared to 60/78/68% and 70/90/84.2% for the respective radiologist classifications. In Group I, the SVM also exceeded radiologist predictions, albeit by a smaller margin and resulted in sensitivity/specificity of 86.49/75.00/81.53% (SVM) compared to 75.7/68.9/73.84% (radiologists). Conclusions: Our results indicate that the addition of multiple longitudinal perfusion time points as well as the combination of structural and perfusion features significantly enhance classification outcome in treatment monitoring of HGGs and machine-learning-assisted diagnosis has potentially superior accuracy than the radiologist's visual evaluation and expertise.

Pattern recognition involves identifying a new instance as a member of a class based on the features extracted from problem space. Not all features contribute to increase in accuracy of classification. Wrapper based methodology selects an optimal subset of reduced features with no much loss in classification accuracy. Inspired by the bio evolutionary behaviour of flowers in optimal reproduction, a wrapper based optimal feature selection methodology is devised. In this paper, a Binary Flower Pollination Algorithm (BFPA) is proposed to address the combinatorial problem of feature selection. Support Vector machine (SVM) is used as a subset evaluator to calculate the fitness function to be optimized. Proposed algorithm is evaluated using 5 datasets from UCI repository. Switching probability is varied to achieve a fair tradeoff between exploration and exploitation. Feature subset chosen by the proposed method serves as input to 6 traditional classifiers to analyse the success of the algorithm. Experimental results are also compared with existing literature and observed that the proposed methodology identifies feature subset with reduced features and improved accuracy.

Motor imagery classification in electroencephalography (EEG)-based brain-computer interface (BCI) systems is an important research area. To simplify the complexity of the classification, selected power bands and electrode channels have been widely used to extract and select features from raw EEG signals, but there is still a loss in classification accuracy in the state-of-the-art approaches. To solve this problem, we propose a discriminative feature extraction algorithm based on power bands with principle component analysis (PCA). First, the raw EEG signals from the motor cortex area were filtered using a bandpass filter with μ and β bands. This research considered the power bands within a 0.4 second epoch to select the optimal feature space region. Next, the total feature dimensions were reduced by PCA and transformed into a final feature vector set. The selected features were classified by applying a support vector machine (SVM). The proposed method was compared with a state-of-art power band feature and shown to improve classification accuracy.

The problem with development of the support vector machine (SVM) classifiers using modified particle swarm optimization (PSO) algorithm and their ensembles has been considered. Solving this problem would allow fulfilling the high-precision data classification, especially Big Data classification, with the acceptable time expenditures. The modified PSO algorithm conducts a simultaneous search of the type of kernel functions, the parameters of the kernel function and the value of the regularization parameter for the SVM classifier. The idea of particles' «regeneration» served as the basis for the modified PSO algorithm. In the implementation of this algorithm, some particles change the type of their kernel function to the one which corresponds to the particle with the best value of the classification accuracy. The offered PSO algorithm allows reducing the time expenditures for the developed SVM classifiers, which is very important for Big Data classification problem. In most cases such SVM classifier provides the high quality of data classification. In exceptional cases the SVM ensembles based on the decorrelation maximization algorithm for the different strategies of the decision-making on the data classification and the majority vote rule can be used. Also, the two-level SVM classifier has been offered. This classifier works as the group of the SVM classifiers at the first level and as the SVM classifier on the base of the modified PSO algorithm at the second level. The results of experimental studies confirm the efficiency of the offered approaches for Big Data classification.

The paper presents an original methodology for the implementation of the Logarithmic Number System (LNS) arithmetic, which uses Reduced Instruction Set Computing (RISC). The core of the proposed method is a newly developed algorithm for conversion between LNS and the floating point (FLP) representations named “looping in sectors”, which brings about reduced memory consumption without a loss of accuracy. The resulting effective RISC conversions use only elementary computer operations without the need to employ multiplication, division, or other functions. Verification of the new concept and related developed algorithms for conversion between the LNS and the FLP representations was realized on Field Programmable Gate Arrays (FPGA), and the conversion accuracy was evaluated via simulation. Using the proposed method, a maximum relative conversion error of less than ±0.001% was achieved with a 22-ns delay and a total of 50 slices of FPGA consumed including memory cells. Promising applications of the proposed method are in embedded systems that are expanding into increasingly demanding applications, such as camera systems, lidars and 2D/3D image processing, neural networks, car control units, autonomous control systems that require more computing power, etc. In embedded systems for real-time control, the developed conversion algorithm can appear in two forms: as RISC conversions or as a simple RISC-based logarithmic addition.

Deep learning approaches for COVID-19 detection based on chest X-ray images

A boosted SVM classifier trained by incremental learning and decremental unlearning approach

In this work we present key elements of DeepChip, a framework that bridges recent trends in machine learning with applicable forward propagation on resource-constrained devices. Main objective of this work is to reduce compute and memory requirements by removing redundancy from neural networks. DeepChip features a flexible quantizer to reduce the bit width of activations to 8-bit fixed-point and weights to an asymmetric ternary representation. In combination with novel algorithms and data compression we leverage reduced precision and sparsity for efficient forward propagation on a wide range of processor architectures. We validate our approach on a set of different convolutional neural networks and datasets: ConvNet on SVHN, ResNet-44 on CIFAR10 and AlexNet on ImageNet. Compared to single-precision floating point, memory requirements can be compressed by a factor of 43, 22 and 10 and computations accelerated by a factor of 5.2, 2.8 and 2.0 on a mobile processor without a loss in classification accuracy. DeepChip allows trading accuracy for efficiency, and for instance tolerating about 2% loss in classification accuracy further reduces memory requirements by a factor of 88, 29 and 13, and speeds up computations by a factor of 6.0, 4.3 and 5.0. Code related to this paper is available at: https://github.com/UniHD-CEG/ECML2018.

This paper presents a video shot boundary detection system based on support vector machine (SVM) classification method. A hardware fully-parallel digital Support Vector Machine (SVM) classifier is used to detect the shot boundary in a continuous video stream. The throughput is increased by employing a pipelined architecture in the feature extraction stage. Hardware SVM can detect both cut and gradual transition in the video stream. Random pseudo-sampling techniques are employed to solve the class imbalance problem in SVM training. The internal wordlength is optimized for performance and hardware complexity. The threshold method in the postprocessing stage merges small subshots to reduce false alarms. The complete system is demonstrated on Xilinx Virtex IV XC4VSX35 FPGA platform to achieve 256 frames per second.

Wavelet SVM in Reproducing Kernel Hilbert Space for hyperspectral remote sensing image classification

We perceive the digital watermark detection as classification problem in image processing. We classify watermarked images as positive class whilst unwatermarked images as negative class. Support Vector Machine (SVM) is used as classifier of the watermarked and unwatermarked digital images. Two watermarking schemes i.e. Cox's spread spectrum (SS) and Single Value Decomposition (SVD) are used to embed watermark into digital images. These algorithms are selected based on their different level of robustness to Stirmark attacks. The payload of the watermark used for both algorithms is consistent at certain number of bits. SVM is trained with both the watermarked and unwatermarked images. Receiver Operating Characteristics (ROC) graphs are plotted to assess the statistical detection behavior of both the correlation detector and SVM classifier. We found that straight forward application of SVM leads to generalization problem. We suggest remedies to preprocess the training data in order to achieve substantially better performance from SVM classifier than those resulting from the straightforward application of SVM. Both watermarked and unwatermarked images are attacked under Stirmark and are then tested with the correlation detectors and SVM classifier. A comparison of the ROC of the correlation detectors and SVM classifier is performed to assess the accuracy of SVM classifier relative to correlation detectors. We found that SVM classifier has higher robustness to Stirmark attacks.

In order to improve the performance of Support Vector Machine (SVM) classifier for imbalanced data, this paper proposes dynamic cost-sensitive SVM classifier based on chaos particle swarm optimization (CPDC_SVM). Firstly, this paper introduces dynamic cost-sensitive thought to SVM classifier, and gives the method for structuring dynamic cost and cost-sensitive SVM model. Secondly, we propose the evaluation methodology performance for classifier, and adopts decimal base to code the particles. At last, chaos thought is introduced in particle swarm optimization algorithm, and the Algorithm of the dynamic cost-sensitive SVM classifier is given, which improves convergent speed and accuracy of particle swarm optimization, and can optimize dynamic cost-sensitive SVM well, so CPDC_SVM adds effectively the convergence speed and accuracy for the particle swarm optimization algorithm. Experimental results show CPDC_SVM has higher precision than traditional SVM classifier, and dynamic cost and chaos particle swarm optimization can improve the performance for classifier.