Deep convolutional cross-connected kernel mapping support vector machine based on SelectDropout

Smoothing methods, extensively used for solving important mathematical programming problems and applications, are applied here to generate and solve an unconstrained smooth reformulation of the support vector machine for pattern classification using a completely arbitrary kernel. We term such reformulation a smooth support vector machine (SSVM). A fast Newton–Armijo algorithm for solving the SSVM converges globally and quadratically. Numerical results and comparisons are given to demonstrate the effectiveness and speed of the algorithm. On six publicly available datasets, tenfold cross validation correctness of SSVM was the highest compared with four other methods as well as the fastest. On larger problems, SSVM was comparable or faster than SVMlight (T. Joachims, in Advances in Kernel Methods—Support Vector Learning, MIT Press: Cambridge, MA, 1999), SOR (O.L. Mangasarian and David R. Musicant, IEEE Transactions on Neural Networks, vol. 10, pp. 1032–1037, 1999) and SMO (J. Platt, in Advances in Kernel Methods—Support Vector Learning, MIT Press: Cambridge, MA, 1999). SSVM can also generate a highly nonlinear separating surface such as a checkerboard.

The inclusion in workplaces of video display terminals has brought multiple benefits for the organization of work. Nevertheless, it also implies a series of risks for the health of the workers, since it can cause ocular and visual disorders, among others. In this research, a group of eye and vision-related problems associated with prolonged computer use (known as computer vision syndrome) are studied. The aim is to select the characteristics of the subject that are most relevant for the occurrence of this syndrome, and then, to develop a classification model for its prediction. The estimate of this problem is made by means of support vector machines for classification. This machine learning technique will be trained with the support of a genetic algorithm. This provides the training of the support vector machine with different patterns of parameters, improving its performance. The model performance is verified in terms of the area under the ROC curve, which leads to a model with high accuracy in the classification of the syndrome.

Dimensionality reduction is one of the key issues of machine learning and data mining, especially for high-dimensional data set. In the literature, there are various dimensionality reduction methods, such as PCA, LDA, and KLDA, and the difference between them mainly lies in the optimization objective. In this paper, we propose a new dimensionality reduction method, whose optimization objective is to maximize the margin between different classes, after projecting the original features into some specific lower-dimensional subspace. The specific subspace is constructed with the help of soft margin support vector machines. Our experiments based on several real-world datasets show that this method improves the performance on classification, and it not only can reduce redundant information in features but also is robust to noise.

Application of the PSO–SVM model for recognition of control chart patterns

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

Automatic classification of the electrocardiogram (ECG) signals is an important subject for clinical diagnosis of heart disease. This study investigates the design of a high-efficient system to classify five types of ECG beat namely normal beats and four manifestations of heart arrhythmia, in twofold. First, we propose a system that includes two main modules: a feature extraction module and a classification module. Feature extraction module extracts a suitable combination of the ECG's morphological characteristics and timing interval features. Discrete wavelet transform is used to extract the morphological features. In the classification module, a multi-class support vector machine (SVM)-based classifier is employed. The parameters of this system are determined based on a trial and error method and its performance is evaluated for the MIT-BIH arrhythmia database. Extensive experiments on the parameters of this system such as classifier kernels and various types of features are conducted. These experiments show that in SVM training, the kernels, kernel parameters, and feature selection have very important roles for SVM classification accuracy. Therefore, most appropriates of these parameters should be used for SVM training. Then at the second fold, a novel hybrid intelligent system (HIS) is proposed that consists of three main modules. In the HIS, further to the two mentioned modules, an optimization module is added. In this module, a genetic algorithm is used for optimization of the relevant parameters of system. These parameters are: wavelet filter type for feature extraction, wavelet decomposition level, and classifier's parameters. Experimental results show that optimization improves the recognition system, efficiently, and HIS is more superior to the system, which as constant parameters.

This article presents improvements in the segmentation module, feature extraction module, and the classification module of a low-cost automated visual inspection (AVI) system for wood defect classification. One of the major drawbacks in the low-cost AVI system was the erroneous segmentation of clear wood regions as defects, which then introduces confusion in the classification module. To reduce this problem, we use the fuzzy min–max neural network for image segmentation (FMMIS). The FMMIS method grows boxes from a set of seed pixels, yielding ideally the minimum bounded rectangle for each defect present in the wood board image. Additional features with texture information are considered for the feature extraction module, and multi-class support vector machines are compared with multilayer perceptron neural networks in the classification module. Results using the FMMIS, additional features, and a pairwise classification support vector machine on a 550 test wood image set containing 11 defect categories show 91% of correct classification, which is significantly better than the original 75% of the low-cost AVI system. The use of computational intelligence techniques improved significantly the overall performance of the proposed automated visual inspection system for wood boards.

Automatic classification of electrocardiogram (ECG) is importance in cardiac disease diagnosis. Support vector machine (SVM) has drawn more and more attention on pattern recognition, including ECG feature extraction and cardiac disease detection. The most prominent advantage of SVM can be represent as its excellent performance on simplification of inner product operation from high dimensional space to low dimensional space, avoiding calculations in high dimension space. In this study, a multi-classification method is proposed utilizing wavelet multi-resolution analysis (WMRA) and SVM. WMRA is applied to eliminate interference with frequency beyond the frequency interval of ECG signals (0.05∼100Hz). Meanwhile, WMRA provides detail coefficients and approximation coefficients of different decomposition levels, which are the input features fed into SVM for classification. After that, SVM is employed to recognize 6 types of cardiac beats from MIT-BIH arrhythmia database. Besides, different parameters C and γ are discussed and tested. Experimental results indicate that the classification performance gets better as C increases and γ decreases. When C and γ are set to be 1000 and 0.1 respectively, an overall classification accuracy, sensitivity and positive predictivity of 95.23%, 97.42% and 97.71% respectively are achieved.

Support vector machine for classification based on fuzzy training data

In the standard support vector machines for classification, training sets with uneven class sizes results in classification biases towards the class with the large training size. That is to say, the larger the training sample size for one class is, the smaller its corresponding classification error rate is, while the smaller the sample size, the larger the classification error rate. The main causes lie in that the penalty of misclassification for each training sample is considered equally. Weighted support vector machines for classification are proposed in this paper where penalty of misclassification for each training sample is different. By setting the equal penalty for the training samples belonging to same class, and setting the ratio of penalties for different classes to the inverse ratio of the training class sizes, the obtained weighted support vector machines compensate for the undesirable effects caused by the uneven training class size, and the classification accuracy for the class with small training size is improved. Experimental simulations on breast cancer diagnosis show the effectiveness of the proposed methods.

Support Vector Machine (SVM) is a fundamental technique in machine learning. A long time challenge facing SVM is how to deal with outliers (caused by mislabeling), as they could make the classes in SVM nonseparable. Existing techniques, such as soft margin SVM, ν-SVM, and Core-SVM, can alleviate the problem to certain extent, but cannot completely resolve the issue. Recently, there are also techniques available for explicit outlier removal. But they suffer from high time complexity and cannot guarantee quality of solution. In this paper, we present a new combinatorial approach, called Random Gradient Descent Tree (or RGD-tree), to explicitly deal with outliers; this results in a new algorithm called RGD-SVM. Our technique yields provably good solution and can be efficiently implemented for practical purpose. The time and space complexities of our approach only linearly depend on the input size and the dimensionality of the space, which are significantly better than existing ones. Experiments on benchmark datasets suggest that our technique considerably outperforms several popular techniques in most of the cases.

We compare L1 and L2 soft margin support vector machines from the standpoint of positive definiteness, the number of support vectors, and uniqueness and degeneracy of solutions. Since the Hessian matrix of L2 SVM is positive definite, the number of support vectors for L2 SVM is larger than or equal to the number of L1 SVM. For L1 SVM, if there are plural irreducible sets of support vectors, the solution of the dual problem is non-unique although the primal problem is unique. Similar to L1 SVM, degenerate solutions, in which all the data are classified into one class, occur for L2 SVM.

Speech recognition systems aim to make human-machine communication quickly and easily. In recent years, various researches and studies have been carried out to develop speech recognition systems. Examples of these studies are speech recognition, speaker recognition and speaker verification. In this study, speech recognition systems were investigated, methods used in the literature were investigated and a Turkish speech recognition application was developed. The application consists of speech coding and speech recognition. Firstly 20 Turkish words which are frequently used on the computer were determined. There are 20 records from each word. A total of 400 words were recorded on the computer with a microphone. In the speech coding section of the application, these words recorded on the computer are encoded by the Linear Pre-estimation Coding (LPC) method and the LPC parameters for each word are obtained. In the speech recognition section of the application, the Support Vector Machines (SVM) method is used. Two types of SVM classifiers are designed. These are the Soft Margin SVM (SM-SVM) classifier and the Least Square SVM (LS-SVM) classifier. Classification consists of training and testing stages. Of the 400 coded words, 200 were used for the training phase and 200 were used for the testing phase. As a result, 91% accurate recognition success for the SM-SVM classifier; 71% correct recognition of the LS-SVM classifier has been achieved.

Over the years, communication through Short Message Service (SMS) has been a primary tool for mobile subscribers. SMS has varied applications in health, industry, finances, education and social networking among others. The growth of mobile devices and SMS usage has consequently increased the attack surface for cyber-criminals culminating to the proliferation of malicious activities introduced using SMS spam, phishing, spyware, malware etc. Ham messages are normal messages people trade with one another and are usually not unwanted by the recipient, while spam messages are unsolicited junk and redundant messages that may be sent to a large number of people at once and are usually unwanted. Various spam detection models have been developed using various traditional machine and deep learning techniques. However, most studies where comparison between deep and traditional machine learning algorithms is done, have unfortunately omitted K-Nearest Neighbors and Support Vector Machine (SVM) which are empirically deemed as the most popular traditional machine learning algorithms. In this study, therefore, we develop a deep learning model based on Recurrent Neural Network (RNN) for Spam and Ham SMS classification and compare its performance against SVM model for the same University of California (UCl) SMS dataset. The results show that RNN has a slightly higher training and validation accuracy of 0.98 compared to SVM at 0.94, however, the false positive rate of SVM is marginally lower. Exploring application of deep learning with better optimization algorithms such as RNN improves accuracy, reduces computational complexities i.e. memory consumption and speed, and thus minimizing false positive rates. For future work, we suggest the use of varied performance metrics to validate the model in a distributed dataset environment.

This paper presents a novel method for Face Recognition (FR) by applying Support Vector Machine (SVM) in addressing this Computer Vision (CV) problem. The SVM is a capable learning classifier capable of training polynomials, neural networks and RBFs. Singular Value Decomposition (SVD) are used for feature extraction and while SVM for classification. The proposed algorithm is tested on four databases, viz., FERET, FRGC Ver. 2.0, CMU-PIE and Indian Face Database. The singular values are filtered, sampled and classified using Gaussian RBF kernal for SVM. The results are compared with other known methods to establish the advantage of SV-M/D. The recall rate for the proposed system is about 90%. The colossal augmentation is due to the simple but efficient feature extraction and the ability to learn in a high dimensional space.