Introduction to multivariate discrimination

Class imbalance is a major problem in machine learning. It occurs when the number of instances in the majority class is significantly more than the number of instances in the minority class. This is a common problem which is recurring in most datasets, including the one used in this paper (i.e. direct marketing dataset). In direct marketing, businesses are interested in identifying potential buyers, or charities wish to identify potential givers. Several solutions have been suggested in the literature to address this problem, amongst which are data-level techniques, algorithmic-level techniques and a combination of both. In this paper, a model is proposed to solve imbalanced data using a Hybrid of Data-level and Algorithmic-level solutions (HybridDA), which involves oversampling the minority class, undersampling the majority class, and additionally, optimising the cost parameter, the gamma and the kernel type of Support Vector Machines (SVM) using a grid search. The proposed model perfomed competitively compared with other models on the same dataset. The dataset used in this work are real-world data collected from a Portuguese marketing campaign for bank-deposit subscriptions and are available from the University of California, Irvine (UCI) Machine Learning Repository.

Support Vector Machine (SVM) has become a popular method in machine learning in recent years. SVM is widely used as a tool in many classifiction areas. The selection optimum values of the parameters (model selection) for SVM is an important step in SVM design. This paper presents the results of research and testing method Particle Swarm Optimization (PSO) in selecting the parameters for SVM. Here we focus on the selection parameter γ of the RBF function and soft margin parameter C of SVM for classification problems. In this paper we propose a Particle Swarm Optimization variation model which combine the cognition-only model with the social-only model and use randomised low discrepancy sequences for initialising particle swarms. We then evaluate the suggested strategies with a series of experiments on 6 benchmark datasets. Experimental results demonstrate the model we proposed better than grid search and full PSO-SVM model.

Over-fitting is a ubiquitous problem in machine learning, and a variety of techniques to avoid over-fitting the training sample have proven highly effective, including early stopping, regularization, and ensemble methods. However, while over-fitting in training is widely appreciated and its avoidance now a standard element of best practice, over-fitting can also occur in model selection. This form of over-fitting can significantly degrade generalization performance, but has thus far received little attention. For example the kernel and regularization parameters of a support vector machine are often tuned by optimizing a cross-validation based model selection criterion. However the cross-validation estimate of generalization performance will inevitably have a finite variance, such that its minimizer depends on the particular sample on which it is evaluated, and this will generally differ from the minimizer of the true generalization error. Therefore if the cross-validation error is aggressively minimized, generalization performance may be substantially degraded. In general, the smaller the amount of data available, the higher the variance of the model selection criterion, and hence the more likely over-fitting in model selection will be a significant problem. Similarly, the more hyper-parameters to be tuned in model selection, the more easily the variance of the model selection criterion can be exploited, which again increases the likelihood of over-fitting in model selection. Over-fitting in model selection is empirically demonstrated to pose a substantial pitfall in the application of kernel learning methods and Gaussian process classifiers. Furthermore, evaluation of machine learning methods can easily be significantly biased unless the evaluation protocol properly accounts for this type of over-fitting. Fortunately the common solutions to avoiding over-fitting in training also appear to be effective in avoiding over-fitting in model selection. Three examples are presented based on regularization of the model selection criterion, early stopping in model selection and minimizing the number of hyper-parameters to be tuned during model selection.

This chapter explains the important concepts and basics requirements of support vector machine (SVM) and apply it to real-life models. It focuses on forecasting the outcome values of time series using an SVM. The objectives of statistical learning are understanding and forecasting. SVM is used as a popular tool for classification of problems in machine learning. SVMs simply separate the given data into different classes by finding the line or hyper-plane and separate the kernel supports to find the hyper-plane in the higher dimension. In regression problems, the support vector regression works on the same principle as SVM. To reduce computational complexity in SVM, J. A. K. Suykensand and J. Vandewalle introduced the LS-SVM classifier in 1999. Though LS-SVM is suitable for time series forecasting, to make it more convenient, it has been modified to DLS-SVM using a slightly improved computation technique.

Decentralized optimization to minimize a finite sum of functions, distributed over a network of nodes, has been a significant area within control and signal-processing research due to its natural relevance to optimal control and signal estimation problems. More recently, the emergence of sophisticated computing and large-scale data science needs have led to a resurgence of activity in this area. In this article, we discuss decentralized first-order gradient methods, which have found tremendous success in control, signal processing, and machine learning problems, where such methods, due to their simplicity, serve as the first method of choice for many complex inference and training tasks. In particular, we provide a general framework of decentralized first-order methods that is applicable to directed and undirected communication networks alike and show that much of the existing work on optimization and consensus can be related explicitly to this framework. We further extend the discussion to decentralized stochastic first-order methods that rely on stochastic gradients at each node and describe how local variance reduction schemes, previously shown to have promise in the centralized settings, are able to improve the performance of decentralized methods when combined with what is known as gradient tracking. We motivate and demonstrate the effectiveness of the corresponding methods in the context of machine learning and signal-processing problems that arise in decentralized environments.

Cassava cultivation is important to the nation because of the products and raw materials that are supplied to industries. However, Its cultivation is affected by diseases such as cassava brown leaf and cassava mosaic, which have been classified using a Support Vector Machine (SVM). SqueezeNet pretrained Convolutional Neural Network (SPCNN) is adjudged a very effective image detection and classification algorithm with limited usage areas. This research, therefore was used to detect and classify cassava leaf diseases using SPCNN and SPCNN with multiclass SVM. Performance evaluation of both techniques showed that SPCNN effectively detected and classified cassava leaf diseases with optimal accuracies of 92.22% and 98.89% for SPCNN and SPCNN-SVM, respectively. Keywords- SqueezeNet, SVM, Cassava diseases, Feature extraction, Transfer learning Arinola I. O., Oke O. A. & Falohun A. S. (2026): Detection and Classification of Cassava Leaf Diseases using Squeezenet Pretrained Convolutional Neural Network and Support Vector Machine. Journal of Advances in Mathematical & Computational Science. Vol. 14, No. 1. Pp 1-12 Available online at www.isteams.net/mathematics-computationaljournal. dx.doi.org/10.22624/AIMS/MATHS/V14N1P1

흡수, 분포, 대사, 배설 특성 및 독성을 예측하기 위한 효과적인 툴을 개발하는 것은 신약개발의 초기단계에서 NCE(new chemical entity)에 대한 가장 중요한 업무 중의 하나이다. 최근에 이런 시도중의 하나로서 ADME/T(absorption, distribution, metabolism, excretion, toxicity)관련 성질들의 예측에 support vector machine(SVM)을 이용하고 있다. 그리고 SVM은 ADME/T 성질들을 정확하게 예측하는데 많이 사용 되고 있다. 그러나 SVM 모델링에 두 가지 문제가 있다. 특성 선택(feature selection) 과 매개변수 설정(parameter setting)은 여전히 해결해야 할 과제이다. 이 두 가지 문제들은 SVM 분류의 효율성과 정확도에 결정적인 영향을 끼친다. 특히 특성 선택과 최적화된 SVM 변수의 설정은 서로 영향을 주기 때문에 동시에 다루어져야 한다. 여기서 우리는 genetic algorithm(GA) – 특성 선택에 사용 – 과 grid search(GS) method– 변수최적화에 사용 – 두 가지를 통합하는 효과적인 해결책을 제시하였다. ADME/T관련 성질 중 하나인 심장부정맥을 야기시키는 hERG 이온채널 저해제 분류 모델이 여기서 제안된 GA-GS-SVM을 위해 할당되고 테스트 되었다. 1891개의 화합물을 가지는 트레이닝 셋으로 단일 모델 3개, 앙상블 모델 3개, 총 6개의 모델을 만들었고 175개의 외부 데이터를 테스트 셋으로 사용하여 검증하였다. 데이터의 불균형 문제를 해결하기 위하여 GA-GS-SVM 단일 모델에 의한 예측 정확도와 GA-GS-SVM 앙상블 모델 예측 정확도를 비교하였으며, 앙상블모델을 사용하여 예측의 정확도를 높일 수 있었다. Developing effective tools for predicting absorption, distribution, metabolism, excretion properties and toxicity (ADME/T) of new chemical entities in the early stage of drug design is one of the most important tasks in drug discovery and development today. As one of these attempts, support vector machines (SVM) has recently been exploited for the prediction of ADME/T related properties. However, two problems in SVM modeling, i.e. feature selection and parameters setting, are still far from solved. The two problems have been shown to be crucial to the efficiency and accuracy of SVM classification. In particular, the feature selection and optimal SVM parameters setting influence each other, which indicates that they should be dealt with simultaneously. In this account, we present an integrated practical solution, in which genetic-based algorithm (GA) is used for feature selection and grid search (GS) method for parameters optimization. hERG ion-channel inhibitor classification models of ADME/T related properties has been built for assessing and testing the proposed GA-GS-SVM. We generated 6 different models that are 3 different single models and 3 different ensemble models using training set - 1891 compounds and validated with external test set - 175 compounds. We compared single model with ensemble model to solve data imbalance problems. It was able to improve accuracy of prediction to use ensemble model.

Using information technology such as search engine for collecting and monitoring of network public opinion is a practical and effective method. This paper puts forward an improved algorithm of SVM classifier based on incremental learning online, and then implements a topic crawler system for network public opinion and to grab the public opinion.

The problem of optimization in machine learning is well established but it entails several approximations. The theory of Hilbert spaces, which is principled and well established, helps solve the representation problem in machine learning by providing a rich (universal) class of functions where the optimization can be conducted. Working with functions is cumbersome, but for the class of reproducing kernel Hilbert spaces (RKHSs) it is still manageable provided the algorithm is restricted to inner products. The best example is the support vector machine (SVM), which is a batch mode algorithm that uses a very efficient (supralinear) optimization procedure. However, the problem of SVMs is that they display large memory and computational complexity. For the large-scale data limit, SVMs are restrictive because for fast operation the Gram matrix, which increases with the square of the number of samples, must fit in computer memory. The computation in this best-case scenario is also proportional to number of samples square. This is not specific to the SVM algorithm and is shared by kernel regression. There are also other relevant data processing scenarios such as streaming data (also called a time series) where the size of the data is unbounded and potentially nonstationary, therefore batch mode is not directly applicable and brings added difficulties.

Beras Miskin (Raskin) Program is a program of social protection, as supporters of other programs such as nutrition improvement, healthy increase, education and productivity improvement of Poor Households. According to Badan Pusat Statistika, there were 14 criteria to determine a household is classified as poor households. Based on these criteria it will be classified of recipient households and non-recipient households of Beras Miskin (Raskin) Program by Support Vector Machine (SVM) method using LibSVM. The concept of classification by SVM is search for the best hyperplane which serves as a separator of two classes of data in the input space. Kernel function is used to convert the data into a higher dimensional space to allow a separation. LibSVM is a package program created by Chih-Chung Chang and Chih-Jen Lin from Department of Computer Science at National Taiwan University. The method used by LibSVM to obtain global solution of duality lagrange problem is decomposition method. To determine the best parameters of kernel function, used k-vold cross validation method and grid search algorithm. In this classification by SVM method using LibSVM, obtain the best accuracy value as 83,1933%, which is the kernel function Radial Basis Function (RBF). Keywords : Beras Miskin (Raskin) Program, Classification, Support Vector Machine (SVM), LibSVM, Kernel Function

Distributed Denial of Service (DDoS) attacks represent a significant danger in network security because they can lead to extensive service interruptions. With these attacks increasingly mirroring regular traffic, smart and effective detection systems are essential. This research seeks to assess the efficacy of the Support Vector Machine (SVM) classification algorithm in identifying DDoS attacks in network traffic. The data utilized is CICIDS2017, focusing on the subset Friday-WorkingHours-Afternoon-DDos.pcap_ISCX.csv, which contains both legitimate traffic and DDoS attacks like DoS-Hulk, DoS-GoldenEye, and DDoS. The preprocessing stage included eliminating duplicates and null entries, label binary encoding, normalization through Min-Max Scaler, and feature selection applying the Chi-Square technique. The data was divided into 80% for training and 20% for testing purposes. The Radial Basis Function (RBF) kernel was utilized to train the SVM model, and hyperparameter optimization was performed with GridSearchCV. The evaluation of the model's performance was conducted through accuracy, precision, recall, F1-score, confusion matrix, and visual representations including ROC and Precision-Recall Curves. The findings indicate that prior to tuning, the model reached an accuracy of 97%, which increased to 99% post-tuning, accompanied by an F1-score of 0.99. This shows that the SVM algorithm, when paired with appropriate preprocessing and optimization, is very efficient in identifying DDoS attacks within network traffic.

Novel Artificial Immune Networks-based optimization of shallow machine learning (ML) classifiers

Image recognition is a feature extraction and pattern matching technique to classify various objectives in computer science. In theory, it mainly relies on the images with distinguishable features as a good starting point. Each image has its own unique characteristics, and image features can be categorized into color features, texture features, and shape features etc. Therefore, using modern recognition techniques, we extract the image features through various algorithms to search for images with high similarities. There are many image recognition algorithms, and some of them are with high recognition rate while others are robust. But not all algorithms can be unconditionally implemented without adjusting to the real situation. In this paper, we introduce the classification algorithm based on Support Vector Machine (SVM) and the feature extraction method based on Principal Components Analysis (PCA). We employ the feature extraction algorithm to characterize facial features and recognize faces by comparing them to those stored in the training data set. Finally, we show the applications of feature extraction and classification algorithms in character recognition. The real characters printed on a cord are preprocessed by PCA, and then classified and identified by SVM with a good recognition rate if properly processed.

Breast cancer is becoming a leading cause of death among women in the whole world, meanwhile, it is confirmed that the early detection and accurate diagnosis of this disease can ensure a long survival of the patients. In this paper, a swarm intelligence technique based support vector machine classifier (PSO_SVM) is proposed for breast cancer diagnosis. In the proposed PSO-SVM, the issue of model selection and feature selection in SVM is simultaneously solved under particle swarm (PSO optimization) framework. A weighted function is adopted to design the objective function of PSO, which takes into account the average accuracy rates of SVM (ACC), the number of support vectors (SVs) and the selected features simultaneously. Furthermore, time varying acceleration coefficients (TVAC) and inertia weight (TVIW) are employed to efficiently control the local and global search in PSO algorithm. The effectiveness of PSO-SVM has been rigorously evaluated against the Wisconsin Breast Cancer Dataset (WBCD), which is commonly used among researchers who use machine learning methods for breast cancer diagnosis. The proposed system is compared with the grid search method with feature selection by F-score. The experimental results demonstrate that the proposed approach not only obtains much more appropriate model parameters and discriminative feature subset, but also needs smaller set of SVs for training, giving high predictive accuracy. In addition, Compared to the existing methods in previous studies, the proposed system can also be regarded as a promising success with the excellent classification accuracy of 99.3% via 10-fold cross validation (CV) analysis. Moreover, a combination of five informative features is identified, which might provide important insights to the nature of the breast cancer disease and give an important clue for the physicians to take a closer attention. We believe the promising result can ensure that the physicians make very accurate diagnostic decision in clinical breast cancer diagnosis.

Diabetic Retinopathy (DR) is a microangiopathic complication of diabetes mellitus that can cause visual impairment to permanent blindness. Early detection of DR is essential to prevent disease progression, but conventional methods require time, cost, and expertise that are not always available. This study aims to compare the performance of the Random Forest (RF) and Support Vector Machine (SVM) algorithms in DR classification based on texture features extracted from retinal fundus images. The dataset used consists of 3,000 retinal fundus images obtained from the Kaggle platform, divided into 2,400 training data and 600 test data. Image preprocessing includes conversion to grayscale, resizing to a resolution of 128×128 pixels, and normalization. Feature extraction is performed using a combination of Local Binary Pattern (LBP) and Gray Level Co-occurrence Matrix (GLCM) to produce a 14-dimensional feature vector. Performance evaluation uses accuracy, precision, recall, F1-score, ROC curve, and 5-fold cross-validation metrics. The results showed that Random Forest significantly outperformed SVM with an accuracy of 96% compared to 64%, an AUC value of 0.99 compared to 0.72, and an average cross-validation accuracy of 94.5% compared to 63.42%. Random Forest also showed balanced performance in both classes with precision, recall, and F1-score of 0.96, while SVM experienced classification imbalance especially in the disease class. This study proves that Random Forest is a more optimal algorithm for an automatic DR detection system based on fundus image texture features and can support increasing the accessibility of DR screening in areas with limited specialist medical personnel.