Multi-class Classification: A Coding Based Space Partitioning

Machine learning has achieved remarkable success across diverse domains. Nevertheless, concerns about interpretability in black-box models, especially within Deep Neural Networks (DNNs), have become pronounced in safety-critical fields like healthcare and finance. Classical machine learning (ML) classifiers, known for their higher interpretability, are preferred in these domains. Similar to DNNs, classical ML classifiers can exhibit bugs that could lead to severe consequences in practice. Test input prioritization has emerged as a promising approach to ensure the quality of an ML system, which prioritizes potentially misclassified tests so that such tests can be identified earlier with limited manual labeling costs. However, when applying to classical ML classifiers, existing DNN test prioritization methods are constrained from three perspectives: 1) Coverage-based methods are inefficient and time-consuming; 2) Mutation-based methods cannot be adapted to classical ML models due to mismatched model mutation rules; 3) Confidence-based methods are restricted to a single dimension when applying to binary ML classifiers, solely depending on the model’s prediction probability for one class. To overcome the challenges, we propose MLPrior, a test prioritization approach specifically tailored for classical ML models. MLPrior leverages the characteristics of classical ML classifiers (i.e., interpretable models and carefully engineered attribute features) to prioritize test inputs. The foundational principles are: 1) tests more sensitive to mutations are more likely to be misclassified, and 2) tests closer to the model’s decision boundary are more likely to be misclassified. Building on the first concept, we design mutation rules to generate two types of mutation features (i.e., <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">model mutation features</b> and <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">input mutation features</b> ) for each test. Drawing from the second notion, MLPrior generates <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">attribute features</b> of each test based on its attribute values, which can indirectly reveal the proximity between the test and the decision boundary. For each test, MLPrior combines all three types of features of it into a final vector. Subsequently, MLPrior employs a pre-trained ranking model to predict the misclassification probability of each test based on its final vector and ranks tests accordingly. We conducted an extensive study to evaluate MLPrior based on 185 subjects, encompassing natural datasets, mixed noisy datasets, and fairness datasets. The results demonstrate that MLPrior outperforms all the compared test prioritization approaches, with an average improvement of 14.74%∼66.93% on natural datasets, 18.55%∼67.73% on mixed noisy datasets, and 15.34%∼62.72% on fairness datasets.

The present study aimed to assess the use of technical-tactical variables and machine learning (ML) classifiers in the automatic classification of the passing difficulty (DP) level in soccer matches and to illustrate the use of the model with the best performance to distinguish the best passing players. We compared eight ML classifiers according to their accuracy performance in classifying passing events using 35 technical-tactical variables based on spatiotemporal data. The Support Vector Machine (SVM) algorithm achieved a balanced accuracy of 0.70 ± 0.04%, considering a multi-class classification. Next, we illustrate the use of the best-performing classifier in the assessment of players. In our study, 2,522 pass actions were classified by the SVM algorithm as low (53.9%), medium (23.6%), and high difficulty passes (22.5%). Furthermore, we used successful rates in low-DP, medium-DP, and high-DP as inputs for principal component analysis (PCA). The first principal component (PC1) showed a higher correlation with high-DP (0.80), followed by medium-DP (0.73), and low-DP accuracy (0.24). The PC1 scores were used to rank the best passing players. This information can be a very rich performance indication by ranking the best passing players and teams and can be applied in offensive sequences analysis and talent identification.

Deep learning has shown promising results in various text-based classification tasks. However, deep learning performance is affected by the number of data, i.e., when the number of data is small, deep learning algorithms do not perform well, and vice versa. Classical machine learning algorithms commonly work well for a few data, and their performance reaches an optimal value and does not increase with the increase in sample data. Therefore, this study aimed to compare the performance of classical machine learning and deep learning methods to detect temperament based on Indonesian Twitter. In this study, the proposed Indonesian Linguistic Inquiry and Word Count were employed to analyze the context of Twitter. The classical machine learning methods implemented were support vector machine and K-nearest neighbor, whereas the deep learning method employed was a convolutional neural network (CNN) with three different architectures. Both learning methods were implemented using multiclass classification and one versus all (OVA) multiclass classification. The highest average f-measure was 58.73%, obtained by CNN OVA with a pool size of 3, a dropout value of 0.7, and a learning rate value of 0.0007.

Significant research has been done in the medical domain using machine learning and clinical data sets. Although there are many interesting and influential clinical research works in the fields of healthcare and health services using machine learning, there is a need to apply machine learning in the field of health human resource planning. This study uses physician billing data and machine learning to identify and classify family physicians with the goal of improving health human resource planning. This research is essential for policy makers because it is important to know the number of family physicians practicing in certain geographical regions for providing timely care. Additionally, this issue becomes particularly important when it comes to serving communities with fewer resources such as the rural areas of Northwestern Ontario, where family physicians need to work to their full scope of practice, provide more services than physicians working in urban areas, to meet the needs of patients. In this study, recursive feature elimination method is used to reduce the number of predictors for the classification problems. As the result of this process, the most important features include physician's rurality, full-time equivalent hours, age, and years of experience. Further, several machine learning models are used to solve binary and multi-class classification problems. Gradient boosting machine learning was the most accurate in predicting family physician practice, with a receiver operating characteristic value, ROC value, of 0.73 and 0.72 for binary and multi-class classification, respectively.

Overlooked pitfalls in multi-class machine learning classification in radiation oncology and how to avoid them

Fall prevention has always been a crucial topic for injury prevention. Research shows that real-time posture monitoring and subsequent fall prevention are important for the prevention of fall-related injuries. In this research, we determine a real-time posture classifier by comparing classical and deep machine learning classifiers in terms of their accuracy and robustness for posture classification. For this, multiple classical classifiers, including classical machine learning, support vector machine, random forest, neural network, and Adaboost methods, were used. Deep learning methods, including LSTM and transformer, were used for posture classification. In the experiment, joint data were obtained using an RGBD camera. The results show that classical machine learning posture classifier accuracy was between 75% and 99%, demonstrating that the use of classical machine learning classification alone is sufficient for real-time posture classification even with missing joints or added noise. The deep learning method LSTM was also effective in classifying the postures with high accuracy, despite incurring a significant computational overhead cost, thus compromising the real-time posture classification performance. The research thus shows that classical machine learning methods are worthy of our attention, at least, to consider for reuse or reinvention, especially for real-time posture classification tasks. The insight of using a classical posture classifier for large-scale human posture classification is also given through this research.

All-in-one multicategory least squares nonparallel hyperplanes support vector machine

Machine learning involves the task of training systems to be able to make decisions without being explicitly programmed. Important among machine learning tasks is classification involving the process of training machines to make predictions from predefined labels. Classification is broadly categorized into three distinct groups: single-label (SL), multi-class, and multi-label (ML) classification. This research work presents an application of a multi-label classification (MLC) technique in automating Quranic verses labeling. MLC has been gaining attention in recent years. This is due to the increasing amount of works based on real-world classification problems of multi-label data. In traditional classification problems, patterns are associated with a single-label from a set of disjoint labels. However, in MLC, an instance of data is associated with a set of labels. In this paper, three standard &lt;em&gt;MLC&lt;/em&gt; methods: &lt;span&gt;binary relevance (BR), classifier chain (CC), and label powerset (LP) algorithms are implemented with four baseline classifiers: support vector machine (SVM), naïve Bayes (NB), k-nearest neighbors (k-NN), and J48. The research methodology adopts the multi-label problem transformation (PT) approach. The results are validated using six conventional performance metrics. These include: hamming loss, accuracy, one error, micro-F1, macro-F1, and avg. precision. From the results, the classifiers effectively achieved above 70% accuracy mark. Overall, SVM achieved the best results with CC and LP algorithms.&lt;/span&gt;

The detection accuracy and model building time of machine learning (ML) classifiers are vital aspects for an intrusion detection system (IDS) to predict attacks in real life. Recently, researchers have introduced feature selection methods to increase the detection accuracy and minimize the model building time of a limited number of ML classifiers. Therefore, identifying more ML classifiers with very high detection accuracy and the lowest possible model building time is necessary. In this study, the authors tested six supervised classifiers on a full NSL-KDD training dataset (a benchmark record for Internet traffic) using 10-fold cross-validation in the Weka tool with and without feature selection/reduction methods. The authors aimed to identify more options to outperform and secure classifiers with the highest detection accuracy and lowest model building time. The results show that the feature selection/reduction methods, including the wrapper method in combination with the discretize filter, the filter method in combination with the discretize filter, and the discretize filter, can significantly decrease model building time without compromising detection accuracy. The suggested ML algorithms and feature selection/reduction methods are automated pattern recognition approaches to detect network attacks, which are within the scope of the Symmetry journal.

Malware detection at the hardware level has emerged recently as a promising solution to improve the security of computing systems. Hardware-based malware detectors take advantage of Machine Learning (ML) classifiers to detect pattern of malicious applications at run-time. These ML classifiers are trained using low-level features such as processor Hardware Performance Counters (HPCs) data which are captured at run-time to appropriately represent the application behaviour. Recent studies show the potential of standard ML-based classifiers for detecting malware using analysis of large number of microarchitectural events, more than the very limited number of HPC registers available in today’s microprocessors which varies from 2 to 8. This results in executing the application more than once to collect the required data, which in turn makes the solution less practical for effective run-time malware detection. Our results show a clear trade-off between the performance of standard ML classifiers and the number and diversity of HPCs available in modern microprocessors. This paper proposes a machine learning-based solution to break this trade-off to realize effective run-time detection of malware. We propose ensemble learning techniques to improve the performance of the hardware-based malware detectors despite using a very small number of microarchitectural events that are captured at run-time by existing HPCs, eliminating the need to run an application several times. For this purpose, eight robust machine learning models and two well-known ensemble learning classifiers applied on all studied ML models (sixteen in total) are implemented for malware detection and precisely compared and characterized in terms of detection accuracy, robustness, performance (accuracy × robustness), and hardware overheads. The experimental results show that the proposed ensemble learning-based malware detection with just 2 HPCs using ensemble technique outperforms standard classifiers with 8 HPCs by up to 17%. In addition, it can match the robustness and performance of standard ML-based detectors with 16 HPCs while using only 4 HPCs allowing effective run-time detection of malware.

Quantitative EEG features and machine learning classifiers for eye-blink artifact detection: A comparative study

This research aims to identify the best machine learning (ML) classification techniques for classifying the flow regimes in vertical gas-liquid two-phase flow. Two-phase flow regime identification is crucial for many operations in the oil and gas industry. Processes such as flow assurance, well control, and production rely heavily on accurate identification of flow regimes for their respective systems' smooth functioning. The primary motivation for the proposed ML classification algorithm selection processes was drilling and well control applications in Deepwater wells. The process started with vertical two-phase flow data collection from literature and two different flow loops. One, a 140 ft. tall vertical flow loop with a centralized inner metal pipe and a larger outer acrylic pipe. Second, an 18-ft long flow loop, also with a centralized, inner metal drill pipe. After extensive experimental and historical data collection, supervised and unsupervised ML classification models such as Multi-class Support vector machine (MCSVM), K-Nearest Neighbor Classifier (KNN), K-means clustering, and hierarchical clustering were fit on the datasets to separate the different flow regions. The next step was fine-tuning the models' parameters and kernels. The last step was to compare the different combinations of models and refining techniques for the best prediction accuracy and the least variance. Among the different models and combinations with refining techniques, the 5- fold cross-validated KNN algorithm, with 37 neighbors, gave the optimal solution with a 98% classification accuracy on the test data. The KNN model distinguished five major, distinct flow regions for the dataset and a few minor regions. These five regions were bubbly flow, slug flow, churn flow, annular flow, and intermittent flow. The KNN-generated flow regime maps matched well with those presented by Hasan and Kabir (2018). The MCSVM model produced visually similar flow maps to KNN but significantly underperformed them in prediction accuracy. The MCSVM training errors ranged between 50% - 60% at normal parameter values and costs but went up to 99% at abnormally high values. However, their prediction accuracy was below 50% even at these highly overfitted conditions. In unsupervised models, both clustering techniques pointed to an optimal cluster number between 10 and 15, consistent with the 14 we have in the dataset. Within the context of gas kicks and well control, a well-trained, reliable two-phase flow region classification algorithm offers many advantages. When trained with well-specific data, it can act as a black box for flow regime identification and subsequent well-control measure decisions for the well. Further advancements with more robust statistical training techniques can render these algorithms as a basis for well-control measures in drilling automation software. On a broader scale, these classification techniques have many applications in flow assurance, production, and any other area with gas-liquid two-phase flow.

The evolution and improvements of deep learning are being used to tackle any research obstacles that could be converted into classification problems in all spheres of life. Each Deep convolutional neural network (DCNN) design's output is determined by the depth and value of the hyperparameters, which explains why so many of them have been proposed. These DCNN architectures must be created entirely from scratch, and they can only be used for the applications for which they were intended. Transfer learning may be used to modify these pre-trained networks so they are more appropriate for particular purposes. This article aims to evaluate the empirical performance of the applicability of pre-trained DCNN models to identify human face presentation threats (FPAD). Human FPAD is one of the most significant and crucial areas of research right now because of the introduction of ambient computing, which necessitates contact-free identification of persons with the help of their biometric traits. Six pre-trained DCNN models are taken into account for an experimental evaluation in human FPAD alias VGG19, VGG16, DensNet121, MobileNet, Xception, and InceptionV3. The investigation makes use of the NUAA and Replay-Attack benchmark FPAD datasets. Thepade's sorted block truncation coding (SBTC) 10-ary features are merged with deep learning features produced from the finest performing finetuned DCNNs to enhance the FPAD capabilities of analyzed machine learning (ML) classifiers. The integration of features of Thepade's SBTC 10-ary and DCNN has considerably increased the FPAD accuracy of ML classifiers with slightly more computations of feature extraction.

Background: Early identification of probable post-traumatic stress disorder (PTSD) can lead to early intervention and treatment.Aims: This study aimed to evaluate supervised machine learning (ML) classifiers for the identification of probable PTSD in those who are serving, or have recently served in the United Kingdom (UK) Armed Forces.Methods: Supervised ML classification techniques were applied to a military cohort of 13,690 serving and ex-serving UK Armed Forces personnel to identify probable PTSD based on self-reported service exposures and a range of validated self-report measures. Data were collected between 2004 and 2009.Results: The predictive performance of supervised ML classifiers to detect cases of probable PTSD were encouraging when compared to a validated measure, demonstrating a capability of supervised ML to detect the cases of probable PTSD. It was possible to identify which variables contributed to the performance, including alcohol misuse, gender and deployment status. A satisfactory sensitivity was obtained across a range of supervised ML classifiers, but sensitivity was low, indicating a potential for false negative diagnoses.Conclusions: Detection of probable PTSD based on self-reported measurement data is feasible, may greatly reduce the burden on public health and improve operational efficiencies by enabling early intervention, before manifestation of symptoms.

Magnetic resonance imaging (MRI) is playing an important role in the classification of breast tumors. MRI can be used to obtain multiparametric (mp) information, such as structural, hemodynamic, and physiological information. Quantitative analysis of mp-MRI data has shown potential in improving the accuracy of breast tumor classification. In general, a large set of quantitative and texture features can be generated depending upon the type of methodology used. A suitable combination of selected quantitative and texture features can further improve the accuracy of tumor classification. Machine learning (ML) classifiers based upon features derived from MRI data have shown potential in tumor classification. There is a need for further research studies on selecting an appropriate combination of features and evaluating the performance of different ML classifiers for accurate classification of breast tumors. The objective of the current study was to develop and optimize an ML framework based upon mp-MRI features for the characterization of breast tumors (malignant vs. benign and low- vs. high-grade). This study included the breast mp-MRI data of 60 female patients with histopathology results. A total of 128 features were extracted from the mp-MRI tumor data followed by features selection. Five ML classifiers were evaluated for tumor classification using 10-fold crossvalidation with 10 repetitions. The support vector machine (SVM) classifier based on optimum features selected using a wrapper method with an adaptive boosting (AdaBoost) technique provided the highest sensitivity (0.96 ± 0.03), specificity (0.92 ± 0.09), and accuracy (94% ± 2.91%) in the classification of malignant versus benign tumors. This method also provided the highest sensitivity (0.94 ± 0.07), specificity (0.80 ± 0.05), and accuracy (90% ± 5.48%) in the classification of low- versus high-grade tumors. These findings suggest that the SVM classifier outperformed other ML methods in the binary classification of breast tumors.