Feature Selection Methods Simultaneously Improve the Detection Accuracy and Model Building Time of Machine Learning Classifiers

In the realm of software development, detecting code smells is a critical task for ensuring good code quality. God class code smell specifically has a specific nature associated with a great deal of subjectivity due to the levels of coupling and cohesion associated to it. Automated techniques for code smell detection aim to resolve this subjectivity. Machine learning techniques have shown promising results that tend to improve accuracy and reduce the bias associated with other techniques for God class identification. This is due to their pattern recognition capabilities making them more objective in identifying patterns that indicate code smells. However, current results need to be further improved in terms of both accuracy and generalizability. The challenge in the use of machine learning is not only in selecting the most appropriate technique but also lies in effectively representing source code as input patterns fed to Machine Learning (ML) classifier(s). Code representation plays a pivotal role in encoding source code for ML algorithms. This study aims improving the accuracy and generalizability of God class code smell detection via exploring the effect of using various code representation techniques, namely, tree-based, metric-based, code embedding, and token-based code representation techniques on the ML detection results. The study is conducted on the MLCQ dataset, and applies various ML algorithms (specifically: Logistic Regression, Random Forest, SVM, Decision Tree, Naive Bayes, Gradient Boosting, XGBoost). The evaluation results show how different code representation techniques influence ML detection outcomes and the comparative performance of ML algorithms. The study findings reveal that the F1-score achieved outperforms prior studies on the MLCQ dataset, indicating the effectiveness of the proposed approach. The presented results reveal how the code representation technique used makes a significant impact on the ML classifier results. This paves the way for developing intelligent IDE plugins for just in time God Class code smell detection among other code smells.

Machine learning has achieved remarkable success across diverse domains. Nevertheless, concerns about interpretability in black-box models, especially within Deep Neural Networks (DNNs), have become pronounced in safety-critical fields like healthcare and finance. Classical machine learning (ML) classifiers, known for their higher interpretability, are preferred in these domains. Similar to DNNs, classical ML classifiers can exhibit bugs that could lead to severe consequences in practice. Test input prioritization has emerged as a promising approach to ensure the quality of an ML system, which prioritizes potentially misclassified tests so that such tests can be identified earlier with limited manual labeling costs. However, when applying to classical ML classifiers, existing DNN test prioritization methods are constrained from three perspectives: 1) Coverage-based methods are inefficient and time-consuming; 2) Mutation-based methods cannot be adapted to classical ML models due to mismatched model mutation rules; 3) Confidence-based methods are restricted to a single dimension when applying to binary ML classifiers, solely depending on the model’s prediction probability for one class. To overcome the challenges, we propose MLPrior, a test prioritization approach specifically tailored for classical ML models. MLPrior leverages the characteristics of classical ML classifiers (i.e., interpretable models and carefully engineered attribute features) to prioritize test inputs. The foundational principles are: 1) tests more sensitive to mutations are more likely to be misclassified, and 2) tests closer to the model’s decision boundary are more likely to be misclassified. Building on the first concept, we design mutation rules to generate two types of mutation features (i.e., <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">model mutation features</b> and <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">input mutation features</b> ) for each test. Drawing from the second notion, MLPrior generates <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">attribute features</b> of each test based on its attribute values, which can indirectly reveal the proximity between the test and the decision boundary. For each test, MLPrior combines all three types of features of it into a final vector. Subsequently, MLPrior employs a pre-trained ranking model to predict the misclassification probability of each test based on its final vector and ranks tests accordingly. We conducted an extensive study to evaluate MLPrior based on 185 subjects, encompassing natural datasets, mixed noisy datasets, and fairness datasets. The results demonstrate that MLPrior outperforms all the compared test prioritization approaches, with an average improvement of 14.74%∼66.93% on natural datasets, 18.55%∼67.73% on mixed noisy datasets, and 15.34%∼62.72% on fairness datasets.

Background Current heart failure guidelines emphasize the importance of timely detection of subclinical left ventricular (LV) remodelling and dysfunction for more precise risk stratification of asymptomatic subjects. Both LV diastolic dysfunction (LVDD) and LV hypertrophy (LVH) as assessed by echocardiography are known independent prognostic markers of future cardiovascular events in the community. However, selective screening strategies of individuals at risk who would benefit most from in-depth cardiac phenotyping are lacking. Purpose We assess the utility of several Machine Learning (ML) classifiers built on clinical and biochemical features for detecting subclinical LV abnormalities. Methods We included 1407 participants (mean age, 51 years, 51% women) randomly recruited from the general population. We used echocardiographic parameters reflecting LV diastolic function and structure to define LV abnormalities (LVDD, n=239; LVH, n=135). After that four supervised ML algorithms (Random Forest (RF), Gradient Boosting (GD), Stochastic Gradient Descent (SGD) and Support Vector Machines (SV)) were built based on routine clinical, hemodynamic and laboratory data (features; n=61) to categorize LVDD and LVH (two prediction tasks). We applied a 10-fold stratified cross-validation set-up. Results ML classifiers exhibited a high area under the ROC (AUC) for predicting LVDD with values between 88.5% and 93.1% (Figure, left panel). Age, BMI, different components of blood pressure, antihypertensive treatment, routine biomarkers such as serum electrolytes, creatinine, blood sugar, leptin, uric acid, lipid profile, as well as blood cell counts were the top selected features for predicting LVDD. Prediction AUC of ML algorithms for detection of LVH was somewhat lower than for LVDD and ranged from 72.5% to 78.7% (Figure, right panel). The top selected features for LVH classifier were similar to those of LVDD, but also included social class, serum gamma-glutamyl transferase, fasting insulin, plasma renin activity and cortisol. ROC curves (sensitivity-1-specificity) Conclusions ML algorithms combining routinely measured clinical and laboratory data have shown high accuracy of LVDD and LVH prediction. These ML classifiers might be useful to preselect individuals at risk for further in depth echocardiographic examination, monitoring and implementation of preventive strategies in order to delay transition to disease symptoms.

Despite the promise for performance and accuracy improvements of quantum inspired (QI) algorithms over classical machine learning (ML) algorithms, such gains have not been realized in practice. The quantum inspired algorithms can theoretically achieve significant speed up based on sampling assumptions and have thus far failed to outperform the existing classical ML models in practical applications. The speedup of quantum machine learning (QML) algorithms assume the access to data in quantum random access memory (QRAM) which is a strong assumption with current quantum architectures. QI algorithms assume sample and query (SQ) access to input vector and norms of matrices using a dynamic data structure. We explore the components of these models and the assumptions in this paper by surveying the recent works in QML and QI Machine learning (QIML) algorithms. We limit our study to QML and QIML models on achieving a speed up over classical ML techniques rather than individual proofs of these algorithms. This study highlights the assumptions being made that are currently not practical for QML and QIML algorithms in achieving performance advantage over classical ML algorithms.

Diabetic Retinopathy (DR) is a highly threatening microvascular complication of diabetes mellitus. Diabetic patients must be screened annually for DR; however, it is practically not viable due to the high volume of patients, lack of resources, economic burden, and cost of the screening procedure. The use of machine learning (ML) classifiers in medical science is an emerging frontier and can help in assisted diagnosis. The few available proposed models perform best when used in similar population cohorts and their external validation has been questioned. Therefore, the purpose of our research is to classify the DR using different ML methods on Saudi diabetic data, propose the best method based on accuracy and identify the most discriminative interpretable features using the socio-demographic and clinical information. This cross-sectional study was conducted among 327 diabetic patients in Almajmaah, Saudi Arabia. Socio-demographic and clinical data were collected using a systematic random sampling technique. For DR classification, ML algorithm including, linear discriminant analysis, support vector machine, K nearest neighbor, random forest and its variate ranger random forest classifiers were used through cross-validation resampling procedure. In classifying DR, ranger random forest outperforms the other methods by accurately classifying 86% of the DR patients on the test data. HbA1c (p<0.001) and duration of diabetes (p<0.001) were the most influential risk factor that best discriminated the DR patients. Other influential risk factors were the body mass index (p<0.001), age-onset (p<0.001), age (p<0.001), systolic blood pressure (p<0.05), and the use of medication (p<0.05) that significantly discriminated the DR patients. Based on the present study findings, integrating ophthalmology and ML can transform diagnosing the disease pattern that can help generate a compelling clinical effect. ML can be used as an added tool for clinical decision-making and must not be the sole substitute for a clinician. We will work to examine the classification performance of multi-class data using more sophisticated ML methods.

Gesture recognition is a topic in computer science and language technology that aims to interpret human gestures with computer programs and many different algorithms. It can be seen as the way computers can understand human body language. Today, the main interaction tools between computers and humans are still the keyboard and mouse. Gesture recognition can be used as a tool for communication with the machine and interaction without any mechanical device such as a keyboard or mouse. In this paper, we present the results of a comparison of eight different machine learning (ML) classifiers in the task of human hand gesture recognition and classification to explore how to efficiently implement one or more tested ML algorithms on an 8-bit AVR microcontroller for on-line human gesture recognition with the intention to gesturally control the mobile robot. The 8-bit AVR microcontrollers are still widely used in the industry, but due to their lack of computational power and limited memory, it is a challenging task to efficiently implement ML algorithms on them for on-line classification. Gestures were recorded by using inertial sensors, gyroscopes, and accelerometers placed at the wrist and index finger. One thousand and eight hundred (1800) hand gestures were recorded and labelled. Six important features were defined for the identification of nine different hand gestures using eight different machine learning classifiers: Decision Tree (DT), Random Forests (RF), Logistic Regression (LR), Linear Discriminant Analysis (LDA), Support Vector Machine (SVM) with linear kernel, Naïve Bayes classifier (NB), K-Nearest Neighbours (KNN), and Stochastic Gradient Descent (SGD). All tested algorithms were ranged according to Precision, Recall, and F1-score (abb.: P-R-F1). The best algorithms were SVM (P-R-F1: 0.9865, 0.9861, and 0.0863), and RF (P-R-F1: 0.9863, 0.9861, and 0.0862), but their main disadvantage is their unusability for on-line implementations in 8-bit AVR microcontrollers, as proven in the paper. The next best algorithms have had only slightly poorer performance than SVM and RF: KNN (P-R-F1: 0.9835, 0.9833, and 0.9834) and LR (P-R-F1: 0.9810, 0.9810, and 0.9810). Regarding the implementation on 8-bit microcontrollers, KNN has proven to be inadequate, like SVM and RF. However, the analysis for LR has proved that this classifier could be efficiently implemented on targeted microcontrollers. Having in mind its high F1-score (comparable to SVM, RF, and KNN), this leads to the conclusion that the LR is the most suitable classifier among tested for on-line applications in resource-constrained environments, such as embedded devices based on 8-bit AVR microcontrollers, due to its lower computational complexity in comparison with other tested algorithms.

Time-interval temporal patterns can beat and explain the malware

Malware detection at the hardware level has emerged recently as a promising solution to improve the security of computing systems. Hardware-based malware detectors take advantage of Machine Learning (ML) classifiers to detect pattern of malicious applications at run-time. These ML classifiers are trained using low-level features such as processor Hardware Performance Counters (HPCs) data which are captured at run-time to appropriately represent the application behaviour. Recent studies show the potential of standard ML-based classifiers for detecting malware using analysis of large number of microarchitectural events, more than the very limited number of HPC registers available in today’s microprocessors which varies from 2 to 8. This results in executing the application more than once to collect the required data, which in turn makes the solution less practical for effective run-time malware detection. Our results show a clear trade-off between the performance of standard ML classifiers and the number and diversity of HPCs available in modern microprocessors. This paper proposes a machine learning-based solution to break this trade-off to realize effective run-time detection of malware. We propose ensemble learning techniques to improve the performance of the hardware-based malware detectors despite using a very small number of microarchitectural events that are captured at run-time by existing HPCs, eliminating the need to run an application several times. For this purpose, eight robust machine learning models and two well-known ensemble learning classifiers applied on all studied ML models (sixteen in total) are implemented for malware detection and precisely compared and characterized in terms of detection accuracy, robustness, performance (accuracy × robustness), and hardware overheads. The experimental results show that the proposed ensemble learning-based malware detection with just 2 HPCs using ensemble technique outperforms standard classifiers with 8 HPCs by up to 17%. In addition, it can match the robustness and performance of standard ML-based detectors with 16 HPCs while using only 4 HPCs allowing effective run-time detection of malware.

Fall prevention has always been a crucial topic for injury prevention. Research shows that real-time posture monitoring and subsequent fall prevention are important for the prevention of fall-related injuries. In this research, we determine a real-time posture classifier by comparing classical and deep machine learning classifiers in terms of their accuracy and robustness for posture classification. For this, multiple classical classifiers, including classical machine learning, support vector machine, random forest, neural network, and Adaboost methods, were used. Deep learning methods, including LSTM and transformer, were used for posture classification. In the experiment, joint data were obtained using an RGBD camera. The results show that classical machine learning posture classifier accuracy was between 75% and 99%, demonstrating that the use of classical machine learning classification alone is sufficient for real-time posture classification even with missing joints or added noise. The deep learning method LSTM was also effective in classifying the postures with high accuracy, despite incurring a significant computational overhead cost, thus compromising the real-time posture classification performance. The research thus shows that classical machine learning methods are worthy of our attention, at least, to consider for reuse or reinvention, especially for real-time posture classification tasks. The insight of using a classical posture classifier for large-scale human posture classification is also given through this research.

In the Industry 4.0 era of smart grids, the real-world problem of blackouts and cascading failures due to cyberattacks is a significant concern and highly challenging because the existing Intrusion Detection System (IDS) falls behind in handling missing rates, response times, and detection accuracy. Addressing this problem with an early attack detection mechanism with a reduced missing rate and decreased response time is critical. The development of an Intelligent IDS is vital to the mission-critical infrastructure of a smart grid to prevent physical sabotage and processing downtime. This paper aims to develop a robust Anomaly-based IDS using a statistical approach with a machine learning classifier to discriminate cyberattacks from natural faults and man-made events to avoid blackouts and cascading failures. The novel mechanism of a statistical approach with a machine learning (SAML) classifier based on Neighborhood Component Analysis, ExtraTrees, and AdaBoost for feature extraction, bagging, and boosting, respectively, is proposed with optimal hyperparameter tuning for the early discrimination of cyberattacks from natural faults and man-made events. The proposed model is tested using the publicly available Industrial Control Systems Cyber Attack Power System (Triple Class) dataset with a three-bus/two-line transmission system from Mississippi State University and Oak Ridge National Laboratory. Furthermore, the proposed model is evaluated for scalability and generalization using the publicly accessible IEEE 14-bus and 57-bus system datasets of False Data Injection (FDI) attacks. The test results achieved higher detection accuracy, lower missing rates, decreased false alarm rates, and reduced response time compared to the existing approaches.

We can state undoubtedly that Bangla language is rich enough to work with and implement various Natural Language Processing (NLP) tasks. Though it needs proper attention, hardly NLP field has been explored with it. In this age of digitalization, large amount of Bangla news contents are generated in online platforms. Some of the contents are inappropriate for the children or aged people. With the motivation to filter out news contents easily, the aim of this work is to perform document level sentiment analysis (SA) on Bangla online news. In this respect, the dataset is created by collecting news from online Bangla newspaper archive. Further, the documents are manually annotated into positive and negative classes. Composite process technique of “Pipeline” class including Count Vectorizer, transformer (TF-IDF) and machine learning (ML) classifiers are employed to extract features and to train the dataset. Six supervised ML classifiers (i.e. Multinomial Naive Bayes (MNB), K-Nearest Neighbor (K-NN), Random Forest (RF), (C4.5) Decision Tree (DT), Logistic Regression (LR) and Linear Support Vector Machine (LSVM)) are used to analyze the best classifier for the proposed model. There has been very few works on SA of Bangla news. So, this work is a small attempt to contribute in this field. This model showed remarkable efficiency through better results in both the validation process of percentage split method and 10-fold cross validation. Among all six classifiers, RF has outperformed others by 99% accuracy. Even though LSVM has shown lowest accuracy of 80%, it is also considered as good output. However, this work has also exhibited surpassing outcome for recent and critical Bangla news indicating proper feature extraction to build up the model.

Quantitative EEG features and machine learning classifiers for eye-blink artifact detection: A comparative study

Novel Artificial Immune Networks-based optimization of shallow machine learning (ML) classifiers

3605 Background: Colorectal cancer (CRC) is the third most common cancer and the second leading cause of cancer-related deaths globally (Xi et al. 2021). In the U.S., it is estimated that 152,810 new cases and 53,010 deaths will occur in 2024, with an increasing incidence among younger adults (ACS, 2024). CRC is a multifactorial disease influenced by both genetic and environmental factors and typically progresses from normal epithelial tissue to adenocarcinoma through a well-established sequence of molecular events. Despite advancements in treatment, early detection remains a key driver of improved survival outcomes. While current screening methods— such as fecal immunochemical testing, fecal DNA testing, and colonoscopy— have reduced CRC mortality and morbidity, low compliance continues to be a significant challenge. In this study, we developed a proof-of-concept machine learning (ML) classifier leveraging active chromatin cell-free DNA (cfDNA ac ) signals in peripheral blood to differentiate CRC patients from healthy individuals. Methods: Plasma samples from treatment-naive colorectal cancer (CRC) patients (stages I–IV, n = 54), advanced adenoma patients (n = 14), and healthy volunteers (n = 40) were processed using Aqtual’s proprietary active chromatin capture workflow to enrich regulatory-active chromatin cfDNA. Samples were split into a training set, which included 31 early-stage CRC samples (stage I: n = 13, stage II: n = 13, stage III: n = 5) and 19 healthy samples, and a hold-out set, comprised 37 disease samples (stage I/II: n = 6, stage III: n = 6, stage IV: n = 11, advanced adenoma: n = 14) and 21 healthy samples. A machine learning classifier was trained using 5-fold cross-validation with 5 repeats on the training set, and the performance was assessed on the hold-out set to ensure robustness and generalizability. Results: A machine learning classifier trained on genome-wide cfDNA ac signals demonstrated robust performance, achieving a mean AUC of 0.94 (95% CI: 0.92 - 0.96). It exhibited 93% sensitivity (95% CI: 84% -100%) for colorectal cancer detection (Stage I/II: 94%, Stage III: 95%, Stage IV: 91%) and 81% sensitivity (95% CI: 67% - 95%) for advanced adenoma, with a specificity of 90% in the hold-out set. Analysis of the top contributing cfDNA ac signals identified 852 promoter and 2,594 exon features derived from 2,678 unique genes. Gene-set enrichment analysis revealed significant associations with key cancer hallmarks, including KRAS signaling and epithelial-mesenchymal transition (EMT). Conclusions: This study highlights the potential of Aqtual’s active chromatin capture assay to identify molecular features in plasma that differentiate CRC and advanced adenoma from healthy individuals. The ML classifier based on these signatures shows promise for future development as a non-invasive tool for early colorectal cancer and advanced adenoma detection and monitoring.

The Internet of Things (IoT) is an interconnected framework of computer devices, software, and sensors to work collectively to automate and monitor different tasks and procedures without human intervention. IoTs provide benefits and ease in performing operations for various industries and home and health stakeholders. Still, it faces security threats and challenges that make the IoT framework unreliable regarding security and privacy. Intrusion detection systems have been designed and deployed to overcome these security challenges. Over the past few years, for making the IDS intelligent artificial intelligence has been integrated with them. Besides this, different feature selection techniques are also being used to make the datasets appropriate for Machine Learning (ML) classifiers. This research uses a filter-based feature selection technique to select the important features from two datasets, UNSW-NB15 and NSL-KDD. After that, datasets with selected features are given to the classifiers as input. This work also evaluates different ML classifiers such as Multiple Layers Perceptron (MLP), CatBoost, Light Gradient Boosting (LGBM), Extra-Tree Classifier (ETC), Random Forest (RF), and K-nearest neighbor (KNN). Different metrics, Accuracy, Precision, Recall, F1, Detection Rate, and FAR, have been used for performance analysis. Evaluation results proved the LGBM classifier effective among other classifiers with 98.70% accuracy on the UNSW-NB15 dataset and 99.8% on the NSL-KDD dataset after feature selection with mean absolute difference (MAD) feature selection technique. Results are also compared with existing models to provide the efficiency of the proposed model.