Machine Learning Algorithms with Parameter Tuning to Predict Students’ Graduation-on-time: A Case Study in Higher Education

Gesture recognition is a topic in computer science and language technology that aims to interpret human gestures with computer programs and many different algorithms. It can be seen as the way computers can understand human body language. Today, the main interaction tools between computers and humans are still the keyboard and mouse. Gesture recognition can be used as a tool for communication with the machine and interaction without any mechanical device such as a keyboard or mouse. In this paper, we present the results of a comparison of eight different machine learning (ML) classifiers in the task of human hand gesture recognition and classification to explore how to efficiently implement one or more tested ML algorithms on an 8-bit AVR microcontroller for on-line human gesture recognition with the intention to gesturally control the mobile robot. The 8-bit AVR microcontrollers are still widely used in the industry, but due to their lack of computational power and limited memory, it is a challenging task to efficiently implement ML algorithms on them for on-line classification. Gestures were recorded by using inertial sensors, gyroscopes, and accelerometers placed at the wrist and index finger. One thousand and eight hundred (1800) hand gestures were recorded and labelled. Six important features were defined for the identification of nine different hand gestures using eight different machine learning classifiers: Decision Tree (DT), Random Forests (RF), Logistic Regression (LR), Linear Discriminant Analysis (LDA), Support Vector Machine (SVM) with linear kernel, Naïve Bayes classifier (NB), K-Nearest Neighbours (KNN), and Stochastic Gradient Descent (SGD). All tested algorithms were ranged according to Precision, Recall, and F1-score (abb.: P-R-F1). The best algorithms were SVM (P-R-F1: 0.9865, 0.9861, and 0.0863), and RF (P-R-F1: 0.9863, 0.9861, and 0.0862), but their main disadvantage is their unusability for on-line implementations in 8-bit AVR microcontrollers, as proven in the paper. The next best algorithms have had only slightly poorer performance than SVM and RF: KNN (P-R-F1: 0.9835, 0.9833, and 0.9834) and LR (P-R-F1: 0.9810, 0.9810, and 0.9810). Regarding the implementation on 8-bit microcontrollers, KNN has proven to be inadequate, like SVM and RF. However, the analysis for LR has proved that this classifier could be efficiently implemented on targeted microcontrollers. Having in mind its high F1-score (comparable to SVM, RF, and KNN), this leads to the conclusion that the LR is the most suitable classifier among tested for on-line applications in resource-constrained environments, such as embedded devices based on 8-bit AVR microcontrollers, due to its lower computational complexity in comparison with other tested algorithms.

Introduction: The aim of this study was to compare various machine learning algorithms for constructing a diabetic retinopathy (DR) prediction model among type 2 diabetes mellitus (DM) patients and to develop a nomogram based on the best model. Methods: This cross-sectional study included DM patients receiving routine DR screening. Patients were randomly divided into training (244) and validation (105) sets. Least absolute shrinkage and selection operator regression was used for the selection of clinical characteristics. Six machine learning algorithms were compared: decision tree (DT), k-nearest neighbours (KNN), logistic regression model (LM), random forest (RF), support vector machine (SVM), and XGBoost (XGB). Model performance was assessed via receiver-operating characteristic (ROC), calibration, and decision curve analyses (DCAs). A nomogram was then developed on the basis of the best model. Results: Compared with the five other machine learning algorithms (DT, KNN, RF, SVM, and XGB), the LM demonstrated the highest area under the ROC curve (AUC, 0.894) and recall (0.92) in the validation set. Additionally, the calibration curves and DCA results were relatively favourable. Disease duration, DPN, insulin dosage, urinary protein, and ALB were included in the LM. The nomogram exhibited robust discrimination (AUC: 0.856 in the training set and 0.868 in the validation set), calibration, and clinical applicability across the two datasets after 1,000 bootstraps. Conclusion: Among the six different machine learning algorithms, the LM algorithm demonstrated the best performance. A logistic regression-based nomogram for predicting DR in type 2 DM patients was established. This nomogram may serve as a valuable tool for DR detection, facilitating timely treatment.

Undernutrition among children under the age of five is a major public health concern, especially in developing countries. This study aimed to use machine learning (ML) algorithms to predict undernutrition and identify its associated factors. Secondary data analysis of the 2017 Multiple Indicator Cluster Survey (MICS) was performed using R and Python. The main outcomes of interest were undernutrition (stunting: height-for-age (HAZ) < -2 SD; wasting: weight-for-height (WHZ) < -2 SD; and underweight: weight-for-age (WAZ) < -2 SD). Seven ML algorithms were trained and tested: linear discriminant analysis (LDA), logistic model, support vector machine (SVM), random forest (RF), least absolute shrinkage and selection operator (LASSO), ridge regression, and extreme gradient boosting (XGBoost). The ML models were evaluated using the accuracy, confusion matrix, and area under the curve (AUC) receiver operating characteristics (ROC). In total, 8564 children were included in the final analysis. The average age of the children was 926 days, and the majority were females. The weighted prevalence rates of stunting, wasting, and underweight were 17%, 7%, and 12%, respectively. The accuracies of all the ML models for wasting were (LDA: 84%; Logistic: 95%; SVM: 92%; RF: 94%; LASSO: 96%; Ridge: 84%, XGBoost: 98%), stunting (LDA: 86%; Logistic: 86%; SVM: 98%; RF: 88%; LASSO: 86%; Ridge: 86%, XGBoost: 98%), and for underweight were (LDA: 90%; Logistic: 92%; SVM: 98%; RF: 89%; LASSO: 92%; Ridge: 88%, XGBoost: 98%). The AUC values of the wasting models were (LDA: 99%; Logistic: 100%; SVM: 72%; RF: 94%; LASSO: 99%; Ridge: 59%, XGBoost: 100%), for stunting were (LDA: 89%; Logistic: 90%; SVM: 100%; RF: 92%; LASSO: 90%; Ridge: 89%, XGBoost: 100%), and for underweight were (LDA: 95%; Logistic: 96%; SVM: 100%; RF: 94%; LASSO: 96%; Ridge: 82%, XGBoost: 82%). Age, weight, length/height, sex, region of residence and ethnicity were important predictors of wasting, stunting and underweight. The XGBoost model was the best model for predicting wasting, stunting, and underweight. The findings showed that different ML algorithms could be useful for predicting undernutrition and identifying important predictors for targeted interventions among children under five years in Ghana.

The efflux transporter multidrug resistance associated protein-2 belongs to ATP-binding cassette superfamily which plays an important role in multidrug resistance and drugdrug interactions. Efflux transporters are considered to be important targets for increasing the efficacy of drugs and importance of computational study of efflux transporters for predicting substrates, non-substrates, inhibitors and non-inhibitors is well documented. Previous work on predictive models for inhibitors of multidrug resistance associated Protein-2 efflux transporter showed that machine learning methods produced good results. The aim of the present work was to develop a machine learning predictive model to classify inhibitors and non-inhibitors of multidrug resistance associated protein-2 transporter using a well refined dataset. In this study, the various algorithms of machine learning were used to develop the predictive models i.e. support vector machine, random forest and k-nearest neighbor. The methods like variance threshold, SelectKBest, random forest, and recursive feature elimination were used to select the features generated by PyDPI. A total of 239 molecules consisting of 124 inhibitors and 115 non-inhibitors were used for model development. The best multidrug resistance associated protein-2 inhibitor model showed prediction accuracies of 0.76, 0.72 and 0.79 for training, 5-fold cross-validation and external sets, respectively. It was observed that support vector machine model built on features selected using recursive feature elimination method shows the best performance. The developed model can be used in the early stages of drug discovery for identifying the inhibitors of multidrug resistance associated protein-2 efflux transporter.

ObjectiveEnhanced technology in computer and internet has driven scale and quality of data to be improved in various areas including healthcare sectors. Machine Learning (ML) has played a pivotal role in efficiently analyzing those big data, but a general misunderstanding of ML algorithms still exists in applying them (e.g., ML techniques can settle a problem of small sample size, or deep learning is the ML algorithm). This paper reviewed the research of diagnosing mental illness using ML algorithm and suggests how ML techniques can be employed and worked in practice.MethodsResearches about mental illness diagnostic using ML techniques were carefully reviewed. Five traditional ML algorithms-Support Vector Machines (SVM), Gradient Boosting Machine (GBM), Random Forest, Naïve Bayes, and K-Nearest Neighborhood (KNN)-frequently used for mental health area researches were systematically organized and summarized.ResultsBased on literature review, it turned out that Support Vector Machines (SVM), Gradient Boosting Machine (GBM), Random Forest, Naïve Bayes, and K-Nearest Neighborhood (KNN) were frequently employed in mental health area, but many researchers did not clarify the reason for using their ML algorithm though every ML algorithm has its own advantages. In addition, there were several studies to apply ML algorithms without fully understanding the data characteristics.ConclusionResearchers using ML algorithms should be aware of the properties of their ML algorithms and the limitation of the results they obtained under restricted data conditions. This paper provides useful information of the properties and limitation of each ML algorithm in the practice of mental health.

Detecting the presence of a disease requires laboratory tests, testing kits, and devices; however, these were not always available on hand. This study proposes a new approach in disease detection using machine learning algorithms by analyzing symptoms experienced by a person without requiring laboratory tests. Six supervised machine learning algorithms such as J48 decision tree, random forest, support vector machine, k-nearest neighbors, naïve Bayes algorithms, and artificial neural networks were applied in the “COVID-19 Symptoms and Presence Dataset” from Kaggle. Through hyperparameter optimization and 10-fold cross validation, we attained the highest possible performance of each algorithm. A comparative analysis was performed according to accuracy, sensitivity, specificity, and area under the ROC curve. Results show that random forest, support vector machine, k-nearest neighbors, and artificial neural networks outweighed other algorithms by attaining 98.84% accuracy, 100% sensitivity, 98.79% specificity, and 98.84% area under the ROC curve. Finally, we developed a web application that will allow users to select symptoms currently being experienced, and use it to predict the presence of COVID-19 through the developed prediction model. Based on this mechanism, the proposed method can effectively predict the presence or absence of COVID-19 in a person immediately without using laboratory tests, kits, and devices in a real-time manner.

Periodontitis is increasingly associated with heart failure, and the goal of this study was to develop and validate a prediction model based on machine learning algorithms for the risk of heart failure in middle-aged and elderly participants with periodontitis. We analyzed data from a total of 2876 participants with a history of periodontitis from the National Health and Nutrition Examination Survey (NHANES) 2009 to 2014, with a training set of 1980 subjects with periodontitis from the NHANES 2009 to 2012 and an external validation set of 896 subjects from the NHANES 2013 to 2014. The independent risk factors for heart failure were identified using univariate and multivariate logistic regression analysis. Machine learning algorithms such as logistic regression, k-nearest neighbor, support vector machine, random forest, gradient boosting machine, and multilayer perceptron were used on the training set to construct the models. The performance of the machine learning models was evaluated using 10-fold cross-validation on the training set and receiver operating characteristic curve (ROC) analysis in the validation set. Based on the results of univariate logistic regression and multivariate logistic regression, it was found that age, race, myocardial infarction, and diabetes mellitus status were independent predictors of the risk of heart failure in participants with periodontitis. Six machine learning models, including logistic regression, K-nearest neighbor, support vector machine, random forest, gradient boosting machine, and multilayer perceptron, were built on the training set, respectively. The area under the ROC for the 6 models was obtained using 10-fold cross-validation with values of 0 848, 0.936, 0.859, 0.889, 0.927, and 0.666, respectively. The areas under the ROC on the external validation set were 0.854, 0.949, 0.647, 0.933, 0.855, and 0.74, respectively. K-nearest neighbor model got the best prediction performance across all models. Out of 6 machine learning models, the K-nearest neighbor algorithm model performed the best. The prediction model offers early, individualized diagnosis and treatment plans and assists in identifying the risk of heart failure occurrence in middle-aged and elderly patients with periodontitis.

Single-cell RNA-seq overcomes the shortcomings of conventional transcriptome sequencing technology and could provide a powerful tool for distinguishing the transcriptome characteristics of various cell types in biological tissues, and comprehensively revealing the heterogeneity of gene expression between cells. Many Intelligent Computing methods have been presented to infer gene regulatory network (GRN) with single-cell RNA-seq data. In this paper, we investigate the performances of seven classifiers including support vector machine (SVM), random forest (RF), Naive Bayesian (NB), GBDT, logical regression (LR), decision tree (DT) and K-Nearest Neighbor (KNN) for solving the binary classification problems of GRN inference with single-cell RNA-seq data (Single_cell_GRN). In SVM, three different kernel functions (linear, polynomial and radial basis function) are utilized, respectively. Three real single-cell RNA-seq datasets from mouse and human are utilized. The experiment results prove that in most cases supervised learning methods (SVM, RF, NB, GBDT, LR, DT and KNN) perform better than unsupervised learning method (GENIE3) in terms of AUC. SVM, RF and KNN have the better performances than other four classifiers. In SVM, linear and polynomial kernels are more fit to model single-cell RNA-seq data.

Characteristics of the mandible structures have been relevant in anthropological and forensic studies for sex prediction. This study aims to evaluate the coronoid process, condyle, and sigmoid notch patterns in sex prediction through supervised machine learning algorithms. Cephalometric radiographs from 410 dental records of patients were screened to investigate the morphology of the coronoid process, condyle, and sigmoid notch and the Co-Gn distance. The following machine learning algorithms were used to build the predictive models: Decision Tree, Gradient Boosting Classifier, K-Nearest Neighbors (KNN), Logistic Regression, Multilayer Perceptron Classifier, Random Forest Classifier, and Support Vector Machine (SVM). A 5-fold cross-validation approach was adopted to validate each model. Metrics such as area under the curve (AUC), accuracy, recall, precision, and F1 Score were calculated for each model, and ROC curves were constructed. All tested variables demonstrated statistical significance (p < 0.10) and were included in the construction of the predictive model. The Co-Gn variable stood out as the most important among the evaluated independent variables, showing greater relevance in three of the four algorithms used in assessing feature importance. In the analysis of the models’ performance, the AUC ranged from 0.82 [95% CI = 0.72–0.93] to 0.66 [95% CI = 0.53–0.76] for the test data, and from 0.83 [95% CI = 0.80–0.87] to 0.71 [95% CI = 0.61–0.75] for cross-validation. The precision of the models ranged from 0.83 [95% CI = 0.75–0.91] to 0.68 [95% CI = 0.58–0.78] in the test phase, and from 0.78 [95% CI = 0.74–0.82] to 0.69 [95% CI = 0.65–0.75] in cross-validation. The SVM, KNN, and Gradient Boosting Classifier algorithms stood out with the highest AUC and precision values in both cross-validation and testing. The use of condyle, coronoid process, and sigmoid notch characteristics, in combination with supervised machine learning predictive models, shows potential for contributing to sex prediction based on morphometric bone characteristics, particularly regarding the distance between the condyle and gnathion. However, given the study’s limitations, these findings should be interpreted with caution.

The problem persists despite the heavy consequences of jail and the death penalty imposed on those found guilty of selling drugs. This situation creates an alarm among academics and managers who work in practice. Hotbeds for drug dealers are on the rise, and their financial impact on the global economy is moving forward. Drug peddlers distract society by warping peace, justice, and order and threaten the Sustainable Development Goals (SDGs). This study evaluated secondary data containing qualities and suspect drug groups to determine whether machine learning techniques could predict the suspect drug group. We developed a prediction model using nine machine learning algorithms: AdaBoost (AB), naïve Bayes (NB), logistic regression (LR), K nearest neighbour (KNN), random forest (RF), decision tree (DT), neural network (NN), CN2 and support vector machine (SVM). The study utilized Orange data mining software to obtain a more accurate perspective on the data. Predictive accuracy was determined using 5-fold stratified cross-validation. Friedman’s test was conducted, and the results showed that the performance of each algorithm was significantly different. Also, the study compared the models and compiled the results. The results reveal that the random forest has the highest accuracy compared to the others. This prediction model implies that high accuracy can help the government make informed decisions by accurately identifying and classifying suspected individuals and offenders. This prediction will help the government refer suspected individuals and offenders who meet the qualifications for specific drug law sections, like prosecution or rehabilitation, to those sections with a faster and more accurate rate. The outcome of this study should be helpful to law enforcement agents, analysts, and other drug practitioners who may find machine learning tools dependable to detect and classify drug offenders.KeywordsDrug groupClassificationMachine learningOffendersDrug peddling

BackgroundEarly identification of Alzheimer’s disease or mild cognitive impairment can help guide direct prevention and supportive treatments, improve outcomes, and reduce medical costs. Existing advanced diagnostic tools are mostly based on neuroimaging and suffer from certain problems in cost, reliability, repeatability, accessibility, ease of use, and clinical integration. To address these problems, we developed, evaluated, and implemented an early diagnostic tool using machine learning and non-imaging factors.Methods and resultsA total of 654 participants aged 65 or older from the Nursing Home in Hangzhou, China were identified. Information collected from these patients includes dementia status and 70 demographic, cognitive, socioeconomic, and clinical features. Logistic regression, support vector machine (SVM), neural network, random forest, extreme gradient boosting (XGBoost), least absolute shrinkage and selection operator (LASSO), and best subset models were trained, tuned, and internally validated using a novel double cross validation algorithm and multiple evaluation metrics. The trained models were also compared and externally validated using a separate dataset with 1,100 participants from four communities in Zhejiang Province, China. The model with the best performance was then identified and implemented online with a friendly user interface. For the nursing dataset, the top three models are the neural network (AUROC = 0.9435), XGBoost (AUROC = 0.9398), and SVM with the polynomial kernel (AUROC = 0.9213). With the community dataset, the best three models are the random forest (AUROC = 0.9259), SVM with linear kernel (AUROC = 0.9282), and SVM with polynomial kernel (AUROC = 0.9213). The F1 scores and area under the precision-recall curve showed that the SVMs, neural network, and random forest were robust on the unbalanced community dataset. Overall the SVM with the polynomial kernel was found to be the best model. The LASSO and best subset models identified 17 features most relevant to dementia prediction, mostly from cognitive test results and socioeconomic characteristics.ConclusionOur non-imaging-based diagnostic tool can effectively predict dementia outcomes. The tool can be conveniently incorporated into clinical practice. Its online implementation allows zero barriers to its use, which enhances the disease’s diagnosis, improves the quality of care, and reduces costs.

This study presents a comparative analysis of the classification performance of facial and emotion recognition systems using Machine Learning (ML) and Deep Learning (DL) algorithms. The primary objective of this work is to evaluate the applicability of emotion recognition in fields such as psychotherapy and crime analysis, using the FER-2013 dataset.The study was conducted with ML algorithms such as Support Vector Machines (SVM), Random Forest (RF), K-Nearest Neighbor (KNN), Decision Trees, and Gradient Boosting, as well as the DL algorithm, Convolutional Neural Network (CNN). Supported by the application of feature selection, preprocessing, and feature extraction techniques, the model’s performance was measured using standard metrics such as accuracy, precision, F1-score, and AUC-ROC.The experimental results showed that the highest classification accuracy (46.61%) was achieved with the CNN model. While the ML models generally offered lower accuracy, they provided advantages in terms of computational efficiency in specific scenarios.This study aims to contribute to the literature by providing a comparative analysis of ML and DL algorithms and by highlighting the effect of data preprocessing on performance. The findings set targets for future work, such as real-time system integration and hybrid model development.

An intrusion detection system (IDS) is an important protection instrument for detecting complex network attacks. Various machine learning (ML) or deep learning (DL) algorithms have been proposed for implementing anomaly-based IDS (AIDS). Our review of the AIDS literature identifies some issues in related work, including the randomness of the selected algorithms, parameters, and testing criteria, the application of old datasets, or shallow analyses and validation of the results. This paper comprehensively reviews previous studies on AIDS by using a set of criteria with different datasets and types of attacks to set benchmarking outcomes that can reveal the suitable AIDS algorithms, parameters, and testing criteria. Specifically, this paper applies 10 popular supervised and unsupervised ML algorithms for identifying effective and efficient ML-AIDS of networks and computers. These supervised ML algorithms include the artificial neural network (ANN), decision tree (DT), k-nearest neighbor (k-NN), naive Bayes (NB), random forest (RF), support vector machine (SVM), and convolutional neural network (CNN) algorithms, whereas the unsupervised ML algorithms include the expectation-maximization (EM), k-means, and self-organizing maps (SOM) algorithms. Several models of these algorithms are introduced, and the turning and training parameters of each algorithm are examined to achieve an optimal classifier evaluation. Unlike previous studies, this study evaluates the performance of AIDS by measuring the true positive and negative rates, accuracy, precision, recall, and F-Score of 31 ML-AIDS models. The training and testing time for ML-AIDS models are also considered in measuring their performance efficiency given that time complexity is an important factor in AIDSs. The ML-AIDS models are tested by using a recent and highly unbalanced multiclass CICIDS2017 dataset that involves real-world network attacks. In general, the k-NN-AIDS, DT-AIDS, and NB-AIDS models obtain the best results and show a greater capability in detecting web attacks compared with other models that demonstrate irregular and inferior results.

Heart disease is one of the leading causes of death worldwide, making early detection essential for improving patient outcomes. With advancements in machine learning (ML), predictive models now offer a powerful way to assist doctors in diagnosing heart disease more accurately and efficiently. This study explores various ML algorithms, including Logistic Regression (LR), Random Forest (RF), Support Vector Machine (SVM), Decision Tree (DT), Naïve Bayes (NB), and K-Nearest Neighbors (KNN), to identify the most effective approach for heart disease prediction. Using the Cleveland Heart Disease Dataset, which contains 1,025 patient records with 14 medical attributes, we preprocessed the data, selected key features, and optimized model parameters. After evaluating the models with 10-fold cross-validation, the Random Forest model achieved the highest accuracy (98%), followed by Decision Tree (97%). These results highlight the potential of ML-based tools in clinical decision-making, helping doctors detect heart disease at an earlier stage and make informed treatment plans.

In sustainable and competitive business management, it is crucial for organizations to consider organizational change and transformational leadership in human resource (HR) management to adapt to the changes in their environment. This capability enables large-scale enterprises to maintain their presence in an increasingly competitive environment through enhanced management capacity. Enterprises that adopt transformational leadership in HR management must equip leadership candidates with competencies such as creating a shared vision, providing appropriate role models, encouraging the adoption of group goals, meeting high performance expectations, and providing individual support and the development of intellectual stimulation. By identifying potential leadership candidates using a decision support model, the necessary competencies can be developed through in-service training and experiential learning during their careers. Innovative and effective approaches to identifying leadership candidates can be developed by analyzing complex big data using advanced artificial intelligence (AI) techniques. In this article, a forecast model using machine learning (ML) algorithms for a human resource management career planning approach was developed for the Turkish Post Corporation (PTT) and it was tested to predict potential leadership candidates by analyzing the big data of 5000 employees. The Turkish Post Corporation ML algorithms were applied to 100 randomly selected data points using the k-Nearest Neighbor (kNN), Random Forest (RF), Gradient Boosting (GB), and Support Vector Machine (SVM) algorithms to predict the types of titles held by the staff employed at PTT. The kNN, GB, RF, and SVM algorithms achieved accuracy rates of 96%, 91%, 73%, and 41%, respectively. The case study results indicate that promotion decisions in large-scale and rooted enterprises can be effectively modeled using big data and ML algorithms, highlighting significant potential for HR management and leadership development practices in the public sector.