The Truth About Labels: Unveiling The Hidden Risk to Supervised Machine Learning Models

Machine learning (ML) often provides applicable high-performance models to facilitate decision-makers in various fields. However, this high performance is achieved at the expense of the interpretability of these models, which has been criticized by practitioners and has become a significant hindrance in their application. Therefore, in highly sensitive decisions, black boxes of ML models are not recommended. We proposed a novel methodology that uses complex supervised ML models and transforms them into simple, interpretable, transparent statistical models. This methodology is like stacking ensemble ML in which the best ML models are used as a base learner to compute relative feature weights. The index of these weights is further used as a single covariate in the simple logistic regression model to estimate the likelihood of an event. We tested this methodology on the primary dataset related to cardiovascular diseases (CVDs), the leading cause of mortalities in recent times. Therefore, early risk assessment is an important dimension that can potentially reduce the burden of CVDs and their related mortality through accurate but interpretable risk prediction models. We developed an artificial neural network and support vector machines based on ML models and transformed them into a simple statistical model and heart risk scores. These simplified models were found transparent, reliable, valid, interpretable, and approximate in predictions. The findings of this study suggest that complex supervised ML models can be efficiently transformed into simple statistical models that can also be validated.

Between December 2020 and February 2022, Mt. Etna produced extraordinary lava fountains which developed into eruptive columns rising several kilometers above the vent. It is crucial to monitor the volcanic clouds produced during these eruptions to assess their impact on the environment, human health, and aviation. Geostationary satellite missions provide high‑frequency thermal infrared data, which are crucial for monitoring volcanic clouds during intense explosive eruptions. However, the large volume of satellite data necessitates automatic and accurate processing algorithms, especially when dealing with global‑scale observations every 5 minutes. In this work, a robust machine learning approach is developed to identify and track volcanic clouds using images from the EUMETSAT MSG SEVIRI (Meteosat Second Generation – Spinning Enhanced Visible and InfraRed Imager). This approach combines two distinct machine learning models: a deep learning (DL) model for volcanic cloud detection and a supervised machine learning (ML) model for identifying its primary components. The DL model segments volcanic clouds in SEVIRI images by analyzing both the spatial and spectral intensity data. The supervised ML model is able to distinguish the main components of a volcanic cloud by classifying the pixels as ash‑rich, SO2‑rich, or characterized by mixed components. Once an accurate mask of the volcanic cloud is obtained, the volcanic plume height is retrieved from satellite observations for further characterization. This integrated ML approach was applied to characterize the volcanic clouds produced during some of the lava fountains occurred at Etna volcano (Italy) between 2020 and 2022.

Using AI to Predict Patient's Atypical Prescription Based on Individual's Feature-Set

The use of machine learning (ML) models in forensic anthropology (FA) has increased in the last half decade; however, there is a lack of a standardized protocol on how to curate, use, and assess ML models. We introduce PUMAA (A Protocol for Utilizing Machine Learning in Forensic Anthropological Analyses), which includes a flowchart and a checklist that forensic practitioners can follow when creating, using, or assessing ML models for forensic research. The five factors that must be assessed in determining an ML model's true performance are explored. The most common types of supervised ML models are also explained in lay terms and accompanied by visuals to increase accessibility of these complex concepts. Although not exhaustive, examples of information that should be reported have been discussed for seven types of ML models. In addition, their various strengths and limitations are evaluated in order to equip researchers with the necessary tools to make decisions regarding when and how to use ML models. This protocol provides an initial standard for the use of ML in FA.

A database of physical and chemical properties from one-hundred different jet fuels was used to create supervised Machine Learning (ML) models that predict Derived Cetane Number (DCN) as well as an unsupervised Self Organizing Map (SOM) of these conventional fuels. The best supervised ML models predict jet fuel DCN with just over one CN average error. The two dominant physical properties in the determination of DCN are fuel density and T50. The various models agree with increases in n-alkanes leading to higher DCN values, while increases in cyclo-aromatic content leading to decreases in DCN. The prediction of eleven newer Sustainable Aviation Fuel (SAF) DCNs showed at best errors of 3 to 5 DCN which corresponds to a 5 to 10 percent prediction error. There were two SAF anomalies with much greater DCN prediction errors (over-prediction), as these two fuels are heavily branched iso-alkanes with very low DCNs. An unsupervised ML approach was also used to characterize fuel similarity. The unsupervised SOM created and trained with the conventional jet fuels was then used to evaluate the SAF fuels. Two of the SAFs which had DCNs similar to the baseline database fuel mean had Quantization Errors (QEs) double the average QE of the database fuels. The two low DCN iso-alkane branched SAF fuels had very high QEs and thus were correctly determined by the SOM ML based approach as most dissimilar to the conventional jet fuels. This unsupervised ML approach shows promise for detecting new fuel differences early in the fuel evaluation process.

Soybean (Glycine max; SB) leaf (SL) is an abundant non-conventional edible resource that possesses value-adding bioactive compounds. We predicted the attributes of SB based on the metabolomes of an SL using targeted metabolomics. The SB was planted in two cities, and SLs were regularly obtained from the SB plant. Nine flavonol glycosides were purified from SLs, and a validated simultaneous quantification method was used to establish rapid separation by ultrahigh-performance liquid chromatography-mass detection. Changes in 31 targeted compounds were monitored, and the compounds were discriminated by various supervised machine learning (ML) models. Isoflavones, quercetin derivatives, and flavonol derivatives were discriminators for cultivation days, varieties, and cultivation sites, respectively, using the combined criteria of supervised ML models. The neural model exhibited higher prediction power of the factors with high fitness and low misclassification rates while other models showed lower. We propose that a set of phytochemicals of SL is a useful predictor for discriminating characteristics of edible plants.

We conduct an in-depth investigation of the structural, electronic, vibrational, thermodynamic, and thermoelectric characteristics of Na–Bi-based compounds, specifically tetragonal NaBi, hexagonal NaBi3, and cubic Na3Bi, using advanced first-principles calculations in conjunction with machine learning (ML) models. We used density functional theory (DFT) with spin–orbit coupling (SOC) to figure out the electronic structure, phonon dispersions, and thermoelectric transport using Boltzmann transport theory. Our findings validate the Dirac semimetal nature of cubic Na3Bi and demonstrate varied topological and thermodynamic properties within the Na–Bi family. To speed up the prediction of the thermoelectric figure of merit (ZT) while enhancing interpretability to understand at the feature level, we trained supervised ML models [Random Forest (RF) and Neural Network (NN)] on thermoelectric results from DFT. It is possible to directly compute the figure of merit (ZT) from DFT-derived transport coefficients such as the Seebeck coefficient, electrical conductivity, and thermal conductivity. However, machine learning (ML) models serve as powerful surrogate predictors, enabling rapid screening of derivative compounds and quantitative assessment of feature importance through SHAP (SHapley Additive exPlanations) analysis. At low temperatures, RF models consistently outperformed NN models, but both performed well at high temperatures. SHAP analysis showed that the Seebeck coefficient has the biggest effect on ZT in all regimes. This integrated, physics-informed, and data-driven methodology demonstrates that machine learning can significantly augment first-principles approaches. It accelerates predictions, guides feature prioritisation, and enhances design capabilities. The developed workflow provides a generalizable and interpretable framework for the predictive modeling of advanced topological thermoelectric materials.

SummarySustained engagement in HIV care and adherence to antiretroviral therapy (ART) are essential for achieving the UNAIDS “95-95-95” targets. Despite increased ART access, disengagement from care remains a significant issue, particularly in sub-Saharan Africa. Traditional machine learning (ML) models have shown moderate success in predicting care disengagement, which would enable early intervention. We develop an enhanced large language model (LLM) fine-tuned with electronic medical records (EMRs) to predict people at risk of disengaging from HIV care in Tanzania and to provide interpretative insights into modifiable risk factors.MethodsWe developed a novel AI model by enhancing a pre-trained LLM (LLaMA 3.1, an open-source pre-trained LLM released by Meta) using routinely collected EMRs from Tanzania’s National HIV Care and Treatment Program from January 1, 2018, to June 30, 2023 (4,809,765 records for 261,192 people) to identify people at risk of disengaging from HIV care or developing adverse outcomes. Outcomes included risk of ART non-adherence, non-suppressed viral load, and loss to follow-up. Models were evaluated internally (Kagera region) and externally (Geita region), with performance compared against state-of-art ML models and zero-shot LLMs. Additionally, a team of HIV physicians in Tanzania assessed the LLM’s predictions along with LLM provided justifications for a subset of patient records to evaluate their clinical relevance and reasoning.FindingsThe enhanced LLMs consistently outperformed the supervised ML model and zero-shot LLMs across all outcomes in both internal and external validation datasets. When focusing on the 25% of PLHIV predicted as most likely to lost-to-follow-up (LTFU), the model correctly identified 78% (2,515 of 3,224) of people living with HIV (PLHIV) genuinely at risk in internal validation and 73% (7,105 of 9,733) in external validation. Attention score analysis indicated that the enhanced LLM focused on keywords such as gaps in follow-up care and ART adherence. The human expert evaluation showed alignment between clinician assessments and the LLM’s predictions in 65% of cases, with experts finding the model’s justifications reasonable and clinically relevant in 92.3% of aligned cases.InterpretationIf implemented in HIV clinics, this LLM-based AI model could help allocate resources efficiently and deliver targeted interventions, improving retention in care and advancing the UNAIDS “95-95-95” targets. By functioning like a clinician–analyzing patient summaries, predicting risks, and offering reasoning–the enhanced LLM could be integrated into clinical workflows to complement human expertise, facilitating timely interventions and informed decision-making. If implemented widely, this human-AI collaboration has the potential to improve health outcomes for people living with HIV and reduce onward transmission.FundingThe study was supported by a grant from the US National Institutes of Health (NIH): NIH NIMH 1R01MH125746.

BackgroundPhysical activity (PA) level is associated with multiple health benefits during early childhood. However, inconsistency in the methods for quantification of PA levels among preschoolers remains a problem.ObjectiveThis study aimed to develop PA intensity cut points for wrist-worn accelerometers by using machine learning (ML) approaches to assess PA in preschoolers.MethodsWrist- and hip-derived acceleration data were collected simultaneously from 34 preschoolers on 3 consecutive preschool days. Two supervised ML models, receiver operating characteristic curve (ROC) and ordinal logistic regression (OLR), and one unsupervised ML model, k-means cluster analysis, were applied to establish wrist-worn accelerometer vector magnitude (VM) cut points to classify accelerometer counts into sedentary behavior, light PA (LPA), moderate PA (MPA), and vigorous PA (VPA). Physical activity intensity levels identified by hip-worn accelerometer VM cut points were used as reference to train the supervised ML models. Vector magnitude counts were classified by intensity based on three newly established wrist methods and the hip reference to examine classification accuracy. Daily estimates of PA were compared to the hip-reference criterion.ResultsIn total, 3600 epochs with matched hip- and wrist-worn accelerometer VM counts were analyzed. All ML approaches performed differently on developing PA intensity cut points for wrist-worn accelerometers. Among the three ML models, k-means cluster analysis derived the following cut points: ≤2556 counts per minute (cpm) for sedentary behavior, 2557-7064 cpm for LPA, 7065-14532 cpm for MPA, and ≥14533 cpm for VPA; in addition, k-means cluster analysis had the highest classification accuracy, with more than 70% of the total epochs being classified into the correct PA categories, as examined by the hip reference. Additionally, k-means cut points exhibited the most accurate estimates on sedentary behavior, LPA, and VPA as the hip reference. None of the three wrist methods were able to accurately assess MPA.ConclusionsThis study demonstrates the potential of ML approaches in establishing cut points for wrist-worn accelerometers to assess PA in preschoolers. However, the findings from this study warrant additional validation studies.

1148 Racial disparities in cesarean delivery rates: can a machine learning model reduce these biases?

This study examines the influence of two data preprocessing techniques, Robust Scaler (RS) and Principal Component Analysis (PCA), on the predictive behaviour of Machine Learning (ML) models used for triage assessment in Internet of Medical Things (IoMT)-based telemedicine environments to enhance the decision support for healthcare, where heterogeneous and high-dimensional clinical data often introduce challenges related to variability and analytical stability. The objective is to evaluate how RS and PCA contribute to the consistency and reliability of triage predictions when integrated with established ML algorithms. A dataset of 55,680 outpatient records was used to assess the integration of RS and PCA with five supervised ML models: Support Vector Machine (SVM), Random Forest (RF), Decision Tree (DT), Logistic Regression (LR), and Naïve Bayes (NB). Performance was evaluated using accuracy, precision, recall, and F-score metrics. The evaluation reveals that each preprocessing technique affects model behaviour in distinct ways, and the RS Decision Tree combination exhibits stable performance in generating triage outcomes. PCA-based models exhibit characteristic patterns associated with dimensionality reduction, affecting interpretability and model response. The findings emphasize the role of preprocessing techniques in shaping ML-driven telemedicine workflows. Applying RS within ML pipelines supports consistent triage prediction, contributing to timely identification of patient conditions and strengthening data-driven remote healthcare services.

Financial institutions nowadays rely heavily on rule engines, such as thresholds, white/black lists, velocity checks to flag suspicious transactions. Machine learning (ML) models, on the other hand, while promising higher accuracy and adaptability, are being dependent on data characteristics, class imbalance, latency constraints, and interpretability requirements. In this paper, I present a controlled evaluation of a configurable rule-based baseline and several supervised ML models (logistic regression, random forest, gradient boosting) on an imbalanced transaction dataset. I measure detection performance (ROC-AUC, PR-AUC, precision/recall at operating points), operational costs (false-positive rate, alerts per 1k transactions), and engineering trade-offs (inference latency, feature complexity, interpretability). Results show that while rules remain competitive at high-precision, low-recall regimes, ML approaches achieve substantially better recall at comparable precision, especially when coupled with calibrated thresholds and class-imbalance handling. I discuss deployment-oriented considerations and outline a hybrid strategy that combines rules for policy compliance with ML for generalization.

ObjectivesThis study aimed to develop and validate machine learning (ML) models to predict all-cause hospital admissions and 90-day readmissions using structured, patient-reported survey data.MethodsA cross-sectional survey was conducted between 3 July 2021 and 18 December 2022, among US adults aged ≥18 years with at least one cardiovascular risk factor. Participants were recruited through social media, community pharmacies and outpatient clinics. The final sample included 1318 participants. Primary outcomes were any all-cause hospitalisation and readmission within 90 days. Eight supervised ML models were trained using an 80:20 train–test split and 10-fold cross-validation. Model performance was evaluated using area under the receiver operating characteristic curve (AUROC), precision, recall, F1 score and calibration metrics. SHapley Additive exPlanations (SHAP) values identified key predictors.ResultsAmong 1318 participants, 35.0% reported at least one hospitalisation and 10.4% reported a 90-day readmission. The Extra Trees (ET) model demonstrated the best performance across both outcomes. For hospitalisation, ET achieved an AUROC of 0.93, precision of 0.83 and recall of 0.87. For readmission, AUROC was 0.99 with precision of 0.95 and recall of 0.96. SHAP analysis identified heart disease, medication burden, race/ethnicity, employment and insurance status as the most influential predictors.DiscussionPatient-reported data reflecting behavioural, social and clinical factors can predict hospitalisations with high accuracy, complementing traditional EHR-based models.ConclusionsIntegrating such patient-reported and behavioural data into electronic health records could enable earlier identification of high-risk individuals and support targeted, preventive interventions to improve healthcare outcomes.

Classifying scattered Bengali text is the primary purpose of this study. Our study classified Bengali news titles with Machine Learning (ML) models, particularly focusing on LSTM networks. The goal was to categorize Bengali news into labeled categories and contributing to the growing field of NLP for low-resource languages. For classification, some supervised ML models are used and LSTM for sequential learning. However, LSTM scores the highest accuracy of 81.70%. Additionally, we incorporated explainable AI techniques to decipher the model’s predictions, ensuring transparency and trustworthiness in the classification process. Our study demonstrates the potentiality of deep learning models in improving text classification tasks for the Bengali language and highlights the importance of explainability in AI-driven solutions.

The complexity of the Internet and the volume of network traffic have dramatically increased in the last few years, making it more challenging to design scalable Network Traffic Monitoring and Analysis (NTMA) systems. Critical NTMA applications such as the detection of network attacks and anomalies require fast mechanisms for on-line analysis of thousands of events per second, as well as efficient techniques for off-line analysis of massive historical data. The high-dimensionality of network data provided by current network monitoring systems opens the door to the massive application of machine learning approaches to improve the detection and classification of network attacks and anomalies, but this higher dimensionality comes with an extra data processing overhead. In this paper we present Big-DAMA, a big data analytics framework (BDAF) for NTMA applications. Big-DAMA is a flexible BDAF, capable to analyze and store big amounts of both structured and unstructured heterogeneous data sources, with both stream and batch processing capabilities. Big-DAMA uses off-the-shelf big data storage and processing engines to offer both stream data processing and batch processing capabilities, decomposing separate engines for stream, batch and query, following a Data Stream Warehouse (DSW) paradigm. Big-DAMA implements several algorithms for anomaly detection and network security using supervised and unsupervised machine learning (ML) models, using off-the-shelf ML libraries. We apply Big-DAMA to the detection of different types of network attacks and anomalies, benchmarking multiple supervised ML models. Evaluations are conducted on top of real network measurements collected at the WIDE backbone network, using the well-known MAWILab dataset for attacks labeling. Big-DAMA can speed up computations by a factor of 10 when compared to a standard Apache Spark cluster, and can be easily deployed in cloud environments, using hardware virtualization technology.