Model Insights: No-Code Platform to Explain and Improve ML Models

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Social media platforms provide valuable insights into mental health trends by capturing user-generated discussions on conditions such as depression, anxiety, and suicidal ideation. Machine learning (ML) and deep learning (DL) models have been increasingly applied to classify mental health conditions from textual data, but selecting the most effective model involves trade-offs in accuracy, interpretability, and computational efficiency. This study evaluates multiple ML models, including logistic regression, random forest, and LightGBM, alongside DL architectures such as ALBERT and Gated Recurrent Units (GRUs), for both binary and multi-class classification of mental health conditions. Our findings indicate that ML and DL models achieve comparable classification performance on medium-sized datasets, with ML models offering greater interpretability through variable importance scores, while DL models are more robust to complex linguistic patterns. Additionally, ML models require explicit feature engineering, whereas DL models learn hierarchical representations directly from text. Logistic regression provides the advantage of capturing both positive and negative associations between features and mental health conditions, whereas tree-based models prioritize decision-making power through split-based feature selection. This study offers empirical insights into the advantages and limitations of different modeling approaches and provides recommendations for selecting appropriate methods based on dataset size, interpretability needs, and computational constraints.

Interpretable vs. noninterpretable machine learning models for data-driven hydro-climatological process modeling

This research addresses the critical challenge of cybersecurity in healthcare by evaluating the effectiveness of machine learning (ML) and deep learning (DL) models in identifying and mitigating five significant cybersecurity threats: distributed denial-of-service (DDoS), man-in-the-middle (MitM), malware, phishing, and SQL injection. The study integrates a secure hardware–software architecture utilizing WireGuard, a lightweight, modern VPN protocol that establishes encrypted tunnels between network nodes, ensuring robust data integrity, confidentiality, and authenticated communication. Two ML models, support vector machine and random forest, and four DL architectures, dense neural networks (DNNs), convolutional neural networks-long short-term memory (CNN-LSTM), and LSTM-gated recurrent unit (LSTM-GRU), are systematically trained and tested using publicly available datasets specific to each threat category. The experimental outcomes demonstrate exceptional detection capabilities for structured network threats, with DNN and CNN-LSTM achieving accuracies and F1-scores from 95% to 97.6% for DDoS and MitM threats. In malware classification, the performance of DNN and CNN maintains precision and recall above 94%. Phishing and SQL injection attacks have lower classification scores of around 82% for most models. Visual analytics, including accuracy, loss plots, and confusion matrices, provide valuable insights into the convergence behaviors and sensitivity of different architectures, highlighting the strong generalization of DNN and variability in recurrent models. Overall, this research highlights the substantial potential of DL, combined with secure communication technologies like WireGuard, in enhancing healthcare cybersecurity, while also identifying areas for further development and optimization.

Machine learning (ML) models provide more choices to patients with diabetes mellitus (DM) to more properly manage blood glucose (BG) levels. However, because of numerous types of ML algorithms, choosing an appropriate model is vitally important. In a systematic review and network meta-analysis, this study aimed to comprehensively assess the performance of ML models in predicting BG levels. In addition, we assessed ML models used to detect and predict adverse BG (hypoglycemia) events by calculating pooled estimates of sensitivity and specificity. PubMed, Embase, Web of Science, and Institute of Electrical and Electronics Engineers Explore databases were systematically searched for studies on predicting BG levels and predicting or detecting adverse BG events using ML models, from inception to November 2022. Studies that assessed the performance of different ML models in predicting or detecting BG levels or adverse BG events of patients with DM were included. Studies with no derivation or performance metrics of ML models were excluded. The Quality Assessment of Diagnostic Accuracy Studies tool was applied to assess the quality of included studies. Primary outcomes were the relative ranking of ML models for predicting BG levels in different prediction horizons (PHs) and pooled estimates of the sensitivity and specificity of ML models in detecting or predicting adverse BG events. In total, 46 eligible studies were included for meta-analysis. Regarding ML models for predicting BG levels, the means of the absolute root mean square error (RMSE) in a PH of 15, 30, 45, and 60 minutes were 18.88 (SD 19.71), 21.40 (SD 12.56), 21.27 (SD 5.17), and 30.01 (SD 7.23) mg/dL, respectively. The neural network model (NNM) showed the highest relative performance in different PHs. Furthermore, the pooled estimates of the positive likelihood ratio and the negative likelihood ratio of ML models were 8.3 (95% CI 5.7-12.0) and 0.31 (95% CI 0.22-0.44), respectively, for predicting hypoglycemia and 2.4 (95% CI 1.6-3.7) and 0.37 (95% CI 0.29-0.46), respectively, for detecting hypoglycemia. Statistically significant high heterogeneity was detected in all subgroups, with different sources of heterogeneity. For predicting precise BG levels, the RMSE increases with a rise in the PH, and the NNM shows the highest relative performance among all the ML models. Meanwhile, current ML models have sufficient ability to predict adverse BG events, while their ability to detect adverse BG events needs to be enhanced. PROSPERO CRD42022375250; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=375250.

Study DesignRetrospective cohort study.ObjectivesProlonged ICU stay is a driver of higher costs and inferior outcomes in Adult Spinal Deformity (ASD) patients. Machine learning (ML) models have recently been seen as a viable method of predicting pre-operative risk but are often ‘black boxes’ that do not fully explain the decision-making process. This study aims to demonstrate ML can achieve similar or greater predictive power as traditional statistical methods and follows traditional clinical decision-making processes.MethodsFive ML models (Decision Tree, Random Forest, Support Vector Classifier, GradBoost, and a CNN) were trained on data collected from a large urban academic center to predict whether prolonged ICU stay would be required post-operatively. 535 patients who underwent posterior fusion or combined fusion for treatment of ASD were included in each model with a 70-20-10 train-test-validation split. Further analysis was performed using Shapley Additive Explanation (SHAP) values to provide insight into each model’s decision-making process.ResultsThe model’s Area Under the Receiver Operating Curve (AUROC) ranged from 0.67 to 0.83. The Random Forest model achieved the highest score. The model considered length of surgery, complications, and estimated blood loss to be the greatest predictors of prolonged ICU stay based on SHAP values.ConclusionsWe developed a ML model that was able to predict whether prolonged ICU stay was required in ASD patients. Further SHAP analysis demonstrated our model aligned with traditional clinical thinking. Thus, ML models have strong potential to assist with risk stratification and more effective and cost-efficient care.

Data is one of the most important limiting factors of deep machine learning (ML) model in drilling applications. Though a big size of historical data can be available, high-quality cleaned and labeled data is usually limited. In this case study, we show that with limited labeled data, physics-based data augmentation combined with unsupervised learning significantly improves both stability and accuracy in bit wear ML model. It provides a pathway to overcome labeled data shortage and field data quality limitations. Labeled bit wear data is usually limited because only the final bit dull state can be labeled using dull photos for the entire drilling distance. To overcome this difficulty, an encoder-decoder unsupervised ML framework based on Bi-directional LSTM architecture is first applied to the data of entire drilling distance to extract and maximize data learnings. Then, a physics-based wear estimator is implemented in the learned latent space to augment labeled wear dataset, guided by true wear labels and offset-well data. Weighting factor in loss function is applied to optimize augmented and true datasets in final supervised learning step. The proposed real-time bit wear model is built on a combination of physics-based data augmentation and unsupervised data learning method. The model is applied on multiple field bit runs. Results show that by implementing the unsupervised data learnings only, the prediction accuracy is improved by 30% compared to the baseline ML model. By combining physics-based augmentation, the accuracy is further improved by 10%. More importantly, adding physics-based data augmentation significantly reduces prediction variance and unphysical wear outputs, therefore improving prediction stability by more than 30%. It should be noted that the prediction stability of AI/ML model is crucial in real-time application and decision making. The results show physics-based data augmentation not only increases the size of label dataset and prevents model overfitting, but also applies physics-based guidance to the ML model effectively. It is learned weighting factor plays a crucial role in balancing loss contributions from true wear labels and physics-based labels. While low weighting factor of physics-based labels diminishes the augmented data, high weighting factor disrespects the true wear labels leading to a high prediction bias. Overall, the bit wear model study shows physic-based data augmentation combined with unsupervised data learning can effectively improve model accuracy, stability, and overcome labeled data shortage difficulty. The proposed paper shows a case study for the bit wear ML model using a combination of physics-based data augmentation and unsupervised data learning. While labeled data is one of the major challenges of ML models in many drilling applications, this study provides a pathway to improve both accuracy and stability of deep ML models with limited labeled data.

To evaluate the diagnostic performance of Machine Learning (ML) and Deep Learning (DL) models for predicting preoperative Lymph Node Metastasis (LNM) in Colorectal Cancer (CRC) patients. A systematic review and meta-analysis were conducted following PRISMA-DTA and AMSTAR-2 guidelines. We searched PubMed, Web of Science, Embase, and Cochrane Library databases until February 16, 2024. Study quality and risk of bias were assessed using the QUADAS-2 tool. Data were analyzed using STATA v18, applying random-effects models to all analyses. Twelve studies involving 8321 patients were included, with most published in 2021-2024 (9/12). The pooled AUC of ML models for predicting LNM in CRC patients was 0.87 (95% CI: 0.82-0.91, I2:86.17) with a sensitivity of 78% (95% CI: 69-87%) and a specificity of 77% (95% CI: 64%-90%). In addition, when assessing the AUC reported by radiologists, both junior and senior radiologists had similar performance, significantly lower than the ML models. (P < 0.001). Subgroup analysis revealed higher AUCs in prospective studies (0.95, 95% CI: 0.87-1) compared to retrospective studies (0.85, 95% CI: 0.81-0.89) (P = 0.03). Studies without external validation exhibited significantly higher AUCs than those with external validation (P < 0.01). While there was no significant difference in AUC and sensitivity between the T1-T2 and T2-T4 stages, specificity was significantly higher in the T2-T4 stages than the low stages of T1 and T2 (95%, 95% CI: 92-98% vs. 61%, 95% CI: 44-78%; P < 0.01). ML models demonstrate strong potential for preoperative LNM staging and treatment planning in CRC, potentially reducing the need for additional surgeries and related health and financial burdens. Further prospective multicenter studies, with standardized reporting of algorithms, modality parameters, and LNM staging, are needed to validate these findings.

Accurate prediction of urban residential rents is of great importance for landlords, tenants, and investors. However, existing rents prediction models face challenges in meeting practical demands due to their limited perspectives and inadequate prediction performance. The existing individual prediction models often lack satisfactory accuracy, while ensemble learning models that combine multiple individual models to improve prediction results often overlook the impact of spatial heterogeneity on residential rents. To address these issues, this paper proposes a novel prediction model called GERPM, which stands for Geographically Weighted Stacking Ensemble Learning-Based Urban Residential Rents Prediction Model. GERPM comprehensively analyzes the influencing factors of residential rents from multiple perspectives and leverages a geographically weighted stacking ensemble learning approach. The model combines multiple machine learning and deep learning models, optimizes parameters to achieve optimal predictions, and incorporates the geographically weighted regression (GWR) model to consider spatial heterogeneity. By combining the strengths of deep learning and machine learning models and taking into account geographical factors, GERPM aims to improve prediction accuracy and provide robust predictions for urban residential rents. The model is evaluated using housing data from Nanjing, a major city in China, and compared with representative individual prediction models, the equal weight combination model, and the ensemble learning model. The experimental results demonstrate that GERPM outperforms other models in terms of prediction performance. Furthermore, the model’s effectiveness and robustness are validated by applying it to other major cities in China, such as Shanghai and Hangzhou. Overall, GERPM shows promising potential in accurately predicting urban residential rents and contributing to the advancement of the rental market.

Uncertainty estimation in healthcare involves quantifying and understanding the inherent uncertainty or variability associated with medical predictions, diagnoses, and treatment outcomes. In this era of Artificial Intelligence (AI) models, uncertainty estimation becomes vital to ensure safe decision-making in the medical field. Therefore, this review focuses on the application of uncertainty techniques to machine and deep learning models in healthcare.A systematic literature review was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Our analysis revealed that Bayesian methods were the predominant technique for uncertainty quantification in machine learning models, with Fuzzy systems being the second most used approach. Regarding deep learning models, Bayesian methods emerged as the most prevalent approach, finding application in nearly all aspects of medical imaging.Most of the studies reported in this paper focused on medical images, highlighting the prevalent application of uncertainty quantification techniques using deep learning models compared to machine learning models. Interestingly, we observed a scarcity of studies applying uncertainty quantification to physiological signals. Thus, future research on uncertainty quantification should prioritize investigating the application of these techniques to physiological signals.Overall, our review highlights the significance of integrating uncertainty techniques in healthcare applications of machine learning and deep learning models. This can provide valuable insights and practical solutions to manage uncertainty in real-world medical data, ultimately improving the accuracy and reliability of medical diagnoses and treatment recommendations.

Stent malapposition (SM) following percutaneous coronary intervention (PCI) for myocardial infarction continues to present significant clinical challenges. In recent years, machine learning (ML) models have demonstrated potential in disease risk stratification and predictive modeling. ML models based on optical coherence tomography (OCT) imaging, laboratory tests, and clinical characteristics can predict the occurrence of SM. We studied 337 patients from the Affiliated Hospital of Zunyi Medical University, China, who had PCI and coronary OCT from May to October 2023. We employed nested cross-validation to partition patients into training and test sets. We developed five ML models: XGBoost, LR, RF, SVM, and NB based on calcification features. Performance was assessed using ROC curves. Lasso regression selected features from 46 clinical and 21 OCT imaging features, which were optimized with the five ML algorithms. In the prediction model based on calcification features, the XGBoost model and SVM model exhibited higher AUC values. Lasso regression identified five key features from clinical and imaging data. After incorporating selected features into the model for optimization, the AUC values of all algorithmic models showed significant improvements. The XGBoost model demonstrated the highest calibration accuracy. SHAP values revealed that the top five ranked features influencing the XGBoost model were calcification length, age, coronary dissection, lipid angle, and troponin. ML models developed using plaque imaging features and clinical characteristics can predict the occurrence of SM. ML models based on clinical and imaging features exhibited better performance.

BackgroundThe large amount of heterogeneous data collected in surgical/endoscopic practice calls for data-driven approaches as machine learning (ML) models. The aim of this study was to develop ML models to predict endoscopic sleeve gastroplasty (ESG) efficacy at 12 months defined by total weight loss (TWL) % and excess weight loss (EWL) % achievement. Multicentre data were used to enhance generalizability: evaluate consistency among different center of ESG practice and assess reproducibility of the models and possible clinical application. Models were designed to be dynamic and integrate follow-up clinical data into more accurate predictions, possibly assisting management and decision-making.MethodsML models were developed using data of 404 ESG procedures performed at 12 centers across Europe. Collected data included clinical and demographic variables at the time of ESG and at follow-up. Multicentre/external and single center/internal and temporal validation were performed. Training and evaluation of the models were performed on Python’s scikit-learn library. Performance of models was quantified as receiver operator curve (ROC-AUC), sensitivity, specificity, and calibration plots.ResultsMulticenter external validation: ML models using preoperative data show poor performance. Best performances were reached by linear regression (LR) and support vector machine models for TWL% and EWL%, respectively, (ROC-AUC: TWL% 0.87, EWL% 0.86) with the addition of 6-month follow-up data.Single-center internal validation: Preoperative data only ML models show suboptimal performance. Early, i.e., 3-month follow-up data addition lead to ROC-AUC of 0.79 (random forest classifiers model) and 0.81 (LR models) for TWL% and EWL% achievement prediction, respectively. Single-center temporal validation shows similar results.ConclusionsAlthough preoperative data only may not be sufficient for accurate postoperative predictions, the ability of ML models to adapt and evolve with the patients changes could assist in providing an effective and personalized postoperative care. ML models predictive capacity improvement with follow-up data is encouraging and may become a valuable support in patient management and decision-making.

Autistic people are often disadvantaged in employment, education, etc. In fact, autistic students/employees face several challenges navigating and communicating with their superiors and colleagues. Mobile applications for people with Autism Spectrum Disorder (ASD apps for short) have been increasingly being adapted to help autistic people manage their conditions and daily activities. User feedback analysis is an effective method that can be used to improve ASD apps' services. In this article, we investigate the usage of ASD apps to improve the quality of life for autistic students/employees based on user feedback analysis. For this purpose, we analyze user reviews suggested on highly ranked ASD apps for college students, and workers. A total of 97,051 reviews have been collected from 13 ASD apps available on Google Play and Apple App stores. The collected reviews have been classified into negative, positive, and neutral opinions. This analysis has been performed using machine learning and deep learning models. The best performances were provided by combining RNN and LSTM models with an accuracy of 96.58% and an AUC of 99.41%. Finally, we provide some recommendations to improve ASD apps to assist developers in upgrading the main services provided by their apps.

Although advancements in remote sensing technology have facilitated quick capture and identification of the source and location of oil spills in water bodies, the presence of other biogenic elements (lookalikes) with similar visual attributes hinder rapid detection and prompt decision making for emergency response. To date, different methods have been applied to distinguish oil spills from lookalikes with limited success. In addition, accurately modeling the trajectory of oil spills remains a challenge. Thus, we aim to provide further insights on the multi-faceted problem by undertaking a holistic review of past and current approaches to marine oil spill disaster reduction as well as explore the potentials of emerging digital trends in minimizing oil spill hazards. The scope of previous reviews is extended by covering the inter-related dimensions of detection, discrimination, and trajectory prediction of oil spills for vulnerability assessment. Findings show that both optical and microwave airborne and satellite remote sensors are used for oil spill monitoring with microwave sensors being more widely used due to their ability to operate under any weather condition. However, the accuracy of both sensors is affected by the presence of biogenic elements, leading to false positive depiction of oil spills. Statistical image segmentation has been widely used to discriminate lookalikes from oil spills with varying levels of accuracy but the emergence of digitalization technologies in the fourth industrial revolution (IR 4.0) is enabling the use of Machine learning (ML) and deep learning (DL) models, which are more promising than the statistical methods. The Support Vector Machine (SVM) and Artificial Neural Network (ANN) are the most used machine learning algorithms for oil spill detection, although the restriction of ML models to feed forward image classification without support for the end-to-end trainable framework limits its accuracy. On the other hand, deep learning models’ strong feature extraction and autonomous learning capability enhance their detection accuracy. Also, mathematical models based on lagrangian method have improved oil spill trajectory prediction with higher real time accuracy than the conventional worst case, average and survey-based approaches. However, these newer models are unable to quantify oil droplets and uncertainty in vulnerability prediction. Considering that there is yet no single best remote sensing technique for unambiguous detection and discrimination of oil spills and lookalikes, it is imperative to advance research in the field in order to improve existing technology and develop specialized sensors for accurate oil spill detection and enhanced classification, leveraging emerging geospatial computer vision initiatives.

Machine learning and deep learning prediction of patient specific quality assurance in breast IMRT radiotherapy plans using Halcyon specific complexity indices