Abstract A40: The SIAMCAT R package enables statistical and machine learning analyses for case-control microbiome datasets

Scientific workflows are one of the well-established pillars of modern large-scale computational science. More recently, scientists have started to leverage machine learning (ML) capabilities in their workflows, leading to a new category of scientific workflows, denoted as scientific ML workflows. ML is not only about training and inference, modern ML workflows also involve complex data processing steps before the training can start, which are not often accounted for in most performance studies. In this work, we consider scientific ML workflows, from data pre-processing to training, inference, and model evaluation. We aim to explore (i) how scientific ML workflows differ from more traditional scientific workflows and; (ii) how we can characterize ML workflows both in terms of execution time and data movements when executing on a target cloud platform. We select three representative workflows, ranging from image classification to natural language processing and image segmentation, which have been executed using the academic cloud platform, Chameleon. We build four realistic deployment scenarios for each workflow, which stress data movements during workflow executions. Then, we compare the performance observed when utilizing these different configurations and study how different settings impact overall workflows performance and efficiency when running on cloud infrastructures. Finally, we summarize our findings and discuss performance impacts when augmenting scientific workflows with ML techniques and how traditional workflow management systems can improve their support for such workflows.

The use of deep learning and machine learning (ML) in medical science is increasing, particularly in the visual, audio, and language data fields. We aimed to build a new optimized ensemble model by blending a DNN (deep neural network) model with two ML models for disease prediction using laboratory test results. 86 attributes (laboratory tests) were selected from datasets based on value counts, clinical importance-related features, and missing values. We collected sample datasets on 5145 cases, including 326,686 laboratory test results. We investigated a total of 39 specific diseases based on the International Classification of Diseases, 10th revision (ICD-10) codes. These datasets were used to construct light gradient boosting machine (LightGBM) and extreme gradient boosting (XGBoost) ML models and a DNN model using TensorFlow. The optimized ensemble model achieved an F1-score of 81% and prediction accuracy of 92% for the five most common diseases. The deep learning and ML models showed differences in predictive power and disease classification patterns. We used a confusion matrix and analyzed feature importance using the SHAP value method. Our new ML model achieved high efficiency of disease prediction through classification of diseases. This study will be useful in the prediction and diagnosis of diseases.

In many domains such as Telecom various scenarios necessitate the processing of large amounts of data using statistical and machine learning algorithms. A noticeable effort has been made to move the data management systems into MapReduce parallel processing environments such as Hadoop and Pig. Nevertheless these systems lack the features of advanced machine learning and statistical analysis. Frame-works such as Mahout on top of Hadoop support machine learning but their implementations are at the preliminary stage. For example Mahout does not provide Support Vector Machine (SVM) algorithms and it is difficult to use. On the other hand traditional statistical software tools such as R containing comprehensive statistical algorithms for advanced analysis are widely used. But such software can only run on a single computer and therefore it is not scalable. In this paper we propose an integrated solution RPig which takes the advantages of R (for machine learning and statistical analysis capabilities) and parallel data processing capabilities of Pig. The RPig framework offers a scalable advanced data analysis solution for machine learning and statistical analysis. Analysis jobs can be easily developed with RPig script in high level languages. We describe the design implementation and an eclipse-based RPigEditor for the RPig framework. Using application scenarios from the Telecom domain we show the usage of RPig and how the framework can significantly reduce the development effort. The results demonstrate the scalability of our framework and the simplicity of deployment for analysis jobs.

Enabled by a rich ecosystem of Machine Learning (ML) libraries, programming using learned models , i.e., Software-2.0 , has gained substantial adoption. However, we do not know what challenges developers encounter when they use ML libraries. With this knowledge gap, researchers miss opportunities to contribute to new research directions, tool builders do not invest resources where automation is most needed, library designers cannot make informed decisions when releasing ML library versions, and developers fail to use common practices when using ML libraries. We present the first large-scale quantitative and qualitative empirical study to shed light on how developers in Software-2.0 use ML libraries, and how this evolution affects their code. Particularly, using static analysis we perform a longitudinal study of 3,340 top-rated open-source projects with 46,110 contributors. To further understand the challenges of ML library evolution, we survey 109 developers who introduce and evolve ML libraries. Using this rich dataset we reveal several novel findings. Among others, we found an increasing trend of using ML libraries: The ratio of new Python projects that use ML libraries increased from 2% in 2013 to 50% in 2018. We identify several usage patterns including the following: (i) 36% of the projects use multiple ML libraries to implement various stages of the ML workflows, (ii) developers update ML libraries more often than the traditional libraries , (iii) strict upgrades are the most popular for ML libraries among other update kinds, (iv) ML library updates often result in cascading library updates, and (v) ML libraries are often downgraded (22.04% of cases). We also observed unique challenges when evolving and maintaining Software-2.0 such as (i) binary incompatibility of trained ML models and (ii) benchmarking ML models. Finally, we present actionable implications of our findings for researchers, tool builders, developers, educators, library vendors, and hardware vendors.

The coding capabilities of large language models (LLMs) have opened up new opportunities for automatic statistical analysis in machine learning and data science. However, before their widespread adoption, it is crucial to assess the accuracy of code generated by LLMs. A major challenge in this evaluation lies in the absence of a benchmark dataset for statistical code. To fill in this gap, this paper introduces StatLLM, an open-source dataset for evaluating the performance of LLMs in statistical analysis. The StatLLM dataset comprises three key components: statistical analysis tasks, LLM-generated SAS code, and human evaluation scores. The first component includes statistical analysis tasks spanning a variety of analyses and datasets, providing problem descriptions, dataset details, and human-verified SAS code. The second component features SAS code generated by GPT-3.5, GPT-4, and Llama-3.1 70B for those tasks. The third component contains evaluation scores from human experts in assessing the correctness, effectiveness, readability, executability, and output accuracy of the LLM-generated code. We also illustrate the unique potential of the established benchmark dataset for (1) evaluating and enhancing natural language processing metrics, (2) assessing and improving LLM performance in statistical coding, and (3) developing and testing of next-generation statistical software – advancements that are crucial for data science and machine learning research. This paper has online supplementary materials.

An automated software methodology for biomedical statistics, data pre-processing, and machine learning.

Machine learning (ML) has become an increasingly popular tool for use in neurosurgical research. The number of publications and interest in the field have recently seen significant expansion in both quantity and complexity. However, this also places a commensurate burden on the general neurosurgical readership to appraise this literature and decide if these algorithms can be effectively translated into practice. To this end, the authors sought to review the burgeoning neurosurgical ML literature and to develop a checklist to help readers critically review and digest this work. The authors performed a literature search of recent ML papers in the PubMed database with the terms "neurosurgery" AND "machine learning," with additional modifiers "trauma," "cancer," "pediatric," and "spine" also used to ensure a diverse selection of relevant papers within the field. Papers were reviewed for their ML methodology, including the formulation of the clinical problem, data acquisition, data preprocessing, model development, model validation, model performance, and model deployment. The resulting checklist consists of 14 key questions for critically appraising ML models and development techniques; these are organized according to their timing along the standard ML workflow. In addition, the authors provide an overview of the ML development process, as well as a review of key terms, models, and concepts referenced in the literature. ML is poised to become an increasingly important part of neurosurgical research and clinical care. The authors hope that dissemination of education on ML techniques will help neurosurgeons to critically review new research better and more effectively integrate this technology into their practices.

Background/Objectives: Vitamin D deficiency (VDD) is a global health concern associated with metabolic disease and immune dysfunction. Despite known risk factors like limited sun exposure, diet, and lifestyle, few studies have explored these factors comprehensively on a large scale. This cross-sectional study aimed to identify VDD-associated factors in South Korea via an integrative approach of machine learning and statistical analyses using Korea National Health and Nutrition Examination Survey (KNHANES) IX-1 data. Methods: Using the KNHANES dataset, six machine learning algorithms were applied to evaluate VDD (serum 25[OH]D3 < 20 ng/mL)-associated factors through feature importance scores. Thereafter, multivariate linear and logistic regression models were applied to the dataset-stratified by sex and age. Results: Among 583 variables, 17 VDD-associated factors were identified using the CatBoost model, which achieved the highest F1 score. When these factors were assessed through statistical analysis, dietary supplement use emerged as a consistent factor associated with VDD across all subgroups (younger men, younger women, older men, and older women). In younger adults, HDL cholesterol, blood and urinary creatinine, water intake, urban residence, and breakfast frequency were significantly associated with VDD. Additionally, blood urea nitrogen and fasting plasma glucose in men and urinary sodium in women showed sex-specific associations with serum 25(OH)D levels. Conclusions: This study identified key VDD-associated factors in the South Korean population, which varied by age or sex. These findings highlight the multifaceted nature of VDD, influenced by dietary, lifestyle, and biochemical factors and underscore the need for strategies integrating machine learning and statistical analysis.

SUMMARYNeuronal oscillations are suggested to play an important role in auditory working memory (WM), but their contribution to content-specific representations has remained unclear. Here, we measure magnetoencephalography during a retro-cueing task with parametric ripple-sound stimuli, which are spectrotemporally similar to speech but resist non-auditory memory strategies. Using machine learning analyses, with rigorous between-subject cross-validation and non-parametric permutation testing, we show that memorized sound content is strongly represented in phase-synchronization patterns between subregions of auditory and frontoparietal cortices. These phase-synchronization patterns predict the memorized sound content steadily across the studied maintenance period. In addition to connectivity-based representations, there are indices of more local, “activity silent” representations in auditory cortices, where the decoding accuracy of WM content significantly increases after task-irrelevant “impulse stimuli.” Our results demonstrate that synchronization patterns across auditory sensory and association areas orchestrate neuronal coding of auditory WM content. This connectivity-based coding scheme could also extend beyond the auditory domain.

Traditional workflows in chemical engineering, from design to characterization, have served the field well. Emerging technologies, such as machine learning, now offer opportunities for faster design iterations and better integration of the various development stages. This study demonstrates how machine learning (ML) can reduce this gap by directly integrating experimental data into predictive models. Focusing on a Additive Manufactured helical micro-distillation column design for separating liquid mixtures, several ML models were trained to predict the number of theoretical stages based on geometric parameters and operating conditions. 11 variants were designed, built, and tested to collect 197 experimental data sets to assess the feasibility of five predictive ML models from experimental data. Among the algorithms tested, the Gradient Boosting Regressor achieved the best performance with a coefficient of determination R 2 = 0 . 7934 . The work highlights the behavior of the model across different regimes, identifies key sources of error, and proposes a hybrid experimental–ML workflow for rapid screening of distillation designs. This approach accelerates process development and reduces the need for extensive experimentation, especially in time-consuming tasks such as distillation. • ML predicts KPIs for complex AM helical distillation columns using real data. • Multiple supervised ML models benchmarked for AM-induced geometric effects. • Multi-output modeling enhances stability and captures physical correlations. • Hybrid ML–experimental workflow identifies high-value regions for testing. • Framework accelerates design and screening of AM-enabled separation units.

Applications of machine learning in metal-organic frameworks

Machine learning application in the oil and gas industry is rapidly becoming popular and in recent years has been applied in the optimization of production for various reservoirs. The objective of this paper is to evaluate the efficacy of advanced machine learning algorithms in reservoir production optimization. A 3-D geological model was constructed based on permeability calculated using a machine learning technique which involved different architectures of algorithms tested using a 5-fold cross-validation to decide the best machine learning algorithm. Sensitivity analysis and a subsequent history matching were conducted using a machine learning workflow. The aquifer properties, permeability heterogeneity in different directions and relative permeability were the control variables assessed. Field development scenarios were exploited with the objective to optimize cumulative oil recovery. The impact of using a normal depletion plan to a secondary recovery plan using waterflooding was investigated. Different injection well placement locations, well patterns as well as the possibility of converting existing oil producing wells to water injection wells were exploited. Considering the outcome of an economic analysis, the optimum development strategy was realized as an outcome for the optimization process. Prior to forecasting cumulative oil production using artificial neural network (ANN) for the optimization process on the generated surrogate model, a sensitivity analysis was performed where the well location, injection rates and bottomhole pressure of both the producer and injector wells were specified as control variables. The water cut as part of the optimization process was utilized as a secondary constraint. Forecasting was performed for a 15-year period. The history-matching results from the constructed geological model showed that the oil rate, water rate, bottom hole pressure, and average reservoir pressure were matched within a 10% deviation from the observed data. In this study, the ANN optimizer was found to provide the best results for the field cumulative oil production. Using a secondary recovery development plan was observed to significantly increase the cumulative oil production. A machine learning based proxy model was built for the prediction of cumulative oil production to reduce computational time. In this study, we propose an approach applied to reservoir production optimization utilizing a machine learning workflow. This was accomplished by utilizing a surrogate model which was calibrated with a number of training simulations and then optimized using advanced machine learning algorithms. A detailed economic analysis was also conducted showing the impact of a variety of field development strategies.

A skilled workforce is the ultimate strength of the economy. COVID-19 has affected the continuity of the regular workforce remarkably across the world. A large number of the workforce had been splashed from operations, and a significant number had been continued operations from remotely. The aim of this research is to analyze the COVID-19 effects on the developing countries workforce like Bangladesh. In this paper, we have performed the statistical analysis and developed a machine learning model based on Bangladeshi IT company daily activities. The dataset has been separated into three parts based on periods such as pre-COVID-19, lockdown and post-lockdown periods. We have used several supervised learning algorithms such as SVM, LR, RF and DT for both classification and regression problem. From statistical analysis, it has been observed that regular operations of pre-COVID-19 have been disrupted during the lockdown period, and to minimize the damages of the lockdown period, company workforces are putting extra effort during the post-lockdown period. From machine learning analysis, it has been observed that random forest (RF) has performed better than other classifiers for both classification and regression problem.KeywordsWorkforceCOVID-19 effectMachine learningRegressionBaseline evaluation

Objective: The present study aimed to predict adolescent risk behavior by examining the combined effects of family monitoring, sensation seeking, and peer deviance using machine learning analytical approaches. Methods and Materials: This cross-sectional predictive study was conducted among 512 adolescents aged 14 to 18 years recruited from secondary schools in Tunisia using multistage cluster random sampling. Data were collected using standardized self-report instruments, including the Youth Risk Behavior Surveillance Questionnaire, the Parental Monitoring Scale, the Brief Sensation Seeking Scale, and the Peer Delinquency Scale, all of which demonstrated acceptable reliability and validity. Statistical analyses were performed using SPSS-27 to compute descriptive indices and Pearson correlations. Machine learning models, including random forest, support vector machine, gradient boosting, and multilayer perceptron neural networks, were implemented in Python. Data preprocessing included normalization and feature scaling, and model performance was evaluated using k-fold cross-validation and metrics such as accuracy, precision, recall, F1-score, and area under the ROC curve. Findings: The results revealed that adolescent risk behavior was significantly negatively associated with family monitoring and positively associated with sensation seeking and peer deviance. Among the predictive models, the neural network demonstrated the highest performance, achieving the greatest accuracy and discriminative power. Feature importance analyses consistently identified peer deviance as the strongest predictor of risk behavior, followed by sensation seeking, while family monitoring showed a weaker but still significant contribution. The models indicated that nonlinear interactions among predictors significantly improved prediction accuracy compared to linear approaches. Conclusion: The findings highlight the critical role of peer deviance and sensation seeking in shaping adolescent risk behavior, while confirming the protective function of family monitoring. The superior performance of machine learning models underscores their utility in capturing complex behavioral patterns and enhancing predictive precision.

Background Previous studies have described machine learning (ML) models to predict how human readers would score disease activity in UC using the endoscopic Mayo Score (eMS). So far, none employed deep human annotation that considers all the endoscopic features making up the eMS. Here we report the results of an ML model that is trained on eMS features using centrally read endoscopies. Methods 793 full-length videos were obtained from 249 patients with UC who participated in NCT02589665, a phase 2 trial with mirikizumab in patients with UC and associated with centrally read (single reader) eMS (CReMS) as the primary dataset. After cleaning for usable frames, the data were split into training, validation and testing subsets. The ML workflow consisted of annotation, segmentation, and classification (e.g., erosions, ulcers, erythema, vascular pattern, and bleeding). Human image classification and segmentation with bounding boxes and was subjected to quality control adjudicated by one of three IBD specialists, generating more than 60,000 eMS-relevant annotation labels. The model was evaluated on a test set of 147 videos using the CReMS, and a consensus set of 94 test videos, where CReMS and annotator reported eMS (AReMS) were in agreement without adjudication. The primary objective of the model was a categorical prediction of endoscopically inactive disease (eMS 0 &amp; 1) compared with active disease (eMS 2 &amp; 3). The secondary objectives of the model were to predict endoscopic healing (eMS 0) and to predict severe disease (eMS 3). Results The model performances are in Table 1. On the full test set of 147 videos, the model predicted inactive disease compared with active disease with an accuracy of 84%, positive predictive value (PPV) of 80%, and negative predictive value (NPV) of 85%. In the subset of 94 videos with CreMS and AReMS consensus, the model predicted inactive disease compared with active disease with an accuracy of 89%, PPV 87%, and NPV of 90%. In this same subset, the model predicted endoscopic healing and severe disease with an accuracy of 95% and 85%, PPVs of 86% and 82% and NPVs of 95% and 87%, respectively. For the secondary objectives in the full set of 147 videos, the model predicted endoscopic healing and severe disease with an accuracy of 90% and 80%, PPVs of 44% and 86%, and NPVs of 95% and 86%, respectively. Conclusion We have developed a ML predictive model of the eMS in UC using centrally read videos and demonstrate excellent distinction between active and inactive disease, and clear discrimination between other levels of endoscopic activity. We propose that this unique ML approach to endoscopic assessment be considered as a substitute to human central reading in future clinical trials.