A Practical Illustration of the Machine Learning Workflow Using a Multiple-Choice Question Dataset

Scientific workflows are one of the well-established pillars of modern large-scale computational science. More recently, scientists have started to leverage machine learning (ML) capabilities in their workflows, leading to a new category of scientific workflows, denoted as scientific ML workflows. ML is not only about training and inference, modern ML workflows also involve complex data processing steps before the training can start, which are not often accounted for in most performance studies. In this work, we consider scientific ML workflows, from data pre-processing to training, inference, and model evaluation. We aim to explore (i) how scientific ML workflows differ from more traditional scientific workflows and; (ii) how we can characterize ML workflows both in terms of execution time and data movements when executing on a target cloud platform. We select three representative workflows, ranging from image classification to natural language processing and image segmentation, which have been executed using the academic cloud platform, Chameleon. We build four realistic deployment scenarios for each workflow, which stress data movements during workflow executions. Then, we compare the performance observed when utilizing these different configurations and study how different settings impact overall workflows performance and efficiency when running on cloud infrastructures. Finally, we summarize our findings and discuss performance impacts when augmenting scientific workflows with ML techniques and how traditional workflow management systems can improve their support for such workflows.

Machine learning application in the oil and gas industry is rapidly becoming popular and in recent years has been applied in the optimization of production for various reservoirs. The objective of this paper is to evaluate the efficacy of advanced machine learning algorithms in reservoir production optimization. A 3-D geological model was constructed based on permeability calculated using a machine learning technique which involved different architectures of algorithms tested using a 5-fold cross-validation to decide the best machine learning algorithm. Sensitivity analysis and a subsequent history matching were conducted using a machine learning workflow. The aquifer properties, permeability heterogeneity in different directions and relative permeability were the control variables assessed. Field development scenarios were exploited with the objective to optimize cumulative oil recovery. The impact of using a normal depletion plan to a secondary recovery plan using waterflooding was investigated. Different injection well placement locations, well patterns as well as the possibility of converting existing oil producing wells to water injection wells were exploited. Considering the outcome of an economic analysis, the optimum development strategy was realized as an outcome for the optimization process. Prior to forecasting cumulative oil production using artificial neural network (ANN) for the optimization process on the generated surrogate model, a sensitivity analysis was performed where the well location, injection rates and bottomhole pressure of both the producer and injector wells were specified as control variables. The water cut as part of the optimization process was utilized as a secondary constraint. Forecasting was performed for a 15-year period. The history-matching results from the constructed geological model showed that the oil rate, water rate, bottom hole pressure, and average reservoir pressure were matched within a 10% deviation from the observed data. In this study, the ANN optimizer was found to provide the best results for the field cumulative oil production. Using a secondary recovery development plan was observed to significantly increase the cumulative oil production. A machine learning based proxy model was built for the prediction of cumulative oil production to reduce computational time. In this study, we propose an approach applied to reservoir production optimization utilizing a machine learning workflow. This was accomplished by utilizing a surrogate model which was calibrated with a number of training simulations and then optimized using advanced machine learning algorithms. A detailed economic analysis was also conducted showing the impact of a variety of field development strategies.

Enabled by a rich ecosystem of Machine Learning (ML) libraries, programming using learned models , i.e., Software-2.0 , has gained substantial adoption. However, we do not know what challenges developers encounter when they use ML libraries. With this knowledge gap, researchers miss opportunities to contribute to new research directions, tool builders do not invest resources where automation is most needed, library designers cannot make informed decisions when releasing ML library versions, and developers fail to use common practices when using ML libraries. We present the first large-scale quantitative and qualitative empirical study to shed light on how developers in Software-2.0 use ML libraries, and how this evolution affects their code. Particularly, using static analysis we perform a longitudinal study of 3,340 top-rated open-source projects with 46,110 contributors. To further understand the challenges of ML library evolution, we survey 109 developers who introduce and evolve ML libraries. Using this rich dataset we reveal several novel findings. Among others, we found an increasing trend of using ML libraries: The ratio of new Python projects that use ML libraries increased from 2% in 2013 to 50% in 2018. We identify several usage patterns including the following: (i) 36% of the projects use multiple ML libraries to implement various stages of the ML workflows, (ii) developers update ML libraries more often than the traditional libraries , (iii) strict upgrades are the most popular for ML libraries among other update kinds, (iv) ML library updates often result in cascading library updates, and (v) ML libraries are often downgraded (22.04% of cases). We also observed unique challenges when evolving and maintaining Software-2.0 such as (i) binary incompatibility of trained ML models and (ii) benchmarking ML models. Finally, we present actionable implications of our findings for researchers, tool builders, developers, educators, library vendors, and hardware vendors.

To broaden the accessibility of cell and gene therapies, it is essential to develop and optimize nonviral, cell type-preferential gene carriers such as lipid nanoparticles (LNPs). While high-throughput screening (HTS) approaches have proven effective in accelerating LNP discovery, they are often costly, labor-intensive, and do not consistently yield actionable design rules that direct screening efforts toward the most relevant chemical and formulation parameters. In this study, we employed a machine learning (ML) workflow, utilizing well-curated plasmid DNA LNP transfection data sets across six cell types, to extract compositional and chemical insights from HTS studies. Our approach achieved prediction errors averaging between 5 and 10%, depending on the cell type. By applying SHapley Additive exPlanations to our ML models, we uncovered key composition-function relationships that govern cell type-preferential LNP transfection efficiency. Notably, we identified consistent LNP composition parameters that enhance in vitro transfection efficiency across diverse cell types, including a helper lipid molar percentage of charged lipids between 9 and 50% and the inclusion of cationic/zwitterionic helper lipids. Additionally, several parameters were found to modulate cell type-preferentiality, such as the total molar percentage of ionizable and helper lipids, N/P ratio, PEGylated lipid molar percentage of uncharged lipids, and hydrophobicity of the helper lipid. This study leverages HTS of compositionally diverse LNP libraries combined with ML analysis to elucidate the interactions between lipid components in LNP formulations, providing insights that contribute to the design of LNP compositions tailored for cell type-preferential transfection.

Organic radical batteries (ORBs) based on the TEMPO (2,2,6,6-tetramethylpiperidin-1-yl oxyl) radical have drawn significant attention, owing to their unique redox properties. A key factor influencing ORB’s redox properties, i.e., the kinetics of the electron transfer between the TEMPO–TEMPO+ pairs, is the communication between the underlying redox-active states as given by the electronic coupling. However, due to the complex structure, predicting accurate electronic couplings for these pairs is computationally expensive and challenging. In this study, we introduce a machine learning (ML) workflow to predict the electronic coupling for TEMPO–TEMPO+ pairs simply by their specific geometric orientations. For the ML models, a data set was generated through time-dependent density functional theory calculations coupled with the Generalized Mulliken Hush method to assess energies, (transition-)dipole moment, and couplings for specific TEMPO–TEMPO+ configurations obtained from classical molecular dynamics simulations that mimic a realistic electrolyte environment. Our results demonstrate that, among the three ML models─linear regression, kernel ridge regression (KRR), and random forest─the KRR model, with its kernel-based approach, most effectively handles the correlated orientation-based descriptors. Moreover, our SHapley Additive exPlanations (SHAP)-based feature importance analysis indicates that multiple orientation factors jointly influence electronic coupling, rather than any single distance or angle dominating, with each parameter’s impact strongly contingent on the values of the others which is in agreement with previous studies computational by the consortium.

19 - Innovative energy solutions: Machine learning-driven lithium-ion battery modeling in electric vehicles

This paper describes how a system that performs Live Learning can be modified to use radar rather than visible light. The pipelined and iterative machine learning (ML) workflow of this system can operate at low network bandwidths for selective transmission of rare unlabeled events embedded in high-bandwidth real-time sensor data. While radar offers greater range and can overcome the severe signal attenuation experienced by visible light under conditions such as rain or fog, it poses a number of challenges for ML. This paper describes how these challenges can be overcome for Live Learning, and identifies some future directions for research.

Alterations in microbiome composition have been linked to many human diseases, including colorectal cancer or precancerous liver diseases. Findings from microbiome-association studies are increasingly explored as promising avenues for clinical applications due to their diagnostic or therapeutic potential. However, microbiome data complexity and the lack of integrated computational tools make it difficult to arrive at robust associations and predictive disease models. Here, we present the SIAMCAT R package as a modular and user-friendly toolbox for machine learning workflows, statistical analysis, and confounder detection for case-control microbiome datasets. Previously, we conducted a machine learning meta-analysis of colorectal cancer metagenomics studies (Wirbel, Pyl et al., Nat Med 2019) using the functionalities of SIAMCAT, which is now available to the wider community via Bioconductor. We showcase how SIAMCAT is a versatile tool by applying it to a large set of metagenomic datasets that have been processed with a wide variety of taxonomic and functional profiling tools. We furthermore demonstrate how SIAMCAT can help to detect confounding factors in microbiome association studies. By making stringent machine learning workflows and analysis pipelines available through a user-friendly and flexible interface, SIAMCAT will facilitate microbiome data analyses, biomarker discovery, and thus the translation of microbiome research to clinical applications, while simultaneously improving statistical rigor and safeguarding against common machine learning pitfalls. Citation Format: Jakob Wirbel, Konrad Zych, Morgan Essex, Nicolai Karcher, Ece Kartal, Guillem Salazar, Peer Bork, Shinichi Sunagawa, Georg Zeller. The SIAMCAT R package enables statistical and machine learning analyses for case-control microbiome datasets [abstract]. In: Proceedings of the AACR Special Conference on the Microbiome, Viruses, and Cancer; 2020 Feb 21-24; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2020;80(8 Suppl):Abstract nr A40.

Seismic subsurface characterization involves interpreting seismic records to delineate geological features and estimate the physical properties of subsurface rocks (such as lithofacies, porosity, and fluid saturation). As an effective remote sensing technology to characterize the large‐scale spatial distribution of subsurface properties, seismic surveys play a crucial role in many fields of Earth, Energy, and Environmental Sciences, including hydrocarbon exploration and development, geological sequestration of CO 2 , underground water management, and geothermal energy exploitation. With the growing need for characterizing geological heterogeneity, the application of machine learning (ML)—capable of nonlinear mapping and feature extraction—has become essential for effective seismic subsurface characterization. To clarify the current state of ML‐based subsurface characterization and promote its application to complex geological formations, we review conventional and machine learning workflows, along with the challenges they face. Subsequently, we delve into ML and deep learning concepts, summarizing recent advances and case studies in seismic subsurface characterization using supervised, semi‐supervised, unsupervised, and transfer learning approaches. Given the non‐unique nature of subsurface inference, we emphasize the importance of physical and geological consistency and of uncertainty quantification, and discuss three actively researched topics: deep generative geomodeling, physics‐guided (or geology‐guided) ML, and uncertainty quantification. Other ML‐based characterization approaches, including large language models, multi‐task learning, multi‐source and multi‐modal data fusion, etc., are further discussed. We also compile openly available data sets and codes, emphasizing their importance for future research. In summary, our objective is to provide the geosciences community with an overview of current research and future perspectives in ML‐based subsurface characterization.

Machine learning (ML) has become an increasingly popular tool for use in neurosurgical research. The number of publications and interest in the field have recently seen significant expansion in both quantity and complexity. However, this also places a commensurate burden on the general neurosurgical readership to appraise this literature and decide if these algorithms can be effectively translated into practice. To this end, the authors sought to review the burgeoning neurosurgical ML literature and to develop a checklist to help readers critically review and digest this work. The authors performed a literature search of recent ML papers in the PubMed database with the terms "neurosurgery" AND "machine learning," with additional modifiers "trauma," "cancer," "pediatric," and "spine" also used to ensure a diverse selection of relevant papers within the field. Papers were reviewed for their ML methodology, including the formulation of the clinical problem, data acquisition, data preprocessing, model development, model validation, model performance, and model deployment. The resulting checklist consists of 14 key questions for critically appraising ML models and development techniques; these are organized according to their timing along the standard ML workflow. In addition, the authors provide an overview of the ML development process, as well as a review of key terms, models, and concepts referenced in the literature. ML is poised to become an increasingly important part of neurosurgical research and clinical care. The authors hope that dissemination of education on ML techniques will help neurosurgeons to critically review new research better and more effectively integrate this technology into their practices.

Unconventional reservoirs, mainly shale oil and natural gas, will continue to significantly help meet the ever-growing energy demands of global markets. Being complex in nature and having ultra-tight producing zones, unconventionals depends on effective well completion and stimulation treatments in order to be successful and economical. Within the last decade, thousands of unconventional wells have been drilled, completed and produced in North America. The scope of this work is exploring the primary impact of completion parameters such as lateral length, frac type, number of stages, proppant and fluid volume effect on the production performance of the wells in unconventional fields. The key attributes in completion, stimulation, and production for the wells were considered in machine learning workflow for building predictive models. Predictive models based on Neural Networks, Support Vector Machines or Decision Tree Based ensemble models, serves as mapping function from completion parameters to production in each well in the field. The completion parameters were analyzed in the workflow with respect to feature engineering and interpretation. This analysis resulted in key performance indicators for the region. Then the optimum values for the best production performing completions were identified for each well. Predictive models in the workflow were analyzed in accuracy and best model is used to understand the impact of completion parameters on the production rates. This study outlines an overall machine learning workflow, from feature engineering to interpretation of the machine learning models to quantify the effects of completion parameters on the production rate of the wells in unconventional fields

This Editorial is intended for materials scientists interested in performing machine learning-centered research.We cover broad guidelines and best practices regarding the obtaining and treatment of data, feature engineering, model training, validation, evaluation and comparison, popular repositories for materials data and benchmarking datasets, model and architecture sharing, and finally publication.In addition, we include interactive Jupyter notebooks with example Python code to demonstrate some of the concepts, workflows, and best practices discussed.Overall, the data-driven methods and machine learning workflows and considerations are presented in a simple way, allowing interested readers to more intelligently guide their machine learning research using the suggested references, best practices, and their own materials domain expertise.

Homomorphic encryption (HE) revolutionizes secure data processing by allowing complex computational operations to be performed on encrypted data, preserving data privacy throughout machine learning workflows especially in cloud computing environments. However, the non-polynomial nature of activation functions like sigmoid poses challenges for HE frameworks, which are limited to basic arithmetic operations. This study explores Hermite polynomial and cubic spline interpolations as efficient methods to approximate the sigmoid function, balancing computational efficiency with approximation accuracy. The techniques are evaluated in both plaintext and encrypted environments. Experiments conducted on neural networks with the Fashion-MNIST and PneumoniaMNIST datasets demonstrate that clamped cubic splines with Chebyshev nodes and mid-degree Hermite polynomials achieve near-perfect approximations while maintaining competitive classification accuracy. Furthermore, an encrypted logistic regression model using Hermite polynomials demonstrates comparable performance to its plaintext counterpart, with only a 0.30% accuracy difference. These findings highlight the feasibility of polynomial approximations for enabling secure, privacy-preserving machine learning, paving the way for more efficient machine learning computations under encryption constraints.

The success of machine learning (ML) techniques implemented in different industries heavily rely on operator expertise and domain knowledge, which is used in manually choosing an algorithm and setting up the specific algorithm parameters for a problem. Due to the manual nature of model selection and parameter tuning, it is impossible to quantify or evaluate the quality of this manual process, which in turn limits the ability to perform comparison studies between different algorithms. In this study, we propose a new hybrid approach for developing machine learning workflows to help automated algorithm selection and hyperparameter optimization. The proposed approach provides a robust, reproducible, and unbiased workflow that can be quantified and validated using different scoring metrics. We have used the most common workflows implemented in the application of artificial intelligence (AI) and ML in engineering problems including grid/random search, Bayesian search and optimization, genetic programming, and compared that with our new hybrid approach that includes the integration of Tree-based Pipeline Optimization Tool (TPOT) and Bayesian optimization. The performance of each workflow is quantified using different scoring metrics such as Pearson correlation (i.e., R2 correlation) and Mean Square Error (i.e., MSE). For this purpose, actual field data obtained from 1567 gas wells in Marcellus Shale, with 121 features from reservoir, drilling, completion, stimulation, and operation is tested using different proposed workflows. A proposed new hybrid workflow is then used to evaluate the type well used for evaluation of Marcellus shale gas production. In conclusion, our automated hybrid approach showed significant improvement in comparison to other proposed workflows using both scoring matrices. The new hybrid approach provides a practical tool that supports the automated model and hyperparameter selection, which is tested using real field data that can be implemented in solving different engineering problems using artificial intelligence and machine learning. The new hybrid model is tested in a real field and compared with conventional type wells developed by field engineers. It is found that the type well of the field is very close to P50 predictions of the field, which shows great success in the completion design of the field performed by field engineers. It also shows that the field average production could have been improved by 8% if shorter cluster spacing and higher proppant loading per cluster were used during the frac jobs.

Traditional workflows in chemical engineering, from design to characterization, have served the field well. Emerging technologies, such as machine learning, now offer opportunities for faster design iterations and better integration of the various development stages. This study demonstrates how machine learning (ML) can reduce this gap by directly integrating experimental data into predictive models. Focusing on a Additive Manufactured helical micro-distillation column design for separating liquid mixtures, several ML models were trained to predict the number of theoretical stages based on geometric parameters and operating conditions. 11 variants were designed, built, and tested to collect 197 experimental data sets to assess the feasibility of five predictive ML models from experimental data. Among the algorithms tested, the Gradient Boosting Regressor achieved the best performance with a coefficient of determination R 2 = 0 . 7934 . The work highlights the behavior of the model across different regimes, identifies key sources of error, and proposes a hybrid experimental–ML workflow for rapid screening of distillation designs. This approach accelerates process development and reduces the need for extensive experimentation, especially in time-consuming tasks such as distillation. • ML predicts KPIs for complex AM helical distillation columns using real data. • Multiple supervised ML models benchmarked for AM-induced geometric effects. • Multi-output modeling enhances stability and captures physical correlations. • Hybrid ML–experimental workflow identifies high-value regions for testing. • Framework accelerates design and screening of AM-enabled separation units.