Secure ML Workflows Using Kubernetes: A CKS-Certified Perspective

In the last 10 years, various automated machine learning (AutoML) systems have been proposed to build end-to-end machine learning (ML) pipelines with minimal human interaction. Even though such automatically synthesized ML pipelines are able to achieve competitive performance, recent studies have shown that users do not trust models constructed by AutoML due to missing transparency of AutoML systems and missing explanations for the constructed ML pipelines. In a requirements analysis study with 36 domain experts, data scientists, and AutoML researchers from different professions with vastly different expertise in ML, we collect detailed informational needs for AutoML. We propose XAutoML , an interactive visual analytics tool for explaining arbitrary AutoML optimization procedures and ML pipelines constructed by AutoML. XAutoML combines interactive visualizations with established techniques from explainable artificial intelligence (XAI) to make the complete AutoML procedure transparent and explainable. By integrating XAutoML with JupyterLab , experienced users can extend the visual analytics with ad-hoc visualizations based on information extracted from XAutoML . We validate our approach in a user study with the same diverse user group from the requirements analysis. All participants were able to extract useful information from XAutoML , leading to a significantly increased understanding of ML pipelines produced by AutoML and the AutoML optimization itself.

Scientific workflows are one of the well-established pillars of modern large-scale computational science. More recently, scientists have started to leverage machine learning (ML) capabilities in their workflows, leading to a new category of scientific workflows, denoted as scientific ML workflows. ML is not only about training and inference, modern ML workflows also involve complex data processing steps before the training can start, which are not often accounted for in most performance studies. In this work, we consider scientific ML workflows, from data pre-processing to training, inference, and model evaluation. We aim to explore (i) how scientific ML workflows differ from more traditional scientific workflows and; (ii) how we can characterize ML workflows both in terms of execution time and data movements when executing on a target cloud platform. We select three representative workflows, ranging from image classification to natural language processing and image segmentation, which have been executed using the academic cloud platform, Chameleon. We build four realistic deployment scenarios for each workflow, which stress data movements during workflow executions. Then, we compare the performance observed when utilizing these different configurations and study how different settings impact overall workflows performance and efficiency when running on cloud infrastructures. Finally, we summarize our findings and discuss performance impacts when augmenting scientific workflows with ML techniques and how traditional workflow management systems can improve their support for such workflows.

Machine learning based prediction of single-frequency viscoelastic brain white matter - A data science framework.

In recent years, a wide variety of automated machine learning (AutoML) methods have been proposed to generate end-to-end ML pipelines. While these techniques facilitate the creation of models, given their black-box nature, the complexity of the underlying algorithms, and the large number of pipelines they derive, they are difficult for developers to debug. It is also challenging for machine learning experts to select an AutoML system that is well suited for a given problem. In this paper, we present the Pipeline Profiler, an interactive visualization tool that allows the exploration and comparison of the solution space of machine learning (ML) pipelines produced by AutoML systems. PipelineProfiler is integrated with Jupyter Notebook and can be combined with common data science tools to enable a rich set of analyses of the ML pipelines, providing users a better understanding of the algorithms that generated them as well as insights into how they can be improved. We demonstrate the utility of our tool through use cases where PipelineProfiler is used to better understand and improve a real-world AutoML system. Furthermore, we validate our approach by presenting a detailed analysis of a think-aloud experiment with six data scientists who develop and evaluate AutoML tools.

Currently, there is no therapy targeting septic cardiomyopathy (SC), a key contributor to organ dysfunction in sepsis. In this study, we used a machine learning (ML) pipeline to explore transcriptomic, proteomic, and metabolomic data from patients with septic shock, and prospectively collected measurements of high-sensitive cardiac troponin and echocardiography. The purposes of the study were to suggest an exploratory methodology to identify and characterise the multiOMICs profile of (i) myocardial injury in patients with septic shock, and of (ii) cardiac dysfunction in patients with myocardial injury. The study included 27 adult patients admitted for septic shock. Peripheral blood samples for OMICS analysis and measurements of high-sensitive cardiac troponin T (hscTnT) were collected at two time points during the ICU stay. A ML-based study was designed and implemented to untangle the relations among the OMICS domains and the aforesaid biomarkers. The resulting ML pipeline consisted of two main experimental phases: recursive feature selection (FS) assessing the stability of biomarkers, and classification to characterise the multiOMICS profile of the target biomarkers. The application of a ML pipeline to circulate OMICS data in patients with septic shock has the potential to predict the risk of myocardial injury and the risk of cardiac dysfunction.

The emergence of the mobile applications that are driven by machine learning (ML) has brought a breakthrough in the world of healthcare, e-commerce, and entertainment with its personalized and smart user experiences. Despite the wide application of ML pipelines in mobile environments, the security and privacy issues that could be raised have become an issue of high concern on a global scale. The storage and processing of data, combined with the deployment of advanced ML models, expose vulnerabilities to risks such as data leakage, adversarial attacks, and unauthorized access. These threats not only compromise user trust but also incur applications with financial and brand damage. The present article deals with the important problems of the safety of mobile ML pipelines, namely adversarial manipulations, insecure storage, and transmitting of data. The vital techniques that can help to work out those problems are described along with the most popular ones, namely secure model storage based on encryption, federated learning for data transfer reduction, and model robustness tactics that will increase adversarial attacks defense. Besides that, new technologies like homomorphic encryption and blockchain are also discussed as ways to secure model updates. Consequently, this paper is involved in the technical and practical levels of the project, and it is meant to show the importance of building secure and privacy-preserving ML pipelines that do not only keep data for user trust but also maintain efficiency. It serves the purpose of opening the way to moving to secure mobile ML solutions in the time of intelligent mobile systems that have started to be of great interest.

Fatal accidents are a major issue hindering the wide acceptance of safety-critical systems that employ machine learning and deep learning models, such as automated driving vehicles. In order to use machine learning in a safety-critical system, it is necessary to demonstrate the safety and security of the system through engineering processes. However, thus far, no such widely accepted engineering concepts or frameworks have been established for these systems. The key to using a machine learning model in a deductively engineered system is decomposing the data-driven training of machine learning models into requirement, design, and verification, particularly for machine learning models used in safety-critical systems. Simultaneously, open problems and relevant technical fields are not organized in a manner that enables researchers to select a theme and work on it. In this study, we identify, classify, and explore the open problems in engineering (safety-critical) machine learning systems—that is, in terms of requirement, design, and verification of machine learning models and systems—as well as discuss related works and research directions, using automated driving vehicles as an example. Our results show that machine learning models are characterized by a lack of requirements specification, lack of design specification, lack of interpretability, and lack of robustness. We also perform a gap analysis on a conventional system quality standard SQuaRE with the characteristics of machine learning models to study quality models for machine learning systems. We find that a lack of requirements specification and lack of robustness have the greatest impact on conventional quality models.

In today’s digital landscape, where data is a critical asset for businesses, building a resilient backup infrastructure is paramount to ensuring data availability and protection. This research investigates the synergy between three essential elements—data redundancy, encryption, and Role-Based Access Control (RBAC) to enhance backup system resilience. Data redundancy, achieved through methods like the 3-2-1 rule and cloud replication, ensures continuous availability and protection against hardware failures. Encryption secures backup data, making it unreadable to unauthorized users and safeguarding sensitive information in compliance with regulatory standards. RBAC, on the other hand, provides granular control over user access, enforcing the principle of least privilege and minimizing the risk of unauthorized data manipulation. By combining these strategies, organizations can mitigate the risks of data loss, breaches, and unauthorized access, creating a robust, secure, and efficient backup infrastructure. This paper provides a comprehensive analysis of how these technologies can be integrated, offers best practices for implementation, and discusses the future of backup systems in an increasingly complex threat environment. Keywords: Backup Infrastructure, Data Redundancy, Data Encryption, Role-Based Access Control, RBAC, Data Security, Disaster Recovery, Data Protection, Cloud Backup, Data Availability, Security Best Practices, IT Resilience, Cybersecurity, Backup Strategies.

In this paper, we detail how damage mechanisms commonly found on marine terminal structures and process equipment can be automatically detected using a machine learning (ML) pipeline. Defects in its varied forms, such as corrosion, steel, wood and concrete damages must be closely monitored in offshore oil and gas platforms and also in marine terminals; the cost of not monitoring and detecting defects can range from unplanned shutdowns, plant damage, to human fatalities. On-site inspection of defects is labor intensive and require frequent travel to offshore and onshore facilities. This process has two shortcomings: firstly, it is costly involving man hours spent on the facility and secondly only a subset of the total defects is monitored during each inspection due to difficulty in obtaining full coverage; specially in this scenario a high-risk defect could be missed completely. We use marine terminals as an example of how defects can be detected using machine learning techniques, however the pipeline for generating the ML models could be extended to other types of defects and structures. Significantly, the output from our ML pipeline can be used as an input to the Risk Based Inspection (RBI) system assisting asset operators in their inspection and maintenance planning. Furthermore, digitized inspections and ML can provide around 95% coverage of the terminal while avoiding problems faced by a human inspector such as producing consistent results across inspections and sites. Our system captured terrestrial spherical imagery and range data (LIDAR), subsea imagery, sonar, and ultrasonic thickness (UT) gauge data using different sensors and equipment on a marine terminal. Equipment used in our data-capture included an unmanned aerial vehicle (UAV), a terrestrial scanner, a remotely operated vehicle for underwater imaging (ROV) and a remotely operated surface vehicle (ROSV). For the purposes of this work, we used the images captured by the terrestrial scanner. We built an automated defect detection pipeline using state of the art machine learning and artificial intelligence (AI) tools taking the cube face images as inputs. The defect detection pipeline was trained with examples of defects labeled by asset integrity engineers to ensure quality of results. We built the pipeline to detect damage mechanisms for steel and wooden structures, however our system could be scaled to incorporate other types of defects and structures. Our AI pipeline was able to achieve a high level of accuracy identifying wood damage with a mean-Average Precision (m-AP) of 0.80 and steel damage with a m-AP of 0.84, where a m-AP of 1.0 indicates perfect detection of damage.

A comparative study of machine learning methods for predicting the evolution of brain connectivity from a baseline timepoint

Enabled by a rich ecosystem of Machine Learning (ML) libraries, programming using learned models , i.e., Software-2.0 , has gained substantial adoption. However, we do not know what challenges developers encounter when they use ML libraries. With this knowledge gap, researchers miss opportunities to contribute to new research directions, tool builders do not invest resources where automation is most needed, library designers cannot make informed decisions when releasing ML library versions, and developers fail to use common practices when using ML libraries. We present the first large-scale quantitative and qualitative empirical study to shed light on how developers in Software-2.0 use ML libraries, and how this evolution affects their code. Particularly, using static analysis we perform a longitudinal study of 3,340 top-rated open-source projects with 46,110 contributors. To further understand the challenges of ML library evolution, we survey 109 developers who introduce and evolve ML libraries. Using this rich dataset we reveal several novel findings. Among others, we found an increasing trend of using ML libraries: The ratio of new Python projects that use ML libraries increased from 2% in 2013 to 50% in 2018. We identify several usage patterns including the following: (i) 36% of the projects use multiple ML libraries to implement various stages of the ML workflows, (ii) developers update ML libraries more often than the traditional libraries , (iii) strict upgrades are the most popular for ML libraries among other update kinds, (iv) ML library updates often result in cascading library updates, and (v) ML libraries are often downgraded (22.04% of cases). We also observed unique challenges when evolving and maintaining Software-2.0 such as (i) binary incompatibility of trained ML models and (ii) benchmarking ML models. Finally, we present actionable implications of our findings for researchers, tool builders, developers, educators, library vendors, and hardware vendors.

BackgroundMagnetic resonance imaging (MRI) can be used to monitor disease progression in Alzheimer's disease (AD). Portable, low‐field MRI (LF) facilitates point‐of‐care assessment and improves patient‐centered access. We developed a machine learning (ML) pipeline to use in conjunction with LF‐MRI to quantify brain morphometry and enable monitoring of AD patients in the outpatient clinic.MethodPatients with mild cognitive impairment (MCI) or dementia due to Alzheimer's disease (AD) were enrolled from the outpatient Massachusetts General Hospital (MGH) Memory Disorders Unit. A separate cohort of subjects with vascular comorbidities (VC) of comparable age were recruited from the Yale New Haven Hospital. LF‐MRI was acquired on a 0.064 T MRI (Hyperfine Inc), with conventional high‐field MRI (HF‐MRI; 1.5‐3 T) scans obtained within 7±11 months of LF‐MRI. T2 FLAIR sequences from HF and LF counterparts were processed through FreeSurfer‐based pipeline WMH‐SynthSeg for quantification of the following regions: whole brain, white matter, cortex, hippocampus, amygdala, putamen, pallidum, caudate, thalamus, accumbens, cerebellum, and ventricles.ResultThe study population included MCI (N = 31, 72±7 years), AD (N = 24, 72±9 years), and VC (N = 22, 64±8 years). Correlations between HF and LF derived brain volumes was high across all brain regions (all p <0.05). The highest correlations were observed in the cortex and white matter (r=0.91; 95% CI 0.86, 0.94), lateral ventricle (r=0.98; 95% CI 0.97, 0.99), 3rd ventricle (r=0.95; 95% CI 0.92, 0.97), hippocampus (r=0.80; 95% CI 0.70, 0.87), caudate (r=0.82; 95% CI 0.73, 0.88), and amygdala (r=0.89; 95% CI 0.83, 0.93). The MCI and AD cohorts showed regional atrophy relative to the VC cohort (all regions p <0.05) except for the caudate and 4th ventricle (p >0.05). Regions with the most significant atrophy included the cortex, hippocampus, and amygdala (all p <0.001).ConclusionLF‐MRI acquisition at the point‐of‐care for MCI or AD patients is feasible, and application of ML algorithms can generate brain volumes comparable to those derived from conventional counterparts, allowing for differentiation between VC and AD/MCI subgroups. Our findings demonstrate that LF‐MRI could be used as an accessible neuroimaging option for disease monitoring among those with cognitive impairment.

Recent technological advances allowed an exponential expansion of biological sequence data, and the extraction of meaningful information through Machine Learning (ML) algorithms. This knowledge improved the understanding of the mechanisms related to several fatal diseases, e.g., Cancer and COVID-19, helping to develop innovative solutions, such as CRISPR-based gene editing, coronavirus vaccine, and precision medicine. These advances benefit our society and economy, directly impacting peoples lives in various areas, such as health care, drug discovery, forensic analysis, and food analysis. Nevertheless, ML approaches applied to biological data require representative, quantitative, and informative features. Necessarily, as many ML algorithms can handle only numerical data, sequences need to be translated into a feature vector. This process, known as feature extraction, is a fundamental step for the elaboration of high-quality ML-based models in bioinformatics, by allowing the feature engineering stage, with the design and selection of suitable features. Feature engineering, ML algorithm selection, and hyperparameter tuning are often manual and time-consuming processes, requiring extensive domain knowledge, and performed manually by a human expert. To deal with this problem, we developed a new package, BioAutoML, which automatically runs an end-to-end ML pipeline. BioAutoML extracts numerical and informative features from biological sequence databases, automating feature selection, recommendation of ML algorithm(s), and tuning of hyperparameters, using Automated ML (AutoML). BioAutoML has two components, divided into four modules, (1) automated feature engineering (feature extraction and selection modules) and (2) Metalearning (algorithm recommendation and hyperparameter tuning modules). Our experimental results, assessing the relevance of our proposal, indicate robust results for different problem domains, such as SARS-CoV-2, anticancer peptides, HIV sequences, and non-coding RNAs. According to our systematic review, our proposal is innovative compared to available studies in the literature, being the first study to propose automated feature engineering and metalearning for biological sequences. BioAutoML has a high potential to significantly reduce the expertise required to use ML pipelines, aiding researchers in combating diseases, particularly in low- and middle-income countries. This initiative can provide biologists, physicians, epidemiologists, and other stakeholders with an opportunity for widespread use of these techniques to enhance the health and well-being of their communities.

Machine Learning (ML) is increasingly being used as an essential component of modern software systems. Also, the maturity of the adopted techniques and the availability of frameworks have changed the way developers approach ML-related development problems. This paper aims at investigating, by analyzing Stack Overflow (SO) posts related to ML, how the questions about ML have been changing over the years, and across six different programming languages. We analyzed 43,950 SO posts in the period 2008-2020, studying (i) how the number of ML-related posts changes over time for each programming language, (ii) how the posts are distributed across different phases of a ML pipeline, and (iii) whether posts belonging to different languages or phases are more or less challenging to address. We found that some programming languages are fading while others are becoming more popular in ML development. While model-building questions are the most discussed in general, the level of challenges posed by the other phases of the ML pipeline appears to be language-dependent. Results of this work could be used to better understand ML challenges in different programming languages, and, possibly, to improve ML tutorials related to different languages.

On taking advantage of opportunistic meta-knowledge to reduce configuration spaces for automated machine learning