Big Data and Java are integrated with machine learning

BackgroundThe exponential growth of publications regarding the application of machine learning (ML) tools in medicine highlights the significant potential for ML to revolutionize the field. Despite the multitude of literature surrounding this topic, there are limited publications addressing the implementation and feasibility of ML models in clinical practice. Currently, Machine Learning Operations (MLOps), a set of practices designed to deploy and maintain ML models in production, is used in various information technology and industrial settings. However, the MLOps pipeline is not well researched in medical settings, where multiple barriers exist to implementing ML pipelines into practice.ObjectiveThis study aims to detail how MLOps is implemented in health care and propose a maturity framework for the health care implementations.MethodsA scoping review search was conducted according to the Joanna Briggs Institute Manual for Evidence Synthesis. Results were synthesized using the 3-stage basic qualitative content analysis. We searched 4 databases (eg, MEDLINE, Embase, Web of Science, and Scopus) to include any studies that involved proof of concept or real-world implementation of MLOps in health care. Studies not reported in English were excluded.ResultsA total of 19 studies were included in this scoping review. The MLOps workflow identified within the studies included (1) data extraction (19/19 studies), (2) data preparation and engineering (18/19 studies), (3) model training (19/19 studies), (4) measured ML metrics and model evaluation (17/19 studies), (5) model validation and test in production (14/19 studies), (6) model serving and deployment (15/19 studies), (7) continuous monitoring (14/19 studies), and (8) continual learning (13/19 studies). We proposed a 3-stage MLOps maturity framework for health care based on existing studies in the field, that is, low (5/19 studies), partial (1/19 studies), and full maturity (13/19 studies). There were 8/19 studies that discussed ethical, legislative, and stakeholder considerations for MLOps implementations in health care settings.ConclusionsWe investigated the implementation of MLOps in health care with a corresponding maturity framework. It is evident that only a limited number of studies reported the implementation of ML in health care contexts. Hence, it is imperative that we shift our focus toward creating an environment that supports the development of ML health care applications, such as improving existing data infrastructure, and engaging partners to support the development of MLOps applications. Specifically, we can include patients, policymakers, and health care professionals in the creation and implementation of ML applications. One of the main limitations includes the varying quality of each extracted study in terms of how the MLOps implementation was presented. Hence, it was difficult to verify the presence and discuss in depth all steps of the MLOps workflow for each study. Furthermore, due to the inherent nature of a scoping review protocol, there may be a compromise on an in-depth discussion of each step within the MLOps workflow.

Aim: The majority of machine learning (ML) models in healthcare are built on retrospective data, much of which is collected without explicit patient consent for use in artificial intelligence (AI) and ML applications. The primary aim of this study was to evaluate whether clinicians and scientific researchers themselves consent to provide their own data for the training of ML models. Materials and Methods: This survey was conducted through an anonymous online survey, utilizing platforms such as Telegram, LinkedIn, and Viber. The target audience comprised specific international groups, primarily Russian, German, and English-speaking, of clinicians and scientific researchers. These participants ranged in their levels of expertise and experience, from beginners to veterans. The survey centered on a singular, pivotal question: “Do You Consent to the Use of Your Biological and Private Data for Training Machine Learning and AI Models?” Respondents had the option to choose from three responses: “Yes” and “No”. Results: The survey was conducted in January 2024. A total of 119 unique and verified individuals participated in the survey. The results revealed that only 50% of respondents (63 persons) expressed consent to provide their own data for the training of ML and AI models. Conclusion: In the development of ML and AI models, particularly open-source ones, it is crucial to ascertain whether participants are willing to provide their private data. While ML algorithms can transform the nature of data, it is important to remember that the primary owner of this data is the individual. Our findings show that in 50% of the cases, even participants from scientific research and clinical backgrounds – individuals typically accountable for ensuring data quality in AI and ML model development – do not consent to the use of their data in AI and ML settings. This highlights the need for more stringent consent processes and ethical considerations in the utilization of personal data in AI and ML research.

The sheer boom in data creation from multiple sources beginning from IoT sensors to social media has changed the technology landscape, rendering predictive analytics in big data environments more important than ever before. This work suggests a hybrid machine learning model for improving predictive accuracy, computational performance, and scalability in big data environments. The Model employs Random Forest (RF) for strong feature selection and Gradient Boosted Decision Trees (GBDT) for enhanced classification accuracy, both optimised using Apache Spark's MLlib in a distributed setting. The Model takes advantage of hybridisation to beat the single-algorithm model drawback in accommodating high-dimensional, heterogeneous data. The architecture is designed to handle data in real time and includes a dynamic pre-processing layer, parallel training pipeline, and continuous evaluation modules. Performance metrics like accuracy, precision, recall, F1 score, and AUC are used to validate performance against benchmark data sets. System-level metrics like latency, Throughput, and scalability are also monitored to validate usability for real-world deployments. Experiments using healthcare and e-commerce industry datasets yield better prediction power and operational effectiveness than single ML models. Visualisation using scatter plots and 3D graphs shows sharp jumps in model accuracy and processing time for different volumes of data. It is a scalable, adaptable, and precise predictive analytics tool for the big data era. Ease of use of the solution with Apache Hadoop and Spark deployment is a willingness to be utilised at the enterprise level.

Study question Can a machine learning (ML) model, developed using modern neural network architecture produce comparable annotation data; utilisable for algorithmic outcome prediction, to manual time-lapse annotations? Summary answer The model automatically annotated unseen embryos with comparable results to manual methods, generating morphokinetic data to enable comparably predictive outputs from an embryo selection algorithm. What is known already The application of artificial intelligence across healthcare industries, including fertility, is increasing. Several ML models are available that seek to generate or analyse embryo images and morphokinetic data, and to determine embryo viability potential. Along with photographic images, the use of time-lapse in IVF laboratories has amassed numeric data, resulting predominantly from annotated manual assessment of images over time. Embryo annotation practice is variable in quality, can be subjective and is time-consuming; commonly taking several minutes per embryo. The development of rapid, accurate automatic annotation would represent a significant time-saving as well as an increase in reproducibility and accuracy. Study design, size, duration Multicentre quality assured annotation data from 63,383 time-lapse monitored embryos (EmbryoScope®), comprising over 400 million individual images, were used to train a ML model to automatically generate morphokinetic annotations. Data was derived from 8 UK clinics within a cohesive group between 2012-2021. Accuracy was assessed using 900 unseen embryos (with live birth outcome) by comparing the output of an established in-house, prospectively validated embryo selection model when the input was either ML-automated, or manual annotations. Participants/materials, setting, methods Multi-focal plane images were processed on the Azure cloud (Microsoft) and resampled to 300x300 pixels. A Laplacian-based focal stacking algorithm merged frames into a single image. The model consisted of an EfficientNetB4 Convolutional Neural Network classifier to extract features and classify the stage of embryo images. A Temporal Convolutional Network interpreted a time-series of image features; producing annotations from pronuclear fading through to blastocyst. Soft localisation loss function used QA data to integrate annotation subjectivities. Main results and the role of chance The ML model rapidly and automatically generated annotations. Efficacy and comparability of the ML model to automate reliable, utilisable annotations was demonstrated by comparison with manual annotation data and the ML model’s ability to auto-generate annotations which could be used to predict live birth by providing annotation data to an established, validated in house embryo selection model. Live birth-predictive capability was measured, and benchmarked against manual annotation, using the area under the receiver operating characteristic curve (AUC). When tested on time-lapse images, collected from pronuclear fading to full blastulation, representing 900 previously unseen, transferred blastocysts where live birth outcomes were blinded, the in-house developed auto-annotation ML model resulted in an AUC of 0.686 compared with 0.661 for manual annotations, for live birth prediction. Auto annotation using the developed model took only milliseconds to complete per embryo. The developed auto-annotation model, built and tested on large data, is considered suitable for productionisation with the aim of being validated and integrated into an application to support IVF laboratory practice. Limitations, reasons for caution Whilst this model was trained to recognise key morphokinetic events, there are other morphokinetic variables that may be useful in the prediction of live birth and further improve embryo selection, or deselection, ability. Akin to manual interpretation, some embryos may fail to be annotated or need second opinion. Wider implications of the findings There is increasing evidence supporting the application of ML to utilise big data from time-lapse imaging and fertility care generally. Whilst promising benefits to IVF clinics and patients, responsible use of data is required alongside large high-quality datasets, and rigorous validation, to ensure safe and robust applications. Trial registration number N/A

Artificial intelligence (AI) is reshaping neurosurgery, offering unprecedented opportunities to enhance diagnostics, personalize treatment, and predict outcomes. At the heart of this transformation is the ability to effectively harness big data (BD) within the electronic medical record. Understanding these data structures is essential for making sense of the vast volumes of information generated in modern neurosurgical practice. Equally important are the machine learning (ML) models driving these advancements. From supervised learning and convolutional neural networks to generative AI, these tools are already making a mark in areas such as brain tumor segmentation and spine surgery outcome predictions. Their versatility highlights the potential of ML to complement clinical expertise and streamline decision-making in neurosurgery. However, adopting BD and ML also brings ethical challenges that cannot be ignored. Bias in algorithms threatens to reinforce health disparities, whereas concerns about data privacy demand vigilance in handling sensitive patient information. In addition, the question of liability looms large as ML increasingly influences clinical decisions. The aim of the study was to provide a roadmap for neurosurgeons navigating the evolving intersection of BD, ML, and ethical responsibility in the AI era.

The medical imaging community has embraced Machine Learning (ML) as evidenced by the rapid increase in the number of ML models being developed, but validating and deploying these models in the clinic remains a challenge. The engineering involved in integrating and assessing the efficacy of ML models within the clinical workflow is complex. This paper presents a general-purpose, end-to-end, clinically integrated ML model deployment and validation system implemented at UCSF. Engineering and usability challenges and results from 3 use cases are presented. A generalized validation system based on free, open-source software (OSS) was implemented, connecting clinical imaging modalities, the Picture Archiving and Communication System (PACS), and an ML inference server. ML pipelines were implemented in NVIDIA’s Clara Deploy framework with results and clinician feedback stored in a customized XNAT instance, separate from the clinical record but linked from within PACS. Prospective clinical validation studies of 3 ML models were conducted, with data routed from multiple clinical imaging modalities and PACS. Completed validation studies provided expert clinical feedback on model performance and usability, plus system reliability and performance metrics. Clinical validation of ML models entails assessing model performance, impact on clinical infrastructure, robustness, and usability. Study results must be easily accessible to participating clinicians but remain outside the clinical record. Building a system that generalizes and scales across multiple ML models takes the concerted effort of software engineers, clinicians, data scientists, and system administrators, and benefits from the use of modular OSS. The present work provides a template for institutions looking to translate and clinically validate ML models in the clinic, together with required resources and expected challenges.

Machine learning (ML) models provide more choices to patients with diabetes mellitus (DM) to more properly manage blood glucose (BG) levels. However, because of numerous types of ML algorithms, choosing an appropriate model is vitally important. In a systematic review and network meta-analysis, this study aimed to comprehensively assess the performance of ML models in predicting BG levels. In addition, we assessed ML models used to detect and predict adverse BG (hypoglycemia) events by calculating pooled estimates of sensitivity and specificity. PubMed, Embase, Web of Science, and Institute of Electrical and Electronics Engineers Explore databases were systematically searched for studies on predicting BG levels and predicting or detecting adverse BG events using ML models, from inception to November 2022. Studies that assessed the performance of different ML models in predicting or detecting BG levels or adverse BG events of patients with DM were included. Studies with no derivation or performance metrics of ML models were excluded. The Quality Assessment of Diagnostic Accuracy Studies tool was applied to assess the quality of included studies. Primary outcomes were the relative ranking of ML models for predicting BG levels in different prediction horizons (PHs) and pooled estimates of the sensitivity and specificity of ML models in detecting or predicting adverse BG events. In total, 46 eligible studies were included for meta-analysis. Regarding ML models for predicting BG levels, the means of the absolute root mean square error (RMSE) in a PH of 15, 30, 45, and 60 minutes were 18.88 (SD 19.71), 21.40 (SD 12.56), 21.27 (SD 5.17), and 30.01 (SD 7.23) mg/dL, respectively. The neural network model (NNM) showed the highest relative performance in different PHs. Furthermore, the pooled estimates of the positive likelihood ratio and the negative likelihood ratio of ML models were 8.3 (95% CI 5.7-12.0) and 0.31 (95% CI 0.22-0.44), respectively, for predicting hypoglycemia and 2.4 (95% CI 1.6-3.7) and 0.37 (95% CI 0.29-0.46), respectively, for detecting hypoglycemia. Statistically significant high heterogeneity was detected in all subgroups, with different sources of heterogeneity. For predicting precise BG levels, the RMSE increases with a rise in the PH, and the NNM shows the highest relative performance among all the ML models. Meanwhile, current ML models have sufficient ability to predict adverse BG events, while their ability to detect adverse BG events needs to be enhanced. PROSPERO CRD42022375250; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=375250.

In recent years, infections and damage caused by malware have increased at exponential rates. At the same time, machine learning (ML) techniques have shown tremendous promise in many domains, often out performing human efforts by learning from large amounts of data. Results in the open literature suggest that ML is able to provide similar results for malware detection, achieving greater than 99% classification accuracy [49]. However, the same detection rates when applied in deployed settings have not been achieved. Malware is distinct from many other domains in which ML has shown success in that (1) it purposefully tries to hide, leading to noisy labels and (2) often its behavior is similar to benign software only differing in intent, among other complicating factors. This report details the reasons for the difficultly of detecting novel malware by ML methods and offers solutions to improve the detection of novel malware. We propose to detect malware by detecting behaviors commonly exhibited by malware such as DLL injection, and process hollowing. This is based on the assumption that there is a set of behaviors that are common to most malware samples and detecting them will generalize to novel malware. Additionally, detected behaviors point analysts toward appropriate handling and mitigation strategies, which is not the case with a binary benign/malicious classification. A behavior labeling method was developed and was used to label an existing malware dataset. Results show that detecting malicious behaviors is much more difficult than simply classifying malware and goodware?achieving 80% accuracy compared to reported 99% accuracy from classifying malware and goodware. This drop is due to several reasons which are detailed in the report. We also propose to evaluate the performance of detecting novel malware by holding out a malware family for testing and training on the other families. Traditional ML evaluation will shuffle the data and then split the data into training and testing. Our method addresses the use-case when novel malware families are encountered and they require more than just a malicious or benign designation. Our results suggest that this type of evaluation is much more difficult than traditional methods and provides more realistic results, albeit, significantly worse. For our behavior detection, accuracy decreases from 80% to 68% across all behaviors when holding out a malware family from training. We show that the degradation in performance is because each malware family has distinct characteristics resulting in high extrapolations by an ML model. Here, an ML model should return an "I do not know" response and request further analysis from an analyst. We run a number of experiments that compare novel malware families to the training data using different feature representations including a genomics-inspired distance measure and features extracted by deep learning. Generally, held-out families are significantly different from the training data, resulting in unpredictable results. This has been observed generally in the ML community [22, 9]. We empirically demonstrate this in the domain of malware detection. In an attempt to improve the detection of malware behaviors, we examine the impact that additional synthetic data has on the performance of an ML model in detecting behaviors in novel malware families. We find that while synthetic data does improve the performance of ML models, often simpler methods perform better than more complicated ones. Two generative modeling techniques were examined to produce synthetic malware samples such that the behaviors present are able to be specified externally. The difficulty is due to finer grained analysis of the executable and modifying the problem from a binary classification problem to a multi-label problem. The addition of synthetic data increases the overall accuracy from 68% to 70%. While far less accurate than measures presented in academic analyses, we believe that this is more representative of real-world performance and allows models to be properly placed within a malware detection system. We suggest that in highly dynamic environments ML pipelines should determine whether an ML model is competent in the area of new data and should involve mechanisms to improve over time with a human in the loop.

Machine learning (ML) models are used to mine inconspicuous information in big data. The model and data quality influence the performance of a ML model. However, modifying the ML model while measuring performance is impractical, and low-quality data causes biased model training. Therefore, improving the data quality is essential. Visual analytics systems supporting DQI (Data Quality Improvement) have been proposed in the past. However, in the studies, it is difficult for users to assess comprehensive data quality improvement methods for machine learning and to determine an appropriate data quality improvement process. In this paper, we propose a novel visual analytics system for managing data quality used in machine learning models.

Optimizing operational decisions, routine actions within some business or operational process, is a key challenge across a variety of domains and application areas. The increasing availability of data, computational power, and advanced machine learning (ML) algorithms offers exciting opportunities for data-driven decision support. To advance the potential of ML for optimizing operational decision-making, we explore two research directions, aiming to develop ML models that are decision-focused and causal. This dissertation presents several developments in machine learning in these two areas. ML is effective at making predictions from historical data: for example, estimating a transaction's fraud probability by comparing it to past cases. However, decision-makers not only need to consider these predictions, but also the operational context. For example, the decision-maker uses predicted fraud probabilities to determine which transactions to investigate, while aiming to minimize monetary losses due to fraud and considering the available capacity of the fraud investigations team. Predictions can help reduce uncertainty (e.g., by predicting the fraud probability), but standard ML models are prediction-focused, instead of decision-focused. This distinction involves two challenges for data-driven decision-making. First, prediction-focused models prioritize predictive accuracy instead of the resulting decision quality (e.g., fraud losses recovered by the bank). Second, these models fail to account for operational constraints, such as the available investigation capacity. Decision-focused learning aims to improve data-driven decision-making by addressing these issues and incorporating the operational context into the optimization of ML models. In this dissertation, we analyze cost-sensitive learning within this prediction-optimization framework and evaluate general strategies for making cost-optimal decisions with ML. Additionally, we propose a novel ML method for optimal decision-making under capacity constraints based on learning to rank. To make effective decisions, a decision-maker has to estimate the causal effect of possible interventions in order to choose actions that achieve the desired outcome. Unfortunately, standard ML models identify correlations in the data instead of causal relationships. Because of this, these models cannot guarantee the effectiveness of decisions made based on their predictions. Causal inference provides a formal framework for reasoning about causality and identifying causal effects from data. This dissertation explores the intersection of causality and ML. First, we illustrate the potential of causal ML for optimizing preventive maintenance. Next, we propose novel causal ML methods for predicting causal effect distributions and for addressing informative sampling when predicting treatment outcomes over time. We also argue for a practical, end-to-end perspective for building ML pipelines for causal inference and propose an automated framework doing so. Finally, we combine decision-focused learning with causal inference by introducing ranking metalearners to optimize treatment decisions under capacity constraints.

Systematic literature review of machine learning based software development effort estimation models

This study focuses on using various machine learning (ML) models to evaluate the shear behaviors of ultra-high-performance concrete (UHPC) beams reinforced with glass fiber-reinforced polymer (GFRP) bars. The main objective of the study is to predict the shear strength of UHPC beams reinforced with GFRP bars using ML models. We use four different ML models: support vector machine (SVM), artificial neural network (ANN), random forest (R.F.), and extreme gradient boosting (XGBoost). The experimental database used in the study is acquired from various literature sources and comprises 54 test observations with 11 input features. These input features are likely parameters related to the composition, geometry, and properties of the UHPC beams and GFRP bars. To ensure the ML models' generalizability and scalability, random search methods are utilized to tune the hyperparameters of the algorithms. This tuning process helps improve the performance of the models when predicting the shear strength. The study uses the ACI318M-14 and Eurocode 2 standard building codes to predict the shear capacity behavior of GFRP bars-reinforced UHPC I-shaped beams. The ML models' predictions are compared to the results obtained from these building code standards. According to the findings, the XGBoost model demonstrates the highest predictive test performance among the investigated ML models. The study employs the SHAP (SHapley Additive exPlanations) analysis to assess the significance of each input parameter in the ML models' predictive capabilities. A Taylor diagram is used to statistically compare the accuracy of the ML models. This study concludes that ML models, particularly XGBoost, can effectively predict the shear capacity behavior of GFRP bars-reinforced UHPC I-shaped beams.

Accurate estimation of reference evapotranspiration (ETo) plays a vital role in irrigation and water resource planning. The Penman–Monteith method recommended by the Food and Agriculture Organization (FAO PM56) is widely used and considered a standard to calculate ETo. However, FAO PM56 cannot be used with limited meteorological variables, so it is compulsory to choose an alternative model for ETo estimation, which requires fewer variables. This study built ten machine learning (ML) models based on multi-function, neural network, and tree-based structure against the FAO PM56 method. For this purpose, time series temperature data on a monthly scale are only used to train ML models. The developed ML models were applied to estimate ETo at different test stations and the obtained results were compared with the FAO PM56 method to verify and validate their performance in ETo estimation for the selected stations. In addition, multiple statistical indicators, including root-mean-square error (RMSE), coefficient of determination (R2), mean absolute error (MAE), Nash–Sutcliffe efficiency (NSE), and correlation coefficient (r) were calculated to compare the performance of each ML model on ETo estimation. Among the applied ML models, the ETo tree boost (TB) ML model outperformed the other ML models in estimating ETo in diverse climatic conditions based on statistical indicators (R2, NSE, r, RMSE, and MAE). Moreover, the observed R2, NSE, and r were the highest for the TB ML model, while RMSE and MAE were found to be the lowest at the study sites compared to other applied ML models. Lastly, ETo point data yielded from the TB ML model was used in an interpolation process to create monthly and annual ETo maps. Based on the ETo maps, this study suggests mainly a focus on areas with high ETo values and proper irrigation scheduling of crops to ensure water sustainability.

Optimisation and interpretation of machine and deep learning models for improved water quality management in Lake Loktak

The emergence of the mobile applications that are driven by machine learning (ML) has brought a breakthrough in the world of healthcare, e-commerce, and entertainment with its personalized and smart user experiences. Despite the wide application of ML pipelines in mobile environments, the security and privacy issues that could be raised have become an issue of high concern on a global scale. The storage and processing of data, combined with the deployment of advanced ML models, expose vulnerabilities to risks such as data leakage, adversarial attacks, and unauthorized access. These threats not only compromise user trust but also incur applications with financial and brand damage. The present article deals with the important problems of the safety of mobile ML pipelines, namely adversarial manipulations, insecure storage, and transmitting of data. The vital techniques that can help to work out those problems are described along with the most popular ones, namely secure model storage based on encryption, federated learning for data transfer reduction, and model robustness tactics that will increase adversarial attacks defense. Besides that, new technologies like homomorphic encryption and blockchain are also discussed as ways to secure model updates. Consequently, this paper is involved in the technical and practical levels of the project, and it is meant to show the importance of building secure and privacy-preserving ML pipelines that do not only keep data for user trust but also maintain efficiency. It serves the purpose of opening the way to moving to secure mobile ML solutions in the time of intelligent mobile systems that have started to be of great interest.