Stabilizing machine learning prediction of dynamics: Novel noise-inspired regularization tested with reservoir computing

Aim: The majority of machine learning (ML) models in healthcare are built on retrospective data, much of which is collected without explicit patient consent for use in artificial intelligence (AI) and ML applications. The primary aim of this study was to evaluate whether clinicians and scientific researchers themselves consent to provide their own data for the training of ML models. Materials and Methods: This survey was conducted through an anonymous online survey, utilizing platforms such as Telegram, LinkedIn, and Viber. The target audience comprised specific international groups, primarily Russian, German, and English-speaking, of clinicians and scientific researchers. These participants ranged in their levels of expertise and experience, from beginners to veterans. The survey centered on a singular, pivotal question: “Do You Consent to the Use of Your Biological and Private Data for Training Machine Learning and AI Models?” Respondents had the option to choose from three responses: “Yes” and “No”. Results: The survey was conducted in January 2024. A total of 119 unique and verified individuals participated in the survey. The results revealed that only 50% of respondents (63 persons) expressed consent to provide their own data for the training of ML and AI models. Conclusion: In the development of ML and AI models, particularly open-source ones, it is crucial to ascertain whether participants are willing to provide their private data. While ML algorithms can transform the nature of data, it is important to remember that the primary owner of this data is the individual. Our findings show that in 50% of the cases, even participants from scientific research and clinical backgrounds – individuals typically accountable for ensuring data quality in AI and ML model development – do not consent to the use of their data in AI and ML settings. This highlights the need for more stringent consent processes and ethical considerations in the utilization of personal data in AI and ML research.

This paper analyzes the optimization features of machine learning (ML) model training procedures using multi-GPU systems to enhance cyber security in telecommunication networks. A key aspect of the study is the use of data parallelism, which allows the distribution of the training load across multiple GPUs, significantly reducing training time and improving model accuracy-critical factors for rapid threat detection in cyberspace. A novel approach for optimizing data batch size using Mutual Information (MI) is proposed, which harmonizes the utilization of computational resources with the information content of the training data. MI helps to determine the optimal data batch size that minimizes training errors and improves model accuracy without a significant increase in training time. Experimental results demonstrate the substantial advantages of multi-GPU configurations compared to single-GPU setups, providing faster training and improved model accuracy. It was particularly emphasized that MI-guided batch size tuning significantly outperforms traditional manual tuning methods, ensuring higher validation accuracy and reducing training time. The study showed that the MI-based approach is an effective tool for addressing the problem of optimizing ML model training processes in real-world scenarios where cyber security is critical. The proposed methods allow ML models to train faster and more accurately identify potential threats, making them particularly relevant for telecommunication networks where a rapid response to new threats in real time is required. The implementation of modern computational technologies such as multi-GPU systems and MI-optimized training enhances the efficiency and accuracy of machine learning models. This, in turn, improves cyber security measures and ensures a more reliable defence of telecommunication networks against malicious attacks. It is noted that the proposed approaches can be adapted not only for cyber security but also for other areas where high model accuracy and fast training are important. Future research prospects include the development of new machine learning methods, particularly deep neural networks, the exploration of alternative computational architectures such as quantum computing or distributed systems, and their integration into real-time systems. Special attention should be paid to the ethical aspects of implementing automated cyber security systems, particularly in preventing bias in algorithms and ensuring fairness in their application.

Accurate estimation of reference evapotranspiration (ETo) plays a vital role in irrigation and water resource planning. The Penman–Monteith method recommended by the Food and Agriculture Organization (FAO PM56) is widely used and considered a standard to calculate ETo. However, FAO PM56 cannot be used with limited meteorological variables, so it is compulsory to choose an alternative model for ETo estimation, which requires fewer variables. This study built ten machine learning (ML) models based on multi-function, neural network, and tree-based structure against the FAO PM56 method. For this purpose, time series temperature data on a monthly scale are only used to train ML models. The developed ML models were applied to estimate ETo at different test stations and the obtained results were compared with the FAO PM56 method to verify and validate their performance in ETo estimation for the selected stations. In addition, multiple statistical indicators, including root-mean-square error (RMSE), coefficient of determination (R2), mean absolute error (MAE), Nash–Sutcliffe efficiency (NSE), and correlation coefficient (r) were calculated to compare the performance of each ML model on ETo estimation. Among the applied ML models, the ETo tree boost (TB) ML model outperformed the other ML models in estimating ETo in diverse climatic conditions based on statistical indicators (R2, NSE, r, RMSE, and MAE). Moreover, the observed R2, NSE, and r were the highest for the TB ML model, while RMSE and MAE were found to be the lowest at the study sites compared to other applied ML models. Lastly, ETo point data yielded from the TB ML model was used in an interpolation process to create monthly and annual ETo maps. Based on the ETo maps, this study suggests mainly a focus on areas with high ETo values and proper irrigation scheduling of crops to ensure water sustainability.

Shear viscosity, though being a fundamental property of all fluids, is computationally expensive to calculate from equilibrium molecular dynamics simulations. Recently, machine learning (ML) methods have been used to augment molecular simulations in many contexts, thus showing promise to estimate viscosity too in a relatively inexpensive manner. However, ML methods face significant challenges—such as overfitting, when the size of the data set is small, as is the case with viscosity. In this work, we train seven ML models to predict the shear viscosity of a Lennard–Jones fluid, with particular emphasis on addressing issues arising from a small data set. Specifically, the issues related to model selection, performance estimation and uncertainty quantification were investigated. First, we show that the widely used performance estimation procedure of using a single unseen data set shows a wide variability—in estimating the errors on—small data sets. In this context, the common practice of using cross validation (CV) to select the hyperparameters (model selection) can be adapted to estimate the generalization error (performance estimation) as well. We compare two simple CV procedures for their ability to do both model selection and performance estimation, and find that k-fold CV based procedure shows a lower variance of error estimates. Also, these CV procedures naturally lead to an ensemble of trained ML models. We discuss the role of performance metrics in training and evaluation and propose a method to rank the ML models based on multiple metrics. Finally, two methods for uncertainty quantification—Gaussian process regression (GPR) and ensemble method—were used to estimate the uncertainty on individual predictions. The uncertainty estimates from GPR were also used to construct an applicability domain using which the ML models provided even more reliable predictions on an independent viscosity data set generated in this work. Overall, the procedures prescribed in this work, together, lead to robust ML models for small data sets.

Summary A meticulous interpretation of steady-state or unsteady-state relative permeability (Kr) experimental data is required to determine a complete set of Kr curves. In this work, different machine learning (ML) models were developed to assist in a faster estimation of these curves from steady-state drainage coreflooding experimental runs. These ML algorithms include gradient boosting (GB), random forest (RF), extreme gradient boosting (XGB), and deep neural network (DNN) with a main focus on and comparison of the two latter algorithms (XGB and DNN). Based on existing mathematical models, a leading-edge framework was developed where a large database of Kr and capillary pressure (Pc) curves were generated. This database was used to perform thousands of coreflood simulation runs representing oil-water drainage steady-state experiments. The results obtained from these simulation runs, mainly pressure drop along with other conventional core analysis data, were used to estimate analytical Kr curves based on Darcy’s law. These analytically estimated Kr curves along with the previously generated Pc curves were fed as features into the ML model. The entire data set was split into 80% for training and 20% for testing. The k-fold cross-validation technique was applied to increase the model’s accuracy by splitting 80% of the training data into 10 folds. In this manner, for each of the 10 experiments, nine folds were used for training and the remaining fold was used for model validation. Once the model was trained and validated, it was subjected to blind testing on the remaining 20% of the data set. The ML model learns to capture fluid flow behavior inside the core from the training data set. In terms of applicability of these ML models, two sets of experimental data were needed as input; the first was the analytically estimated Kr curves from the steady-state drainage coreflooding experiments, while the other was the Pc curves estimated from centrifuge or mercury injection capillary pressure (MICP) measurements. The trained/tested model was then able to estimate Kr curves based on the experimental results fed as input. Furthermore, to test the performance of the ML model when only one set of experimental data is available to an end user, a recurrent neural network (RNN) algorithm was trained/tested to predict Kr curves in the absence of Pc curves as an input. The performance of the three developed models (XGB, DNN, and RNN) was assessed using the values of the coefficient of determination (R2) along with the loss calculated during training/validation of the model. The respective crossplots along with comparisons of ground truth vs. artificial intelligence (AI)-predicted curves indicated that the model is capable of making accurate predictions with an error percentage between 0.2% and 0.6% on history-matching experimental data for all three tested ML techniques. This implies that the AI-based model exhibits better efficiency and reliability in determining Kr curves when compared to conventional methods. The developed ML models by no means replace the need to conduct drainage coreflooding or centrifuge experiments but act as an alternative to existing commercial platforms that are used to interpret experimental data to predict Kr curves. The two main advantages of the developed ML models are their capability of predicting Kr curves within a matter of a few minutes as well as with limited intervention from the end user. The results also include a comparison between classical ML approaches, shallow neural networks, and DNNs in terms of accuracy in predicting the final Kr curves. The research presented here is an extension of the state-of-the-art framework proposed by Mathew et al. (2021). However, the two main aspects of the current study are the application of deep learning for the prediction of Kr curves and the application of feature engineering. The latter not only reduces the training/testing time for the ML models but also enables the end user to obtain the final predictions with the least set of experimental data. The various models discussed in this research work currently focus on the prediction of Kr curves for drainage steady-state experiments; however, the work can be extended to capture the imbibition cycle as well.

This paper provides an overview of the following three modeling approaches – Physics-based Modeling, Machine Learning, and Hybrid Modeling (Physics-based Modeling & Machine Learning combined), and addresses their applicability for multiphase flow simulations in specific use-cases. Physics-based modeling builds on well understood concepts in, e.g., thermodynamics, fluid dynamics, fluid modeling and optimization techniques. It requires deep domain knowledge as well as accurate fluid data and may incur significant computational cost. Machine Learning systems are based on learning algorithms, which find relationships between sensor data and output variables in a training dataset. The approach requires a good understanding of the learning algorithms and statistics. Small datasets or changes in operational conditions limit the suitability of this approach. Hybrid Models combine Physics with Machine Learning, and these models are on a sliding scale between pure Physics-based Models and pure Machine Learning Models. The individual use-case defines how the Hybrid Model is configured and where it sits on the sliding scale. Here, we will investigate three different use-cases: Physics supporting Machine Learning: Use physics to generate data to support training of machine learning models when data is sparse. Machine Learning supporting Physics: Use machine learning to provide additional input to physics modeling when available data does not suffice to achieve accurate numerical solutions. True Hybrid: Close the circle and use physics for feature engineering to improve the machine learning models, which then provide synthetic data as input to physics-based simulations. We observe that there is not one universal solution. On the one hand, flow simulations may address situations which are stochastic in nature and not deterministic. Here, a machine learning model based on physics-based simulations fits the purpose well. On the other hand, there are situations when machine learning can deduce relations between parameters such that you can provide additional input into physics-based models. One such example is to address the oil-water split at very high water cuts. By adopting the appropriate combination, the hybrid approach results in superior accuracy and offers the ability to address a broader range of applications than physics or machine learning alone.

PD27-01 DEVELOPMENT OF A MACHINE LEARNING (ML) MODEL TO AUTOMATICALLY AND PRECISELY IDENTIFY KIDNEY STONES FROM URETEROSCOPY VIDEO RECORDINGS

<p>Background: The prediction of aneurysm treatment outcomes can help to optimize the treatment strategies. Machine learning (ML) has shown positive results in many clinical areas. However, the development of such models requires expertise in ML, which is not an easy task for surgeons.</p><p>Objectives: The recently emerged automated machine learning (AutoML) has shown promise in making ML more accessible to non-computer experts. We aimed to evaluate the feasibility of applying AutoML to develop the ML models for treatment outcome prediction.</p><p>Methods: The patients with aneurysms treated by endovascular treatment were prospectively recruited from 2016 to 2020. Treatment was considered successful if angiographic complete occlusion was achieved at follow-up. A statistical prediction model was developed using multivariate logistic regression. In addition, two ML models were developed. One was developed manually and the other was developed by AutoML. Three models were compared based on their area under the precision-recall curve (AUPRC) and area under the receiver operating characteristic curve (AUROC).</p><p>Results: The aneurysm size, stent-assisted coiling (SAC), and posterior circulation were the three significant and independent variables associated with treatment outcome. The statistical model showed an AUPRC of 0.432 and AUROC of 0.745. The conventional manually trained ML model showed an improved AUPRC of 0.545 and AUROC of 0.781. The AutoML derived ML model showed the best performance with AUPRC of 0.632 and AUROC of 0.832, significantly better than the other two models.</p><p>Conclusions: This study demonstrated the feasibility of using AutoML to develop a high-quality ML model, which may outperform the statistical model and manually derived ML models. AutoML could be a useful tool that makes ML more accessible to the clinical researchers.</p>

The pooled cohort equations (PCE) predict atherosclerotic cardiovascular disease (ASCVD) risk in patients with characteristics within prespecified ranges and has uncertain performance among Asians or Hispanics. It is unknown if machine learning (ML) models can improve ASCVD risk prediction across broader diverse, real-world populations. We developed ML models for ASCVD risk prediction for multi-ethnic patients using an electronic health record (EHR) database from Northern California. Our cohort included patients aged 18 years or older with no prior CVD and not on statins at baseline (n = 262,923), stratified by PCE-eligible (n = 131,721) or PCE-ineligible patients based on missing or out-of-range variables. We trained ML models [logistic regression with L2 penalty and L1 lasso penalty, random forest, gradient boosting machine (GBM), extreme gradient boosting] and determined 5-year ASCVD risk prediction, including with and without incorporation of additional EHR variables, and in Asian and Hispanic subgroups. A total of 4309 patients had ASCVD events, with 2077 in PCE-ineligible patients. GBM performance in the full cohort, including PCE-ineligible patients (area under receiver-operating characteristic curve (AUC) 0.835, 95% confidence interval (CI): 0.825–0.846), was significantly better than that of the PCE in the PCE-eligible cohort (AUC 0.775, 95% CI: 0.755–0.794). Among patients aged 40–79, GBM performed similarly before (AUC 0.784, 95% CI: 0.759–0.808) and after (AUC 0.790, 95% CI: 0.765–0.814) incorporating additional EHR data. Overall, ML models achieved comparable or improved performance compared to the PCE while allowing risk discrimination in a larger group of patients including PCE-ineligible patients. EHR-trained ML models may help bridge important gaps in ASCVD risk prediction.

A machine learning model exploring creep performance of dental composites.

Fatal accidents are a major issue hindering the wide acceptance of safety-critical systems that employ machine learning and deep learning models, such as automated driving vehicles. In order to use machine learning in a safety-critical system, it is necessary to demonstrate the safety and security of the system through engineering processes. However, thus far, no such widely accepted engineering concepts or frameworks have been established for these systems. The key to using a machine learning model in a deductively engineered system is decomposing the data-driven training of machine learning models into requirement, design, and verification, particularly for machine learning models used in safety-critical systems. Simultaneously, open problems and relevant technical fields are not organized in a manner that enables researchers to select a theme and work on it. In this study, we identify, classify, and explore the open problems in engineering (safety-critical) machine learning systems—that is, in terms of requirement, design, and verification of machine learning models and systems—as well as discuss related works and research directions, using automated driving vehicles as an example. Our results show that machine learning models are characterized by a lack of requirements specification, lack of design specification, lack of interpretability, and lack of robustness. We also perform a gap analysis on a conventional system quality standard SQuaRE with the characteristics of machine learning models to study quality models for machine learning systems. We find that a lack of requirements specification and lack of robustness have the greatest impact on conventional quality models.

<p><strong>Abstract: </strong></p> <p><strong> </strong>The increasing development of GNSS techniques enables solving geodetic problems on both local and global scales. Parallelly, complex algorithms have been proposed and can also be solved well by Machine Learning (ML). However, ML techniques are sometimes not sensitive enough to gain results with a high probability for some cases, like sparse data or non-stationary GNSS time series. In this study, we use a combination of Human and Machine learning (H&M) to improve the classification performance of continuous GNSS stations. First, 427 permanent GNSS stations are obtained from the EUREF network to train ML models. The models are then applied to classify the quality of 939 continuous observation stations from two projects, EIFEL and IPOC, carried out by the German Research Centre for Geosciences (GFZ), Potsdam, Germany. Next, we independently validate the ML models' reality through a MATLAB program, GNSS metadata, and seismic data. Finally, all data of these 1366 stations are used to re-train the ML models. The main criteria to classify are the number of outliers, jumps in GNSS time series, root mean square errors, observation time-spans, and stability of the crustal motion velocity fields. Applying the approach of the H&M combination improves the performance of the ML models up to 92% while using only ML methods remains ~68%. These ML-based classification models can be applied to estimate the quality of permanent GNSS stations and to manage big databases. The result is the basis for selecting suitable control and monitoring stations in crustal deformation monitoring as well as in civil and industrial applications.</p> <p><strong>Keywords: </strong></p> <p>GNSS station classification, Machine learning, Human & Machine learning combination.</p>

Machine learning (ML) models provide more choices to patients with diabetes mellitus (DM) to more properly manage blood glucose (BG) levels. However, because of numerous types of ML algorithms, choosing an appropriate model is vitally important. In a systematic review and network meta-analysis, this study aimed to comprehensively assess the performance of ML models in predicting BG levels. In addition, we assessed ML models used to detect and predict adverse BG (hypoglycemia) events by calculating pooled estimates of sensitivity and specificity. PubMed, Embase, Web of Science, and Institute of Electrical and Electronics Engineers Explore databases were systematically searched for studies on predicting BG levels and predicting or detecting adverse BG events using ML models, from inception to November 2022. Studies that assessed the performance of different ML models in predicting or detecting BG levels or adverse BG events of patients with DM were included. Studies with no derivation or performance metrics of ML models were excluded. The Quality Assessment of Diagnostic Accuracy Studies tool was applied to assess the quality of included studies. Primary outcomes were the relative ranking of ML models for predicting BG levels in different prediction horizons (PHs) and pooled estimates of the sensitivity and specificity of ML models in detecting or predicting adverse BG events. In total, 46 eligible studies were included for meta-analysis. Regarding ML models for predicting BG levels, the means of the absolute root mean square error (RMSE) in a PH of 15, 30, 45, and 60 minutes were 18.88 (SD 19.71), 21.40 (SD 12.56), 21.27 (SD 5.17), and 30.01 (SD 7.23) mg/dL, respectively. The neural network model (NNM) showed the highest relative performance in different PHs. Furthermore, the pooled estimates of the positive likelihood ratio and the negative likelihood ratio of ML models were 8.3 (95% CI 5.7-12.0) and 0.31 (95% CI 0.22-0.44), respectively, for predicting hypoglycemia and 2.4 (95% CI 1.6-3.7) and 0.37 (95% CI 0.29-0.46), respectively, for detecting hypoglycemia. Statistically significant high heterogeneity was detected in all subgroups, with different sources of heterogeneity. For predicting precise BG levels, the RMSE increases with a rise in the PH, and the NNM shows the highest relative performance among all the ML models. Meanwhile, current ML models have sufficient ability to predict adverse BG events, while their ability to detect adverse BG events needs to be enhanced. PROSPERO CRD42022375250; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=375250.

Machine learning (ML) models based on a large data set of in-cylinder flow fields of an IC engine obtained by high-speed particle image velocimetry allow the identification of relevant flow structures underlying cycle-to-cycle variations of engine performance. To this end, deep feature learning is employed to train ML models that predict cycles of high and low in-cylinder maximum pressure. Deep convolutional autoencoders are self-supervised-trained to encode flow field features in low dimensional latent space. Without the limitations ascribable to manual feature engineering, ML models based on these learned features are able to classify high energy cycles already from the flow field during late intake and the compression stroke as early as 290 crank angle degrees before top dead center ([Formula: see text]) with a mean accuracy above chance level. The prediction accuracy from [Formula: see text] to [Formula: see text] is comparable to baseline ML approaches utilizing an extensive set of engineered features. Relevant flow structures in the compression stroke are revealed by feature analysis of ML models and are interpreted using conditional averaged flow quantities. This analysis unveils the importance of the horizontal velocity component of in-cylinder flows in predicting engine performance. Combining deep learning and conventional flow analysis techniques promises to be a powerful tool for ultimately revealing high-level flow features relevant to the prediction of cycle-to-cycle variations and further engine optimization.

Data reformation – A novel data processing technique enhancing machine learning applicability for predicting streamflow extremes