Towards Accountability in Machine Learning Applications: A System-testing Approach

Aim: The majority of machine learning (ML) models in healthcare are built on retrospective data, much of which is collected without explicit patient consent for use in artificial intelligence (AI) and ML applications. The primary aim of this study was to evaluate whether clinicians and scientific researchers themselves consent to provide their own data for the training of ML models. Materials and Methods: This survey was conducted through an anonymous online survey, utilizing platforms such as Telegram, LinkedIn, and Viber. The target audience comprised specific international groups, primarily Russian, German, and English-speaking, of clinicians and scientific researchers. These participants ranged in their levels of expertise and experience, from beginners to veterans. The survey centered on a singular, pivotal question: “Do You Consent to the Use of Your Biological and Private Data for Training Machine Learning and AI Models?” Respondents had the option to choose from three responses: “Yes” and “No”. Results: The survey was conducted in January 2024. A total of 119 unique and verified individuals participated in the survey. The results revealed that only 50% of respondents (63 persons) expressed consent to provide their own data for the training of ML and AI models. Conclusion: In the development of ML and AI models, particularly open-source ones, it is crucial to ascertain whether participants are willing to provide their private data. While ML algorithms can transform the nature of data, it is important to remember that the primary owner of this data is the individual. Our findings show that in 50% of the cases, even participants from scientific research and clinical backgrounds – individuals typically accountable for ensuring data quality in AI and ML model development – do not consent to the use of their data in AI and ML settings. This highlights the need for more stringent consent processes and ethical considerations in the utilization of personal data in AI and ML research.

Accurate estimation of reference evapotranspiration (ETo) plays a vital role in irrigation and water resource planning. The Penman–Monteith method recommended by the Food and Agriculture Organization (FAO PM56) is widely used and considered a standard to calculate ETo. However, FAO PM56 cannot be used with limited meteorological variables, so it is compulsory to choose an alternative model for ETo estimation, which requires fewer variables. This study built ten machine learning (ML) models based on multi-function, neural network, and tree-based structure against the FAO PM56 method. For this purpose, time series temperature data on a monthly scale are only used to train ML models. The developed ML models were applied to estimate ETo at different test stations and the obtained results were compared with the FAO PM56 method to verify and validate their performance in ETo estimation for the selected stations. In addition, multiple statistical indicators, including root-mean-square error (RMSE), coefficient of determination (R2), mean absolute error (MAE), Nash–Sutcliffe efficiency (NSE), and correlation coefficient (r) were calculated to compare the performance of each ML model on ETo estimation. Among the applied ML models, the ETo tree boost (TB) ML model outperformed the other ML models in estimating ETo in diverse climatic conditions based on statistical indicators (R2, NSE, r, RMSE, and MAE). Moreover, the observed R2, NSE, and r were the highest for the TB ML model, while RMSE and MAE were found to be the lowest at the study sites compared to other applied ML models. Lastly, ETo point data yielded from the TB ML model was used in an interpolation process to create monthly and annual ETo maps. Based on the ETo maps, this study suggests mainly a focus on areas with high ETo values and proper irrigation scheduling of crops to ensure water sustainability.

Covalent inhibition offers many advantages over non-covalent inhibition, but covalent warhead reactivity must be carefully balanced to maintain potency while avoiding unwanted side effects. While warhead reactivities are commonly measured with assays, a computational model to predict warhead reactivities could be useful for several aspects of the covalent inhibitor design process. Studies have shown correlations between covalent warhead reactivities and quantum mechanic (QM) properties that describe important aspects of the covalent reaction mechanism. However, the models from these studies are often linear regression equations and can have limitations associated with their usage. Applications of machine learning (ML) models to predict covalent warhead reactivities with QM descriptors are not extensively seen in the literature. This study uses QM descriptors, calculated at different levels of theory, to train ML models to predict reactivities of covalent acrylamide warheads. The QM/ML models are compared with linear regression models built upon the same QM descriptors and with ML models trained on structure-based features like Morgan fingerprints and RDKit descriptors. Experiments show that the QM/ML models outperform the linear regression models and the structure-based ML models, and literature test sets demonstrate the power of the QM/ML models to predict reactivities of unseen acrylamide warhead scaffolds. Ultimately, these QM/ML models are effective, computationally feasible tools that can expedite the design of new covalent inhibitors.

Reconstructing the emergence of the modern continental crust is crucial to understand the evolution of the crust, the onset of plate tectonics, elemental cycling, and long-term climate. However, it remains highly contentious about when and how the major subaerial continental crust emerged over time. Here, we used a machine learning (ML) model (XGBoost) to reveal the exposed history of continental crust over the last 3.8 billion years ago (Ga). First, we compiled ~10,000 modern subaerial or submarine basalts with major and trace elements to train the ML model. Then, the trained ML model (with resampling) was utilized to predict and calculate the mean proportions of subaerially erupted continental basaltic rocks since 3.8 Ga. The result suggested that the subaerial proportion only reached about 50% at ~2.5 Ga, indicating the exposure of the continental crust was far from the present-day level at the end of the Archean era. On the other hand, since ~1.8 Ga, the subaerial proportion of the continental crust exhibited a dynamic balance between ~60% and 80%, reaching the present-day level.

HADA: An automated tool for hardware dimensioning of AI applications

Federated learning enables privacy-preserving and data-efficient dimension prediction and part qualification across additive manufacturing factories

This study presents the first application of Machine Learning (ML) models to optimise Powder Bed Fusion using Laser Beam (PBF-LB) process parameters for H13 steel fabricated on a 350 °C preheated building platform. A total of 189 cylindrical specimens were produced for training and testing machine learning (ML) models using variable process parameters: laser power (250-350 W), scanning speed (1050-1300 mm/s), and hatch spacing (65-90 μm). Eight ML models were investigated: 1. Support Vector Regression (SVR), 2. Kernel Ridge Regression (KRR), 3. Stochastic Gradient Descent Regressor, 4. Random Forest Regressor (RFR), 5. Extreme Gradient Boosting (XGBoost), 6. Extreme Gradient Boosting with limited depth (XGBoost LD), 7. Extra Trees Regressor (ETR) and 8. Light Gradient Boosting Machine (LightGBM). All models were trained using the Fast Library for Automated Machine Learning & Tuning (FLAML) framework to predict the relative density of the fabricated samples. Among these, the XGBoost model achieved the highest predictive accuracy, with a coefficient of determination R2=0.977, mean absolute percentage error MAPE = 0.002, and mean absolute error MAE = 0.017. Experimental validation was conducted on 27 newly fabricated samples using ML predicted process parameters. Relative densities exceeding 99.6% of the theoretical value (7.76 g/cm3) for all models except XGBoost LD and KRR. The lowest MAE = 0.004 and the smallest difference between the ML-predicted and PBF-LB validated density were obtained for samples made with LightGBM-predicted parameters. Those samples exhibited a hardness of 604 ± 13 HV0.5, which increased to approximately 630 HV0.5 after tempering at 550 °C. The LightGBM optimised parameters were further applied to fabricate a part of a forging die incorporating internal through-cooling channels, demonstrating the efficacy of machine learning-guided optimisation in achieving dense, defect-free H13 components suitable for industrial applications.

Machine learning (ML) models based on a large data set of in-cylinder flow fields of an IC engine obtained by high-speed particle image velocimetry allow the identification of relevant flow structures underlying cycle-to-cycle variations of engine performance. To this end, deep feature learning is employed to train ML models that predict cycles of high and low in-cylinder maximum pressure. Deep convolutional autoencoders are self-supervised-trained to encode flow field features in low dimensional latent space. Without the limitations ascribable to manual feature engineering, ML models based on these learned features are able to classify high energy cycles already from the flow field during late intake and the compression stroke as early as 290 crank angle degrees before top dead center ([Formula: see text]) with a mean accuracy above chance level. The prediction accuracy from [Formula: see text] to [Formula: see text] is comparable to baseline ML approaches utilizing an extensive set of engineered features. Relevant flow structures in the compression stroke are revealed by feature analysis of ML models and are interpreted using conditional averaged flow quantities. This analysis unveils the importance of the horizontal velocity component of in-cylinder flows in predicting engine performance. Combining deep learning and conventional flow analysis techniques promises to be a powerful tool for ultimately revealing high-level flow features relevant to the prediction of cycle-to-cycle variations and further engine optimization.

Background The prevalence of atrial fibrillation (AF), the most common type of cardiac arrhythmia, has increased globally. AF is also the leading cardiac factor for several cardiovascular and cerebrovascular events. Early prediction and treatment of AF prevents major cardiovascular and cerebrovascular events. Purpose Our objective is to predict AF risk within 7-years follow-up using only twelve basic factors and applying machine learning (ML) models. Methods Data from the German epidemiological trial on ankle brachial index (getABI) were utilized in this work. GetABI includes 6,457 patients with mean age of 72.53 years. 3,985 individuals were assessed for AF of which 393 patients suffered from AF at 7-years follow-up. The feature selection was initially done manually, ending up in 45 features. The 10 best features were extracted from this dataset according to the SHAP (SHapley Additive exPlanations) values. The final features were body mass index, sensitive CRP, cholesterol, LDL-cholesterol, troponin I, NT-pro-BNP, glomerular filtration rate, ratio of triglycerides and HDL-cholesterol, gamma-GT, heart rate, including age and sex as major factors. The handling of missing values was achieved by SimpleImputer, estimating the mean values. XGBoost and Random Forest predicted AF. Logistic Regression was also used as a baseline model. The data were split into training and test set and ML models were applied for 100 iterative runs, extracting results with mean values. Results XGBoost was the most accurate model, predicting the AF with 71.19 % mean accuracy, 66.37 % mean sensitivity and 75.61 % mean specificity. Figure 1 depicts the performance of all models. Additionally, Receiver Operating Characteristic Curve (ROC) and Area Under Curve (AUC) for each model are presented in Figure 2. XGBoost also outperformed all the others models, achieving the highest mean AUC value, equal to 70.99 %. Contrariwise, logistic regression had the lowest overall performance. Conclusion ML models, specifically XGBoost, achieved accurate AF prediction in 3,985 individuals utilizing only twelve clinical-routine data.Evaluation results of each used model.ROC curve and AUC value of each model.

Batteries, recognized as effective energy storage solutions, are considered the main facilitators of the world-wide transition towards clean and renewable energy sources. Among different types of batteries, lithium-ion (Li-ion) variants offer higher energy densities and relatively longer life spans when compared to other types. Nonetheless, a primary concern with these batteries is their lifetime. Batteries undergo various degradation mechanisms under storage and use, significantly impacting their lifespan. To this end, it is crucial to predict the degradation and lifetime of Li-ion batteries under given conditions.Researchers use three main methodologies to perform battery health diagnostics and to predict the lifetime of batteries. One approach revolves around using mechanistic, first-principle electrochemical models, also known as physics-based models. If equipped with proper thermodynamic theories, such models can show promising capabilities; however, holistic degradation prediction with these models is still challenging due to the computational complexities and the multitude of parameters that need to be fine-tuned in this approach. The other common strategy is to use empirical models to predict battery degradation. These models generally entail fewer parameters to be identified and are computationally less intensive to solve. Nonetheless, empirical models can suffer from the accuracy point-of-view as they are constrained to predict degradation trends introduced by certain degradation modes. Another common technique in battery life prediction is to utilize purely data-driven methods, such as machine learning (ML) algorithms, which also have shown promising results in the literature on rapid health predictions. However, these methods require large volumes of experimental data for training and testing ML models to ensure accuracy. In addition, data-driven methods are likely to extrapolate poorly to conditions beyond their training data and are indifferent towards the underlying degradation mechanisms. Recently, physics-informed machine learning (PI-ML) methods have garnered significant attention. They integrate physics-based or empirical models (developed based on physics) with a data-driven approach and allow one to train ML models on a smaller set of experimental data.To the best of the authors’ knowledge, the performance comparison between first-principle and empirical models when integrated within PI-ML remains unclear. Therefore, in this work, we aim to compare these two models when applied to prognostics (capacity forecasting and remaining useful life prediction) of a set of Li-ion batteries. To perform this study, we generate aging data for 40 Li-ion coin cells cycled under randomized conditions. Each cell undergoes a three-step charging stage followed by a two-step discharge stage. After obtaining the aging data, we will develop two PI-ML models, one equipped with a physics-based model and another with a set of empirical models. Both PI-ML models in this work will follow the sequential integration approach, where the training data for the final PI-ML model come from both experimental and computational data, the latter of which are obtained from the physics-based or empirical models. The parameters for the physics-based and empirical models are identified from another set of experimental data. Finally, the PI-ML models will be tested with experimental data obtained at different cycling conditions. The data flow for the sequential architecture of PI-ML is shown in the attached figure. This comparative study will help identify the performance of physics-based and empirical models when integrated into PI-ML. The main performance metric considered in this work is each model’s ability to extrapolate beyond the experimental training data set, hence aiding the final PI-ML model in generalizing to conditions not covered by its experimental training data set. Figure 1

The future 6G network will be able to connect massive and various unmanned vehicles (UxVs) into the network, bringing novel requirements toward UxV network security and data privacy. Deploying machine learning (ML)-based intrusion detection on UxVs can be one promising approach. The conventional approach cannot fulfill security and privacy requirements, as the structure can be more flexible, and there may be a lack of a central node in the 6G-based UxV network. Moreover, UxV clients can exhibit high heterogeneity with extremely class-imbalanced training samples. With the presence of federated learning (FL), UxVs can collaborate to train and update ML models while preserving data privacy in the UxV security critical scenario. However, most past works focused on centralized approaches, using cloud servers to assist intrusion detection and service deployments. This article introduces an FL framework with a decentralized design for training ML models on UxVs using intrusion detection as a study case. This scheme can be more flexible, as UxV clients can collaborate to train and synchronize models without the central server. Simulation experiments were conducted to evaluate the efficiency of the proposed approach. Experimental results and theoretical analysis confirm that the proposed method outperforms baseline models with FedAvg and local-trained client models.

In recent years, the microbiome has been recognized as an important factor associated with cardiovascular disease (CVD), which is the leading cause of human mortality worldwide. Disparities in gut microbial compositions between individuals with and without CVD were reported, whereby, we hypothesized that utilizing such microbiome-based data for training with supervised machine learning (ML) models could be exploited as a new strategy for evaluation of cardiovascular health. To test our hypothesis, we analyzed the metagenomics data extracted from the American Gut Project. Specifically, 16S rRNA reads from stool samples of 478 CVD and 473 non-CVD control samples were analyzed using five supervised ML algorithms: random forest (RF), support vector machine with radial kernel (svmRadial), decision tree (DT), elastic net (ENet) and neural networks (NN). Thirty-nine differential bacterial taxa (LEfSe: LDA > 2) were identified between CVD and non-CVD groups. ML classifications, using these taxonomic features, achieved an AUC (area under the receiver operating characteristic curve) of ~0.58 (RF). However, by choosing the top 500 high-variance features of operational taxonomic units (OTUs) for training ML models, an improved AUC of ~0.65 (RF) was achieved. Further, by limiting the selection to only the top 25 highly contributing OTU features to reduce the dimensionality of feature space, the AUC was further significantly enhanced to ~0.70 (RF). In summary, this study is the first to demonstrate the successful development of a ML model using microbiome-based datasets for a systematic diagnostic screening of CVD.

The rising machine learning (ML) models have become the preferred way for landslide detection based on remote sensing images, but the performance of these models in a sample-free area are rarely concerned in many studies. In this study, we used a cross-validation method (training model in one area and validation in another) to compare the model portability of trained ML models applied in an “off-site” area, as a consideration of the landslide detection ability of these models in sample-free areas. We integrate nighttime light imagery, multi-seasonal optical Landsat time-series and digital elevation data, and we employed support vector machines (SVM), artificial neural networks (ANN) and random forest (RF) models to classify the satellite imagery and identify landslides. Samples of two scenarios generated from two subareas of the Jiuzhaigou disaster-stricken region are used for the cross-application and accuracy evaluation of three ML models. The results revealed that when the trained models are applied in areas outside those in which they were developed, the landslide identification accuracy of these three models has declined. Especially for the SVM and ANN models, the accuracy is greatly reduced and there appears a seriously imbalanced user’s and producer’s accuracy. However, although the performance of the RF model is lower than that of SVM and ANN models in their local area, the RF model exhibits stable portability, and retains the original performance and achieves a satisfactory balance between overestimation and underestimation in “off-site” areas. An additional validation from a new area proved that the landslide detection performance of the RF model with stable portability is higher than that of the SVM and ANN models in “off-site” areas. The results suggest that evaluating the model portability through cross-application can be a useful way to determine the most suitable model for landslide detection in “off-site” areas with a similar geographic environment to model development areas, so as to maximize the accuracy of landslide detection based on limited samples.

This paper analyzes the optimization features of machine learning (ML) model training procedures using multi-GPU systems to enhance cyber security in telecommunication networks. A key aspect of the study is the use of data parallelism, which allows the distribution of the training load across multiple GPUs, significantly reducing training time and improving model accuracy-critical factors for rapid threat detection in cyberspace. A novel approach for optimizing data batch size using Mutual Information (MI) is proposed, which harmonizes the utilization of computational resources with the information content of the training data. MI helps to determine the optimal data batch size that minimizes training errors and improves model accuracy without a significant increase in training time. Experimental results demonstrate the substantial advantages of multi-GPU configurations compared to single-GPU setups, providing faster training and improved model accuracy. It was particularly emphasized that MI-guided batch size tuning significantly outperforms traditional manual tuning methods, ensuring higher validation accuracy and reducing training time. The study showed that the MI-based approach is an effective tool for addressing the problem of optimizing ML model training processes in real-world scenarios where cyber security is critical. The proposed methods allow ML models to train faster and more accurately identify potential threats, making them particularly relevant for telecommunication networks where a rapid response to new threats in real time is required. The implementation of modern computational technologies such as multi-GPU systems and MI-optimized training enhances the efficiency and accuracy of machine learning models. This, in turn, improves cyber security measures and ensures a more reliable defence of telecommunication networks against malicious attacks. It is noted that the proposed approaches can be adapted not only for cyber security but also for other areas where high model accuracy and fast training are important. Future research prospects include the development of new machine learning methods, particularly deep neural networks, the exploration of alternative computational architectures such as quantum computing or distributed systems, and their integration into real-time systems. Special attention should be paid to the ethical aspects of implementing automated cyber security systems, particularly in preventing bias in algorithms and ensuring fairness in their application.

Surrogate-assisted hydraulic fracture optimization workflow with applications for shale gas reservoir development: a comparative study of machine learning models