Efficient Integration of Machine Learning into District Heating Predictive Models

<p>Soil spectroscopy in the mid-infrared (MIR) allows the fast and cost-effective derivation of multiple physical and chemical soil properties, e.g., soil organic carbon (SOC) and soil texture, from a single reflectance spectrum. The recent development of extensive soil spectral libraries and field-portable handheld FTIR spectrometers have opened up new opportunities for the widespread application of soil reflectance spectroscopy in the geo- and environmental sciences. Compared to laboratory measurements on pre-treated soil material, field recordings of MIR spectra are impacted by in situ environmental conditions that modify and degrade the measured reflectance signal, most prominently variations in soil moisture and particle size across samples. These conditions prevent leveraging available MIR soil spectral libraries to build predictive models of soil properties directly.</p><p>We evaluated the capacity of the External Parameter Orthogonalization (EPO) algorithm to compensate for moisture and particle size-induced effects on MIR reflectance spectra recorded in the field to generate laboratory-equivalent spectra from the in-situ data, which would allow calibrations of predictive soil property models from soil spectral library data to be transferred to field-recorded spectra. An archive of 230 soils collected across five soil regions in Germany covering a broad range of parent materials, soil texture classes and organic carbon contents was used to evaluate the approach. For each soil sample, MIR reflectance spectra had been acquired both in the field, i.e., measured in situ on the soil surface, and in the laboratory on pre-treated (sieved and ground) soil material. Field spectra were corrected for environmental effects by EPO and used to predict SOC and soil texture with predictive models developed on the laboratory spectra.</p><p>Analysis of the EPO-transformed spectra showed that the algorithm could compensate for some of the significant environmental effects present in the field data, e.g., non-linear baseline shifts and large-scale water absorption features, effectively reducing variation across the soil samples that is not linked to the physical and chemical soil properties of interest. EPO-transformation of the spectra further allowed a robust transfer of calibrations developed on laboratory spectra of pre-treated soils to the field spectra. Predictive accuracies for SOC and soil texture were lower than for pure laboratory applications but generally in line with models developed with an extensive regional calibration sample directly on the field MIR spectra.</p><p>The correction of field MIR spectra with the EPO algorithm thus represents a promising approach to integrating existing soil spectral libraries into the development of predictive soil property models for in-situ MIR reflectance spectra as it would allow the development of predictive models without requiring a large number of additional regional calibration samples for field application of MIR soil spectroscopy.</p>

Background and purpose — External validation of machine learning (ML) prediction models is an essential step before clinical application. We assessed the proportion, performance, and transparent reporting of externally validated ML prediction models in orthopedic surgery, using the Transparent Reporting for Individual Prognosis or Diagnosis (TRIPOD) guidelines. Material and methods — We performed a systematic search using synonyms for every orthopedic specialty, ML, and external validation. The proportion was determined by using 59 ML prediction models with only internal validation in orthopedic surgical outcome published up until June 18, 2020, previously identified by our group. Model performance was evaluated using discrimination, calibration, and decision-curve analysis. The TRIPOD guidelines assessed transparent reporting. Results — We included 18 studies externally validating 10 different ML prediction models of the 59 available ML models after screening 4,682 studies. All external validations identified in this review retained good discrimination. Other key performance measures were provided in only 3 studies, rendering overall performance evaluation difficult. The overall median TRIPOD completeness was 61% (IQR 43–89), with 6 items being reported in less than 4/18 of the studies. Interpretation — Most current predictive ML models are not externally validated. The 18 available external validation studies were characterized by incomplete reporting of performance measures, limiting a transparent examination of model performance. Further prospective studies are needed to validate or refute the myriad of predictive ML models in orthopedics while adhering to existing guidelines. This ensures clinicians can take full advantage of validated and clinically implementable ML decision tools.

Computational cell metabolism models seek to provide metabolic explanations of cell behavior under different conditions or following genetic alterations, help in the optimization of in vitro cell growth environments, or predict cellular behavior in vivo and in vitro. In the extremes, mechanistic models can include highly detailed descriptions of a small number of metabolic reactions or an approximate representation of an entire metabolic network. To date, all mechanistic models have required details of individual metabolic reactions, either kinetic parameters or metabolic flux, as well as information about extracellular and intracellular metabolite concentrations. Despite the extensive efforts and the increasing availability of high-quality data, required in vivo data are not available for the majority of known metabolic reactions; thus, mechanistic models are based primarily on ex vivo kinetic measurements and limited flux information. Machine learning approaches provide an alternative for derivation of functional dependencies from existing data. The increasing availability of metabolomic and lipidomic data, with growing feature coverage as well as sample set size, is expected to provide new data options needed for derivation of machine learning models of cell metabolic processes. Moreover, machine learning analysis of longitudinal data can lead to predictive models of cell behaviors over time. Conversely, machine learning models trained on steady-state data can provide descriptive models for the comparison of metabolic states in different environments or disease conditions. Additionally, inclusion of metabolic network knowledge in these analyses can further help in the development of models with limited data.This chapter will explore the application of machine learning to the modeling of cell metabolism. We first provide a theoretical explanation of several machine learning and hybrid mechanistic machine learning methods currently being explored to model metabolism. Next, we introduce several avenues for improving these models with machine learning. Finally, we provide protocols for specific examples of the utilization of machine learning in the development of predictive cell metabolism models using metabolomic data. We describe data preprocessing, approaches for training of machine learning models for both descriptive and predictive models, and the utilization of these models in synthetic and systems biology. Detailed protocols provide a list of software tools and libraries used for these applications, step-by-step modeling protocols, troubleshooting, as well as an overview of existing limitations to these approaches.

SummaryAccurate prediction of the solubility of chemical substances in solvents remains a challenge. The sparsity of high-quality solubility data is recognized as the biggest hurdle in the development of robust data-driven methods for practical use. Nonetheless, the effects of the quality and quantity of data on aqueous solubility predictions have not yet been scrutinized. In this study, the roles of the size and the quality of data sets on the performances of the solubility prediction models are unraveled, and the concepts of actual and observed performances are introduced. In an effort to curtail the gap between actual and observed performances, a quality-oriented data selection method, which evaluates the quality of data and extracts the most accurate part of it through statistical validation, is designed. Applying this method on the largest publicly available solubility database and using a consensus machine learning approach, a top-performing solubility prediction model is achieved.

ObjectivesMissing data is a common problem during the development, evaluation, and implementation of prediction models. Although machine learning (ML) methods are often said to be capable of circumventing missing data, it is unclear how these methods are used in medical research. We aim to find out if and how well prediction model studies using machine learning report on their handling of missing data. Study design and settingWe systematically searched the literature on published papers between 2018 and 2019 about primary studies developing and/or validating clinical prediction models using any supervised ML methodology across medical fields. From the retrieved studies information about the amount and nature (e.g. missing completely at random, potential reasons for missingness) of missing data and the way they were handled were extracted. ResultsWe identified 152 machine learning-based clinical prediction model studies. A substantial amount of these 152 papers did not report anything on missing data (n = 56/152). A majority (n = 96/152) reported details on the handling of missing data (e.g., methods used), though many of these (n = 46/96) did not report the amount of the missingness in the data. In these 96 papers the authors only sometimes reported possible reasons for missingness (n = 7/96) and information about missing data mechanisms (n = 8/96). The most common approach for handling missing data was deletion (n = 65/96), mostly via complete-case analysis (CCA) (n = 43/96). Very few studies used multiple imputation (n = 8/96) or built-in mechanisms such as surrogate splits (n = 7/96) that directly address missing data during the development, validation, or implementation of the prediction model. ConclusionThough missing values are highly common in any type of medical research and certainly in the research based on routine healthcare data, a majority of the prediction model studies using machine learning does not report sufficient information on the presence and handling of missing data. Strategies in which patient data are simply omitted are unfortunately the most often used methods, even though it is generally advised against and well known that it likely causes bias and loss of analytical power in prediction model development and in the predictive accuracy estimates. Prediction model researchers should be much more aware of alternative methodologies to address missing data.

A fundamental part of medical research is the development and validation of diagnostic and prognostic prediction models [1,2]. These prediction models aim to predict the absolute probability that a certain disease or condition is currently present (diagnostic models) or that an outcome will occur within a specific follow-up period (prognostic models) for an individual subject. Prediction models typically rely on multiple predictors, which can include demographic characteristics, medical history and physical examination items, or more complex measurements from, for example, medical imaging, electrophysiology, pathology, and biomarkers. Also for diagnostic models, estimates of probabilities are rarely based on a single test, and doctors naturally integrate several patient characteristics and symptoms [3]. A broad range of prediction modeling techniques exist, like regression approaches, neural network models, decision tree models, genetic programming models, and support vector machine learning models, although prediction models developed by a multivariable regression approach are by far prevailing. It is widely recommended that a developed prediction model should not be used in practice before being externally validated—at least once—in other individuals than those used for model development [4–7]. Unfortunately, most prediction models are poorly or not at all validated, rendering interpretation of their generalizability difficult. In addition, many systematic reviews showed that for the same outcome or same target population, numerous competing models exist [8–10]. Generally speaking, researchers often ignore existing prediction models and develop yet another prediction model from their own data [2]. This practice sustains a cycle of underpowered prediction model development studies and poor knowledge about the generalizability and applicability of developed prediction models. Evidence synthesis and meta-analysis of individual participant data (IPD) from multiple studies seems to be a unique opportunity to address these problems, as it allows researchers to develop and directly validate models on large datasets and across a wide range of populations and settings, to directly test a model’s generalizability (Fig 1) [11–13]. Fig 1 Trends in publications of IPD-MA studies focusing on the development and/or validation of diagnostic or prognostic prediction models. There is currently little guidance on how to conduct an IPD meta-analysis (IPD-MA) for developing and/or validating diagnostic or prognostic prediction models [15]. To date, most IPD-MA articles focus on estimating relative quantities, like a risk ratio, hazard ratio, or odds ratio for a specific treatment or a specific etiologic factor. In contrast, prediction modeling research is focused on developing and validating multivariable models aimed at calculating an absolute risk estimate of the combined variables, rather than estimating the relative effect of a specific treatment or etiologic factor. Furthermore, prediction modeling studies focus entirely on the role and joint contribution of multiple covariates, whereas intervention studies in principle rely on randomization to reduce the role of covariates (Table 1). Hence, IPD-MAs of randomized intervention and etiological studies, which are beyond the scope of this paper and are instead addressed in the accompanying paper [16], differ from IPD-MAs of multivariable prediction models, which are the focus of this paper. Table 1 The main differences between IPD-MA of treatment intervention studies and of multivariable prediction modeling studies. We provide an overview of the advantages and limitations of IPD-MAs aiming to develop a novel prediction model or to validate one or more existing models across multiple datasets. This overview is based on published guidelines and existing recommendations for the conduct of prediction modeling studies and of IPD-MA research. We illustrate this overview with examples of recently published IPD-MAs of prediction models across various medical domains. Our aim is to help researchers, readers, reviewers, and editors to identify and understand the key issues involved with such IPD-MA projects.

Accurate prediction models can potentially transform businesses, organizations, governments, and industries. Data-driven prediction methods and applications have recently become very popular. One of the novel method of building prediction models is to use data-driven methods and knowledge discovery on the web contents. This includes the news and media as well as social networks contents. This method uses advanced technologies of big data, machine learning, deep learning and intelligent optimization for finding patterns in big data to build prediction models. This article presents a state of the art survey on the latest technological advancements, novel methods, and applications in developing prediction models.

Machine learning (ML) and large-scale big data are key factors in developing an accurate prediction model for cardiovascular disease (CVD). Although the CVD risk often depends on the race and ethnicity, most previous studies considered only US or European populations for the CVD risk prediction. In this work, to complement previous researches, we analyzed the Korean National Health Insurance Service-National Health Sample Cohort (KNHSC) data and studied the characteristics of ML and big data for predicting the CVD risk. More specifically, we assessed the effectiveness of various ML methods in predicting the 2-year and 10-year risk of CVD such as atrial fibrillation, coronary artery disease, heart failure, and strokes. To develop prediction models, we considered the usual medical examination data, questionnaire survey results, comorbidities, and past medication information available in the KNHSC data. We developed various ML-based prediction models using logistic regression, deep neural networks, random forests, and LightGBM, and validated them using various metrics such as receiver operating characteristic curves, precision-recall curves, sensitivity, specificity, and F1 score. Experimental results showed that all ML models outperformed the baseline method derived from the ACC/AHA guidelines for estimating the 10-year CVD risk, demonstrating the usefulness of ML methods. In addition, in our analysis, whether we included the past medication information as a feature or not, the prediction accuracy of all ML models was comparable to each other. Since the use of medications by the physicians provided important information on the occurrence of diseases, when we included it as a feature, all prediction models achieved a slightly higher prediction accuracy.

In recent years, the discovery and utilization of the millimeter-wave for the next generation of mobile communications have fuelled the demand to develop site specific and more accurate wireless channel models. Due to the dynamic and high interactive nature of the millimeter waves, accurate models which takes into consideration the effects of the interacting objects in the propagation environments are needed. This paper presents a development of a machine learning (ML) based prediction model for large scale channel parameter (LSCP) estimation by analyzing the interaction of the reflected multipath rays information with propagation environments. This is to capture the shape of the changing propagation environments more accurately. Utilizing the robustness of ML, a dynamic LSCP prediction model can be realized. The input parameters to the prediction model are transmitter (TX) and receiver (RX) positional coordinates, and the reflected rays’ information such as the delay, angle of arrival, angle of departure, elevation angle of arrival, elevation angle of departure, and power gain. The proposed model was implemented using Random Forests. Ray tracing (RT) simulation was performed to calculate the input dataset of the LSCPs, and also the input datasets of the reflected rays. Cross validation was utilized to validate the model.

Incorporating crumb rubber in slag-based geopolymer: Experimental work and predictive modelling

Accurate prediction of energy consumption in district heating systems plays an important role in supporting effective and clean energy production and distribution in dense urban areas. Predictive models are needed for flexible and cost-effective operation of energy production and usage, e.g., using peak shaving or load shifting to compensate for heat losses in the pipeline. This helps to avoid exceedance of power plant capacity. The purpose of this study is to automate the process of building machine learning (ML) models to solve a short-term power demand prediction problem. The dataset contains a district heating network’s measured hourly power consumption and ambient temperature for 415 days. In this paper, we propose a hybrid evolutionary-based algorithm, named GA-SHADE, for the simultaneous optimization of ML models and feature selection. The GA-SHADE algorithm is a hybrid algorithm consisting of a Genetic Algorithm (GA) and success-history-based parameter adaptation for differential evolution (SHADE). The results of the numerical experiments show that the proposed GA-SHADE algorithm allows the identification of simplified ML models with good prediction performance in terms of the optimized feature subset and model hyperparameters. The main contributions of the study are (1) using the proposed GA-SHADE, ML models with varying numbers of features and performance are obtained. (2) The proposed GA-SHADE algorithm self-adapts during operation and has only one control parameter. There is no fine-tuning required before execution. (3) Due to the evolutionary nature of the algorithm, it is not sensitive to the number of features and hyperparameters to be optimized in ML models. In conclusion, this study confirms that each optimized ML model uses a unique set and number of features. Out of the six ML models considered, SVR and NN are better candidates and have demonstrated the best performance across several metrics. All numerical experiments were compared against the measurements and proven by the standard statistical tests.

Three machine learning classifiers: random forest, decision tree and support vector machine were used to build predictive models of an anti-mycobacterial ChEMBL database and evaluated for their predictive capability. Before the development of predictive models, data pre-processing was carried out to fix the class imbalance problem by applying cost-sensitive classifier, and filtration of data instance by supervised synthetic minority oversampling technique (SMOTE), spread subsample and resample method. The statistical evaluation indicated that random forest model was the best model as it showed the best accuracy 93.83%, specificity 90.5%, receiver operating characteristic (ROC) 0.984, MCC 0.772 and kappa statistics 0.768 in comparison to other models whereas LibSVM showed the highest sensitivity 94.4% compared with others. Additionally, toxicity predictive models based on SingleCellcall DSSTox carcinogenicity database (AID1189) was developed which resulted in random forest model as the best model. The deployment of both RF predictive models on two unknown datasets resulted in 1317 compounds out of 1554 approved drugs and 2234 compounds out of 18,746 ChEMBL anti-malarial dataset as non-toxic and anti-mycobacterial compounds. Thus machine learning models present highly efficient methods to find out novel hit anti-mycobacterial compounds. We suggest that such machine learning techniques could be very useful to screen drug candidates not only for tuberculosis but also for other diseases.

Background: Cardiac amyloidosis (CA) in conjunction with atrial fibrillation (AF) presents unique management challenges. CHA2DS2VASc score in these patients is believed to underestimate the risk of ischemic stroke, necessitating a better predictive model in these patients. Methods: Data was obtained from the National Readmission Database (NRD). Outcomes between CA-AF and no-CA-AF were compared using multivariate regression analysis to calculate adjusted odds ratios (aOR). AutoScore; an interpretable machine learning framework, was used to develop a stroke risk prediction model, the predictive accuracy of which was evaluated with an area under the curve (AUC) using the receiver operating characteristic analysis. Results: A total of 11,860,804 (CA-AF 22,687 [0.19%] and no-CA-AF 11,838,117) patients were identified from 2015-2019. The adjusted odds of mortality (aOR 1.41 and 1.29), stroke (aOR 1.78 and 1.74), non-intracranial hemorrhage (aOR 2.10 and aOR 1.85), and intracranial hemorrhage (aOR 14.4 and aOR 4.26) were significantly higher in CA-AF compared with non-CA-AF at both index admission and 30-days, respectively. The CHA 2 DS 2 VASc score had a poor discriminative accuracy for stroke at 30-days in CA-AF (AUC 49%, 95%CI 47%-51%, p=0.54). The machine learning autoscore integrative model revealed that the predictive ability of our newly proposed E-CHADS score (end-stage renal disease (ESRD), congestive heart failure, hypertension, active cancer, dementia, and diabetes mellitus) for 30-day risk of ischemic stroke in CA-AF was excellent (for a cutoff of 52 points random forest score) with an AUC of 80% (95%CI 74%-86%) Conclusion: Cardiac amyloidosis carries a high risk of ischemic stroke that is not accurately predicted by the CHA 2 DS 2 VASc score. Our proposed model (E-CHADS) identifies 3 new variables (ESRD, dementia, and cancer) that have higher discriminative accuracy for ischemic stroke in these patients.

Background and ObjectivesWe sought to summarize the study design, modelling strategies, and performance measures reported in studies on clinical prediction models developed using machine learning techniques. MethodsWe search PubMed for articles published between 01/01/2018 and 31/12/2019, describing the development or the development with external validation of a multivariable prediction model using any supervised machine learning technique. No restrictions were made based on study design, data source, or predicted patient-related health outcomes. ResultsWe included 152 studies, 58 (38.2% [95% CI 30.8–46.1]) were diagnostic and 94 (61.8% [95% CI 53.9–69.2]) prognostic studies. Most studies reported only the development of prediction models (n = 133, 87.5% [95% CI 81.3–91.8]), focused on binary outcomes (n = 131, 86.2% [95% CI 79.8–90.8), and did not report a sample size calculation (n = 125, 82.2% [95% CI 75.4–87.5]). The most common algorithms used were support vector machine (n = 86/522, 16.5% [95% CI 13.5–19.9]) and random forest (n = 73/522, 14% [95% CI 11.3–17.2]). Values for area under the Receiver Operating Characteristic curve ranged from 0.45 to 1.00. Calibration metrics were often missed (n = 494/522, 94.6% [95% CI 92.4–96.3]). ConclusionOur review revealed that focus is required on handling of missing values, methods for internal validation, and reporting of calibration to improve the methodological conduct of studies on machine learning–based prediction models. Systematic review registrationPROSPERO, CRD42019161764.

In recent years, machine learning models for clinical prediction have become increasingly prevalent in the neurosurgical literature. However, little is known about the quality of these models, and their translation to clinical care has been limited. The aim of this systematic review was to empirically determine the adherence of machine learning models in neurosurgery with standard reporting guidelines specific to clinical prediction models. Studies describing the development or validation of machine learning predictive models published between January 1, 2020, and January 10, 2023, across five neurosurgery journals (Journal of Neurosurgery, Journal of Neurosurgery: Spine, Journal of Neurosurgery: Pediatrics, Neurosurgery, and World Neurosurgery) were included. Studies where the TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) guidelines were not applicable, radiomic studies, and natural language processing studies were excluded. Forty-seven studies featuring a machine learning-based predictive model in neurosurgery were included. The majority (53%) of studies were single-center studies, and only 15% of studies externally validated the model in an independent cohort of patients. The median compliance across all 47 studies was 82.1% (IQR 75.9%-85.7%). Giving details of treatment (n = 17 [36%]), including the number of patients with missing data (n = 11 [23%]), and explaining the use of the prediction model (n = 23 [49%]) were identified as the TRIPOD criteria with the lowest rates of compliance. Improved adherence to TRIPOD guidelines will increase transparency in neurosurgical machine learning predictive models and streamline their translation into clinical care.