Physics-Aware Learning for Environmental Systems: Surrogate Modeling and Constraint-Driven Machine Intelligence

<div class="section abstract"><div class="htmlview paragraph">In recent years, diesel engine emissions regulations have been strengthened worldwide, necessitating the evaluation of regulatory values under transient conditions. Consequently, the need to assess transient states in the development of diesel engines has increased significantly. The evaluation using MBD (Model Based Development) is considered a promising method for achieving both low fuel consumption and simultaneous reduction of NOx and soot emissions. However, the mechanism of soot formation is complex, making it challenging to model mathematically directly. In this paper, hybrid machine learning approaches combining a physical model and a machine learning model are used to validate the prediction of soot emissions under transient conditions in a diesel engine with an EGR system. Various parameters such as fuel consumption and emissions predicted by the physical model are compared with measurements to validate the accuracy of the physical model. The prediction of soot emissions by the physical model is based on the Hiroyasu model. From these results, it is demonstrated that the physical model has sufficient accuracy to be used in hybrid machine learning approaches. However, it is shown that the physical model is inadequate as a prediction approach for soot emissions. Gaussian Process Regression (GPR), Support Vector Regression (SVR), Random Forest (RF), and Gradient Boosting Decision Tree (GBDT) are used to develop the machine learning models, and each model is trained on data under steady-state conditions. The prediction accuracy of each model and the physical model is compared and validated. The results show that the hybrid machine learning approaches have higher predictive accuracy than the physical model for soot emissions predictions in both steady-state and transient conditions. The GPR model with the highest prediction accuracy shows a test R<sup>2</sup> of 0.87 under steady-state conditions and relative errors with the measured values of less than 10% for both Non-load Transient Cycle (NRTC) and Low Load Cycle (LLC), which are engine test cycles.</div></div>

The sea surface temperature (SST) is an environmental indicator closely related to climate, weather, and atmospheric events worldwide. Its forecasting is essential for supporting the decision of governments and environmental organizations. Literature has shown that single machine learning (ML) models are generally more accurate than traditional statistical models for SST time series modeling. However, the parameters tuning of these ML models is a challenging task, mainly when complex phenomena, such as SST forecasting, are addressed. Issues related to misspecification, overfitting, or underfitting of the ML models can lead to underperforming forecasts. This work proposes using hybrid systems (HS) that combine (ML) models using residual forecasting as an alternative to enhance the performance of SST forecasting. In this context, two types of combinations are evaluated using two ML models: support vector regression (SVR) and long short-term memory (LSTM). The experimental evaluation was performed on three datasets from different regions of the Atlantic Ocean using three well-known measures: mean square error (MSE), mean absolute percentage error (MAPE), and mean absolute error (MAE). The best HS based on SVR improved the MSE value for each analyzed series by 82.26%, 98.93%, and 65.03% compared to its respective single model. The HS employing the LSTM improved 92.15%, 98.69%, and 32.41% concerning the single LSTM model. Compared to literature approaches, at least one version of HS attained higher accuracy than statistical and ML models in all study cases. In particular, the nonlinear combination of the ML models obtained the best performance among the proposed HS versions.

Stream temperature (Ts) is an important water quality parameter that affects ecosystem health and human water use for beneficial purposes. Accurate Ts predictions at different spatial and temporal scales can inform water management decisions that account for the effects of changing climate and extreme events. In particular, widespread predictions of Ts in unmonitored stream reaches can enable decision makers to be responsive to changes caused by unforeseen disturbances. In this study, we demonstrate the use of classical machine learning (ML) models, support vector regression and gradient boosted trees (XGBoost), for monthly Ts predictions in 78 pristine and human-impacted catchments of the Mid-Atlantic and Pacific Northwest hydrologic regions spanning different geologies, climate, and land use. The ML models were trained using long-term monitoring data from 1980–2020 for three scenarios: (1) temporal predictions at a single site, (2) temporal predictions for multiple sites within a region, and (3) spatiotemporal predictions in unmonitored basins (PUB). In the first two scenarios, the ML models predicted Ts with median root mean squared errors (RMSE) of 0.69–0.84 °C and 0.92–1.02 °C across different model types for the temporal predictions at single and multiple sites respectively. For the PUB scenario, we used a bootstrap aggregation approach using models trained with different subsets of data, for which an ensemble XGBoost implementation outperformed all other modeling configurations (median RMSE 0.62 °C).The ML models improved median monthly Ts estimates compared to baseline statistical multi-linear regression models by 15–48% depending on the site and scenario. Air temperature was found to be the primary driver of monthly Ts for all sites, with secondary influence of month of the year (seasonality) and solar radiation, while discharge was a significant predictor at only 10 sites. The predictive performance of the ML models was robust to configuration changes in model setup and inputs, but was influenced by the distance to the nearest dam with RMSE <1 °C at sites situated greater than 16 and 44 km from a dam for the temporal single site and regional scenarios, and over 1.4 km from a dam for the PUB scenario. Our results show that classical ML models with solely meteorological inputs can be used for spatial and temporal predictions of monthly Ts in pristine and managed basins with reasonable (<1 °C) accuracy for most locations.

This study highlights drought forecasting for understanding the semi-arid area in India, where drought phenomena play vital role in the irrigation, drinking water supplies, and sustaining the ecological with economic balance for every nation. Therefore, drought forecasting is important for the future drought planning based on the machine learning (ML) models. Hence, The Standardized Precipitation Index (SPI) at 3- and 6-month periods have been selected and used for future drought forecasting scenarios in area. The combinations of ten inputs SPI-1- and SPI-10 were used for predicting modeling for SPI-3 and SPI-6 timescales, that modeling developed based on the historical SPI datasets from 1989 to 2019 years. The SPI-3 and SPI-6 maximum and minimum values are shown SPI-3 (2.03 and -5.522) and SPI-6 (1.94 and -6.93). The SPI is a popular method for estimating the drought analysis and has been used everywhere at global level. The developed models have been compared with each other, with the best combination of input variables selected using subset regression models and sensitivity studies. After that, the active input parameters were used for forecasting of SPI-3 and SPI-6 values to understanding of drought in semi-arid area. The finest input variables combination have been used in the Ml models and established the novel five models such as robust linear regression, bagged trees, boosted trees, support vector regression (SVM-Linear), and Matern Gaussian Process Regression (Matern GPR) models. Such kind of models first time has been applied for the forecasting of future drought conditions. Whole models were fine and improved modeling by using hyperparameters tuning, bagging, and boosting models. Entire ML models’ accuracy was compared using different statistical metrics. Compared with five ML models accuracy, we have found that the Matern GPR model better accuracy than other ML models. The best model accuracy is R2 = 0.95 and 0.93, RMSE, MSE, MAE, MARE, and NSE values, respectively, for predicting SPI-3 and SPI-6 values in the area. Therefore, the Matern GPR model was identified as the finest ML algorithm for predicting SPI-3 and SPI-6 associated with other algorithms. This research demonstrates the Matern GPR model's efficacy in predicting multiscale SPI-3 and SPI-6 under climate variations. It can be helpful in soil and water resource conservation planning and management and understanding droughts in the entire basin areas of the country India.

The application of support vector regression and virtual sample generation technique in the optimization design of transonic compressor

A long-standing problem at the frontier of biomechanical studies is to develop fast methods capable of estimating material properties from clinical data. In this paper, we have studied three surrogate models based on machine learning (ML) methods for fast parameter estimation of left ventricular (LV) myocardium. We use three ML methods named K-nearest neighbour (KNN), XGBoost and multi-layer perceptron (MLP) to emulate the relationships between pressure and volume strains during the diastolic filling. Firstly, to train the surrogate models, a forward finite-element simulator of LV diastolic filling is used. Then the training data are projected in a low-dimensional parametrized space. Next, three ML models are trained to learn the relationships of pressure–volume and pressure–strain. Finally, an inverse parameter estimation problem is formulated by using those trained surrogate models. Our results show that the three ML models can learn the relationships of pressure–volume and pressure–strain very well, and the parameter inference using the surrogate models can be carried out in minutes. Estimated parameters from both the XGBoost and MLP models have much less uncertainties compared with the KNN model. Our results further suggest that the XGBoost model is better for predicting the LV diastolic dynamics and estimating passive parameters than other two surrogate models. Further studies are warranted to investigate how XGBoost can be used for emulating cardiac pump function in a multi-physics and multi-scale framework.

Segregation of granular materials is a critical challenge in many industries, often aimed at being controlled or minimised. The discrete element method (DEM) offers valuable insights into this phenomenon. However, calibrating DEM models is a crucial, albeit time-consuming, step. Recently, using machine learning (ML)-based surrogate models (SMs) in the calibration process has emerged as a promising solution. Nevertheless, developing such SMs is challenging due to the high number of DEM simulations required for training. Additionally, choosing a suitable ML model is not trivial. This study aims to develop SMs that effectively link particle-particle and particle-wall DEM interaction parameters to segregation of a multi-component mixture. We evaluate several ML models, ranging from artificial neural networks to ensemble learning, that are trained on a very cost-effective dataset, employing Bayesian optimisation with cross-validation to tune their hyperparameters. Next, we introduce a novel transfer learning (TL)-based approach that leverages knowledge from a few scenarios to handle new “unseen” ones. This method enables the construction of adaptive SMs for unseen scenarios, such as a new initial configuration (IC) of granular mixtures, without the need for a full-sized dataset. Our findings indicate that Gaussian process regression (GPR) efficiently builds accurate SMs on a very small dataset. We also demonstrate that only a few samples are required to build an accurate SM for the unseen IC, which significantly reduces the data preparation burden. By incorporating one and five samples from unseen scenarios to update the TL-GPR-based surrogate model, the SM’s performance (based on ) on unseen scenarios improves by 17 and 47%, respectively. The insights and methodology presented in this study will facilitate and accelerate the development of accurate SMs for DEM calibration, assisting in developing reliable DEM models in a shorter timeframe.

The prediction of wind energy generation is important to enhance the performance and dependability of renewable energy systems due to the rising demand for wind-generated electricity and advancements in wind energy technology competitiveness. This study leverages advanced machine learning (ML) and some other statistical and deep learning based time series forecasting models to enhance the accuracy of wind energy predictions. This comprehensive analysis includes nine ML models—Linear Regression, Random Forests (RF), Gradient Boosting Machines (GBM), Support Vector Machines (SVM), K-Nearest Neighbors (KNN), AdaBoost, XGBoost, Support Vector Regression (SVR), and Neural Networks—as well as Four time-series forecasting models—ARIMA, Temporal Convolutional Networks (TCNs), Long Short-Term Memory (LSTM) networks and GRU. Each ML model underwent rigorous cross-validation to ensure optimal performance. The assessment criteria utilized here comprised the Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and the R² Score. It was found that among the nine ML models, Random Forests, GBM and KNN consistently provided superior accuracy and robustness, making them the top choices for wind energy prediction whereas the performance of linear regression, SVM and SVR were very poor for the considered dataset. From the experiment, Random Forest, GBM, and KNN showed the best performance with low MSE values of 0.77, 1.95, and 1.51 respectively, while other models had MSEs above 7.5, with AdaBoost reaching 30. Their RMSEs (0.88, 1.40, 1.23) and MAEs (0.093, 0.73, 0.10) also indicate strong predictive accuracy compared to the rest.In this paper, time series forecasting, TCNs, LSTM and GRU networks showed strong capabilities in capturing temporal dependencies and trends within the wind energy data. Visualization techniques were employed to compare model performances comprehensively, providing clear insights into their predictive power. Therefore, this present study offers a robust framework for researchers and practitioners aiming to leverage machine learning and time series forecasting in the realm of renewable energy prediction.

Computational Fluid Dynamics (CFD) is regarded as an important tool for analyzing the flow field, thermal environment, and air quality around the built environment. However, for built environment applications, the high computational cost of CFD hinders large-scale scenario simulation and efficient design optimization. In the field of built environment research, surrogate modeling has become a key technology to connect the needs of high-fidelity CFD simulation and rapid prediction, whereas the low-dimensional nature of traditional surrogate models is unable to match the physical complexity and prediction needs of built flow fields. Therefore, combining machine learning (ML) with CFD to predict flow fields in built environments offers a promising way to increase simulation speed while maintaining reasonable accuracy. This review briefly reviews traditional surrogate models and focuses on ML-based surrogate models, especially the specific application of neural network architectures in rapidly predicting flow fields in the built environment. The review indicates that ML accelerates the three core aspects of CFD, namely mesh preprocessing, numerical solving, and post-processing visualization, in order to achieve efficient coupled CFD simulation. Although ML surrogate models still face challenges such as data availability, multi-physics field coupling, and generalization capability, the emergence of physical information-driven data enhancement techniques effectively alleviates the above problems. Meanwhile, the integration of traditional methods with ML can further enhance the comprehensive performance of surrogate models. Notably, the online ministry of trained ML models using transfer learning strategies deserves further research. These advances will provide an important basis for advancing efficient and accurate operational solutions in sustainable building design and operation.

A numerical comparison of simplified Galerkin and machine learning reduced order models for vaginal deformations

A simple chemical surrogate fuel model may not be able to fully reproduce the chemical behavior in real fuel combustion. A structure–chemical reactivity relationship at the molecular level is believed to be useful in tuning the chemical composition of reported surrogate fuel models. This work proposes an approach to predict a component fraction of a RP-3 surrogate fuel model with a combined method of ReaxFF molecular dynamics (MD) simulations and machine learning. There are four major steps to get a refined surrogate fuel model on the basis of parent RP-3 fuel models with two, three, or four components. Step 1 helps to prepare chemical reactivity data as the input of machine learning. The chemical reactivity data are described by the oxidation reactions in terms of dynamic species concentration with time that were prepared with ReaxFF MD simulations for each derived fuel model generated from the RP-3 parent fuel model by randomly changing its component fraction. Step 2 fits a component fraction prediction model between a single surrogate component fraction and its chemical reactivity data among the common reaction space in oxidation of possible RP-3 surrogates and a 45-component RP-3 fuel with the machine-learning method. The best performance model of LightGBM was selected among the machine-learning models of linear regression, support vector regression, and LightGBM based on the training error evaluation. Each refined component fraction of RP-3 surrogate models was predicted one by one using the trained LightGBM model with the input of dynamic species concentration data of the 45-component RP-3 fuel model better representing real RP-3 fuel. Step 3 helps to select one refined surrogate model among the predicted surrogate models with two, three, or four components by comparing the chemical reactivity deviation of important species with additional ReaxFF MD simulations under the conditions of heat-up, isothermal oxidation, and isothermal pyrolysis. The optimal model of the refined three-component RP-3 surrogate is validated in step 4 where its predicted ignition delay time is found to be closer to the reported experimental data of the real RP-3 fuel than to its parent surrogate fuel model. This work suggests that the proposed computational approach by combining ReaxFF MD simulations and machine learning should be potentially useful in search for refined RP-3 chemical surrogate fuel models directly on the basis of chemical reactivity data in oxidation at the molecular level.

Inverse scattering problems can be seen as an optimization problem in which a cost function has to be minimized. The cost function measures the discrepancy between a physical model written as a function of the sought parameters and the measured data. Due to the high computational cost of physical models, metamodels (also known as surrogate models) have gradually appeared as alternatives. Polynomial chaos (PC) expansions are a type of metamodel based on an explicit spectral decomposition of the physical model, allowing for an exact computation of the gradient of the metamodel. The gradient of the metamodel is then used to improve the performance of the global algorithm Particle Swarm Optimization (PSO).

Two-dimensional depth-averaged (2DH) hydrodynamic models are often used to obtain insight in the consequences of a flood. However, the long computation times of these models do not allow for real-time flood forecasting. Surrogate models such as machine learning models or conceptual models promise much lower computation times while achieving reasonable accuracy. In this study, we develop and compare a machine learning model and a conceptual model for the prediction of overland flow propagation after a dike breach. We find a high accuracy of around 95% by the machine learning model, and a reasonable accuracy of around 70–80% by the conceptual model. We discuss trade-offs of each method regarding practical use in a real-time flood forecasting system and conclude that both machine learning and conceptual models have a growing capability to be applied during time-sensitive emergency situations.

An easy-to-implement methodology to develop accurate, fast and thermodynamically consistent surrogate machine learning (ML) models for multicomponent phase equilibria is proposed. The methodology is successfully applied to predict the vapour-liquid equilibrium (VLE) behavior of a mixture containing CO2, monoethanolamine (MEA), and water (H2O). The accuracy of the surrogate model predictions of VLE for this system is found to be satisfactory as the results provide an average absolute relative difference of 0.50% compared to the estimates obtained with a rigorous thermodynamic model (eNRTL + Peng-Robinson).It is further demonstrated that the integration of Gibbs phase rule and physical constraints into the development of the ML models is necessary, as it ensures that the models comply with fundamental thermodynamic relationships.Finally, it is shown that the speed of ML based surrogate models can be ~10 times faster than interpolation methods and ~1000 times faster than rigorous VLE calculations.

A data-driven surrogate model to rapidly predict microstructure morphology during physical vapor deposition