Non‐intrusive reduced‐order modeling using convolutional autoencoders

Conventional reduced order modeling techniques such as the reduced basis (RB) method (relying, e.g., on proper orthogonal decomposition (POD)) may incur in severe limitations when dealing with nonlinear time-dependent parametrized PDEs, as these are strongly anchored to the assumption of modal linear superimposition they are based on. For problems featuring coherent structures that propagate over time such as transport, wave, or convection-dominated phenomena, the RB method may yield inefficient reduced order models (ROMs) when very high levels of accuracy are required. To overcome this limitation, in this work, we propose a new nonlinear approach to set ROMs by exploiting deep learning (DL) algorithms. In the resulting nonlinear ROM, which we refer to as DL-ROM, both the nonlinear trial manifold (corresponding to the set of basis functions in a linear ROM) as well as the nonlinear reduced dynamics (corresponding to the projection stage in a linear ROM) are learned in a non-intrusive way by relying on DL algorithms; the latter are trained on a set of full order model (FOM) solutions obtained for different parameter values. We show how to construct a DL-ROM for both linear and nonlinear time-dependent parametrized PDEs. Moreover, we assess its accuracy and efficiency on different parametrized PDE problems. Numerical results indicate that DL-ROMs whose dimension is equal to the intrinsic dimensionality of the PDE solutions manifold are able to efficiently approximate the solution of parametrized PDEs, especially in cases for which a huge number of POD modes would have been necessary to achieve the same degree of accuracy.

Model order reduction for large-scale structures with local nonlinearities

POD reduced-order unstructured mesh modeling applied to 2D and 3D fluid flow

Common trends in model reduction of large nonlinear finite element (FE)-discretized systems involve Galerkin projection of the governing equations onto a low-dimensional linear subspace. Though this reduces the number of unknowns in the system, the computational cost for obtaining the reduced solution could still be high due to the prohibitive computational costs involved in the evaluation of nonlinear terms. Hyper-reduction methods are then used for fast approximation of these nonlinear terms. In the finite element context, the energy conserving sampling and weighing (ECSW) method has emerged as an effective tool for hyper-reduction of Galerkin-projection-based reduced-order models (ROMs). More recent trends in model reduction involve the use of nonlinear manifolds, which involves projection onto the tangent space of the manifold. While there are many methods to identify such nonlinear manifolds, hyper-reduction techniques to accelerate computation in such ROMs are rare. In this work, we propose an extension to ECSW to allow for hyper-reduction using nonlinear mappings, while retaining its desirable stability and structure-preserving properties. As a proof of concept, the proposed hyper-reduction technique is demonstrated over models of a flat plate and a realistic wing structure, whose dynamics have been shown to evolve over a nonlinear (quadratic) manifold. An online speed-up of over one thousand times relative to the full system has been obtained for the wing structure using the proposed method, which is higher than its linear counterpart using the ECSW.

Adaptive non-intrusive reduced order modeling for compressible flows

Study of non-intrusive model order reduction of neutron transport problems

Chlorophyll content is one of the essential parameters to assess the growth process of the fruit trees. This present study developed a model for estimation of canopy averaged chlorophyll content (CACC) of pear trees using the convolutional autoencoder (CAE) features of hyperspectral (HS) data. This study also demonstrated the inspection of anomaly among the trees by employing multidimensional scaling on the CAE features and detected outlier trees prior to fit nonlinear regression models. These outlier trees were excluded from the further experiments that helped in improving the prediction performance of CACC. Gaussian process regression (GPR) and support vector regression (SVR) techniques were investigated as nonlinear regression models and used for prediction of CACC. The CAE features were proven to be providing better prediction of CACC when compared with the direct use of HS bands or vegetation indices as predictors. The CACC prediction performance was improved with the exclusion of the outlier trees during training of the regression models. It was evident from the experiments that GPR could predict the CACC with better accuracy compared to SVR. In addition, the reliability of the tree canopy masks, which were utilized for averaging the features' values for a particular tree, was also evaluated.

Non-intrusive reduced order modeling of nonlinear problems using neural networks

In this thesis we analyze and develop methods based on model order reduction (MOR) for the solution of optimization problems constrained by time-dependent partial differential equations (PDEs). The methods combine a direct solution approach with model reduction via proper orthogonal decomposition (POD) and the discrete empirical interpolation method (DEIM). The reduced-order models (ROMs) are used to approximate the high-dimensional dynamic systems originating from a spatial discretization of a PDE. However, when used in an optimization algorithm, conventional POD/DEIM ROMs often lack the ability to give adequate approximations of the gradient. We propose methods for a suitable enhancement of the ROMs for the optimization purpose which are based on the inclusion of derivative information in the POD and DEIM subspaces. We distinguish two types of error between quantities evaluated with the high-dimensional model and its ROM approximation in dependency on the optimization variable q: The reconstruction error which is evaluated with the same q0 which is used constructing the ROM and the prediction error which assesses approximations at q with a ROM constructed at q0 different to q. The novel reconstruction results we present include estimates for solutions of the adjoint equation and the sensitivity equations as well as for the gradient of the objective function. Based on the estimates we explain how the POD and DEIM bases should be extended with either adjoint or sensitivity information. The enhanced ROMs allow control of the reconstruction error for the objective and its gradient up to machine precision. Moreover, we propose a POD prediction estimate for the objective of the optimization problem in a neighborhood of q where the ROM is constructed. In case of sensitivity-extended POD and DEIM bases we give an analogous result for solutions of the states. The derivative-extended ROMs are then used to develop adaptive algorithms for the solution of optimal control and parameter estimation problems which results in great runtime improvements for the optimization while ensuring high approximation quality of the solution of the original problem. For the parameter estimation case a novel a posteriori error estimate is proposed which assesses the quality of suboptimal solutions obtained with the ROM. A further fundamental contribution is a discussion of discretize-then-optimize (DTO) vs. optimize-then-discretize (OTD) approaches in the context of MOR for optimization. We analyze advantages and disadvantages of both approaches and discuss to which extent our methods exhibit properties of either strategy. We also give examples of representative optimization problems in which standard POD/DEIM ROMs show an inacceptable behavior and can be successfully solved by derivative-extended ROMs. We have further implemented the developed methods emphasizing an efficient realization which is important for the investigation of the MOR potential. We showcase the practical performance of the proposed algorithms and the superiority of derivative-extended over conventional ROMs on two academic and one industry-relevant application which exhibit a variety of challenges for the model reduction approach in optimization.

This report summarizes fiscal year (FY) 2017 progress towards developing and implementing within the SPARC in-house finite volume flow solver advanced fluid reduced order models (ROMs) for compressible captive-carriage flow problems of interest to Sandia National Laboratories for the design and qualification of nuclear weapons components. The proposed projection-based model order reduction (MOR) approach, known as the Proper Orthogonal Decomposition (POD)/Least- Squares Petrov-Galerkin (LSPG) method, can substantially reduce the CPU-time requirement for these simulations, thereby enabling advanced analyses such as uncertainty quantification and de- sign optimization. Following a description of the project objectives and FY17 targets, we overview briefly the POD/LSPG approach to model reduction implemented within SPARC . We then study the viability of these ROMs for long-time predictive simulations in the context of a two-dimensional viscous laminar cavity problem, and describe some FY17 enhancements to the proposed model reduction methodology that led to ROMs with improved predictive capabilities. Also described in this report are some FY17 efforts pursued in parallel to the primary objective of determining whether the ROMs in SPARC are viable for the targeted application. These include the implemen- tation and verification of some higher-order finite volume discretization methods within SPARC (towards using the code to study the viability of ROMs on three-dimensional cavity problems) and a novel structure-preserving constrained POD/LSPG formulation that can improve the accuracy of projection-based reduced order models. We conclude the report by summarizing the key takeaways from our FY17 findings, and providing some perspectives for future work.

Inverse viscoelastic material characterization using POD reduced-order modeling in acoustic–structure interaction

We address the problem of model order reduction (MOR) of parametrized dynamical systems. Motivated by reduced basis (RB) methods for partial differential equations, we show that some characteristic components can be transferred to model reduction of parametrized linear dynamical systems. We assume an affine parameter dependence of the system components, which allows an offline/online decomposition and is the basis for efficient reduced simulation. Additionally, error control is possible by a posteriori error estimators for the state vector and output vector, based on residual analysis and primal-dual techniques. Experiments demonstrate the applicability of the reduced parametrized systems, the reliability of the error estimators and the runtime gain by the reduction technique. The a posteriori error estimation technique can straightforwardly be applied to all traditional projection-based reduction techniques of non-parametric and parametric linear systems, such as model reduction, balanced truncation, moment matching, proper orthogonal decomposition (POD) and so on.

The proper orthogonal decomposition (POD) and the discrete empirical interpolation method (DEIM) are applied to coupled Burgers equations to develop its reduced-order model (ROM) by the Galerkin projection. A calibrated POD ROM is developed in the current study through adding and multiplying a set of time-dependent random parameters to recover the loss of accuracy due to the truncation of the POD modes. Calibrating the ROM becomes essentially a high-dimensional statistical inverse inference problem. To reduce the computational effort, the polynomial chaos based ensemble Kalman filter (PC-EnKF) is adopted in this work. By using a sparse optimization algorithm, a sparse PC expansion is obtained to facilitate further calculation of statistical moments used in ensemble Kalman filter. We apply the well-defined calibrated POD ROM for the coupled Burgers equations with the Reynolds numberRe= 10 000. The numerical results show that the PC-EnKF method is efficient in reducing the uncertainty included in the initial guess of input parameters and feasible in correcting the behavior of the POD ROM. The study suggests that the PC-EnKF is quite general as a calibration tool for calibrating the POD ROM.

In this paper, a novel reduced order model based on a convolutional auto-encoder with self-attention (SACAE ROM) is proposed. The proposed model is a non-intrusive reduced order model, which uses a convolutional neural network and a long short-term memory network to extract temporal feature relationships from high-fidelity numerical solutions. The self-attention is introduced into the convolutional neural network to enhance the non-local information perception ability of the convolutional neural network and improve the feature extraction ability of the network. The model adopts a joint construction method, which overcomes the problem of propagating error in each process of the model. The model proposed in this paper has been verified on the problem of the flow around a cylinder. The experimental results indicate that the SACAE ROM has higher robustness and accuracy. Compared with the ROM based on a convolutional auto-encoder, the prediction error of the SACAE ROM is reduced by 42.9%. As with other ROMs based on deep neural networks, the SACAE ROM takes a long time to train. To solve this problem, the transfer and generalization ability of the model is studied in this paper. In the experiment, the flow velocity and spoiler of the flow around the cylinder were changed, and the training time of the transfer model was reduced by about 50% to 60%. This result demonstrates that the problem of too long training time can be solved by transfer learning.

The application of machine learning (ML) techniques for the control and development of digital twins for a fluidized bed reactors represents a significant advancement in process engineering. In this study, the integration of data-driven models trained using computational fluid dynamics (CFD) simulations, is explored for developing and optimizing the lab-scale fluidized bed reactor operations. By leveraging the collection of data generated from CFD simulations, data-driven algorithms, based on the Singular Value Decomposition (SVD) and Gaussian processes for regression, are trained to predict the gas-solid flow patterns under different operating condition. The data-driven models presented, serve as efficient reduced order model (ROM) surrogate for computationally expensive CFD simulations, enabling real-time predictions and control strategies for fluidized bed reactors, facilitating continuous monitoring, optimization, and predictive maintenance. Moreover, the ROM can effectively capture the complex relationships within the reactor system, with an overall error < 10% even without precise knowledge of the underlying physical phenomena. The synergistic combination of ML techniques and CFD simulations offers valuable insights into complex multiphase flow phenomena and reactor dynamics, leading to improved process control, energy efficiency, and overall performance of fluidized bed reactors. This approach holds great promise for accelerating innovation and sustainability in chemical and energy industries.