Automated optimization of numerical methods for partial differential equations

We study numerical methods for time-dependent partial differential equations describing wave propagation, primarily applied to problems in quantum dynamics governed by the time-dependent Schrodinger equation (TDSE). We consider both methods for spatial approximation and for time stepping. In most settings, numerical solution of the TDSE is more challenging than solving a hyperbolic wave equation. This is mainly because the dispersion relation of the TDSE makes it very sensitive to dispersion error, and infers a stringent time step restriction for standard explicit time stepping schemes. The TDSE is also often posed in high dimensions, where standard methods are intractable.The sensitivity to dispersion error makes spectral methods advantageous for the TDSE. We use spectral or pseudospectral methods in all except one of the included papers. In Paper III we improve and analyse the accuracy of the Fourier pseudospectral method applied to a problem with limited regularity, and in Paper V we construct a matrix-free spectral method for problems with non-trivial boundary conditions. Due to its stiffness, the TDSE is most often solved using exponential time integration. In this thesis we use exponential operator splitting and Krylov subspace methods. We rigorously prove convergence for force-gradient operator splitting methods in Paper IV. One way of making high-dimensional problems computationally tractable is low-rank approximation. In Paper VI we prove that a splitting method for dynamical low-rank approximation is robust to singular values in the approximation approaching zero, a situation which is difficult to handle since it implies strong curvature of the approximation space.

A new higher order finite element method for elliptic partial differential equations on a stationary smooth surface $\Gamma$ is introduced and analyzed. We assume that $\Gamma$ is characterized as the zero level of a level set function $\phi$ and only a finite element approximation $\phi_h$ (of degree $k \geq 1$) of $\phi$ is known. For the discretization of the partial differential equation, finite elements (of degree $m\geq 1$) on a piecewise linear approximation of $\Gamma$ are used. The discretization is lifted to $\Gamma_h$, which denotes the zero level of $\phi_h$, using a quasi-orthogonal coordinate system that is constructed by applying a gradient recovery technique to $\phi_h$. A complete discretization error analysis is presented in which the error is split into a geometric error, a quadrature error, and a finite element approximation error. The main result is a $H^1(\Gamma)$-error bound of the form $c(h^m + h^{k+1})$. Results of numerical experiments illustrate the higher order convergence of t...

An iterated pseudospectral method for delay partial differential equations

RETRACTED: Quartic spline collocation methods for elliptic partial differential equations

Over the last couple of decades the demand for high precision and enhanced performance of physical systems has been steadily increasing. This demand often results in miniaturization and complex design, thus increasing the need for complex nonlinear control methods. Some of the state of the art nonlinear methods are stymied by the requirement of full state information, model and parameter uncertainties, mathematical complexity, etc. For many scenarios it is nearly impossible to consider all the uncertainties during the design of a feedback controller. Additionally, while designing a modelbased nonlinear control there is no standard mechanism to incorporate performance measures. Some of the mentioned issues can be addressed by using online learning. Animals and humans have the ability to share, explore, act or respond, memorize the outcome and repeat the task to achieve a better outcome when they encounter the same or a similar scenario. This is called learning from interaction. One instance of this approach is reinforcement learning (RL). However, RL methods are hindered by the curse of dimensionality, non-interpretability and non-monotonic convergence of the learning algorithms. This can be attributed to the intrinsic characteristics of RL, as it is a modelfree approach and hence no standard mechanism exists to incorporate a priori model information. In this thesis, learning methods are proposed which explicitly use the available system knowledge. This can be seen as a new class of approaches that bridge model-based and model-free methods. These methods can address some of the hurdles mentioned earlier. For example, i) a prior system information can speed up the learning, ii) new control objectives can be achieved which otherwise would be extremely difficult to attain using only model-based methods, iii) physical meaning can be attributed to the learned controller. The developed approach is as follows: themodel of the given physical system is represented in the port-Hamiltonian (PH) form. For the system dynamics in PH form a passivity-based control (PBC) law is formulated, which often requires the solution to a set of partial differential equations (PDEs). Instead of finding an analytical solution, the PBC control law is parameterized using an unknown parameter vector. Then, by using a variation of the standard actor-critic learning algorithm, the unknown parameters can be learned online. Using the principles of stochastic approximation theory, a proof of convergence for the developed method is shown. The proposedmethods are evaluated for the stabilization and regulation ofmechanical and electro-mechanical systems. The simulation and experimental results show comparable learning curves. In the final part of the thesis a novel integral reinforcement learning approach is developed to solve for the optimal output tracking control problem for a set of linear heterogeneous multi-agent systems. Unlike existing methods, this approach does not need to solve either the output regulator equation or requires a p-copy of the leader’s dynamics in the agent’s control law. A detailed numerical evaluation has been conducted to show the feasibility of the developed method.

Tremendous progress has been achieved in image analysis and processing, and in particular, the use of partial differential equation (PDE) methods in image analysis has proliferated in recent years. However, PDE methods have seen little application to optical image-based structural damage detection for civil systems. This paper applies the Chan-Vese active contour model and its level set representation to detect concrete surface damage, in this case concrete cracks, in optical images. The detected cracks can be further characterized by extracted geometric quantities including their width, length and area of coverage, and associated with engineering design. With regard to crack width extraction, an approximate method is proposed, which relies on solving for a signed distance function. We test these methods by using synthetic images as well as real multi-temporal images from a laboratory experiment. This paper illustrates that using the proposed methods, cracks with complex topographical patterns can be successfully detected with sufficient accuracy.

Different numerical methods have been proposed for the solution of partial differential equations (PDE). Most of them are based on a variational principle which recasts the PDE into an equivalent integral equation. One of the most common principles is the Galerkin method, which has some specific disadvantages for some types of PDE. In this work an alternative variational principle, the least squares finite element method, will be tested with respect to its application for transient fluid-structure interaction problems. The accurracy of different formulations which were proposed for the Navier-Stokes equations in literature will be tested. In a next step these formulations will be coupled with a standard Galerkin approach for the structure. After that a new formulation for the linear equations of elastodynamics is developed and analysed with respect to its stability and accuracy. With this formulation it is possible do develop a pure least squares formulation for the strongly coupled fluid-structure problem. Finally the different formulations are tested with respect to their accuracy and efficiency.

Most instructors manage two opposing currents in classroom dynamics. The first is a desire to describe the subject in its broadest sense including all the subtleties and nuances. It gives the students a road map to the field, and if they can appreciate it at the time, it provides structure to a course that might otherwise appear to be a loosely connected series of topics. The second is the desire to ground the subject with concrete examples that students can grasp at the finest level. The examples link prerequisite knowledge with one of the new subjects and provide students with a foundation upon which to construct a deeper understanding. Few courses draw together so many disparate topics as numerical methods for partial differential equations (PDEs). When first exposed to finite differences for numerical PDEs, students will readily notice the regular structures in matrices, but few texts address these structures systematically and connect them directly to the analytical properties of the solution. There are many good examples, but it's hard to see the broader landscape. In this Education section article, “Functions of Difference Matrices Are Toeplitz Plus Hankel,” authors Gil Strang and Shev MacNamara fill in some of the big picture by presenting some new results and observations about the structure of difference matrices for approximating solutions to PDEs. A common approach in the analysis of numerical methods for PDEs is to discretize the spatial operator and leave time continuous so that one studies the semidiscrete system. The exact solution to these systems will be functions of the underlying difference matrices, hence the structure of functions of difference matrices is a natural point of interest for students, instructors, and researchers. Toeplitz matrices, having entries that are constant along diagonals, arise naturally in difference matrices. As the authors point out, this is to be expected because of the translation-invariance of the differentiation operator. In an unbounded domain, if initial condition $u_0$ leads to a solution $u$, a translation of $u_0$ will lead to a solution that is a translation of $u$. The authors discover that solutions to the semidiscrete systems are Toeplitz plus Hankel. The role of Hankel matrices, matrices that are constant along antidiagonals, is a little more subtle to explain. If one considers the role of boundary conditions for canonical problems like the heat or wave equations, one can see how this might arise. In the case of a single Neumann boundary condition for the heat equation, the classic method of images provides some insight. Shifting the initial condition toward the boundary shifts the image solution toward the boundary outside the domain, which shifts the influence of the image away from the boundary within the domain. Thus, shifting the initial condition has both a Toeplitz part from transition invariance and a Hankel part from the influence of the boundaries. The Toeplitz plus Hankel decomposition paves the way for exact solutions to semidiscrete approximations, allowing better error estimates and a deeper general understanding of spatial discretization. While it makes sense intuitively that functions of difference matrices should be Toeplitz plus Hankel, the authors enthusiastically explore the topic using a variety of methods and techniques from advanced undergraduate or introductory graduate mathematics including spectral decompositions, Riemann sums, and Fourier analysis. In other words, the destination is interesting, but the voyage is even more fun! This paper would serve as an excellent module for a numerical analysis course or an integrative seminar for undergraduates or graduate students interested in numerical methods.

: Complex physical phenomena involving chemically reacting systems or the transport of heat or fluids are often modeled by coupled systems of time-dependent, nonlinear partial differential equations. The difficulties in understanding the stability of the differential equation systems and in designing efficient, accurate numerical methods for their solution are widely recognized and were the focus of this research. We have worked on four general aspects of the analysis and numerical approximation of systems of partial differential equations. These areas of research are: (1) modeling aspects and stability analysis for nonlinear time-dependent partial differential equations; (2) use and analysis of finite element or finite difference methods to discretize coupled systems of nonlinear differential equations; (3) development of adaptive or local grid refinement capabilities to resolve local phenomena in large-scale applications; and (4) development of data structures, preconditioners, and efficient solution algorithms for large-scale problems on new computer architectures. Emphasis has been placed upon multiphase or multicomponent, transport-dominated flow processes with dynamic local phenomena. The research also involved a mix of analysis, algorithm development, and large-scale computation using newer computer architectures.

Some multiscale methods for partial differential equations by J. Xu and A. Zhou Terascale implicit methods for partial differential equations by D. E. Keyes Boundary conditions for fully implicit two-phase flow models by M. Peszynska, E. W. Jenkins, and M. F. Wheeler Phase-field models of solidification by G. B. McFadden Singularity formation in free-surface Stokes flows by Q. Nie, S. Tanveer, T. F. Dupont, and X. Li A note on cache memory methods for multigrid in three dimensions by C. C. Douglas and D. T. Thorne.

In this thesis we analyze and develop methods based on model order reduction (MOR) for the solution of optimization problems constrained by time-dependent partial differential equations (PDEs). The methods combine a direct solution approach with model reduction via proper orthogonal decomposition (POD) and the discrete empirical interpolation method (DEIM). The reduced-order models (ROMs) are used to approximate the high-dimensional dynamic systems originating from a spatial discretization of a PDE. However, when used in an optimization algorithm, conventional POD/DEIM ROMs often lack the ability to give adequate approximations of the gradient. We propose methods for a suitable enhancement of the ROMs for the optimization purpose which are based on the inclusion of derivative information in the POD and DEIM subspaces. We distinguish two types of error between quantities evaluated with the high-dimensional model and its ROM approximation in dependency on the optimization variable q: The reconstruction error which is evaluated with the same q0 which is used constructing the ROM and the prediction error which assesses approximations at q with a ROM constructed at q0 different to q. The novel reconstruction results we present include estimates for solutions of the adjoint equation and the sensitivity equations as well as for the gradient of the objective function. Based on the estimates we explain how the POD and DEIM bases should be extended with either adjoint or sensitivity information. The enhanced ROMs allow control of the reconstruction error for the objective and its gradient up to machine precision. Moreover, we propose a POD prediction estimate for the objective of the optimization problem in a neighborhood of q where the ROM is constructed. In case of sensitivity-extended POD and DEIM bases we give an analogous result for solutions of the states. The derivative-extended ROMs are then used to develop adaptive algorithms for the solution of optimal control and parameter estimation problems which results in great runtime improvements for the optimization while ensuring high approximation quality of the solution of the original problem. For the parameter estimation case a novel a posteriori error estimate is proposed which assesses the quality of suboptimal solutions obtained with the ROM. A further fundamental contribution is a discussion of discretize-then-optimize (DTO) vs. optimize-then-discretize (OTD) approaches in the context of MOR for optimization. We analyze advantages and disadvantages of both approaches and discuss to which extent our methods exhibit properties of either strategy. We also give examples of representative optimization problems in which standard POD/DEIM ROMs show an inacceptable behavior and can be successfully solved by derivative-extended ROMs. We have further implemented the developed methods emphasizing an efficient realization which is important for the investigation of the MOR potential. We showcase the practical performance of the proposed algorithms and the superiority of derivative-extended over conventional ROMs on two academic and one industry-relevant application which exhibit a variety of challenges for the model reduction approach in optimization.

HiFlow3 Version 2.0 continues the path as a multi-purpose finite element software, which provides powerful tools for efficient and accurate solution of a wide range of problems modeled by partial differen- tial equations (PDEs). New features and functionalities, which allow to run numerical simulations with more advanced solution algorithms and discretizations in comparison to previous releases of HiFlow3, have been implemented. These comprise Uncertainty Quantification (UQ), energy-efficient multigrid techniques, Schur complement solvers for saddle-point problems, extended third-party library support, and adaptive local mesh refinement in a parallel computing environment. Furthermore, HiFlow3 has been successfully integrated into advanced and state-of-the-art simulation environments by means of the Medical Simulation Markup Language (MSML), for example.The presented new algorithms and features as well as general under-the-hood improvements have enabled excellent and relevant research activities in the fields of both medical engineering and meteorol- ogy and environmental sciences. The described show cases demonstrate the potential and advantages, which HiFlow3 can offer in performing a numerical simulation by means of finite element methods. Especially, the high performance computing capabilities of HiFlow3 – not only in the mentioned fields of applications, but also in general – have been significantly improved in Version 2.0.

Preface. Part 1: Modeling of Complex Systems with Asymptotic-Enhanced Numerical Methods. Part 2: Asymptotic-Induced Domain Decomposition Methods. Part 3: Multiple-Scale Problems in Scientific Computing. Part 4: Applied and Asymptotic Analysis. Part 5: Symbolic Manipulation Tools for Asymptotic Analysis. Part 6: Numerical Methods, Algorithms, and Architectures. Index.

A recently proposed mathematical model for turor angiogenesis consists of a coupled system of ordinary and partial differential equations, some of which are strongly convection dominated diffusion equations. A numerical method based on the use of characterics, which is mass conserving, can be used to effectively handle this feature. The model and the numerical method are presented.

Comparison method of partial functional differential equations and its application