Deep mixed residual method for solving PDE-constrained optimization problems

Distributed control and constraint preconditioners

Interior point methods provide an attractive class of approaches for solving linear, quadratic and nonlinear programming problems, due to their excellent efficiency and wide applicability. In this paper, we consider PDE-constrained optimization problems with bound constraints on the state and control variables, and their representation on the discrete level as quadratic programming problems. To tackle complex problems and achieve high accuracy in the solution, one is required to solve matrix systems of huge scale resulting from Newton iteration, and hence fast and robust methods for these systems are required. We present preconditioned iterative techniques for solving a number of these problems using Krylov subspace methods, considering in what circumstances one may predict rapid convergence of the solvers in theory, as well as the solutions observed from practical computations.

Piecewise constant level set method for topology optimization of unilateral contact problems

Waveform-agile tracking performance of cognitive tracking radar is significantly affected by the target maneuverability and increasing radio frequency interference (RFI) of other in-band systems. Therefore, a joint spectrum management and waveform optimization (JSMWO) policy is proposed to improve the maneuvering target tracking performance while mitigating RFI. First, the JSMWO policy is mathematically formulated to simultaneously optimize the spectrum occupancy and waveform parameters by minimizing a JSMWO-oriented three-term objective function. Specifically, the first two terms consider the tradeoff between interference avoidance and bandwidth utilization, and the impact of current spectrum occupancy action on future multistep rewards. The last subreward term incorporates the criterion-specific conditional predictive Bayesian risk, waveform library limitation, and real-time spectrum constraint generated by spectrum occupancy action. Second, to resolve the underlying mixed-integer, nonlinear, nonconvex multivariable optimization problem, by proper coupling analysis, we develop a four-stage iterative method to decompose the original JSMWO policy into two subpolicies and tackle them sequentially, which encompasses the deep Q-learning (DQL) based spectrum occupancy optimization, particle swarm optimization (PSO) based waveform optimization, and feedback information-based online learning. Specifically, the dual-network architecture and experience replay mechanism equipped in DQL can effectively address the delay reward and future multistep effects during the spectrum occupancy subpolicy optimization. Waveform optimization subpolicy is subsequently reformulated as an equivalent constrained optimization problem, wherein the PSO can effectively optimize the waveform parameters through interparticle interactions and iterations while satisfying the spectrum constraint and waveform limitation. Simulation results demonstrate the effectiveness and superiority of the above-mentioned JSMWO policy.

We consider equality-constrained optimization problems, where a given solution may not satisfy any constraint qualification but satisfies the standard second-order sufficient condition for optimality. Based on local identification of the rank of the constraints degeneracy via the singular-value decomposition, we derive a modified primal-dual optimality system whose solution is locally unique, nondegenerate, and thus can be found by standard Newton-type techniques. Using identification of active constraints, we further extend our approach to mixed equality- and inequality-constrained problems, and to mathematical programs with complementarity constraints (MPCC). In particular, for MPCC we obtain a local algorithm with quadratic convergence under the second-order sufficient condition only, without any constraint qualifications, not even the special MPCC constraint qualifications.

In this paper, a recurrent neural network for both convex and nonconvex equality-constrained optimization problems is proposed, which makes use of a cost gradient projection onto the tangent space of the constraints. The proposed neural network constructs a generically nonfeasible trajectory, satisfying the constraints only as t --> infinity. Local convergence results are given that do not assume convexity of the optimization problem to be solved. Global convergence results are established for convex optimization problems. An exponential convergence rate is shown to hold both for the convex case and the nonconvex case. Numerical results indicate that the proposed method is efficient and accurate.

In this paper, a class of additive block triangular preconditioners are constructed for solving block two-by-two linear systems with symmetric positive (semi-)definite sub-matrices. Convergence analysis of the related splitting iteration method shows that it is almost unconditionally convergent and behaves problem independent with a convergence rate less than 0.5 under a practical parameter choice. Optimization of the preconditioned matrices, which have real and tight eigenvalue distributions, shows that it can result in an upper bound less than 2 for the condition number of the preconditioned matrices. Moreover, we also give a special consideration about the feasibility of the proposed preconditioner for solving more general problems with indefinite sub-matrices. Numerical experiments based on examples arising from complex symmetric linear systems and PDE-constrained optimization problems are presented to show the robustness and effectiveness of the proposed preconditioners compared with some other existing preconditioners.

In this article, we motivate, derive, and test effective preconditioners to be used with the Minres algorithm for solving a number of saddle point systems which arise in PDE-constrained optimization problems. We consider the distributed control problem involving the heat equation and the Neumann boundary control problem involving Poisson's equation and the heat equation. Crucial to the effectiveness of our preconditioners in each case is an effective approximation of the Schur complement of the matrix system. In each case, we state the problem being solved, propose the preconditioning approach, prove relevant eigenvalue bounds, and provide numerical results which demonstrate that our solvers are effective for a wide range of regularization parameter values, as well as mesh sizes and time-steps.

This paper deals with the solution of some multi-objective optimal control problems for several PDEs: linear and semilinear elliptic equations and stationary Navier-Stokes systems. Specifically, we look for Nash equilibria associated with standard cost functionals. For linear and semilinear elliptic equations, we prove the existence of equilibria and we deduce related optimality systems. For stationary Navier-Stokes equations, we prove the existence of Nash quasi-equilibria, i.e. solutions to the optimality system. In all cases, we present some iterative algorithms and, in some of them, we establish convergence results. For the existence and characterization of Nash quasi-equilibria in the Navier-Stokes case, we use the formalism of Dubovitskii and Milyutin. In this context, we also present a finite element approximation and we illustrate the techniques with numerical experiments.

In this work, we investigate the performance CutFEM as a high fidelity solver as well as we construct a competent and economical reduced order solver for PDE-constrained optimization problems in parametrized domains that live in a fixed background geometry and mesh. Its effectiveness and reliability will be assessed through its application for the numerical solution of quadratic optimization problems with elliptic equations as constraints, examining an archetypal case. The reduction strategy will be via Proper Orthogonal Decomposition of suitable FE snapshots, using an aggregated state and adjoint test space, while the efficiency of the offline-online decoupling will be ensured by means of Discrete Empirical Interpolation of the optimality system matrix and right-hand side, enabling thus a rapid resolution of the reduced order model for each new spatial configuration.

At present, large-scale sky surveys have obtained a large volume of stellar spectra. Analyzing these spectra by manual methods cannot meet the actual needs, so it is necessary to explore automatic classification methods of stellar spectra. In this study, a multitask residual neural network is proposed, which adopts the idea of multitask learning to process two tasks simultaneously by using the correlation between the luminosity class and spectral subtype of stars. Moreover, the structure of residual neural network can reduce the parameters in the model to improve the calculation efficiency. We applied the multitask residual neural network to the classification of LAMOST DR4 M-type star spectra, and achieved good results in both luminosity class classification and spectral subtype classification. We further compared the performance of multitask residual neural network in stellar classification with that of random forest (RF) and eXtreme Gradient Boosting (XGBoost). The performance of multitask residual neural network is better than the other two algorithms.

This paper gives details on how to obtain mathematically rigorous results for global unconstrained and equality constrained optimization problems, as well as for finding all roots of a nonlinear function within a box. When trying to produce mathematically rigorous results for such problems of global nature, the main issue is to mathematically verify that a certain sub-box cannot contain a solution to the problem, i.e. to discard boxes. The presented verification methods are based on mathematical theorems, the assumptions of which are verified using Algorithmic Differentiation and interval arithmetic. In contrast to traditional numerical algorithms, the main problem of verification methods is how to formulate those assumptions. We present mathematical and implementation details on how to obtain fast verification algorithms in pure Matlab/Octave code. The methods are implemented in INTLAB, the Matlab/Octave toolbox for Reliable Computing. Several examples together with executable code show advantages and weaknesses of the proposed methods. New results are included, however, the main goal is to introduce and give enough details to understand black-box Matlab/Octave routines to solve the mentioned problems. An outlook on current research on verification methods for large problems with several million variables and several tens of thousands of constraints based on conic programming is given as well. The latter can be regarded as an extension of interval arithmetic.

Bioluminescence tomography (BLT) can three-dimensionally and quantitatively resolve the molecular processes in small animals in vivo. In this paper, we propose a BLT reconstruction algorithm based on duality and variable splitting. By using duality and variable splitting to obtain a new equivalent constrained optimization problem and updating the primal variable as the Lagrangian multiplier in the dual augmented Lagrangian problem, the proposed method can obtain fast and stable source reconstruction even without the permissible source region and multispectral measurements. Numerical simulations on a mouse atlas and in vivo mouse experiments were conducted to validate the effectiveness and potential of the method.

A real-time EIT imaging system based on the split augmented Lagrangian shrinkage algorithm

Chinese calligraphy, as a writing art of Chinese characters, plays an important role in the inheritance of traditional culture. Compared with simple English fonts, the generation of Chinese calligraphy fonts is more challenging. This paper proposes Chinese calligraphy generation based on residual dense network. The generator network combines residual network structure and dense network structure, and uses local residual learning and global feature fusion to enhance feature reuse and information continuous transmission. The discriminator network adopts an autoencoder structure, and restores the similarity between the distributions by an approximate convergence strategy to achieve fast and stable training. At the same time, the structural similarity loss is introduced to quantify the similarity between the generated font image and the real font image. Experiments show that the proposed method achieves good calligraphy font generation effect, which effectively improves the quality of the generated font images and the rate of training.