An Inexact Double Primal-Dual Algorithm for Saddle Point Problems

In this paper, we further investigate the local Hermitian and skew-Hermitian splitting (LHSS) iteration method and the modified LHSS (MLHSS) iteration method for solving generalized nonsymmetric saddle point problems with nonzero (2,2) blocks. When A is non-symmetric positive definite, the convergence conditions are obtained, which generalize some results of Jiang and Cao [M.-Q. Jiang and Y. Cao, On local Hermitian and Skew-Hermitian splitting iteration methods for generalized saddle point problems, J. Comput. Appl. Math., 2009(231): 973-982] for the generalized saddle point problems to generalized nonsymmetric saddle point problems with nonzero (2,2) blocks. Numerical experiments show the effectiveness of the iterative methods.

In this paper, wavelet techniques are employed for the fast numerical solution of a control problem governed by an elliptic boundary value problem with boundary control. A quadratic cost functional involving natural norms of the state and the control is to be minimized. Firstly the constraint, the elliptic boundary value problem, is formulated in an appropriate weak form that allows to handle varying boundary conditions explicitly: the boundary conditions are treated by Lagrange multipliers, leading to a saddle point problem. This is combined with a fictitious domain approach in order to cover also more complicated boundaries. Deviating from standard approaches, we then use (biorthogonal) wavelets to derive an equivalent infinite discretized control problem which involves only e2-norms and -operators. Classical methods from optimization yield the corresponding optimality conditions in terms of two weakly coupled (still infinite) saddle point problems for which a unique solution exists. For deriving finite-dimensional systems which are uniformly invertible, stability of the discretizations has to be ensured. This together with the e2-setting circumvents the problem of preconditioning: all operators have uniformly bounded condition numbers independent of the discretization. In order to numerically solve the resulting (finite-dimensional) linear system of the weakly coupled saddle point problems, a fully iterative method is proposed which can be viewed as an inexact gradient scheme. It consists of a gradient algorithm as an outer iteration which alternatingly picks the two saddle point problems, and an inner iteration to solve each of the saddle point problems, exemplified in terms of the Uzawa algorithm. It is proved here that this strategy converges, provided that the inner systems are solved sufficiently well. Moreover, since the system matrix is well-conditioned, it is shown that in combination with a nested iteration strategy this iteration is asymptotically optimal in the sense that it provides the solution on discretization level J with an overall amount of arithmetic operations that is proportional to the number of unknows NJ on that level. Finally, numerical results are provided.

In this paper, the nonlinear inexact Uzawa (NIU) methods for saddle point problems are studied. A further improved convergence result of the NIU algorithm for stabilized saddle point problems is presented, which results in much smaller convergence factor. This result can also be viewed as a generalization of some previous work for classical saddle point problems. Based on this result, we generalize some adaptive versions of the NIU algorithm with variable iteration parameters for classical saddle point problems to solve stabilized saddle point problems. Convergence analyses of these algorithms are presented. It shows that they converge under similar conditions as those in classical cases, which are more practical and without estimates on the extreme eigenvalues of the involved preconditioned systems. Numerical experiments are given to demonstrate the effectiveness of these algorithms over some original NIU algorithms.

In this work, we consider numerical methods for solving a class of block three-by-three saddle point problems, which arise from finite element methods for solving time-dependent Maxwell equations and a class of quadratic programs. We present a variant of Uzawa method with two variable parameters for the saddle point problems. These two parameters can be updated easily in each iteration, similar to the evaluation of the two iteration parameters in the conjugate gradient method. We show that the new iterative method converges to the unique solution of the saddle point problems under a reasonable condition. Numerical experiments highlighting the performance of the proposed method for problems are presented.

Saddle point problems appear in many scientific and engineering applications. Hence, it is important to solve them fast with high accuracy. We have proposed the method for saddle point problems using block structure in order to solve them fast. In our proposed method, first, a linear system with multiple right-hand sides is solved instead of solving the saddle point problems directly. After that, a small linear system with a dense matrix is solved. In our previous work, it has been observed that our method is faster than the conventional approach, but the accuracy of the obtained approximate solutions is worse than the conventional one. In this paper, we propose to improve the accuracy of the approximate solutions of the saddle point problems by improving the accuracy of the solution of the small linear system using the mixed precision iterative refinement technique. Numerical experiments illustrate that the proposed approach improves the accuracy of the approximate solutions to the same extent as the conventional approach.

In this work we study convergence properties of sparse polynomial approximations for a class of affine parametric saddle point problems, including the Stokes equations for viscous incompressible flow, mixed formulation of diffusion equations for groundwater flow, time-harmonic Maxwell equations for electromagnetics, etc. Due to the lack of knowledge or intrinsic randomness, the (viscosity, diffusivity, permeability, permittivity, etc.) coefficients of such problems are uncertain and can often be represented or approximated by high- or countably infinite-dimensional random parameters equipped with suitable probability distributions, and the coefficients affinely depend on a series of either globally or locally supported basis functions, e.g., Karhunen–Loève expansion, piecewise polynomials, or adaptive wavelet approximations. We consider sparse polynomial approximations of the parametric solutions, in particular sparse Taylor approximations, and study their convergence properties for these parametric problems. Under suitable sparsity assumptions on the parametrization of the random coefficients, we show the algebraic convergence rates O(N−r) for the sparse polynomial approximations of the parametric solutions based on the results for affine parametric elliptic PDEs (Cohen, A. et al.: Anal. Appl. 9, 11–47, 2011), (Bachmayr, M., et al.: ESAIM Math. Model. Numer. Anal. 51, 321–339, 2017), (Cohen, A., DeVore, R.: Acta Numer. 24, 1–159, 2015), (Chkifa, A., et al.: J. Math. Pures Appl. 103, 400–428, 2015), (Chkifa, A., et al.: ESAIM Math. Model. Numer. Anal. 47, 253–280, 2013), (Cohen, A., Migliorati, G.: Contemp. Comput. Math., 233–282, 2018), with the rate r depending only on a sparsity parameter in the parametrization, not on the number of active parameter dimensions or the number of polynomial terms N. We note that parametric saddle point problems were considered in (Cohen, A., DeVore, R.: Acta Numer. 24, 1–159, 2015, Section 2.2) with the anticipation that the same results on the approximation of the solution map obtained for elliptic PDEs can be extended to more general saddle point problems. In this paper, we consider a general formulation of saddle point problems, different from that presented in (Cohen, A., DeVore, R.: Acta Numer. 24, 1–159, 2015, Section 2.2), and obtain convergence rates for the two variables, e.g., velocity and pressure in Stokes equations, which are different for the case of locally supported basis functions.

An adaptive algorithm based on wavelets is proposed for the fast numerical solution of control problems governed by elliptic boundary value problems with Dirichlet boundary control. A quadratic cost functional representing Sobolev norms of the state and a regularization in terms of the control is to be minimized subject to linear constraints in weak form. In particular, the constraints are formulated as a saddle point problem that allows to handle the varying boundary conditions explicitly. In the framework of (biorthogonal) wavelets, a representer for the functional is derived in terms of l2-norms of wavelet expansion coefficients and the constraints are written in form of an l2 automorphism. Standard techniques from optimization are then used to deduce the resulting first order necessary conditions as a (still infinite) system in l2. Applying the machinery developed in [8,9] which has been extended to control problems in [14], an adaptive method is proposed which can be interpreted as an inexact gradient method for the control. In each iteration step, in turn the primal and the adjoint saddle point system are solved up to a prescribed accuracy by an adaptive iterative Uzawa algorithm for saddle point problems which has been proposed in [10]. Under these premises, it can be shown that the adaptive algorithm containing now three layers of iterations is asymptotically optimal. This means that the convergence rate achieved for computing the solution up to a desired target tolerance is asymptotically the same as the wavelet-best N-term approximation of the solution, and the total computational work is proportional to the number of computational unknowns.

ABSTRACTIn this paper, we present a parameterized matrix splitting (PMS) preconditioner for the large sparse saddle point problems. The preconditioner is based on a parameterized splitting of the saddle point matrix, resulting in a fixed-point iteration. The convergence theorem of the new iteration method for solving large sparse saddle point problems is proposed by giving the restrictions imposed on the parameter. Based on the idea of the parameterized splitting, we further propose a modified PMS preconditioner. Some useful properties of the preconditioned matrix are established. Numerical implementations show that the resulting preconditioner leads to fast convergence when it is used to precondition Krylov subspace iteration methods such as generalized minimal residual method.

In the framework of a special block alternating splitting implicit (BASI) iteration scheme for generalized saddle point problems, we establish some new iteration methods for solving double saddle point problems by means of a suitable partitioning strategy. Convergence analysis of the corresponding BASI iteration methods indicates that they are convergent unconditionally under certain weak requirements for the related matrix splittings, which are satisfied directly for our specific application to double saddle point problems. Numerical examples for liquid crystal director and time‐harmonic eddy current models are presented to demonstrate the efficiency of the proposed BASI preconditioners to accelerate the GMRES method.

Overlapping Schwarz methods for mixed linear elasticity and Stokes problems

In this work we aim to solve a convex-concave saddle point problem, where the convex-concave coupling function is smooth in one variable and nonsmooth in the other and not assumed to be linear in either. The problem is augmented by a nonsmooth regulariser in the smooth component. We propose and investigate a novel algorithm under the name of OGAProx, consisting of an optimistic gradient ascent step in the smooth variable coupled with a proximal step of the regulariser, and which is alternated with a proximal step in the nonsmooth component of the coupling function. We consider the situations convex-concave, convex-strongly concave and strongly convex-strongly concave related to the saddle point problem under investigation. Regarding iterates we obtain (weak) convergence, a convergence rate of order mathcal {O}(frac{1}{K}) and linear convergence like mathcal {O}(theta ^{K}) with theta < 1, respectively. In terms of function values we obtain ergodic convergence rates of order mathcal {O}(frac{1}{K}), mathcal {O}(frac{1}{K^{2}}) and mathcal {O}(theta ^{K}) with theta < 1, respectively. We validate our theoretical considerations on a nonsmooth-linear saddle point problem, the training of multi kernel support vector machines and a classification problem incorporating minimax group fairness.

This paper studies the convergence rate of a second-order dynamical system associated with a nonsmooth bilinearly coupled convex-concave saddle point problem, as well as the convergence rate of its corresponding discretizations. We derive the convergence rate of the primal-dual gap for the second-order dynamical system with asymptotically vanishing damping term. Based on an implicit discretization scheme, we propose a primal-dual algorithm and provide a non-ergodic convergence rate under a general setting for the inertial parameters when one objective function is continuously differentiable and convex and the other is a proper, convex and lower semicontinuous function. For this algorithm we derive a $$O\left( 1/k^2 \right) $$ O 1 / k 2 convergence rate under three classical rules proposed by Nesterov, Chambolle-Dossal and Attouch-Cabot without assuming strong convexity, which is compatible with the results of the continuous-time dynamic system. For the case when both objective functions are continuously differentiable and convex, we further present a primal-dual algorithm based on an explicit discretization. We provide a corresponding non-ergodic convergence rate for this algorithm and show that the sequence of iterates generated weakly converges to a primal-dual optimal solution. Finally, we present numerical experiments that indicate the superior numerical performance of both algorithms.

Finite element approximation of an elliptic variational inequality with quaisilinear operator and constraints to the solution and its gradient is constructed. Corresponding discrete problem is splitted into subproblems by non-overlapping domain decomposition technique and constrained saddle point problem is constructed for the splitted problem. Block relaxation-Uzawa iterative solution method is applied to this resulting saddle point problem. Existence of a solution to saddle point problem and convergence of the iterative method are proved.

This paper focuses on saddle point problems with a 2-by-2 block coefficient matrix. When the number of columns in the upper-right block and the number of rows in the lower-left block of the coefficient matrix is large, the convergence behavior of Krylov subspace methods for the saddle point problems tends to be poor even if the upper-left block is a well-conditioned matrix. In this paper, an efficient approach for solving the saddle point problems using block structure of the problems is proposed. The most time-consuming part of our proposed approach is the solution of a linear system with multiple right-hand sides. To solve the linear system with multiple right-hand sides efficiently, we propose to apply Block Krylov subspace methods to this linear system. Numerical experiments show that the proposed approach with Block Krylov subspace methods can solve the saddle point problems more efficiently than the conventional approach in terms of the number of iterations and the computation time.

A new SOR-Like method for the saddle point problems