Modified Uzawa methods for saddle point problems

In this paper, the nonlinear inexact Uzawa (NIU) methods for saddle point problems are studied. A further improved convergence result of the NIU algorithm for stabilized saddle point problems is presented, which results in much smaller convergence factor. This result can also be viewed as a generalization of some previous work for classical saddle point problems. Based on this result, we generalize some adaptive versions of the NIU algorithm with variable iteration parameters for classical saddle point problems to solve stabilized saddle point problems. Convergence analyses of these algorithms are presented. It shows that they converge under similar conditions as those in classical cases, which are more practical and without estimates on the extreme eigenvalues of the involved preconditioned systems. Numerical experiments are given to demonstrate the effectiveness of these algorithms over some original NIU algorithms.

Recently, by combining the preconditioned accelerated Hermitian and skew-Hermitian splitting (PAHSS) and the parameterized Uzawa (PU) methods, Zheng and Ma (Appl Math Comput 273, 217–225, 2016b) presented the PAHSS–PU method for saddle point problems. By adding a block lower triangular matrix to the coefficient matrix on two sides of the first equation of the PAHSS–PU iterative scheme, the modified PAHSS–PU (MPAHSS–PU) iteration method is proposed in this paper, which has a faster convergence rate than the PAHSS–PU one. Furthermore, changing the position of the parameters in the MPAHSS–PU method, we develop another new method referred to as the modified PPHSS-SOR (MPPHSS-SOR) iteration method for solving saddle point problems. We provide the convergence properties of the MPAHSS–PU and the MPPHSS-SOR iteration methods, which show that the new methods are convergent if the related parameters satisfy suitable conditions. Meanwhile, practical ways to choose iteration parameters for the proposed methods are developed. Finally, numerical experiments demonstrate that the MPAHSS–PU and the MPPHSS-SOR methods outperform some existing ones both on the number of iterations and the computational times.

In this paper we consider additive Schwarz-type iteration methods for saddle point problems as smoothers in a multigrid method. Each iteration step of the additive Schwarz method requires the solutions of several small local saddle point problems. This method can be viewed as an additive version of a (multiplicative) Vanka-type iteration, well-known as a smoother for multigrid methods in computational fluid dynamics. It is shown that, under suitable conditions, the iteration can be interpreted as a symmetric inexact Uzawa method. In the case of symmetric saddle point problems the smoothing property, an important part in a multigrid convergence proof, is analyzed for symmetric inexact Uzawa methods including the special case of the additive Schwarz-type iterations. As an example the theory is applied to the Crouzeix-Raviart mixed finite element for the Stokes equations and some numerical experiments are presented.

Saddle point problems appear in many scientific and engineering applications. Hence, it is important to solve them fast with high accuracy. We have proposed the method for saddle point problems using block structure in order to solve them fast. In our proposed method, first, a linear system with multiple right-hand sides is solved instead of solving the saddle point problems directly. After that, a small linear system with a dense matrix is solved. In our previous work, it has been observed that our method is faster than the conventional approach, but the accuracy of the obtained approximate solutions is worse than the conventional one. In this paper, we propose to improve the accuracy of the approximate solutions of the saddle point problems by improving the accuracy of the solution of the small linear system using the mixed precision iterative refinement technique. Numerical experiments illustrate that the proposed approach improves the accuracy of the approximate solutions to the same extent as the conventional approach.

A new SOR-Like method for the saddle point problems

On unsymmetric block overrelaxation-type methods for saddle point problems

In this work, we consider numerical methods for solving a class of block three-by-three saddle point problems, which arise from finite element methods for solving time-dependent Maxwell equations and a class of quadratic programs. We present a variant of Uzawa method with two variable parameters for the saddle point problems. These two parameters can be updated easily in each iteration, similar to the evaluation of the two iteration parameters in the conjugate gradient method. We show that the new iterative method converges to the unique solution of the saddle point problems under a reasonable condition. Numerical experiments highlighting the performance of the proposed method for problems are presented.

In this paper two classes of iterative methods for saddle point problems are considered: inexact Uzawa algorithms and a class of methods with symmetric preconditioners. In both cases the iteration matrix can be transformed to a symmetric matrix by block diagonal matrices, a simple but essential observation which allows one to estimate the convergence rate of both classes by studying associated eigenvalue problems. The obtained estimates apply for a wider range of situations and are partially sharper than the known estimates in literature. A few numerical tests are given which confirm the sharpness of the estimates.

We design and analyze multigrid methods for the saddle point problems resulting from Raviart–Thomas–Nedelec mixed finite element methods for the Darcy system in porous media flow. Uniform convergence of the W-cycle algorithm in a nonstandard energy norm is established. Extensions to general second order elliptic problems are also addressed.

The paper overviews and compares some block preconditioners for the solution of saddle point systems, especially systems arising from the Brinkman model of porous media flow. The considered preconditioners involve different Schur complements as inverse free Schur complement in HSS (Hermitian - Skew Hermitian Splitting preconditioner), Schur complement to the velocity matrix and finally Schur complement to a regularization block in the augmented matrix preconditioner. The inverses appearing in most of the considered Schur complements are approximated by simple sparse approximation techniques as element-by-element and Frobenius norm minimization approaches. A special interest is devoted to problems involving various Darcy, Stokes and Brinkman flow regions, the efficiency of preconditioners in this case is demonstrated by some numerical experiments.

Multigrid methods for saddle point problems: Optimality systems

Analysis of a generalized MASSOR method for saddle point problems

This paper introduces the Multiple Greedy Quasi-Newton (MGSR1-SP) method, a novel approach to solving strongly-convex-strongly-concave (SCSC) saddle point problems. Our method enhances the approximation of the squared indefinite Hessian matrix inherent in these problems, significantly improving both stability and efficiency through iterative greedy updates. We provide a thorough theoretical analysis of MGSR1-SP, demonstrating its linear-quadratic convergence rate. Numerical experiments conducted on AUC maximization and adversarial debiasing problems, compared with state-of-the-art algorithms, underscore our method's enhanced convergence rate. These results affirm the potential of MGSR1-SP to improve performance across a broad spectrum of machine learning applications where efficient and accurate Hessian approximations are crucial.

Convergence behavior of a nested iterative scheme presented in a paper by Bank, Welfert and Yserentant is studied. It is shown that this scheme converges under conditions weaker than that stated in their paper.

In this paper, wavelet techniques are employed for the fast numerical solution of a control problem governed by an elliptic boundary value problem with boundary control. A quadratic cost functional involving natural norms of the state and the control is to be minimized. Firstly the constraint, the elliptic boundary value problem, is formulated in an appropriate weak form that allows to handle varying boundary conditions explicitly: the boundary conditions are treated by Lagrange multipliers, leading to a saddle point problem. This is combined with a fictitious domain approach in order to cover also more complicated boundaries. Deviating from standard approaches, we then use (biorthogonal) wavelets to derive an equivalent infinite discretized control problem which involves only e2-norms and -operators. Classical methods from optimization yield the corresponding optimality conditions in terms of two weakly coupled (still infinite) saddle point problems for which a unique solution exists. For deriving finite-dimensional systems which are uniformly invertible, stability of the discretizations has to be ensured. This together with the e2-setting circumvents the problem of preconditioning: all operators have uniformly bounded condition numbers independent of the discretization. In order to numerically solve the resulting (finite-dimensional) linear system of the weakly coupled saddle point problems, a fully iterative method is proposed which can be viewed as an inexact gradient scheme. It consists of a gradient algorithm as an outer iteration which alternatingly picks the two saddle point problems, and an inner iteration to solve each of the saddle point problems, exemplified in terms of the Uzawa algorithm. It is proved here that this strategy converges, provided that the inner systems are solved sufficiently well. Moreover, since the system matrix is well-conditioned, it is shown that in combination with a nested iteration strategy this iteration is asymptotically optimal in the sense that it provides the solution on discretization level J with an overall amount of arithmetic operations that is proportional to the number of unknows NJ on that level. Finally, numerical results are provided.