A Symmetry-Preserving Extrapolated Primal-Dual Hybrid Gradient Method for Saddle-Point Problems

The in-line X-ray phase contrast imaging technique relies on the measurement of Fresnel diffraction intensity patterns due to the phase shift and the absorption induced by the object. The recovery of both phase and absorption is an ill-posed non-linear inverse problem. In this work, we address this problem with an iterative algorithm based on a primal-dual method, which allows us to introduce the non-linearity of the forward operator. We used a variational approach with different regularizations for the phase and absorption, in order to take into account the specificities of each quantity. Assuming the solution to be piecewise constant, the functional used involves the Total Generalized Variation (TGV) as well as the classical Total Variation (TV), which enables a compromise between sharp discontinuities and smoothness in the solution. This optimization problem is solved efficiently using primal-dual approach such as Primal-Dual Hybrid Gradient Method (PDHGM). From this approach, we propose an algorithm called PDGHGM-CTF, which is based on the linearized Contrast Transfer Function model, that we generalize for the nonlinear problem to get the Non-Linear Primal-Dual Hybrid Gradient Method (NL-PDHGM). The proposed iterative algorithms are able to recover simultaneously the phase and absorption from a single diffraction pattern without homogeneity assumption or support constraint, and the nonlinear version is valid without restriction on the object. Moreover, we show that the approach is robust with respect to the initialization. While giving a good approximation as starting point reduced the convergence time, it did not improve the reconstruction results. We demonstrate the potential of the proposed algorithms on simulated datasets. We show that it produces reconstructions with fewer artifacts and improved normalized mean squared error compared to a gradient descent scheme. We evaluate the robustness of the proposed algorithm by evaluating the reconstruction on simulated images of 1 000 random objects, given the same hyperparameters.

First-order primal-dual methods are appealing for their low memory overhead, fast iterations, and effective parallelization. However, they are often slow at finding high accuracy solutions, which creates a barrier to their use in traditional linear programming (LP) applications. This paper exploits the sharpness of primal-dual formulations of LP instances to achieve linear convergence using restarts in a general setting that applies to alternating direction method of multipliers (ADMM), primal-dual hybrid gradient method (PDHG) and extragradient method (EGM). In the special case of PDHG, without restarts we show an iteration count lower bound of \(\Omega (\kappa ^2 \log (1/\epsilon ))\), while with restarts we show an iteration count upper bound of \(O(\kappa \log (1/\epsilon ))\), where \(\kappa \) is a condition number and \(\epsilon \) is the desired accuracy. Moreover, the upper bound is optimal for a wide class of primal-dual methods, and applies to the strictly more general class of sharp primal-dual problems. We develop an adaptive restart scheme and verify that restarts significantly improve the ability of PDHG, EGM, and ADMM to find high accuracy solutions to LP problems.

A first-order optimization algorithm has been introduced by Liu, Liu, Osher, and Li [J. Comput. Phys. 500 (2024), 19 pp.] to solve time-implicit schemes of reaction-diffusion equations. In this research, we conduct theoretical studies on this first-order algorithm equipped with a quadratic regularization term. We provide sufficient conditions under which the proposed algorithm and its time-continuous limit converge exponentially fast to a desired time-implicit numerical solution. We show both theoretically and numerically that the convergence rate is independent of the grid size, which makes our method suitable for large scale problems. The efficiency of our algorithm has been verified via a series of numerical examples conducted on various types of reaction-diffusion equations. The choice of optimal hyperparameters as well as comparisons with some classical root-finding algorithms is also discussed in the numerical section.

The distance between sets is a long-standing computational geometry problem. In robotics, the distance between convex sets with Minkowski sum structure plays a fundamental role in collision detection. However, it is typically nontrivial to be computed, even if the projection onto each component set admits explicit formula. In this paper, we explore the problem of calculating the distance between convex sets arising from robotics. Upon the recent progress in convex optimization community, the proposed model can be efficiently solved by the recent hot-investigated first-order methods, e.g., alternating direction method of multipliers or primal-dual hybrid gradient method. Preliminary numerical results demonstrate that those first-order methods are fairly efficient in solving distance problems in robotics.

Based on the local correntropy-based K-means clustering active contour model, this paper proposes a fast two-stage segmentation method for intensity inhomogeneous images. Under our framework, the segmentation process is split into two stages. In the first stage, we preliminary segment the down-sampled images by the proposed relaxed anisotropic–isotropic local correntropy-based K-means clustering (AILCK) model, which can obtain a coarse segmentation result quickly. Subsequently, in the second stage, we further segment original images by an improved AILCK model, where we use the up-sampled coarse contour obtained by the first stage as the initialization. Following it, to obtain the global minima of energy functions fast, we incorporate a weighted difference of anisotropic and isotropic total variations into relaxed formulation of the two-stage active contour models. And then, we minimize them utilizing the difference-of-convex algorithm and the primal–dual hybrid gradient method. The experimental results on synthetic and real-world images demonstrate that the proposed method can achieve accurate segmentation results for intensity inhomogeneous images in a fast way.

In this paper, the nonlinear inexact Uzawa (NIU) methods for saddle point problems are studied. A further improved convergence result of the NIU algorithm for stabilized saddle point problems is presented, which results in much smaller convergence factor. This result can also be viewed as a generalization of some previous work for classical saddle point problems. Based on this result, we generalize some adaptive versions of the NIU algorithm with variable iteration parameters for classical saddle point problems to solve stabilized saddle point problems. Convergence analyses of these algorithms are presented. It shows that they converge under similar conditions as those in classical cases, which are more practical and without estimates on the extreme eigenvalues of the involved preconditioned systems. Numerical experiments are given to demonstrate the effectiveness of these algorithms over some original NIU algorithms.

Numerous diffusion model (DM)-based methods have been proposed for solving inverse imaging problems. Among these, a recent line of work has demonstrated strong performance by formulating sampling as an optimization procedure that enforces measurement consistency, forward diffusion consistency, and both step-wise and backward diffusion consistency. However, these methods have only considered 2D reconstruction tasks and do not directly extend to 3D image reconstruction problems, such as in Computed Tomography (CT). To bridge this gap, we propose NEtwork-Regularized diffusion sampling for 3D CT (NERD) by incorporating an $\ell_{1}$ regularization into the optimization objective. This regularizer encourages spatial continuity across adjacent slices, reducing inter-slice artifacts and promoting coherent volumetric reconstructions. Additionally, we introduce two efficient optimization strategies to solve the resulting objective: one based on the Alternating Direction Method of Multipliers (ADMM) and another based on the Primal-Dual Hybrid Gradient (PDHG) method. Experiments on medical 3D CT data demonstrate that our approach achieves either state-of-the-art or highly competitive results.

In this paper, we present a unified analysis of methods for such a wide class of problems as variational inequalities, which includes minimization problems and saddle point problems. We develop our analysis on the modified Extra-Gradient method (the classic algorithm for variational inequalities) and consider the strongly monotone and monotone cases, which corresponds to strongly-convex-strongly-concave and convex-concave saddle point problems. The theoretical analysis is based on parametric assumptions about Extra-Gradient iterations. Therefore, it can serve as a strong basis for combining the already existing type methods and also for creating new algorithms. In particular, to show this we develop new robust methods, which include methods with quantization, coordinate methods, distributed randomized local methods, and others. Most of these approaches have never been considered in the generality of variational inequalities and have previously been used only for minimization problems. The robustness of the new methods is also confirmed by numerical experiments with GANs.

Decomposition methods have been well studied for solving two-stage and multi-stage stochastic programming problems, see Rockafellar and Wets (Math. Oper. Res. 16:119–147, 1991), Ruszczynski and Shapiro (Stochastic Programming, Handbook in OR & MS, North-Holland Publishing Company, Amsterdam, 2003) and Ruszczynski (Math. Program. 79:333–353, 1997). In this paper, we propose an algorithmic framework based on the fundamental ideas of the methods for solving two-stage minimax distributionally robust optimization (DRO) problems where the underlying random variables take a finite number of distinct values. This is achieved by introducing nonanticipativity constraints for the first stage decision variables, rearranging the minimax problem through Lagrange decomposition and applying the well-known primal-dual hybrid gradient (PDHG) method to the new minimax problem. The algorithmic framework does not depend on specific structure of the ambiguity set. To extend the algorithm to the case that the underlying random variables are continuously distributed, we propose a discretization scheme and quantify the error arising from the discretization in terms of the optimal value and the optimal solutions when the ambiguity set is constructed through generalized prior moment conditions, the Kantorovich ball and $$\phi$$ -divergence centred at an empirical probability distribution. Some preliminary numerical tests show the proposed decomposition algorithm featured with parallel computing performs well.

Alternative scoring of the Patient Health Questionnaire-9 in neurological populations: an approach based on a predictive algorithm deriving from individual item scores

In this paper we consider additive Schwarz-type iteration methods for saddle point problems as smoothers in a multigrid method. Each iteration step of the additive Schwarz method requires the solutions of several small local saddle point problems. This method can be viewed as an additive version of a (multiplicative) Vanka-type iteration, well-known as a smoother for multigrid methods in computational fluid dynamics. It is shown that, under suitable conditions, the iteration can be interpreted as a symmetric inexact Uzawa method. In the case of symmetric saddle point problems the smoothing property, an important part in a multigrid convergence proof, is analyzed for symmetric inexact Uzawa methods including the special case of the additive Schwarz-type iterations. As an example the theory is applied to the Crouzeix-Raviart mixed finite element for the Stokes equations and some numerical experiments are presented.

In this work, we consider numerical methods for solving a class of block three-by-three saddle point problems, which arise from finite element methods for solving time-dependent Maxwell equations and a class of quadratic programs. We present a variant of Uzawa method with two variable parameters for the saddle point problems. These two parameters can be updated easily in each iteration, similar to the evaluation of the two iteration parameters in the conjugate gradient method. We show that the new iterative method converges to the unique solution of the saddle point problems under a reasonable condition. Numerical experiments highlighting the performance of the proposed method for problems are presented.

In this work we aim to solve a convex-concave saddle point problem, where the convex-concave coupling function is smooth in one variable and nonsmooth in the other and not assumed to be linear in either. The problem is augmented by a nonsmooth regulariser in the smooth component. We propose and investigate a novel algorithm under the name of OGAProx, consisting of an optimistic gradient ascent step in the smooth variable coupled with a proximal step of the regulariser, and which is alternated with a proximal step in the nonsmooth component of the coupling function. We consider the situations convex-concave, convex-strongly concave and strongly convex-strongly concave related to the saddle point problem under investigation. Regarding iterates we obtain (weak) convergence, a convergence rate of order mathcal {O}(frac{1}{K}) and linear convergence like mathcal {O}(theta ^{K}) with theta < 1, respectively. In terms of function values we obtain ergodic convergence rates of order mathcal {O}(frac{1}{K}), mathcal {O}(frac{1}{K^{2}}) and mathcal {O}(theta ^{K}) with theta < 1, respectively. We validate our theoretical considerations on a nonsmooth-linear saddle point problem, the training of multi kernel support vector machines and a classification problem incorporating minimax group fairness.

BDDC algorithms have previously been extended to the saddle point problems arising from mixed formulations of elliptic and incompressible Stokes problems. In these two-level BDDC algorithms, all iterates are required to be in a benign space, a subspace in which the preconditioned operators are positive definite. This requirement can lead to large coarse problems, which have to be generated and factored by a direct solver at the beginning of the computation and they can ultimately become a bottleneck. An additional level is introduced in this paper to solve the coarse problem approximately and to remove this difficulty. This three-level BDDC algorithm keeps all iterates in the benign space and the conjugate gradient methods can therefore be used to accelerate the convergence. This work is an extension of the three-level BDDC methods for standard finite element discretization of elliptic problems and the same rate of convergence is obtained for the mixed formulation of the same problems. Estimate of the condition number for this three-level BDDC methods is provided and numerical experiments are discussed.

Saddle point problems appear in many scientific and engineering applications. Hence, it is important to solve them fast with high accuracy. We have proposed the method for saddle point problems using block structure in order to solve them fast. In our proposed method, first, a linear system with multiple right-hand sides is solved instead of solving the saddle point problems directly. After that, a small linear system with a dense matrix is solved. In our previous work, it has been observed that our method is faster than the conventional approach, but the accuracy of the obtained approximate solutions is worse than the conventional one. In this paper, we propose to improve the accuracy of the approximate solutions of the saddle point problems by improving the accuracy of the solution of the small linear system using the mixed precision iterative refinement technique. Numerical experiments illustrate that the proposed approach improves the accuracy of the approximate solutions to the same extent as the conventional approach.