Central Paths, Generalized Proximal Point Methods, and Cauchy Trajectories in Riemannian Manifolds

We consider an application of the proximal point method to variational inequality problems subject to box constraints, whose cost mappings possess order monotonicity properties instead of the usual monotonicity ones. Usually, convergence results of such methods require the additional boundedness assumption of the solutions set. We suggest another approach to obtaining convergence results for proximal point methods which is based on the assumption that the dual variational inequality is solvable. Then the solutions set may be unbounded. We present classes of economic equilibrium problems which satisfy such assumptions.

This paper studies the nonlinear inverse problem of identifying certain material parameters in the fourth-order boundary value problem representing the beam model. The inverse problem is solved by posing a convex optimization problem whose solution is an approximation of the sought parameters. The optimization problem is solved by the gradient based approaches, and in this setting, the most challenging aspect is the computation of the gradient of the objective functional. We present a detailed treatment of the adjoint stiffness matrix based approach for the gradient computation. We employ recently proposed self-adaptive inexact proximal point methods by Hager and Zhang [6] to solve the inverse problem. It is known that the regularization features of the proximal point methods are quite different from that of the Tikhonov regularization. We present a comparative analysis of the numerical efficiency of the used proximal point methods without using the Tikhonov regularization.

There has been a long history of using ordinary differential equations (ODEs) to understand the dynamics of discrete-time algorithms (DTAs). Surprisingly, there are still two fundamental and unanswered questions: (i) it is unclear how to obtain a suitable ODE from a given DTA, and (ii) it is unclear the connection between the convergence of a DTA and its corresponding ODEs. In this paper, we propose a new machinery—an $$O(s^r)$$ -resolution ODE framework—for analyzing the behavior of a generic DTA, which (partially) answers the above two questions. The framework contains three steps: 1. To obtain a suitable ODE from a given DTA, we define a hierarchy of $$O(s^r)$$ -resolution ODEs of a DTA parameterized by the degree r, where s is the step-size of the DTA. We present a principal approach to construct the unique $$O(s^r)$$ -resolution ODEs from a DTA; 2. To analyze the resulting ODE, we propose the $$O(s^r)$$ -linear-convergence condition of a DTA with respect to an energy function, under which the $$O(s^r)$$ -resolution ODE converges linearly to an optimal solution; 3. To bridge the convergence properties of a DTA and its corresponding ODEs, we define the properness of an energy function and show that the linear convergence of the $$O(s^r)$$ -resolution ODE with respect to a proper energy function can automatically guarantee the linear convergence of the DTA. To better illustrate this machinery, we utilize it to study three classic algorithms—gradient descent ascent (GDA), proximal point method (PPM) and extra-gradient method (EGM)—for solving the unconstrained minimax problem $$\min _{x\in \mathbb {R}^n} \max _{y\in \mathbb {R}^m} L(x,y)$$ . Their O(s)-resolution ODEs explain the puzzling convergent/divergent behaviors of GDA, PPM and EGM when L(x, y) is a bilinear function, and showcase that the interaction terms help the convergence of PPM/EGM but hurts the convergence of GDA. Furthermore, their O(s)-linear-convergence conditions not only unify the known scenarios when PPM and EGM have linear convergence, but also showcase that these two algorithms exhibit linear convergence in much broader contexts, including when solving a class of nonconvex-nonconcave minimax problems. Finally, we show how this ODE framework can help design new optimization algorithms for minimax problems, by studying the difference between the O(s)-resolution ODE of GDA and that of PPM/EGM.

The proximal point method (PPM) for solving maximal monotone operator inclusion problem is a highly powerful tool for algorithm design, analysis and interpretation. To accelerate convergence of the PPM, inertial PPM (iPPM) was proposed in the literature. In this note, we point out that some of the attractive properties of the PPM, e.g., the generated sequence is contractive with the set of solutions, do not hold anymore for iPPM. To partially inherit the advantages of the PPM and meanwhile incorporate inertial extrapolation steps, we propose an iPPM with alternating inertial steps. Our analyses show that the even subsequence generated by the proposed iPPM is contractive with the set of solutions. Moreover, we establish global convergence result under much relaxed conditions on the inertial extrapolation stepsizes, e.g., monotonicity is no longer needed and the stepsizes are significantly enlarged compared to existing methods. Furthermore, we establish certain $o(1/k)$ convergence rate results, where $k$ denotes the iteration counter. These features are new to inertial type PPMs.

The EM algorithm is a popular method for maximum likelihood estimation from incomplete data. This method may be viewed as a proximal point method for maximizing the log-likelihood function using an integral form of the Kullback-Leibler distance function. Motivated by this interpretation, we consider a proximal point method using an integral form of entropy-like distance function. We give a convergence analysis of the resulting proximal point method in the case where the cluster points lie in the interior of the objective function domain. This result is applied to a normal/independent example and a Gaussian mixture example to establish convergence of the EM algorithm on these examples. Further convergence analysis of the method for maximization over an orthant is given in low dimensions. Sublinear convergence and schemes for accelerating convergence are also discussed.

Abstract. In this paper, we consider and study a new class of general-ized vector quasi-variational type inequalities and obtain some existencetheorems for both under compact and noncompact assumptions in topo-logical vector spaces without using monotonicity. For the noncompactcase, we use the concept of escaping sequences. 1. IntroductionThe vector variational inequality problem was initiated by Giannessi [11] in nite dimensional Euclidean spaces with applications. Later on many authors[6, 7, 12, 15, 24] generalized vector variational inequalities in abstract spacesin several ways. The vector variational-like inequality is one of the generalizedforms of vector variational inequalities [11]. In 1989, Parida et al [19] stud-ied the existence of solutions for variational-like inequalities in R n space andhave shown a relationship between variational like inequalities problems andconvex programming as well as with complementarity problems. The vectorvariational like inequality and generalized vector variational like inequality arepowerful tools to study non-convex vector optimization problems and convexand nondi erentiable vector optimization problems respectively, see [10, 18, 21].In 1973, Bensoussan and Lions [3] introduced the quasi-variational inequalityproblem (QVIP). Since then, many generalizations of the QVIP have beenappeared in the literature (see, for example, [1, 2]). In the last decade, becauseof applications in the optimization problems, mathematical programming andequilibrium problems, the QVIP has been intensively studied by many authors[5, 8, 16, 20, 22, 23, 25].In this paper, we consider the generalized vector quasivariational type in-equality problem (GVQVTIP) and obtain some existence theorems for solutions

We study the properties of the weighted central paths in linear programming. We consider each path as the function of the parameter 0 where the value at D 0 corresponds to the limit point at the boundary of the feasible set. We calculate the recursive formulas for the central path derivatives of all orders valid at each 0. We establish the geometric growth of the derivatives and, consequently, the analyticity of the weighted central path at 0 . This paper also provides the analysis of limiting behavior of those projection operators which often appear in the interior point methods. The central trajectory is one of the most important concepts in the interior point methods for linear programming. The central trajectory (or the central path) is a distinguished curve that is interior to the feasible set and tends to an optimal point. Most interior points algorithms follow, or are related to, the central trajectory. Although the central path concept is essential for the design and analysis of algorithms relatively few papers study properties of this curve itself. For a review on this topic we refer to (3). Recently Zhao and Zhu (16) have studied some analytical properties of the central trajectory in the interior of the feasible set. They have established an upper bound for the high order derivatives of the weighted central path and estimated the convergence radius of the corresponding Taylor series. They have also pointed out the relationship of these properties with the complexity of a path following algorithm where the number of iterations is estimated by a "curvature integral". (See also (15).) Further, Guler (3) has analysed some limiting properties of the central trajectory derivatives with respect to the parameter >0. He has established the existence of finite limits for derivatives of all orders as tends to infinity (that is, the central path stays in the interior) and to zero (the central path tends to the boundary). These results have applications to the construction of polynomial- time algorithms which are also locally fast. A surprising result of this study is that all derivatives of order > 1t end to 0 as!1, which helps to explain why interior point methods for linear programming require so few iterations (see (3)).

We present a unified convergence rate analysis of iterative methods for solving the variational inequality problem. Our results are based on certain error bounds; they subsume and extend the linear and sublinear rates of convergence established in several previous studies. We also derive a new error bound for $\gamma$-strictly monotone variational inequalities. The class of algorithms covered by our analysis in fairly broad. It includes some classical methods for variational inequalities, e.g., the extragradient, matrix splitting, and proximal point methods. For these methods, our analysis gives estimates not only for linear convergence (which had been studied extensively), but also sublinear, depending on the properties of the solution. In addition, our framework includes a number of algorithms to which previous studies are not applicable, such as the infeasible projection methods, a separation-projection method, (inexact) hybrid proximal point methods, and some splitting techniques. Finally, our analysis covers certain feasible descent methods of optimization, for which similar convergence rate estimates have been recently obtained by Luo [14].

Metric subregularity and the proximal point method

This paper is concerned with the mixed linear complementarity problem and the role it and its variants play in the stability analysis of the nonlinear complementarity problem and the Karush-Kuhn-Tucker system of a variational inequality problem. Under a nonsingular assumption, the mixed linear complementarity problem can be converted to the standard problem; in this case, the rich theory of the latter can be directly applied to the former. In this work, we employ degree theory to derive some sufficient conditions for the existence of a solution to the mixed linear complementarity problem in the absence of the nonsingularity property. Next, we extend this existence theory to the mixed nonlinear complementarity problem and establish a main stability result under a certain degree-theoretic assumption concerning the linearized problem. We then specialize this stability result and its consequences to the parametric variational inequality problem under the assumption of a unique set of multipliers. Finally, we consider the latter problem with the uniqueness assumption of the multipliers replaced by a convexity assumption and obtain stability results under some weak second-order conditions. In addition to the new existence results for the mixed linear complementarity problem, the main contributions of this paper in the stability category are the following: a resolution to a conjecture concerning the local solvability of a parametric variational inequality, the use of the generalized linear complementarity problem as a tool to broaden the second-order conditions, the characterization of the solution stability of the linear complementarity problem and the affine variational inequality problem in terms of the solution isolatedness under some weak hypotheses, and various stability theorems under some weak second-order conditions.

A popular approach for addressing uncertainty in variational inequality problems is by solving the expected residual minimization (ERM) problem. This avenue necessitates distributional information associated with the uncertainty and requires minimizing nonconvex expectation-valued functions. We consider a distinctly different approach in the context of uncertain linear complementarity problems with a view towards obtaining robust solutions. Specifically, we define a robust solution to a complementarity problem as one that minimizes the worst-case of the gap function. In what we believe is amongst the first efforts to comprehensively address such problems in a distribution-free environment, we show that under specified assumptions on the uncertainty sets, the robust solutions to uncertain monotone linear complementarity problem can be tractably obtained through the solution of a single convex program. We also define uncertainty sets that ensure that robust solutions to non-monotone generalizations can also be obtained by solving convex programs. More generally, robust counterparts of uncertain non-monotone LCPs are proven to be low-dimensional nonconvex quadratically constrained quadratic programs. We show that these problems may be globally resolved by customizing an existing branching scheme. We further extend the tractability results to include uncertain affine variational inequality problems defined over uncertain polyhedral sets as well as to hierarchical regimes captured by mathematical programs with uncertain complementarity constraints. Preliminary numerics on uncertain linear complementarity and traffic equilibrium problems suggest that the presented avenues hold promise.

We consider the concept of so-called central path in nonlinear optimization that lies at the heart of interior point methods. Using singularity theory and real algebraic geometry, we provide conditions for the existence, convergence, and analyticity of central paths of nonlinear optimization problems. We also prove quantitative bounds on the convergence rate and analyticity of the central path for the special case of semi-definite optimization.

A hybrid entropic proximal decomposition method with self-adaptive strategy for solving variational inequality problems

This paper establishes a linear convergence rate for a class of epsilon-subgradient descent methods for minimizing certain convex functions. Currently prominent methods belonging to this class include the resolvent (proximal point) method and the bundle method in proximal form (considered as a sequence of serious steps). Other methods, such as the recently proposed descent proximal level method, may also fit this framework depending on implementation. The convex functions covered by the analysis are those whose conjugates have subdifferentials that are locally upper Lipschitzian at the origin, a class introduced by Zhang and Treiman. We argue that this class is a natural candidate for study in connection with minimization algorithms.

We consider a general equilibrium problem defined on a convex set, whose cost bifunction may not be monotone. We show that this problem can be solved by the inexact proximal point method if there exists a solution to the dual problem. An application of this approach to nonlinearly constrained problems is also suggested.