On The Behavior of Subgradient Projections Methods for Convex Feasibility Problems in Euclidean Spaces

We study the convergence behavior of a class of projection methods for solving convex feasibility and optimization problems. We prove that the algorithms in this class converge to solutions of the consistent convex feasibility problem, and that their convergence is stable under summable perturbations. Our class is a subset of the class of string-averaging projection methods, large enough to contain, among many other procedures, a version of the Cimmino algorithm, as well as the cyclic projection method. A variant of our approach is proposed to approximate the minimum of a convex functional subject to convex constraints. This variant is illustrated on a problem in image processing: namely, for optimization in tomography.

The notion of relaxation is well understood for orthogonal projections onto convex sets. For general Bregman projections it was considered only for hyperplanes, and the question of how to relax Bregman projections onto convex sets that are not linear (i.e., not hyperplanes or half-spaces) has remained open. A definition of the underrelaxation of Bregman projections onto general convex sets is given here, which includes as special cases the underrelaxed orthogonal projections and the underrelaxed Bregman projections onto linear sets as given by De Pierro and Iusem [J. Optim. Theory Appl., 51 (1986), pp. 421--440]. With this new definition, we construct a block-iterative projection algorithmic scheme and prove its convergence to a solution of the convex feasibility problem. The practical importance of relaxation parameters in the application of such projection algorithms to real-world problems is demonstrated on a problem of image reconstruction from projections.

Many real-world problems can often be cast as the optimization of DR-submodular functions defined over a convex domain. These functions play an important role with applications in many areas of applied mathematics, such as machine learning, computer vision, operation research, communication systems or economics. In addition, they capture a subclass of non-convex optimization that provides both practical and theoretical guarantees. In this paper, we show that for maximizing non-monotone DR-submodular functions over a general convex set (such as up-closed convex sets, conic convex set, etc) the Frank-Wolfe algorithm achieves an approximation guarantee which depends on the convex set. To the best of our knowledge, this is the first approximation guarantee. Finally we benchmark our algorithm on problems arising in machine learning domain with the real-world datasets.

In this paper, we study properties of general closed convex sets that determine the closedness and polyhedrality of the convex hull of integer points contained in it. We first present necessary and sufficient conditions for the convex hull of integer points contained in a general convex set to be closed. This leads to useful results for special classes of convex sets such as pointed cones, strictly convex sets, and sets containing integer points in their interior. We then present a sufficient condition for the convex hull of integer points in general convex sets to be a polyhedron. This result generalizes the well-known result due to Meyer (Math Program 7:223–225, 1974). Under a simple technical assumption, we show that these sufficient conditions are also necessary for the convex hull of integer points contained in general convex sets to be a polyhedron.

To efficiently solve online problems with complicated constraints, projection-free algorithms including online frank-wolfe (OFW) and its variants have received significant interest recently. However, in the general case, existing efficient projection-free algorithms only achieved the regret bound of O(T^{3/4}), which is worse than the regret of projection-based algorithms, where T is the number of decision rounds. In this paper, we study the special case of online learning over strongly convex sets, for which we first prove that OFW can enjoy a better regret bound of O(T^{2/3}) for general convex losses. The key idea is to refine the decaying step-size in the original OFW by a simple line search rule. Furthermore, for strongly convex losses, we propose a strongly convex variant of OFW by redefining the surrogate loss function in OFW. We show that it achieves a regret bound of O(T^{2/3}) over general convex sets and a better regret bound of O(T^{1/2}) over strongly convex sets.

In this paper we generalize the factorization theorem of Gouveia, Parrilo and Thomas to a broader class of convex sets. Given a general convex set, we define a slack operator associated to the set and its polar according to whether the convex set is full dimensional, whether it is a translated cone and whether it contains lines. We strengthen the condition of a cone lift by requiring not only the convex set is the image of an affine slice of a given closed convex cone, but also its recession cone is the image of the linear slice of the closed convex cone. We show that the generalized lift of a convex set can also be characterized by the cone factorization of a properly defined slack operator.

A general iterative scheme including relaxation and a corresponding problem class are presented. Some global convergence results are given. The acceleration of convergence is discussed, The scheme comprises a lot of known iterative methods such as subgradient methods and methods of successive orthogonal projections with relaxation. Applications to convex optimization, convex feasibility problems, systems of convex inequalities, variational inequalities, operator equations and systems of linear equations are given.

The method of projections onto convex sets to find a point in the intersection of a finite number of closed convex sets in a Euclidean space, may lead to slow convergence of the constructed sequence when that sequence enters some narrow “corridor” between two or more convex sets. A way to leave such corridor consists in taking a big step at different moments during the iteration, because in that way the monotoneous behaviour that is responsible for the slow convergence may be interrupted. In this paper we present a technique that may introduce interruption of the monotony for a sequential algorithm, but that at the same time guarantees convergence of the constructed sequence to a point in the intersection of the sets. We compare experimentally the behaviour concerning the speed of convergence of the new algorithm with that of an existing monotoneous algorithm.

This book describes and analyzes all available alternating projection methods for solving the general problem of finding a point in the intersection of several given sets belonging to a Hilbert space. For each method the authors describe and analyze convergence, speed of convergence, acceleration techniques, stopping criteria, and applications. Different types of algorithms and applications are studied for subspaces, linear varieties, and general convex sets. The authors also unify these algorithms into a common theoretical framework. Alternating Projection Methods is a comprehensive and accessible source of information, providing readers with the theoretical and practical aspects of the most relevant alternating projection methods. It features several acceleration techniques for every method it presents and analyzes, including schemes that cannot be found in other books. It also provides full descriptions of several important mathematical problems and specific applications for which the alternating projection methods represent an efficient option. Examples and problems that illustrate this material are also included. Audience: This book can be used as a textbook for advanced undergraduate or first-year graduate students. Because it is comprehensive, it can also be used as a tutorial or a reference by mathematicians and nonmathematicians from many fields of application who need to solve alternating projection problems in their work. Contents: Preface; Chapter 1: Introduction; Chapter 2: Overview on Spaces; Chapter 3: The MAP on Subspaces; Chapter 4: Row-Action Methods; Chapter 5: Projecting on Convex Sets; Chapter 6: Applications of MAP for Matrix Problems; Bibliography; Author Index; Subject Index.

It is known that the curvature of the feasible set in convex optimization allows for algorithms with better convergence rates, and there is renewed interest in this topic for both off-line and online problems. In this paper, leveraging results on geometry and convex analysis, we further our understanding of the role of curvature in optimization: We first show the equivalence of two notions of curvature, namely, strong convexity and gauge bodies, proving a conjecture of Abernethy et al. As a consequence, this shows that the Frank–Wolfe–type method of Wang and Abernethy has accelerated convergence rate [Formula: see text] over strongly convex feasible sets without additional assumptions on the (convex) objective function. In online linear optimization, we identify two main properties that help explaining why/when follow the leader (FTL) has only logarithmic regret over strongly convex sets. This allows one to directly recover and slightly extend a recent result of Huang et al., and to show that FTL has logarithmic regret over strongly convex sets whenever the gain vectors are nonnegative. We provide an efficient procedure for approximating convex bodies by strongly convex ones while smoothly trading off approximation error and curvature. This allows one to extend the improved algorithms over strongly convex sets to general convex sets. As a concrete application, we extend results on online linear optimization with hints to general convex sets. Funding: This work was supported by Conselho Nacional de Desenvolvimento Científico e Tecnológico [Grant Bolsa de Produtividade em Pesquisa #4310516/2017-0] Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Finance Code 001) Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro (Jovem Cientista do Nosso Estado), Bolsa de Produtividade em Pesquisa #312751/2021-4 from CNPq, FAPERJ [Grant “Jovem Cientista do Nosso Estado”].

The convex feasibility problem asks to find a point in the intersection of a collection of nonempty closed convex sets. This problem is of basic importance in mathematics and the physical sciences, and projection (or splitting) methods solve it by employing the projection operators associated with the individual sets to generate a sequence which converges to a solution. Motivated by an application in road design, we present the method of cyclic intrepid projections (CycIP) and provide a rigorous convergence analysis. We also report on very promising numerical experiments in which CycIP is compared to a commerical state-of-the-art optimization solver.KeywordsConvex setFeasibility problemHalfspaceIntrepid projectionLinear inequalitiesProjectionRoad designAMS 2010 Subject ClassificationPrimary 65K0590C25Secondary 90C05

Generalized distances give rise to generalized projections into convex sets. An important question is whether or not one can use within the same projection algorithm different types of such generalized projections. This question has practical consequences in the area of signal detection and image recovery in situations that can be formulated mathematically as a convex feasibility problem. Using an extension of Pierra's product space formalism, we show here that a multiprojection algorithm converges. Our algorithm is fully simultaneous, i.e., it uses in each iterative stepall sets of the convex feasibility problem. Different multiprojection algorithms can be derived from our algorithmic scheme by a judicious choice of the Bregman functions which govern the process. As a by-product of our investigation we also obtain blockiterative schemes for certain kinds of linearly constraned optimization problems.

Problems in signal detection and image recovery can sometimes be formulated as a convex feasibility problem (CFP) of finding a vector in the intersection of a finite family of closed convex sets. Algorithms for this purpose typically employ orthogonal or generalized projections onto the individual convex sets. The simultaneous multiprojection algorithm of Censor and Elfving for solving the CFP, in which different generalized projections may be used at the same time, has been shown to converge for the case of nonempty intersection; still open is the question of its convergence when the intersection of the closed convex sets is empty.

Fixed-Point Algorithms for Inverse Problems in Science and Engineering presents some ofthe most recent work from top-notch researchers studying projection and other first-order fixed-point algorithms in several areas of mathematics and the applied sciences. The material presented provides a survey of the state-of-the-art theory and practice in fixed-point algorithms, identifying emerging problems driven by applications, and discussing new approaches for solving these problems. This book incorporates diverse perspectives from broad-ranging areas of research including, variational analysis, numerical linear algebra, biotechnology, materials science, computational solid-state physics, and chemistry. Topics presented include: Theory of Fixed-point algorithms: convex analysis, convex optimization, subdifferential calculus, nonsmooth analysis, proximal point methods, projection methods, resolvent and related fixed-point theoretic methods, and monotone operator theory. Numerical analysis of fixed-point algorithms: choice of step lengths, of weights, of blocks for block-iterative and parallel methods, and of relaxation parameters; regularization of ill-posed problems; numerical comparison of various methods. Areas of Applications: engineering (image and signal reconstruction and decompression problems), computer tomography and radiation treatment planning (convex feasibility problems), astronomy (adaptive optics), crystallography (molecular structure reconstruction), computational chemistry (molecular structure simulation) and other areas. Because of the variety of applications presented, this book can easily serve as a basis for new and innovated research and collaboration.

In image coding, image information can often be transmitted with relatively little bandwidth requirement. This is the case in the pixel domain if there are large blocks of homogeneous intensity within a single frame, or when comparing adjacent frames there are large blocks of unchanged pixels. In the frequency domain, this is the case when only a few of the frequency coefficients are deemed significant. This paper presents an algorithm for improving the bandwidth compression of the error pattern generated by typical motion-compensated coding techniques. The algorithm involves two well established techniques; conventional transform coding and the method of successive Projection onto Convex Sets.