Projection-free Online Learning over Strongly Convex Sets

Many real-world problems can often be cast as the optimization of DR-submodular functions defined over a convex domain. These functions play an important role with applications in many areas of applied mathematics, such as machine learning, computer vision, operation research, communication systems or economics. In addition, they capture a subclass of non-convex optimization that provides both practical and theoretical guarantees. In this paper, we show that for maximizing non-monotone DR-submodular functions over a general convex set (such as up-closed convex sets, conic convex set, etc) the Frank-Wolfe algorithm achieves an approximation guarantee which depends on the convex set. To the best of our knowledge, this is the first approximation guarantee. Finally we benchmark our algorithm on problems arising in machine learning domain with the real-world datasets.

In this paper, we study properties of general closed convex sets that determine the closedness and polyhedrality of the convex hull of integer points contained in it. We first present necessary and sufficient conditions for the convex hull of integer points contained in a general convex set to be closed. This leads to useful results for special classes of convex sets such as pointed cones, strictly convex sets, and sets containing integer points in their interior. We then present a sufficient condition for the convex hull of integer points in general convex sets to be a polyhedron. This result generalizes the well-known result due to Meyer (Math Program 7:223–225, 1974). Under a simple technical assumption, we show that these sufficient conditions are also necessary for the convex hull of integer points contained in general convex sets to be a polyhedron.

In this paper we generalize the factorization theorem of Gouveia, Parrilo and Thomas to a broader class of convex sets. Given a general convex set, we define a slack operator associated to the set and its polar according to whether the convex set is full dimensional, whether it is a translated cone and whether it contains lines. We strengthen the condition of a cone lift by requiring not only the convex set is the image of an affine slice of a given closed convex cone, but also its recession cone is the image of the linear slice of the closed convex cone. We show that the generalized lift of a convex set can also be characterized by the cone factorization of a properly defined slack operator.

The notion of relaxation is well understood for orthogonal projections onto convex sets. For general Bregman projections it was considered only for hyperplanes, and the question of how to relax Bregman projections onto convex sets that are not linear (i.e., not hyperplanes or half-spaces) has remained open. A definition of the underrelaxation of Bregman projections onto general convex sets is given here, which includes as special cases the underrelaxed orthogonal projections and the underrelaxed Bregman projections onto linear sets as given by De Pierro and Iusem [J. Optim. Theory Appl., 51 (1986), pp. 421--440]. With this new definition, we construct a block-iterative projection algorithmic scheme and prove its convergence to a solution of the convex feasibility problem. The practical importance of relaxation parameters in the application of such projection algorithms to real-world problems is demonstrated on a problem of image reconstruction from projections.

We study some methods of subgradient projections for solving a convex feasibility problem with general (not necessarily hyperplanes or half-spaces) convex sets in the inconsistent case and propose a strategy that controls the relaxation parameters in a specific self-adapting manner. This strategy leaves enough user-flexibility but gives a mathematical guarantee for the algorithm's behavior in the inconsistent case. We present numerical results of computational experiments that illustrate the computational advantage of the new method.

This book describes and analyzes all available alternating projection methods for solving the general problem of finding a point in the intersection of several given sets belonging to a Hilbert space. For each method the authors describe and analyze convergence, speed of convergence, acceleration techniques, stopping criteria, and applications. Different types of algorithms and applications are studied for subspaces, linear varieties, and general convex sets. The authors also unify these algorithms into a common theoretical framework. Alternating Projection Methods is a comprehensive and accessible source of information, providing readers with the theoretical and practical aspects of the most relevant alternating projection methods. It features several acceleration techniques for every method it presents and analyzes, including schemes that cannot be found in other books. It also provides full descriptions of several important mathematical problems and specific applications for which the alternating projection methods represent an efficient option. Examples and problems that illustrate this material are also included. Audience: This book can be used as a textbook for advanced undergraduate or first-year graduate students. Because it is comprehensive, it can also be used as a tutorial or a reference by mathematicians and nonmathematicians from many fields of application who need to solve alternating projection problems in their work. Contents: Preface; Chapter 1: Introduction; Chapter 2: Overview on Spaces; Chapter 3: The MAP on Subspaces; Chapter 4: Row-Action Methods; Chapter 5: Projecting on Convex Sets; Chapter 6: Applications of MAP for Matrix Problems; Bibliography; Author Index; Subject Index.

It is known that the curvature of the feasible set in convex optimization allows for algorithms with better convergence rates, and there is renewed interest in this topic for both off-line and online problems. In this paper, leveraging results on geometry and convex analysis, we further our understanding of the role of curvature in optimization: We first show the equivalence of two notions of curvature, namely, strong convexity and gauge bodies, proving a conjecture of Abernethy et al. As a consequence, this shows that the Frank–Wolfe–type method of Wang and Abernethy has accelerated convergence rate [Formula: see text] over strongly convex feasible sets without additional assumptions on the (convex) objective function. In online linear optimization, we identify two main properties that help explaining why/when follow the leader (FTL) has only logarithmic regret over strongly convex sets. This allows one to directly recover and slightly extend a recent result of Huang et al., and to show that FTL has logarithmic regret over strongly convex sets whenever the gain vectors are nonnegative. We provide an efficient procedure for approximating convex bodies by strongly convex ones while smoothly trading off approximation error and curvature. This allows one to extend the improved algorithms over strongly convex sets to general convex sets. As a concrete application, we extend results on online linear optimization with hints to general convex sets. Funding: This work was supported by Conselho Nacional de Desenvolvimento Científico e Tecnológico [Grant Bolsa de Produtividade em Pesquisa #4310516/2017-0] Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Finance Code 001) Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro (Jovem Cientista do Nosso Estado), Bolsa de Produtividade em Pesquisa #312751/2021-4 from CNPq, FAPERJ [Grant “Jovem Cientista do Nosso Estado”].

The Stop Operator Related to a Convex Polyhedron

Suppose X is a random vector, that is distributed uniformly in some n-dimensional convex set. It was conjectured that when the dimension n is very large, there exists a non-zero vector u, such that the distribution of the real random variable <X,u> is close to the gaussian distribution. A well-understood situation, is when X is distributed uniformly over the n-dimensional cube. In this case, <X,u> is approximately gaussian for, say, the vector u = (1,...,1) / sqrt(n), as follows from the classical central limit theorem. We prove the conjecture for a general convex set. Moreover, when the expectation of X is zero, and the covariance of X is the identity matrix, we show that for 'most' unit vectors u, the random variable <X,u> is distributed approximately according to the gaussian law. We argue that convexity - and perhaps geometry in general - may replace the role of independence in certain aspects of the phenomenon represented by the central limit theorem.

A classical theorem of Spencer shows that any set system with $n$ sets and $n$ elements admits a coloring of discrepancy $O(\sqrt{n})$. Recent exciting work of Bansal, Lovett, and Meka shows that such colorings can be found in polynomial time. In fact, the Lovett--Meka algorithm finds a half integral point in any “large enough” polytope. However, their algorithm crucially relies on the facet structure and does not apply to general convex sets. We show that for any symmetric convex set $K$ with Gaussian measure at least $e^{-n/500}$, the following algorithm finds a point $y \in K \cap [-1,1]^n$ with $\Omega(n)$ coordinates in $\pm 1$: (1) take a random Gaussian vector $x$; (2) compute the point $y$ in $K \cap [-1,1]^n$ that is closest to $x$; (3) return $y$. This provides another truly constructive proof of Spencer's theorem and the first constructive proof of a theorem of Gluskin and Giannopoulos.

The theory of abstract convexity exists in many mathematical branches such as algebra, topology and order. The fuzzification is one of important directions towards the discussion of abstract convexity. The theory of ( L , M )-convex spaces is one very general fuzzy convexity since it includes many other fuzzy convexities as its special case. In this paper, we shall introduce a subcategory of ( L , M )-convex spaces and study its property. This subcategory is called enriched ( L , M )-convex spaces and denoted by ELMCS . The main results are: (1) ELMCS is a topological category; (2) ELMCS is a coreflective subcategory of the category of ( L , M )-convex spaces; (3) the category of M -fuzzifying convex spaces (denoted by MCS ) can be embedded in the category ELMCS as a reflective subcategory.

This paper is devoted to deriving the first circumcenter iteration scheme that does not employ a product space reformulation for finding a point in the intersection of two closed convex sets. We introduce a so-called centralized version of the circumcentered-reflection method (CRM). Developed with the aim of accelerating classical projection algorithms, CRM is successful for tracking a common point of a finite number of affine sets. In the case of general convex sets, CRM was shown to possibly diverge if Pierra's product space reformulation is not used. In this work, we prove that there exists an easily reachable region consisting of what we refer to as centralized points, where pure circumcenter steps possess properties yielding convergence. The resulting algorithm is called centralized CRM (cCRM). In addition to having global convergence, cCRM converges linearly under an error bound condition, and superlinearly if the two target sets are so that their intersection have nonempty interior and their boundaries are locally differentiable manifolds. We also run numerical experiments with successful results.

This article is concerned with the approximation of unbounded convex sets by polyhedra. While there is an abundance of literature investigating this task for compact sets, results on the unbounded case are scarce. We first point out the connections between existing results before introducing a new notion of polyhedral approximation called left( varepsilon , delta right) -approximation that integrates the unbounded case in a meaningful way. Some basic results about left( varepsilon , delta right) -approximations are proved for general convex sets. In the last section, an algorithm for the computation of left( varepsilon , delta right) -approximations of spectrahedra is presented. Correctness and finiteness of the algorithm are proved.

On generalized convex sets and their applications

Generalized convex sets in the plane