Strong duality in Lasserre’s hierarchy for polynomial optimization

This issue's SIGEST paper, from the SIAM Journal on Optimization (SIOPT), is an excellent example of deep mathematical analysis, involving a range of mathematical techniques, which is having significant practical algorithmic and computational consequences. The title of the paper, “A Sum of Squares Approximation of Nonnegative Polynomials,” by Jean B. Lasserre, combines two concepts that are known to all SIAM readers: sums of squares, the types of functions that arise in many data fitting and other applications; and nonnegative polynomials, polynomials whose values are greater than or equal to zero for all inputs. It is obvious that any polynomial that can be written as a sum of squares is nonnegative. It is far from obvious whether any nonnegative polynomial has a representation as, or arbitrarily close to, a sum of squares of polynomials. In this SIGEST paper, Lasserre shows that this is the case. The nomination from the editors of SIOPT captures the immediate and more general contributions of this paper eloquently, in the context of a series of three papers of which this is the second: “Jean Lasserre, in a series of papers (starting from the first paper in SIOPT Vol. 11, pp. 796–817, and concluding with the third paper in SIOPT Vol. 15, pp. 383–393) by exploiting deep results from real algebraic geometry and theory of moments for multivariate polynomials, showed that the optimal value of a nonconvex polynomial optimization problem (i.e., an optimization problem whose objective function and constraints are specified by multivariate polynomials) can asymptotically be approximated by an increasing sequence of optimal values from increasing tighter semidefinite programming relaxation problems. Even though the result is asymptotic in the most general case, fortuitously, practical experience has revealed that very often, the first few levels of semidefinite programming relaxations typically can achieve the global optimum value. The theoretical connection that Jean Lasserre (and Pablo Parrilo at about the same time from the viewpoint of sum of squares representations of multivariate polynomials) had established between polynomial optimizations with real algebraic geometry and theory of moments has led to exciting developments in combinatorial optimization. In particular, many hard combinatorial problems such as the maximum clique problem of a graph and the quadratic assignment problem are now actively revisited by researchers under the framework of polynomial optimization to derive increasingly tighter bounds.” That is, a consequence of this and related work is the ability to use a sequence of semidefinite programming problems to (nearly) solve much more difficult optimization problems. Semidefinite programming problems are problems with a linear objective function and linear constraints in a matrix variable X, plus the additional constraint that X be positive semidefinite. Since a rich array of efficient algorithms for solving semidefinite programming problems has been developed in recent years, reducing more general problems to a sequence of semidefinite programming problems is an important advance. We hope that readers of SIAM Review will enjoy this glimpse into the sophisticated mathematics behind an important algorithmic innovation in optimization.

We propose a multigrid approach for the global optimization of polynomial optimization problems with sparse support. The problems we consider arise from the discretization of infinite dimensional optimization problems, such as PDE optimization problems, boundary value problems, and some global optimization applications. In many of these applications, the level of discretization can be used to obtain a hierarchy of optimization models that capture the underlying infinite dimensional problem at different degrees of fidelity. This approach, inspired by multigrid methods, has been successfully used for decades to solve large systems of linear equations. However, multigrid methods are difficult to apply to semidefinite programming (SDP) relaxations of polynomial optimization problems. The main difficulty is that the information between grids is lost when the original problem is approximated via an SDP relaxation. Despite the loss of information, we develop a multigrid approach and propose prolongation operator...

We propose a hierarchy of semidefinite programming (SDP) relaxations for polynomial optimization with sparse patterns over unbounded feasible sets. The convergence of the proposed SDP hierarchy is established for a class of polynomial optimization problems. This is done by employing known sums-of-squares sparsity techniques of Kojima and Muramatsu Comput Optim Appl 42(1):31---41, (2009) and Lasserre SIAM J Optim 17:822---843, (2006) together with a representation theorem for polynomials over unbounded sets obtained recently in Jeyakumar et al. J Optim Theory Appl 163(3):707---718, (2014). We demonstrate that the proposed sparse SDP hierarchy can solve some classes of large scale polynomial optimization problems with unbounded feasible sets using the polynomial optimization solver SparsePOP developed by Waki et al. ACM Trans Math Softw 35:15 (2008).

Identification Aspects of SDP Based Polynomial Optimization Relaxations

We consider a property of positive polynomials on a compact set with a small perturbation. When applied to a polynomial optimization problem (POP), the property implies that the optimal value of the corresponding SemiDefinite Programming (SDP) relaxation with sufficiently large relaxation order is bounded from below by and from above by , where is the optimal value of the POP. We propose new SDP relaxations for POP based on modifications of existing sums-of-squares representation theorems. An advantage of our SDP relaxations is that in many cases they are of considerably smaller dimension than those originally proposed by Lasserre. We present some applications and the results of our computational experiments.

A SemiDefinite Programming (SDP) problem is one of the most central problems in mathematical optimization. SDP provides an effective computation framework for many research fields. Some applications, however, require solving a large-scale SDP whose size exceeds the capacity of a single processor both in terms of computation time and available memory. SDPARA (SemiDefinite Programming Algorithm paRAllel package) [Yamashita et al. 2003b] was designed to solve such large-scale SDPs. Its parallel performance is outstanding for general SDPs in most cases. However, the parallel implementation is less successful for some sparse SDPs obtained from applications such as Polynomial Optimization Problems (POPs) or Sensor Network Localization (SNL) problems, since this version of SDPARA cannot directly handle sparse Schur Complement Matrices (SCMs). In this article we improve SDPARA by focusing on the sparsity of the SCM and we propose a new parallel implementation using the formula-cost-based distribution along with a replacement of the dense Cholesky factorization. We verify numerically that these features are key to solving SDPs with sparse SCMs more quickly on parallel computing systems. The performance is further enhanced by multithreading and the new SDPARA attains considerable scalability in general. It also finds solutions for extremely large-scale SDPs arising from POPs which cannot be obtained by other solvers.

A polynomial optimization problem whose objective function is represented as a sum of positive and even powers of polynomials, called a polynomial least squares problem, is considered. Methods to transform a polynomial least square problem to polynomial semidefinite programs to reduce degrees of the polynomials are discussed. Computational efficiency of solving the original polynomial least squares problem and the transformed polynomial semidefinite programs is compared. Numerical results on selected polynomial least square problems show better computational performance of a transformed polynomial semidefinite program, especially when degrees of the polynomials are larger.

This short note extends the sparse SOS (sum of squares) and SDP (semidefinite programming) relaxation proposed by Waki, Kim, Kojima and Muramatsu for normal POPs (polynomial optimization problems) to POPs over symmetric cones, and establishes its theoretical convergence based on the recent convergence result by Lasserre on the sparse SOS and SDP relaxation for normal POPs. A numerical example is also given to exhibit its high potential.

We consider semidefinite programming (SDP) formulations of certain truss topology optimization problems, where a lower bound is imposed on the fundamental frequency of vibration of the truss structure. These SDP formulations were introduced in Ohsaki et al. (Comp. Meth. Appl. Mech. Eng. 180:203–217, 1999). We show how one may automatically obtain symmetric designs, by eliminating the ‘redundant’ symmetry in the SDP problem formulation. This has the advantage that the original SDP problem is substantially reduced in size for trusses with large symmetry groups.

We consider the problem of estimating the discrete clustering structures under the sub-Gaussian mixture model. Our main results establish a hidden integrality property of a semidefinite programming (SDP) relaxation for this problem: while the optimal solution to the SDP is not integer-valued in general, its estimation error can be upper bounded by that of an idealized integer program. The error of the integer program, and hence that of the SDP, are further shown to decay exponentially in the signal-to-noise ratio. In addition, we show that the SDP relaxation is robust under the semirandom setting in which an adversary can modify the data generated from the mixture model. In particular, we generalize the hidden integrality property to the semirandom model and thereby show that SDP achieves the optimal error bound in this setting. These results together highlight the “global-to-local” mechanism that drives the performance of the SDP relaxation. To the best of our knowledge, our result is the first exponentially decaying error bound for convex relaxations of mixture models. A corollary of our results shows that in certain regimes, the SDP solutions are in fact integral and exact. More generally, our results establish sufficient conditions for the SDP to correctly recover the cluster memberships of [Formula: see text] fraction of the points for any [Formula: see text]. As a special case, we show that under the [Formula: see text]-dimensional stochastic ball model, SDP achieves nontrivial (sometimes exact) recovery when the center separation is as small as [Formula: see text], which improves upon previous exact recovery results that require constant separation.

SparsePOP is a Matlab implementation of the sparse semidefinite programming (SDP) relaxation method for approximating a global optimal solution of a polynomial optimization problem (POP) proposed by Waki et al. [2006]. The sparse SDP relaxation exploits a sparse structure of polynomials in POPs when applying “a hierarchy of LMI relaxations of increasing dimensions” Lasserre [2006]. The efficiency of SparsePOP to approximate optimal solutions of POPs is thus increased, and larger-scale POPs can be handled.

Semialgebraic computations, i.e., the manipulation of sets and logical conditions defined by polynomial inequalities in real variables, are an essential ”primitive” in the analysis and design of hybrid dynamical systems. Fundamental tasks such as reachability analysis, abstraction verification, and the computation of stability and performance certificates, all use these operations extensively, and can quickly become the computational bottleneck in the design process. Although there is a well-developed body of both basic theory and algorithms for these tasks, the practical performance of most available methods is still far from being satisfactory on real-world problems. While there are several possible causes for this (besides the NP-hardness of the task), a sensible explanation lies in the purely algebraic nature of the usual methods, as well as the insistence on exact (as opposed to approximate or ”relaxed”) solutions. For these reasons, there is a strong interest in the development of efficient techniques for (perhaps restricted) classes of semialgebraic problems. In this talk we review the basic elements and present several new results on the SOS approach to semialgebraic computations, that combines symbolic and numerical techniques from real algebra and convex optimization. Its main defining feature is the use and computation of sum of squares (SOS) decompositions for multivariate polynomials via semidefinite programming. These are extended, using the Positivstellensatz, to structured infeasibility certificates for polynomial equations and inequalities. The developed techniques unify and generalize many well-known existing methods. In particular, we will discuss semialgebraic problems with at most two quantifier alternations (i.e., classical polynomial optimization problems as well as the related games and minimax problems). As an example, we will solve in detail a class of zero-sum two-person games with an infinite number of pure strategies, where the payoff function is a polynomial expression of the actions of the players. Although particular emphasis will be given to the hybrid systems viewpoint, the basic ideas and algorithms, as well as these recent extensions, will be illustrated with examples drawn from a broad range of related domains, including dynamical systems and geometric theorem proving.

Statistical inference problems arising within signal processing, data mining, and machine learning naturally give rise to hard combinatorial optimization problems. These problems become intractable when the dimensionality of the data is large, as is often the case for modern datasets. A popular idea is to construct convex relaxations of these combinatorial problems, which can be solved efficiently for large-scale datasets. Semidefinite programming (SDP) relaxations are among the most powerful methods in this family and are surprisingly well suited for a broad range of problems where data take the form of matrices or graphs. It has been observed several times that when the statistical noise is small enough, SDP relaxations correctly detect the underlying combinatorial structures. In this paper we develop asymptotic predictions for several detection thresholds, as well as for the estimation error above these thresholds. We study some classical SDP relaxations for statistical problems motivated by graph synchronization and community detection in networks. We map these optimization problems to statistical mechanics models with vector spins and use nonrigorous techniques from statistical mechanics to characterize the corresponding phase transitions. Our results clarify the effectiveness of SDP relaxations in solving high-dimensional statistical problems.

In this chapter we consider polynomial optimization problems, asking to minimize a polynomial function over a compact semialgebraic set, defined by polynomial inequalities. This models a great variety of (in general, nonlinear nonconvex) optimization problems. Various hierarchies of (lower and upper) bounds have been introduced, having the remarkable property that they converge asymptotically to the global minimum. These bounds exploit algebraic representations of positive polynomials in terms of sums of squares and can be computed using semidefinite optimization. Our focus in this chapter lies in the performance analysis of these hierarchies of bounds, namely, in how far the bounds are from the global minimum as the degrees of the sums of squares they involve tend to infinity. We present the main state-of-the-art results and offer a gentle introductory overview over the various techniques that have been recently developed to establish them, stemming from the theory of orthogonal polynomials, approximation theory, Fourier analysis, and more.

A Semidefinite Programming (SDP) relaxation is an effective computational method to solve a Sensor Network Localization problem, which attempts to determine the locations of a group of sensors given the distances between some of them. In this paper, we analyze and determine new sufficient conditions and formulations that guarantee that the SDP relaxation is exact, i.e., gives the correct solution. These conditions can be useful for designing sensor networks and managing connectivities in practice. Our main contribution is threefold: First, we present the first non-asymptotic bound on the connectivity (or radio) range requirement of randomly distributed sensors in order to ensure the network is uniquely localizable with high probability. Determining this range is a key component in the design of sensor networks, and we provide a result that leads to a correct localization of each sensor, for any number of sensors. Second, we introduce a new class of graphs that can always be correctly localized by an SDP relaxation. Specifically, we show that adding a simple objective function to the SDP relaxation model will ensure that the solution is correct when applied to a triangulation graph. Since triangulation graphs are very sparse, this is informationally efficient, requiring an almost minimal amount of distance information. Finally, we analyze a number of objective functions for the SDP relaxation to solve the localization problem for a general graph.Key wordsSensor network localizationGraph realizationSemidefinite programmingSubject Classifications90C2290C4690C9090B18