Solving polynomial least squares problems via semidefinite programming relaxations

We propose a multigrid approach for the global optimization of polynomial optimization problems with sparse support. The problems we consider arise from the discretization of infinite dimensional optimization problems, such as PDE optimization problems, boundary value problems, and some global optimization applications. In many of these applications, the level of discretization can be used to obtain a hierarchy of optimization models that capture the underlying infinite dimensional problem at different degrees of fidelity. This approach, inspired by multigrid methods, has been successfully used for decades to solve large systems of linear equations. However, multigrid methods are difficult to apply to semidefinite programming (SDP) relaxations of polynomial optimization problems. The main difficulty is that the information between grids is lost when the original problem is approximated via an SDP relaxation. Despite the loss of information, we develop a multigrid approach and propose prolongation operator...

We propose a hierarchy of semidefinite programming (SDP) relaxations for polynomial optimization with sparse patterns over unbounded feasible sets. The convergence of the proposed SDP hierarchy is established for a class of polynomial optimization problems. This is done by employing known sums-of-squares sparsity techniques of Kojima and Muramatsu Comput Optim Appl 42(1):31---41, (2009) and Lasserre SIAM J Optim 17:822---843, (2006) together with a representation theorem for polynomials over unbounded sets obtained recently in Jeyakumar et al. J Optim Theory Appl 163(3):707---718, (2014). We demonstrate that the proposed sparse SDP hierarchy can solve some classes of large scale polynomial optimization problems with unbounded feasible sets using the polynomial optimization solver SparsePOP developed by Waki et al. ACM Trans Math Softw 35:15 (2008).

A Semidefinite Programming (SDP) relaxation is an effective computational method to solve a Sensor Network Localization problem, which attempts to determine the locations of a group of sensors given the distances between some of them. In this paper, we analyze and determine new sufficient conditions and formulations that guarantee that the SDP relaxation is exact, i.e., gives the correct solution. These conditions can be useful for designing sensor networks and managing connectivities in practice. Our main contribution is threefold: First, we present the first non-asymptotic bound on the connectivity (or radio) range requirement of randomly distributed sensors in order to ensure the network is uniquely localizable with high probability. Determining this range is a key component in the design of sensor networks, and we provide a result that leads to a correct localization of each sensor, for any number of sensors. Second, we introduce a new class of graphs that can always be correctly localized by an SDP relaxation. Specifically, we show that adding a simple objective function to the SDP relaxation model will ensure that the solution is correct when applied to a triangulation graph. Since triangulation graphs are very sparse, this is informationally efficient, requiring an almost minimal amount of distance information. Finally, we analyze a number of objective functions for the SDP relaxation to solve the localization problem for a general graph.Key wordsSensor network localizationGraph realizationSemidefinite programmingSubject Classifications90C2290C4690C9090B18

This issue's SIGEST paper, from the SIAM Journal on Optimization (SIOPT), is an excellent example of deep mathematical analysis, involving a range of mathematical techniques, which is having significant practical algorithmic and computational consequences. The title of the paper, “A Sum of Squares Approximation of Nonnegative Polynomials,” by Jean B. Lasserre, combines two concepts that are known to all SIAM readers: sums of squares, the types of functions that arise in many data fitting and other applications; and nonnegative polynomials, polynomials whose values are greater than or equal to zero for all inputs. It is obvious that any polynomial that can be written as a sum of squares is nonnegative. It is far from obvious whether any nonnegative polynomial has a representation as, or arbitrarily close to, a sum of squares of polynomials. In this SIGEST paper, Lasserre shows that this is the case. The nomination from the editors of SIOPT captures the immediate and more general contributions of this paper eloquently, in the context of a series of three papers of which this is the second: “Jean Lasserre, in a series of papers (starting from the first paper in SIOPT Vol. 11, pp. 796–817, and concluding with the third paper in SIOPT Vol. 15, pp. 383–393) by exploiting deep results from real algebraic geometry and theory of moments for multivariate polynomials, showed that the optimal value of a nonconvex polynomial optimization problem (i.e., an optimization problem whose objective function and constraints are specified by multivariate polynomials) can asymptotically be approximated by an increasing sequence of optimal values from increasing tighter semidefinite programming relaxation problems. Even though the result is asymptotic in the most general case, fortuitously, practical experience has revealed that very often, the first few levels of semidefinite programming relaxations typically can achieve the global optimum value. The theoretical connection that Jean Lasserre (and Pablo Parrilo at about the same time from the viewpoint of sum of squares representations of multivariate polynomials) had established between polynomial optimizations with real algebraic geometry and theory of moments has led to exciting developments in combinatorial optimization. In particular, many hard combinatorial problems such as the maximum clique problem of a graph and the quadratic assignment problem are now actively revisited by researchers under the framework of polynomial optimization to derive increasingly tighter bounds.” That is, a consequence of this and related work is the ability to use a sequence of semidefinite programming problems to (nearly) solve much more difficult optimization problems. Semidefinite programming problems are problems with a linear objective function and linear constraints in a matrix variable X, plus the additional constraint that X be positive semidefinite. Since a rich array of efficient algorithms for solving semidefinite programming problems has been developed in recent years, reducing more general problems to a sequence of semidefinite programming problems is an important advance. We hope that readers of SIAM Review will enjoy this glimpse into the sophisticated mathematics behind an important algorithmic innovation in optimization.

We study two instances of polynomial optimization problem over a single sphere. The first problem is to compute the best rank-1 tensor approximation. We show the equivalence between two recent semidefinite relaxations methods. The other one arises from Bose-Einstein condensates (BEC), whose objective function is a summation of a probably nonconvex quadratic function and a quartic term. These two polynomial optimization problems are closely connected since the BEC problem can be viewed as a structured fourth-order best rank-1 tensor approximation. We show that the BEC problem is NP-hard and propose a semidefinite relaxation with both deterministic and randomized rounding procedures. Explicit approximation ratios for these rounding procedures are presented. The performance of these semidefinite relaxations are illustrated on a few preliminary numerical experiments.

Statistical inference problems arising within signal processing, data mining, and machine learning naturally give rise to hard combinatorial optimization problems. These problems become intractable when the dimensionality of the data is large, as is often the case for modern datasets. A popular idea is to construct convex relaxations of these combinatorial problems, which can be solved efficiently for large-scale datasets. Semidefinite programming (SDP) relaxations are among the most powerful methods in this family and are surprisingly well suited for a broad range of problems where data take the form of matrices or graphs. It has been observed several times that when the statistical noise is small enough, SDP relaxations correctly detect the underlying combinatorial structures. In this paper we develop asymptotic predictions for several detection thresholds, as well as for the estimation error above these thresholds. We study some classical SDP relaxations for statistical problems motivated by graph synchronization and community detection in networks. We map these optimization problems to statistical mechanics models with vector spins and use nonrigorous techniques from statistical mechanics to characterize the corresponding phase transitions. Our results clarify the effectiveness of SDP relaxations in solving high-dimensional statistical problems.

We consider a property of positive polynomials on a compact set with a small perturbation. When applied to a polynomial optimization problem (POP), the property implies that the optimal value of the corresponding SemiDefinite Programming (SDP) relaxation with sufficiently large relaxation order is bounded from below by and from above by , where is the optimal value of the POP. We propose new SDP relaxations for POP based on modifications of existing sums-of-squares representation theorems. An advantage of our SDP relaxations is that in many cases they are of considerably smaller dimension than those originally proposed by Lasserre. We present some applications and the results of our computational experiments.

A polynomial optimization problem (POP) consists of minimizing a multivariate real polynomial on a semi-algebraic set \(K\) described by polynomial inequalities and equations. In its full generality it is a non-convex, multi-extremal, difficult global optimization problem. More than an decade ago, J. B. Lasserre proposed to solve POPs by a hierarchy of convex semidefinite programming (SDP) relaxations of increasing size. Each problem in the hierarchy has a primal SDP formulation (a relaxation of a moment problem) and a dual SDP formulation (a sum-of-squares representation of a polynomial Lagrangian of the POP). In this note, we show that there is no duality gap between each primal and dual SDP problem in Lasserre’s hierarchy, provided one of the constraints in the description of set \(K\) is a ball constraint. Our proof uses elementary results on SDP duality, and it does not assume that \(K\) has a strictly feasible point.

Identification Aspects of SDP Based Polynomial Optimization Relaxations

We consider the problem of estimating the discrete clustering structures under the sub-Gaussian mixture model. Our main results establish a hidden integrality property of a semidefinite programming (SDP) relaxation for this problem: while the optimal solution to the SDP is not integer-valued in general, its estimation error can be upper bounded by that of an idealized integer program. The error of the integer program, and hence that of the SDP, are further shown to decay exponentially in the signal-to-noise ratio. In addition, we show that the SDP relaxation is robust under the semirandom setting in which an adversary can modify the data generated from the mixture model. In particular, we generalize the hidden integrality property to the semirandom model and thereby show that SDP achieves the optimal error bound in this setting. These results together highlight the “global-to-local” mechanism that drives the performance of the SDP relaxation. To the best of our knowledge, our result is the first exponentially decaying error bound for convex relaxations of mixture models. A corollary of our results shows that in certain regimes, the SDP solutions are in fact integral and exact. More generally, our results establish sufficient conditions for the SDP to correctly recover the cluster memberships of [Formula: see text] fraction of the points for any [Formula: see text]. As a special case, we show that under the [Formula: see text]-dimensional stochastic ball model, SDP achieves nontrivial (sometimes exact) recovery when the center separation is as small as [Formula: see text], which improves upon previous exact recovery results that require constant separation.

A method for quickly bounding the optimal objective value of an OPF problem using a semidefinite relaxation and a local solution

Cooperative beamforming in relay networks is considered, in which a source transmits to its destination with the help of a set of cooperating nodes. The source first transmits locally. The cooperating nodes that receive the source signal retransmit a weighted version of it in an amplify-and-forward (AF) fashion. Assuming knowledge of the second-order statistics of the channel state information, beamforming weights are determined so that the signal-to-noise ratio (SNR) at the destination is maximized subject to two different power constraints, i.e., a total (source and relay) power constraint, and individual relay power constraints. For the former constraint, the original problem is transformed into a problem of one variable, which can be solved via Newton's method. For the latter constraint, the original problem is transformed into a homogeneous quadratically constrained quadratic programming (QCQP) problem. In this case, it is shown that when the number of relays does not exceed three the global solution can always be constructed via semidefinite programming (SDP) relaxation and the matrix rank-one decomposition technique. For the cases in which the SDP relaxation does not generate a rank one solution, two methods are proposed to solve the problem: the first one is based on the coordinate descent method, and the second one transforms the QCQP problem into an infinity norm maximization problem in which a smooth finite norm approximation can lead to the solution using the augmented Lagrangian method.

This short note extends the sparse SOS (sum of squares) and SDP (semidefinite programming) relaxation proposed by Waki, Kim, Kojima and Muramatsu for normal POPs (polynomial optimization problems) to POPs over symmetric cones, and establishes its theoretical convergence based on the recent convergence result by Lasserre on the sparse SOS and SDP relaxation for normal POPs. A numerical example is also given to exhibit its high potential.

Many important NP-hard combinatorial problems can be efficiently approximated using semidefinite programming relaxations. We propose a new hierarchy of semidefinite relaxations for classes of such problems that are based on graphs and for which the projection of the problem onto a subgraph shares the same structure as the original problem. This includes the well-studied max-cut and stable-set problems. Each level k of the proposed hierarchy consists of the basic semidefinite relaxation of the problem augmented by the constraints enforcing the structural projection condition on every k-node subgraph of the problem. This hierarchy has the distinguishing feature that all the relaxations are formulated in the space of the original semidefinite relaxation. Because the size of the relaxations increases rapidly with the number of subgraphs, we explore the possibility of adding the projection constraints only for selected subgraphs. Preliminary computational results show that the proposed hierarchy yields improved bounds when compared to the initial relaxation for benchmark instances of the max-cut and stable-set problems, and that the improved bounds result in significantly smaller enumeration trees when the relaxation is used in a branch-and-bound scheme.

A SemiDefinite Programming (SDP) problem is one of the most central problems in mathematical optimization. SDP provides an effective computation framework for many research fields. Some applications, however, require solving a large-scale SDP whose size exceeds the capacity of a single processor both in terms of computation time and available memory. SDPARA (SemiDefinite Programming Algorithm paRAllel package) [Yamashita et al. 2003b] was designed to solve such large-scale SDPs. Its parallel performance is outstanding for general SDPs in most cases. However, the parallel implementation is less successful for some sparse SDPs obtained from applications such as Polynomial Optimization Problems (POPs) or Sensor Network Localization (SNL) problems, since this version of SDPARA cannot directly handle sparse Schur Complement Matrices (SCMs). In this article we improve SDPARA by focusing on the sparsity of the SCM and we propose a new parallel implementation using the formula-cost-based distribution along with a replacement of the dense Cholesky factorization. We verify numerically that these features are key to solving SDPs with sparse SCMs more quickly on parallel computing systems. The performance is further enhanced by multithreading and the new SDPARA attains considerable scalability in general. It also finds solutions for extremely large-scale SDPs arising from POPs which cannot be obtained by other solvers.