Copositive Programming – a Survey

In this paper, we present a model based on copositive programming and semidefinite relaxations of it to prove properties on discrete-time piecewise affine systems. We consider invariant i.e. properties represented by a set and formulated as all reachable values are included in the set. Also, we restrict the analysis to sublevel sets of quadratic forms i.e. ellipsoids. In this case, to check the property is equivalent to solve a quadratic maximization problem under the constraint that the decision variable belongs to the reachable values set. This maximization problem is relaxed using an abstraction of reachable values set by a union of truncated ellipsoids.

The Lovasz theta number of a graph G can be viewed as a semidefinite programming relaxation of the stability number of G. It has recently been shown that a copositive strengthening of this semidefinite program in fact equals the stability number of G. We introduce a related strengthening of the Lovasz theta number toward the chromatic number of G, which is shown to be equal to the fractional chromatic number of G. Solving copositive programs is NP-hard. This motivates the study of tractable approximations of the copositive cone. We investigate the Parrilo hierarchy to approximate this cone and provide computational simplifications for the approximation of the chromatic number of vertex transitive graphs. We provide some computational results indicating that the Lovasz theta number can be strengthened significantly toward the fractional chromatic number of G on some Hamming graphs.

Semidefinite programming (SDP) covers a wide range of applications such as robust optimization, polynomial optimization, combinatorial optimization, system and control theory, financial engineering, machine learning, quantum information and quantum chemistry. In those applications, SDP problems can be large scale easily. Such large scale SDP problems often satisfy a certain sparsity characterized by a chordal graph structure. This sparsity is classified in two types. The one is the domain space sparsity (d-space sparsity) for positive semidefinite symmetric matrix variables involved in SDP problems, and the other the range space sparsity (r-space sparsity) for matrix-inequality constraints in SDP problems. In this short note, we survey how we exploit these two types of sparsities to solve large scale linear and nonlinear SDP problems. We refer to the paper [7] for more details.

Many of the currently best-known approximation algorithms for NP-hard optimization problems are based on Linear Programming (LP) and Semi-definite Programming (SDP) relaxations. Given its power, this class of algorithms seems to contain the most favourable candidates for outperforming the current state-of-the-art approximation guarantees for NP-hard problems, for which there still exists a gap between the inapproximability results and the approximation guarantees that we know how to achieve in polynomial time. In this thesis, we address both the power and the limitations of these relaxations, as well as the connection between the shortcomings of these relaxations and the inapproximability of the underlying problem. In the first part, we study the limitations of LP relaxations of well-known graph problems such as the Vertex Cover problem and the Independent Set problem. We prove that any small LP relaxation for the aforementioned problems, cannot have an integrality gap strictly better than $2$ and $\omega(1)$, respectively. Furthermore, our lower bound for the Independent Set problem also holds for any SDP relaxation. Prior to our work, it was only known that such LP relaxations cannot have an integrality gap better than $1.5$ for the Vertex Cover Problem, and better than $2$ for the Independent Set problem. In the second part, we study the so-called knapsack cover inequalities that are used in the current best relaxations for numerous combinatorial optimization problems of covering type. In spite of their widespread use, these inequalities yield LP relaxations of exponential size, over which it is not known how to optimize exactly in polynomial time. We address this issue and obtain LP relaxations of quasi-polynomial size that are at least as strong as that given by the knapsack cover inequalities. In the last part, we show a close connection between structural hardness for k-partite graphs and tight inapproximability results for scheduling problems with precedence constraints. This connection is inspired by a family of integrality gap instances of a certain LP relaxation. Assuming the hardness of an optimization problem on k-partite graphs, we obtain a hardness of $2-\varepsilon$ for the problem of minimizing the makespan for scheduling with preemption on identical parallel machines, and a super constant inapproximability for the problem of scheduling on related parallel machines. Prior to this result, it was only known that the first problem does not admit a PTAS, and the second problem is NP-hard to approximate within a factor strictly better than 2, assuming the Unique Games Conjecture.

A SemiDefinite Programming (SDP) problem is one of the most central problems in mathematical optimization. SDP provides an effective computation framework for many research fields. Some applications, however, require solving a large-scale SDP whose size exceeds the capacity of a single processor both in terms of computation time and available memory. SDPARA (SemiDefinite Programming Algorithm paRAllel package) [Yamashita et al. 2003b] was designed to solve such large-scale SDPs. Its parallel performance is outstanding for general SDPs in most cases. However, the parallel implementation is less successful for some sparse SDPs obtained from applications such as Polynomial Optimization Problems (POPs) or Sensor Network Localization (SNL) problems, since this version of SDPARA cannot directly handle sparse Schur Complement Matrices (SCMs). In this article we improve SDPARA by focusing on the sparsity of the SCM and we propose a new parallel implementation using the formula-cost-based distribution along with a replacement of the dense Cholesky factorization. We verify numerically that these features are key to solving SDPs with sparse SCMs more quickly on parallel computing systems. The performance is further enhanced by multithreading and the new SDPARA attains considerable scalability in general. It also finds solutions for extremely large-scale SDPs arising from POPs which cannot be obtained by other solvers.

The research presented here is an improved row-by-row (RBR) algorithm for the solution of boolean quadratic programming (BQP) problems. While a faster and implementable RBR method has been widely used for semidefinite programming (SDP) relaxed BQPs, it can be challenged by SDP relaxations because of the fact that it produce a tighter lower bounds than RBR on BQPs. On the other hand, solving SDP by interior point method (IPM) is computationally expensive for large scale problems. Departing from IPM, our methods provides better lower bound than the RBR algorithm by Wai et al. (IEEE international conference on acoustics, speech and signal processing ICASSP, 2011) and competitive with SDP solved by IPM. The method includes the SDP cut relaxation on the SDP and is solved by a modified RBR method. The algorithm has been tested on MATLAB platform and applied to several BQPs from BQPLIB (a library by the authors). Numerical experiments show that the proposed method outperform the previous RBR method proposed by several authors and the solution of BQP by IPM as well.

The nonnegative rank of an entrywise nonnegative matrix \(A \in \mathbb {R}^{m \times n}_+\) is the smallest integer \(r\) such that \(A\) can be written as \(A=UV\) where \(U \in \mathbb {R}^{m \times r}_+\) and \(V \in \mathbb {R}^{r \times n}_+\) are both nonnegative. The nonnegative rank arises in different areas such as combinatorial optimization and communication complexity. Computing this quantity is NP-hard in general and it is thus important to find efficient bounding techniques especially in the context of the aforementioned applications. In this paper we propose a new lower bound on the nonnegative rank which, unlike most existing lower bounds, does not solely rely on the matrix sparsity pattern and applies to nonnegative matrices with arbitrary support. The idea involves computing a certain nuclear norm with nonnegativity constraints which allows to lower bound the nonnegative rank, in the same way the standard nuclear norm gives lower bounds on the standard rank. Our lower bound is expressed as the solution of a copositive programming problem and can be relaxed to obtain polynomial-time computable lower bounds using semidefinite programming. We compare our lower bound with existing ones, and we show examples of matrices where our lower bound performs better than currently known ones.

This issue's SIGEST paper, from the SIAM Journal on Optimization (SIOPT), is an excellent example of deep mathematical analysis, involving a range of mathematical techniques, which is having significant practical algorithmic and computational consequences. The title of the paper, “A Sum of Squares Approximation of Nonnegative Polynomials,” by Jean B. Lasserre, combines two concepts that are known to all SIAM readers: sums of squares, the types of functions that arise in many data fitting and other applications; and nonnegative polynomials, polynomials whose values are greater than or equal to zero for all inputs. It is obvious that any polynomial that can be written as a sum of squares is nonnegative. It is far from obvious whether any nonnegative polynomial has a representation as, or arbitrarily close to, a sum of squares of polynomials. In this SIGEST paper, Lasserre shows that this is the case. The nomination from the editors of SIOPT captures the immediate and more general contributions of this paper eloquently, in the context of a series of three papers of which this is the second: “Jean Lasserre, in a series of papers (starting from the first paper in SIOPT Vol. 11, pp. 796–817, and concluding with the third paper in SIOPT Vol. 15, pp. 383–393) by exploiting deep results from real algebraic geometry and theory of moments for multivariate polynomials, showed that the optimal value of a nonconvex polynomial optimization problem (i.e., an optimization problem whose objective function and constraints are specified by multivariate polynomials) can asymptotically be approximated by an increasing sequence of optimal values from increasing tighter semidefinite programming relaxation problems. Even though the result is asymptotic in the most general case, fortuitously, practical experience has revealed that very often, the first few levels of semidefinite programming relaxations typically can achieve the global optimum value. The theoretical connection that Jean Lasserre (and Pablo Parrilo at about the same time from the viewpoint of sum of squares representations of multivariate polynomials) had established between polynomial optimizations with real algebraic geometry and theory of moments has led to exciting developments in combinatorial optimization. In particular, many hard combinatorial problems such as the maximum clique problem of a graph and the quadratic assignment problem are now actively revisited by researchers under the framework of polynomial optimization to derive increasingly tighter bounds.” That is, a consequence of this and related work is the ability to use a sequence of semidefinite programming problems to (nearly) solve much more difficult optimization problems. Semidefinite programming problems are problems with a linear objective function and linear constraints in a matrix variable X, plus the additional constraint that X be positive semidefinite. Since a rich array of efficient algorithms for solving semidefinite programming problems has been developed in recent years, reducing more general problems to a sequence of semidefinite programming problems is an important advance. We hope that readers of SIAM Review will enjoy this glimpse into the sophisticated mathematics behind an important algorithmic innovation in optimization.

Finding global optimum of a non-convex quadratic function is in general a very difficult task even when the feasible set is a polyhedron. We show that when the feasible set of a quadratic problem consists of orthogonal matrices from $${\mathbb{R}^{n\times k}}$$ , then we can transform it into a semidefinite program in matrices of order kn which has the same optimal value. This opens new possibilities to get good lower bounds for several problems from combinatorial optimization, like the Graph partitioning problem (GPP), the Quadratic assignment problem (QAP) etc. In particular we show how to improve significantly the well-known Donath-Hoffman eigenvalue lower bound for GPP by semidefinite programming. In the last part of the paper we show that the copositive strengthening of the semidefinite lower bounds for GPP and QAP yields the exact values.

The traveling salesman problem (TSP) is a fundamental problem in combinatorial optimization. Several semidefinite programming relaxations have been proposed recently that exploit a variety of mathematical structures including, for example, algebraic connectivity, permutation matrices, and association schemes. The main results of this paper are twofold. First, de Klerk and Sotirov [de Klerk E, Sotirov R (2012) Improved semidefinite programming bounds for quadratic assignment problems with suitable symmetry. Math. Programming 133(1):75–91.] present a semidefinite program (SDP) based on permutation matrices and symmetry reduction; they show that it is incomparable to the subtour elimination linear program but generally dominates it on small instances. We provide a family of simplicial TSP instances that shows that the integrality gap of this SDP is unbounded. Second, we show that these simplicial TSP instances imply the unbounded integrality gap of every SDP relaxation of the TSP mentioned in the survey on SDP relaxations of the TSP in section 2 of Sotirov [Sotirov R (2012) SDP relaxations for some combinatorial optimization problems. Anjos MF, Lasserre JB, eds., Handbook on Semidefinite, Conic and Polynomial Optimization (Springer, New York), 795–819.]. In contrast, the subtour linear program performs perfectly on simplicial instances. The simplicial instances thus form a natural litmus test for future SDP relaxations of the TSP.

Exploiting special structure in semidefinite programming: A survey of theory and applications

Determining if a solution is optimal or near optimal is fundamental in optimization theory, algorithms, and computation. For instance, Karush-Kuhn-Tucker conditions provide necessary and sufficient optimality conditions for certain classes of problems, and bounds on optimality gaps are frequently used as part of optimization algorithms. Such bounds are obtained through Lagrangian, integrality, or semidefinite program- ming relaxations. An alternative approach in stochastic programming is to use Monte Carlo sampling-based estimators on the optimality gap. In this tutorial, we present a simple, easily implemented procedure that forms a point and interval estimator on the optimality gap of a given candidate solution. We then discuss methods to reduce the computational effort, bias, and variance of our simplest estimator. We also provide a framework that allows the use these optimality gap estimators in an algorithmic way by providing rules to iteratively increase the sample sizes and to terminate. This scheme can be used as a stand-alone sequential sampling procedure, or it can be used in conjunction with a variety of sampling-based algorithms to obtain a solution to a stochastic program with a priori control on the quality of that solution.

Entanglement is a key resource in the emerging field of Quantum Information. The strong correlations between systems described by an entangled state allow us to perform certain tasks more efficiently than it would be possible by using only classical resources. This is why the characterization of entanglement is one of the most important problems in Quantum Information. In this thesis, we analyze several aspects of entanglement. First, we introduce a new family of criteria to determine if a bipartite mixed state is entangled or not. This family consists of a sequence of tests that can be implemented efficiently, and has the property that all entangled states can be detected by some test in the sequence. Each test in the family can be stated as a semidefinite program, which is a class of convex optimization problems. The duality structure of these programs allows us to explicitly construct an entanglement witness that proves entanglement of a state, whenever the state fails one of the tests in the sequence. The entanglement witnesses constructed in this manner have well-defined algebraic properties that can be used to give a characterization of the interior of the set of all possible entanglement witnesses, as well as the set of strictly positive bihermitian forms and the set of strictly positive maps. We also study deterministic transformations of three-qubit pure state when only local operations and classical communication (LOCC) are allowed. We derive strong constraints that the operations and states involved must satisfy, and we apply these results to characterize the set of real states that can be obtained from the GHZ state by LOCC.

A truncated moment sequence (tms) in n variables and of degree d is a finite sequence y=(yź) indexed by nonnegative integer vectors ź:=(ź1,ź,źn) such that ź1+ź+źn≤d. Let K⊆źn be a semialgebraic set. The truncated K-moment problem (TKMP) is: How can one check if a tms y admits a K-measure μ (a nonnegative Borel measure supported in K) such that $y_{\alpha}= \int_{K} x_{1}^{\alpha_{1}}\cdots x_{n}^{\alpha_{n}}\,\mathrm{d}\mu$ for every ź? This paper proposes a semidefinite programming (SDP) approach for solving TKMP. When K is compact, we get the following results: whether a tms admits a K-measure or not can be checked via solving a sequence of SDP problems; when y admits no K-measure, a certificate for the nonexistence can be found; when y admits one, a representing measure for y can be obtained from solving the SDP problems under some necessary and some sufficient conditions. Moreover, we also propose a practical SDP method for finding flat extensions, which in our numerical experiments always found a finitely atomic representing measure when it exists.

In this paper, we propose an efficient semidefinite programming (SDP) approach to worst case linear discriminant analysis (WLDA). Compared with the traditional LDA, WLDA considers the dimensionality reduction problem from the worst case viewpoint, which is in general more robust for classification. However, the original problem of WLDA is non-convex and difficult to optimize. In this paper, we reformulate the optimization problem of WLDA into a sequence of semidefinite feasibility problems. To efficiently solve the semidefinite feasibility problems, we design a new scalable optimization method with a quasi-Newton method and eigen-decomposition being the core components. The proposed method is orders of magnitude faster than standard interior-point SDP solvers. Experiments on a variety of classification problems demonstrate that our approach achieves better performance than standard LDA. Our method is also much faster and more scalable than standard interior-point SDP solvers-based WLDA. The computational complexity for an SDP with m constraints and matrices of size d by d is roughly reduced from O(m(3)+md(3)+m(2)d(2)) to O(d(3)) (m>d in our case).