A conditional gradient homotopy method with applications to semidefinite programming

Semidefinite programming (SDP) solvers are increasingly used as primitives in many program verification tasks to synthesize and verify polynomial invariants for a variety of systems including programs, hybrid systems and stochastic models. On one hand, they provide a tractable alternative to reasoning about semi-algebraic constraints. However, the results are often unreliable due to “numerical issues” that include a large number of reasons such as floating-point errors, ill-conditioned problems, failure of strict feasibility, and more generally, the specifics of the algorithms used to solve SDPs. These issues influence whether the final numerical results are trustworthy or not. In this paper, we briefly survey the emerging use of SDP solvers in the static analysis community. We report on the perils of using SDP solvers for common invariant synthesis tasks, characterizing the common failures that can lead to unreliable answers. Next, we demonstrate existing tools for guaranteed semidefinite programming that often prove inadequate to our needs. Finally, we present a solution for verified semidefinite programming that can be used to check the reliability of the solution output by the solver and a padding procedure that can check the presence of a feasible nearby solution to the one output by the solver. We report on some successful preliminary experiments involving our padding procedure.

Semi-supervised kernel learning methods have been received much more attention in the past few years. Traditional semi-supervised Non-Parametric Kernel Learning (NPKL) methods usually formulate the learning task as a Semi-Definite Programming (SDP) problem, which is very time consuming. Although some fast semi-supervised NPKL methods have been proposed recently, they usually scale very poorly. Furthermore, many semi-supervised NPKL methods are developed based on the manifold assumption. But, such an assumption might be invalid when handling some high-dimensional and sparse data, which has severely negative effect on the performance of learning algorithms. In this paper, we propose a more efficient semi-supervised NPKL method, which can effectively learn a low-rank kernel matrix from must-link and cannot-link constraints. Specially, by virtue of the nonlinear spectral embedded technique based on extreme learning machine (ELM), the proposed method has the ability of coping with data points that do not have a clear manifold structure in a low dimensional space. The proposed method is formulated as a trace ratio optimization problem, which is combined with dimensionality reduction in ELM feature space and aims to find optimal low-rank kernel matrices. The proposed optimization problem can be solved much more efficiently than SDP solvers. Extensive experiments have validated the superior performance of the proposed method compared to state-of-the-art semi-supervised kernel learning methods.

The sensor network localization (SNL) problem in embedding dimension r consists of locating the positions of wireless sensors, given only the distances between sensors that are within radio range and the positions of a subset of the sensors (called anchors). Current solution techniques relax this problem to a weighted, nearest, (positive) semidefinite programming (SDP) completion problem by using the linear mapping between Euclidean distance matrices (EDM) and semidefinite matrices. The resulting SDP is solved using primal-dual interior point solvers, yielding an expensive and inexact solution. This relaxation is highly degenerate in the sense that the feasible set is restricted to a low dimensional face of the SDP cone, implying that the Slater constraint qualification fails. Cliques in the graph of the SNL problem give rise to this degeneracy in the SDP relaxation. In this paper, we take advantage of the absence of the Slater constraint qualification and derive a technique for the SNL problem, with exact data, that explicitly solves the corresponding rank restricted SDP problem. No SDP solvers are used. For randomly generated instances, we are able to efficiently solve many huge instances of this NP-hard problem to high accuracy by finding a representation of the minimal face of the SDP cone that contains the SDP matrix representation of the EDM. The main work of our algorithm consists in repeatedly finding the intersection of subspaces that represent the faces of the SDP cone that correspond to cliques of the SNL problem.

In contrast to many other convex optimization classes, state-of-the-art semidefinite programming solvers are still unable to efficiently solve large-scale instances. This work aims to reduce this scalability gap by proposing a novel proximal algorithm for solving general semidefinite programming problems. The key characteristic of the proposed algorithm is to be able to exploit the low-rank property inherent to several semidefinite programming problems. Exploiting the low-rank structure provides a substantial speedup and allows the operator splitting method to efficiently scale to larger instances. As opposed to other low-rank based methods, the proposed algorithm has convergence guarantees for general semidefinite programming problems. Additionally, an open-source semidefinite programming solver called ProxSDP is made available and its implementation details are discussed. Case studies are presented in order to evaluate the performance of the proposed methodology.

The complexity-performance trade-off is a fundamental aspect of the design of low-density parity-check (LDPC) codes. In this paper, we consider LDPC codes for the binary erasure channel (BEC), use code rate for performance metric, and number of decoding iterations to achieve a certain residual erasure probability for complexity metric. We first propose a quite accurate approximation of the number of iterations for the BEC. Moreover, a simple but efficient utility function corresponding to the number of iterations is developed. Using the aforementioned approximation and the utility function, two optimization problems w.r.t. complexity are formulated to find the code degree distributions. We show that both optimization problems are convex. In particular, the problem with the proposed approximation belongs to the class of semi-infinite problems which are computationally challenging to be solved. However, the problem with the proposed utility function falls into the class of semi-definite programming (SDP) and thus, the global solution can be found efficiently using available SDP solvers. Numerical results reveal the superiority of the proposed code design compared to existing code designs from literature.

An optimal experimental design represents a structured approach to collecting data with the aim of maximizing the information gleaned. Achieving this requires defining an optimality criterion tailored to the specific model under consideration and the purpose of the investigation. However, it is often observed that a design optimized for one criterion may not perform optimally when applied to another. To mitigate this, one strategy involves employing compound designs. These designs balance multiple criteria to create robust experimental plans that are versatile across different applications. In our study, we systematically tackle the challenge of constructing compound approximate optimal experimental designs using Semidefinite Programming. We focus on discretized design spaces, with the objective function being the geometric or the arithmetic mean of design efficiencies relative to individual criteria. We address two combinations of two criteria: concave-concave (illustrated by DE–optimality) and convex-concave (such as DA–optimality). To handle the latter, we reformulate the problem as a bilevel problem. Here, the outer problem is solved using Surrogate Based Optimization, while the inner problem is addressed with a Semidefinite Programming solver. We demonstrate our formulations using both linear and nonlinear models (for the response) of the Beta class, previously linearized to facilitate analysis and comparison.

In this paper, we consider a least square semidefinite programming problem under ellipsoidal data uncertainty. We show that the robustification of this uncertain problem can be reformulated as a semidefinite linear programming problem with an additional second-order cone constraint. We then provide an explicit quantitative sensitivity analysis on how the solution under the robustification depends on the size/shape of the ellipsoidal data uncertainty set. Next, we prove that, under suitable constraint qualifications, the reformulation has zero duality gap with its dual problem, even when the primal problem itself is infeasible. The dual problem is equivalent to minimizing a smooth objective function over the Cartesian product of second-order cones and the Euclidean space, which is easy to project onto. Thus, we propose a simple variant of the spectral projected gradient method (Birgin et al. in SIAM J. Optim. 10:1196---1211, 2000) to solve the dual problem. While it is well-known that any accumulation point of the sequence generated from the algorithm is a dual optimal solution, we show in addition that the dual objective value along the sequence generated converges to a finite value if and only if the primal problem is feasible, again under suitable constraint qualifications. This latter fact leads to a simple certificate for primal infeasibility in situations when the primal feasible set lies in a known compact set. As an application, we consider robust correlation stress testing where data uncertainty arises due to untimely recording of portfolio holdings. In our computational experiments on this particular application, our algorithm performs reasonably well on medium-sized problems for real data when finding the optimal solution (if exists) or identifying primal infeasibility, and usually outperforms the standard interior-point solver SDPT3 in terms of CPU time.

We consider optimization problems expressed as a linear program with a cone constraint. Cone-LP's subsume ordinary linear programs, and semidefinite programs. We study the notions of basic solutions, nondegeneracy, and feasible directions, and propose a generalization of the simplex method for a large class including LP's and SDP's. One key feature of our approach is considering feasible directions as a sum of two directions. In LP, these correspond to variables leaving and entering the basis, respectively. The resulting algorithm for SDP inherits several important properties of the LP-simplex method, in particular, the linesearch can be done in the current face of the cone, similarly to LP, where the linesearch must determine only the variable leaving the basis.

This issue's SIGEST paper, from the SIAM Journal on Optimization (SIOPT), is an excellent example of deep mathematical analysis, involving a range of mathematical techniques, which is having significant practical algorithmic and computational consequences. The title of the paper, “A Sum of Squares Approximation of Nonnegative Polynomials,” by Jean B. Lasserre, combines two concepts that are known to all SIAM readers: sums of squares, the types of functions that arise in many data fitting and other applications; and nonnegative polynomials, polynomials whose values are greater than or equal to zero for all inputs. It is obvious that any polynomial that can be written as a sum of squares is nonnegative. It is far from obvious whether any nonnegative polynomial has a representation as, or arbitrarily close to, a sum of squares of polynomials. In this SIGEST paper, Lasserre shows that this is the case. The nomination from the editors of SIOPT captures the immediate and more general contributions of this paper eloquently, in the context of a series of three papers of which this is the second: “Jean Lasserre, in a series of papers (starting from the first paper in SIOPT Vol. 11, pp. 796–817, and concluding with the third paper in SIOPT Vol. 15, pp. 383–393) by exploiting deep results from real algebraic geometry and theory of moments for multivariate polynomials, showed that the optimal value of a nonconvex polynomial optimization problem (i.e., an optimization problem whose objective function and constraints are specified by multivariate polynomials) can asymptotically be approximated by an increasing sequence of optimal values from increasing tighter semidefinite programming relaxation problems. Even though the result is asymptotic in the most general case, fortuitously, practical experience has revealed that very often, the first few levels of semidefinite programming relaxations typically can achieve the global optimum value. The theoretical connection that Jean Lasserre (and Pablo Parrilo at about the same time from the viewpoint of sum of squares representations of multivariate polynomials) had established between polynomial optimizations with real algebraic geometry and theory of moments has led to exciting developments in combinatorial optimization. In particular, many hard combinatorial problems such as the maximum clique problem of a graph and the quadratic assignment problem are now actively revisited by researchers under the framework of polynomial optimization to derive increasingly tighter bounds.” That is, a consequence of this and related work is the ability to use a sequence of semidefinite programming problems to (nearly) solve much more difficult optimization problems. Semidefinite programming problems are problems with a linear objective function and linear constraints in a matrix variable X, plus the additional constraint that X be positive semidefinite. Since a rich array of efficient algorithms for solving semidefinite programming problems has been developed in recent years, reducing more general problems to a sequence of semidefinite programming problems is an important advance. We hope that readers of SIAM Review will enjoy this glimpse into the sophisticated mathematics behind an important algorithmic innovation in optimization.

Solving optimization problems with sparse or low-rank optimal solutions has been an important topic since the recent emergence of compressed sensing and its matrix extensions such as the matrix rank minimization and robust principal component analysis problems. Compressed sensing enables one to recover a signal or image with fewer observations than the “length” of the signal or image, and thus provides potential breakthroughs in applications where data acquisition is costly. However, the potential impact of compressed sensing cannot be realized without efficient optimization algorithms that can handle extremely large-scale and dense data from real applications. Although the convex relaxations of these problems can be reformulated as either linear programming, second-order cone programming or semidefinite programming problems, the standard methods for solving these relaxations are not applicable because the problems are usually of huge size and contain dense data. In this dissertation, we give efficient algorithms for solving these “sparse” optimization problems and analyze the convergence and iteration complexity properties of these algorithms. Chapter 2 presents algorithms for solving the linearly constrained matrix rank minimization problem. The tightest convex relaxation of this problem is the linearly constrained nuclear norm minimization. Although the latter can be cast and solved as a semidefinite programming problem, such an approach is computationally expensive when the matrices are large. In Chapter 2, we propose fixed-point and Bregman iterative algorithms for solving the nuclear norm minimization problem and prove convergence of the first of these algorithms. By using a homotopy approach together with an approximate singular value decomposition procedure, we get a very fast, robust and powerful algorithm, which we call FPCA (Fixed Point Continuation with Approximate SVD), that can solve very large matrix rank minimization problems. Our numerical results on randomly generated and real matrix completion problems demonstrate that this algorithm is much faster and provides much better recoverability than semidefinite programming solvers such as SDPT3. For example, our algorithm can recover 1000 × 1000 matrices of rank 50 with a relative error of 10−5 in about 3 minutes by sampling only 20 percent of the elements. We know of no other method that achieves as good recoverability. Numerical experiments on online recommendation, DNA microarray data set and image inpainting problems demonstrate the effectiveness of our algorithms. In Chapter 3, we study the convergence/recoverability properties of the fixed-point continuation algorithm and its variants for matrix rank minimization. Heuristics for determining the rank of the matrix when its true rank is not known are also proposed. Some of these algorithms are closely related to greedy algorithms in compressed sensing. Numerical results for these algorithms for solving linearly constrained matrix rank minimization problems are reported. Chapters 4 and 5 considers alternating direction type methods for solving composite convex optimization problems. We present in Chapter 4 alternating linearization algorithms that are based on an alternating direction augmented Lagrangian approach for minimizing the sum of two convex functions. Our basic methods require at most O(1/e) iterations to obtain an e-optimal solution, while our accelerated (i.e., fast) versions require at most O(1/ 3 ) iterations, with little change in the computational effort required at each iteration. For more general problem, i.e., minimizing the sum of K convex functions, we propose multiple-splitting algorithms for solving them. We propose both basic and accelerated algorithms with O(1/e) and O(1/ 3 ) iteration complexity bounds for obtaining an e-optimal solution. To the best of our knowledge, the complexity results presented in these two chapters are the first ones of this type that have been given for splitting and alternating direction type methods. Numerical results on various applications in sparse and low-rank optimization, including compressed sensing, matrix completion, image deblurring, robust principal component analysis, are reported to demonstrate the efficiency of our methods.

Semidefinite programming (SDP) covers a wide range of applications such as robust optimization, polynomial optimization, combinatorial optimization, system and control theory, financial engineering, machine learning, quantum information and quantum chemistry. In those applications, SDP problems can be large scale easily. Such large scale SDP problems often satisfy a certain sparsity characterized by a chordal graph structure. This sparsity is classified in two types. The one is the domain space sparsity (d-space sparsity) for positive semidefinite symmetric matrix variables involved in SDP problems, and the other the range space sparsity (r-space sparsity) for matrix-inequality constraints in SDP problems. In this short note, we survey how we exploit these two types of sparsities to solve large scale linear and nonlinear SDP problems. We refer to the paper [7] for more details.

This work studies the problem of analyzing and, subsequently, optimizing the stabilization capabilities of a class of controllers for input constrained nonlinear systems. The proposed techniques apply to continuous, state feedback controllers which are defined in a subset of the state space where the time derivative of a known, candidate Control Lyapunov Function (CLF) can be made negative definite under the input constraints. This set is associated with the domain of this class of controllers and depends on the CLF. The analysis problem concerns approximating the domain and is posed via appropriately formulated set containment relationships through the generalized S-procedure with sum of squares (SOS) constraints. The optimization problem is concerned with the adjustment or enlargement of the domain and constitutes a way of controller synthesis. These objectives are pursued via optimizing over the coefficients of polynomial CLFs, through a sequence of semidefinite programming (SDP) problems with SOS constraints. By partitioning the state space based on the structure of the input value set and building upon earlier results on SOS methods, the SDP problems are subject to only convex constraints, rendering thus the proposed techniques computationally viable. The capabilities of the proposed algorithms are demonstrated through numerical examples.

Semidefinite relaxation method for unified near-Field and far-Field localization by AOA

Antenna array pattern synthesis with mask constraints can be formulated as a convex optimization problem with semiinfinite trigonometric polynomial constraints. The current approach uses a Linear Matrix Inequality (LMI) characterization of the semi-infinite constraints to convert the original problem into a semidefinite programming (SDP) problem. However, an important drawback of this approach is the large number of additional variables incurred in the equivalent SDP representation, which in turn prohibits its use in the design of large antenna arrays that arise in many modern applications. This paper presents an efficient method for the synthesis of large antenna arrays via a novel LMI characterization of semi-infinite constraints that only involves a minimal number of additional variables. Subsequently, the design of patterns for arrays with hundreds of elements can be easily achieved on a standard personal computer using existing SDP solvers.

Most moment-based distributionally robust optimization (DRO) problems can be reformulated as semidefinite programming (SDP) problems, which can be solved in polynomial time. However, solving high-dimensional SDPs is often time-consuming. Existing approximation methods typically reduce the dimensionality of random parameters before solving the approximated SDPs. This sequential approach relies solely on statistical information to reduce the high-dimensional uncertainty space, which may not yield the best approximation performance. Jiang et al. (2025) introduce an optimized dimensionality reduction (ODR) approach that integrates the dimensionality reduction of random parameters with subsequent optimization problems. This integration enables two outer approximations and one inner approximation of the original problem, all represented as low-dimensional SDPs that can be solved efficiently, providing two lower bounds and one upper bound, respectively. More importantly, these approximations can theoretically achieve the optimal value of the original high-dimensional SDPs, resulting in a zero gap.