SoS certification for symmetric quadratic functions and its connection to constrained boolean hypercube optimization

This issue's SIGEST paper, from the SIAM Journal on Optimization (SIOPT), is an excellent example of deep mathematical analysis, involving a range of mathematical techniques, which is having significant practical algorithmic and computational consequences. The title of the paper, “A Sum of Squares Approximation of Nonnegative Polynomials,” by Jean B. Lasserre, combines two concepts that are known to all SIAM readers: sums of squares, the types of functions that arise in many data fitting and other applications; and nonnegative polynomials, polynomials whose values are greater than or equal to zero for all inputs. It is obvious that any polynomial that can be written as a sum of squares is nonnegative. It is far from obvious whether any nonnegative polynomial has a representation as, or arbitrarily close to, a sum of squares of polynomials. In this SIGEST paper, Lasserre shows that this is the case. The nomination from the editors of SIOPT captures the immediate and more general contributions of this paper eloquently, in the context of a series of three papers of which this is the second: “Jean Lasserre, in a series of papers (starting from the first paper in SIOPT Vol. 11, pp. 796–817, and concluding with the third paper in SIOPT Vol. 15, pp. 383–393) by exploiting deep results from real algebraic geometry and theory of moments for multivariate polynomials, showed that the optimal value of a nonconvex polynomial optimization problem (i.e., an optimization problem whose objective function and constraints are specified by multivariate polynomials) can asymptotically be approximated by an increasing sequence of optimal values from increasing tighter semidefinite programming relaxation problems. Even though the result is asymptotic in the most general case, fortuitously, practical experience has revealed that very often, the first few levels of semidefinite programming relaxations typically can achieve the global optimum value. The theoretical connection that Jean Lasserre (and Pablo Parrilo at about the same time from the viewpoint of sum of squares representations of multivariate polynomials) had established between polynomial optimizations with real algebraic geometry and theory of moments has led to exciting developments in combinatorial optimization. In particular, many hard combinatorial problems such as the maximum clique problem of a graph and the quadratic assignment problem are now actively revisited by researchers under the framework of polynomial optimization to derive increasingly tighter bounds.” That is, a consequence of this and related work is the ability to use a sequence of semidefinite programming problems to (nearly) solve much more difficult optimization problems. Semidefinite programming problems are problems with a linear objective function and linear constraints in a matrix variable X, plus the additional constraint that X be positive semidefinite. Since a rich array of efficient algorithms for solving semidefinite programming problems has been developed in recent years, reducing more general problems to a sequence of semidefinite programming problems is an important advance. We hope that readers of SIAM Review will enjoy this glimpse into the sophisticated mathematics behind an important algorithmic innovation in optimization.

This work studies the problem of analyzing and, subsequently, optimizing the stabilization capabilities of a class of controllers for input constrained nonlinear systems. The proposed techniques apply to continuous, state feedback controllers which are defined in a subset of the state space where the time derivative of a known, candidate Control Lyapunov Function (CLF) can be made negative definite under the input constraints. This set is associated with the domain of this class of controllers and depends on the CLF. The analysis problem concerns approximating the domain and is posed via appropriately formulated set containment relationships through the generalized S-procedure with sum of squares (SOS) constraints. The optimization problem is concerned with the adjustment or enlargement of the domain and constitutes a way of controller synthesis. These objectives are pursued via optimizing over the coefficients of polynomial CLFs, through a sequence of semidefinite programming (SDP) problems with SOS constraints. By partitioning the state space based on the structure of the input value set and building upon earlier results on SOS methods, the SDP problems are subject to only convex constraints, rendering thus the proposed techniques computationally viable. The capabilities of the proposed algorithms are demonstrated through numerical examples.

In this chapter we consider polynomial optimization problems, asking to minimize a polynomial function over a compact semialgebraic set, defined by polynomial inequalities. This models a great variety of (in general, nonlinear nonconvex) optimization problems. Various hierarchies of (lower and upper) bounds have been introduced, having the remarkable property that they converge asymptotically to the global minimum. These bounds exploit algebraic representations of positive polynomials in terms of sums of squares and can be computed using semidefinite optimization. Our focus in this chapter lies in the performance analysis of these hierarchies of bounds, namely, in how far the bounds are from the global minimum as the degrees of the sums of squares they involve tend to infinity. We present the main state-of-the-art results and offer a gentle introductory overview over the various techniques that have been recently developed to establish them, stemming from the theory of orthogonal polynomials, approximation theory, Fourier analysis, and more.

Semialgebraic computations, i.e., the manipulation of sets and logical conditions defined by polynomial inequalities in real variables, are an essential ”primitive” in the analysis and design of hybrid dynamical systems. Fundamental tasks such as reachability analysis, abstraction verification, and the computation of stability and performance certificates, all use these operations extensively, and can quickly become the computational bottleneck in the design process. Although there is a well-developed body of both basic theory and algorithms for these tasks, the practical performance of most available methods is still far from being satisfactory on real-world problems. While there are several possible causes for this (besides the NP-hardness of the task), a sensible explanation lies in the purely algebraic nature of the usual methods, as well as the insistence on exact (as opposed to approximate or ”relaxed”) solutions. For these reasons, there is a strong interest in the development of efficient techniques for (perhaps restricted) classes of semialgebraic problems. In this talk we review the basic elements and present several new results on the SOS approach to semialgebraic computations, that combines symbolic and numerical techniques from real algebra and convex optimization. Its main defining feature is the use and computation of sum of squares (SOS) decompositions for multivariate polynomials via semidefinite programming. These are extended, using the Positivstellensatz, to structured infeasibility certificates for polynomial equations and inequalities. The developed techniques unify and generalize many well-known existing methods. In particular, we will discuss semialgebraic problems with at most two quantifier alternations (i.e., classical polynomial optimization problems as well as the related games and minimax problems). As an example, we will solve in detail a class of zero-sum two-person games with an infinite number of pure strategies, where the payoff function is a polynomial expression of the actions of the players. Although particular emphasis will be given to the hybrid systems viewpoint, the basic ideas and algorithms, as well as these recent extensions, will be illustrated with examples drawn from a broad range of related domains, including dynamical systems and geometric theorem proving.

We revisit the problem of protein structure determination from geometrical restraints from NMR, using convex optimization. It is well-known that the NP-hard distance geometry problem of determining atomic positions from pairwise distance restraints can be relaxed into a convex semidefinite program (SDP). However, often the NOE distance restraints are too imprecise and sparse for accurate structure determination. Residual dipolar coupling (RDC) measurements provide additional geometric information on the angles between atom-pair directions and axes of the principal-axis-frame. The optimization problem involving RDC is highly non-convex and requires a good initialization even within the simulated annealing framework. In this paper, we model the protein backbone as an articulated structure composed of rigid units. Determining the rotation of each rigid unit gives the full protein structure. We propose solving the non-convex optimization problems using the sum-of-squares (SOS) hierarchy, a hierarchy of convex relaxations with increasing complexity and approximation power. Unlike classical global optimization approaches, SOS optimization returns a certificate of optimality if the global optimum is found. Based on the SOS method, we proposed two algorithms-RDC-SOS and RDC-NOE-SOS, that have polynomial time complexity in the number of amino-acid residues and run efficiently on a standard desktop. In many instances, the proposed methods exactly recover the solution to the original non-convex optimization problem. To the best of our knowledge this is the first time SOS relaxation is introduced to solve non-convex optimization problems in structural biology. We further introduce a statistical tool, the Cramér-Rao bound (CRB), to provide an information theoretic bound on the highest resolution one can hope to achieve when determining protein structure from noisy measurements using any unbiased estimator. Our simulation results show that when the RDC measurements are corrupted by Gaussian noise of realistic variance, both SOS based algorithms attain the CRB. We successfully apply our method in a divide-and-conquer fashion to determine the structure of ubiquitin from experimental NOE and RDC measurements obtained in two alignment media, achieving more accurate and faster reconstructions compared to the current state of the art.

Sums of squares based approximation algorithms for MAX-SAT

The field of Metaheuristics has produced a large number of algorithms for continuous, black-box optimization. In contrast, there are few standard benchmark problem sets, limiting our ability to gain insight into the empirical performance of these algorithms. Clustering problems have been used many times in the literature to evaluate optimization algorithms. However, much of this work has occurred independently on different problem instances and the various experimental methodologies used have produced results which are frequently incomparable and provide little knowledge regarding the difficulty of the problems used, or any platform for comparing and evaluating the performance of algorithms. This paper discusses sum of squares clustering problems from the optimization viewpoint. Properties of the fitness landscape are analysed and it is proposed that these problems are highly suitable for algorithm benchmarking. A set of 27 problem instances (from 4-D to 40-D), based on three well-known datasets, is specified. Baseline experimental results are presented for the Covariance Matrix Adaptation-Evolution Strategy and several other standard algorithms. A web-repository has also been created for this problem set to facilitate future use for algorithm evaluation and comparison.

The prominent feature is the nonlinear characteristics as the squaring and rectification functions, which are observed in the retinal and visual cortex networks. Conventional model for motion processing in cortex, uses a symmetric quadratic functions with Gabor filters. This paper proposes a new motion processing model in the asymmetric networks. First, the asymmetric network is analyzed using Wiener kernels. It is shown that the asymmetric network with nonlinearities is effective and general for generating the directional movement compared with the conventional quadratic model. Second, independence maximization of data is an important issue in computational neural networks. To make clear the characteristics of the asymmetric network with Gabor functions, orthogonality is computed, which shows independent characteristics of the asymmetric network without maximizing optimization of independence in the quadratic model. The orthogonal analyses for the independence of the asymmetric networks are applied to the V1 and MT neural networks to generate independent subspaces by using selective Gabor functions.

The Quantum Max Cut (QMC) problem has emerged as a test-problem for designing approximation algorithms for local Hamiltonian problems. In this paper we attack this problem using the algebraic structure of QMC, in particular the relationship between the quantum max cut Hamiltonian and the representation theory of the symmetric group.The first major contribution of this paper is an extension of non-commutative Sum of Squares (ncSoS) optimization techniques to give a new hierarchy of relaxations to Quantum Max Cut. The hierarchy we present is based on optimizations over polynomials in the qubit swap operators. This is in contrast to the "standard" quantum Lasserre Hierarchy, which is based on polynomials expressed in terms of the Pauli matrices. To prove correctness of this hierarchy, we exploit a finite presentation of the algebra generated by the qubit swap operators. This presentation allows for the use of computer algebraic techniques to manipulate and simplify polynomials written in terms of the swap operators, and may be of independent interest. Surprisingly, we find that level-2 of this new hierarchy is numerically exact (up to tolerance 10−7) on all QMC instances with uniform edge weights on graphs with at most 8 vertices.The second major contribution of this paper is a polynomial-time algorithm that computes (in exact arithmetic) the maximum eigenvalue of the QMC Hamiltonian for certain graphs, including graphs that can be "decomposed" as a signed combination of cliques. A special case of the latter are complete bipartite graphs with uniform edge-weights, for which exact solutions are known from the work of Lieb and Mattis \cite{lieb1962ordering}. Our methods, which use representation theory of the symmetric group, can be seen as a generalization of the Lieb-Mattis result.

This paper proposes a novel maximum-force-per-ampere strategy for the current distribution of planar switched reluctance motors (PSRMs) for efficiency improvement. This strategy is the first of its kind for planar motors, and it is used to generate the desired thrust force with the minimum sum of squares of the three-phase current. To formulate this strategy, a constrained optimization problem with time-varying parameters is first developed. Then, the problem is transformed into an unconstrained problem with a barrier function. Additionally, a self-designed adaptive genetic algorithm is introduced to solve the unconstrained optimization problem for locating the optimal currents. Comparative studies of the proposed and conventional strategies for a PSRM system are carried out via simulation and experiment, and planar trajectory tracking for the system with the proposed strategy is experimentally performed. The validity of the proposed strategy is also verified.

This article presents a rational polynomial tracking control approach to a common system representation for unmanned aerial vehicles (UAVs). First, we newly provide a common system representation of kinematic models for straight and orbit paths. A polynomial representation is introduced to describe the common system. To stabilize the polynomial system, we design a rational polynomial controller using a sum-of-squares (SOS)-based design framework. A set of stabilization conditions considering a real actuator saturation is represented in terms of SOS, where a relaxation for the SOS design conditions is brought by considering practical operation domains. The SOS-based design framework is applied to path stabilization for a real UAV platform (a parafoil wing-type UAV). Experimental results demonstrate the capabilities of the proposed SOS-based design framework in path stabilization of real-world complex UAVs.

Dissipative Dynamical Systems

We consider the problem of minimizing a form on the standard simplex [equivalently, the problem of minimizing an even form on the unit sphere]. Converging hierarchies of approximations for this problem can be constructed, that are based, respectively, on results by Schmudgen-Putinar and by Polya about representations of positive polynomials in terms of sums of squares. We show that the two approaches yield, in fact, the same approximations. The same type of argument also permits to establish some representation results a la Polya for positive polynomials on semi-algebraic cones.

Optimization means finding that argument which minimizes or maximizes a given function such as a likelihood function or a sum of squares. Many optimization algorithms are derived from algorithms that solve the nonlinear equations defined by setting the derivative of the objective function equal to zero. This article discusses a number of methods for unconstrained optimization, including bisection and golden search in the univariate case and Newton's method and quasi‐Newton algorithms in the multivariate case. Applications to maximum likelihood estimation, Fisher's method of scoring, nonlinear regression, and generalized linear models are described. Restricted step modifications are discussed for preventing divergence of Newton‐type algorithms. A number of derivative‐free methods are discussed, including the Nelder–Mead simplex method, methods with numeric derivatives, conjugate methods, and the EM algorithm.

Recently, inspired by quantum annealing, many solvers specialized for unconstrained binary quadratic programming problems have been developed. For further improvement and application of these solvers, it is important to clarify the differences in their performance for various types of problems. In this study, the performance of four quadratic unconstrained binary optimization problem solvers, namely D-Wave Hybrid Solver Service (HSS), Toshiba Simulated Bifurcation Machine (SBM), Fujitsu Digital Annealer (DA), and simulated annealing on a personal computer, was benchmarked. The problems used for benchmarking were instances of real problems in MQLib, instances of the SAT-UNSAT phase transition point of random not-all-equal 3-SAT (NAE 3-SAT), and the Ising spin glass Sherrington-Kirkpatrick (SK) model. Concerning MQLib instances, the HSS performance ranked first; for NAE 3-SAT, DA performance ranked first; and regarding the SK model, SBM performance ranked first. These results may help understand the strengths and weaknesses of these solvers.