Robust convergence analysis of distributed optimization algorithms

We consider semidefinite programming (SDP) formulations of certain truss topology optimization problems, where a lower bound is imposed on the fundamental frequency of vibration of the truss structure. These SDP formulations were introduced in Ohsaki et al. (Comp. Meth. Appl. Mech. Eng. 180:203–217, 1999). We show how one may automatically obtain symmetric designs, by eliminating the ‘redundant’ symmetry in the SDP problem formulation. This has the advantage that the original SDP problem is substantially reduced in size for trusses with large symmetry groups.

An optimal experimental design represents a structured approach to collecting data with the aim of maximizing the information gleaned. Achieving this requires defining an optimality criterion tailored to the specific model under consideration and the purpose of the investigation. However, it is often observed that a design optimized for one criterion may not perform optimally when applied to another. To mitigate this, one strategy involves employing compound designs. These designs balance multiple criteria to create robust experimental plans that are versatile across different applications. In our study, we systematically tackle the challenge of constructing compound approximate optimal experimental designs using Semidefinite Programming. We focus on discretized design spaces, with the objective function being the geometric or the arithmetic mean of design efficiencies relative to individual criteria. We address two combinations of two criteria: concave-concave (illustrated by DE–optimality) and convex-concave (such as DA–optimality). To handle the latter, we reformulate the problem as a bilevel problem. Here, the outer problem is solved using Surrogate Based Optimization, while the inner problem is addressed with a Semidefinite Programming solver. We demonstrate our formulations using both linear and nonlinear models (for the response) of the Beta class, previously linearized to facilitate analysis and comparison.

A polynomial optimization problem (POP) consists of minimizing a multivariate real polynomial on a semi-algebraic set \(K\) described by polynomial inequalities and equations. In its full generality it is a non-convex, multi-extremal, difficult global optimization problem. More than an decade ago, J. B. Lasserre proposed to solve POPs by a hierarchy of convex semidefinite programming (SDP) relaxations of increasing size. Each problem in the hierarchy has a primal SDP formulation (a relaxation of a moment problem) and a dual SDP formulation (a sum-of-squares representation of a polynomial Lagrangian of the POP). In this note, we show that there is no duality gap between each primal and dual SDP problem in Lasserre’s hierarchy, provided one of the constraints in the description of set \(K\) is a ball constraint. Our proof uses elementary results on SDP duality, and it does not assume that \(K\) has a strictly feasible point.

In the past decade or so, semi-definite programming (SDP) has emerged as a powerful tool capable of handling a remarkably wide range of problems. This article describes an innovative application of SDP techniques to quadratic inverse eigenvalue problems (QIEPs). The notion of QIEPs is of fundamental importance because its ultimate goal of constructing or updating a vibration system from some observed or desirable dynamical behaviors while respecting some inherent feasibility constraints well suits many engineering applications. Thus far, however, QIEPs have remained challenging both theoretically and computationally due to the great variations of structural constraints that must be addressed. Of notable interest and significance are the uniformity and the simplicity in the SDP formulation that solves effectively many otherwise very difficult QIEPs.

Exploiting special structure in semidefinite programming: A survey of theory and applications

For a given set of input-output pairs of quantum states or observables, we ask the question whether there exists a physically implementable transformation that maps each of the inputs to the corresponding output. The physical maps on quantum states are trace-preserving completely positive maps, but we also consider variants of these requirements. We generalize the definition of complete positivity to linear maps defined on arbitrary subspaces, then formulate this notion as a semidefinite program, and relate it by duality to approximative extensions of this map. This gives a characterization of the maps which can be approximated arbitrarily well as the restriction of a map that is completely positive on the whole algebra, also yielding the familiar extension theorems on operator spaces. For quantum channel extensions and extensions by probabilistic operations we obtain semidefinite characterizations, and we also elucidate the special case of Abelian inputs or outputs. Finally, revisiting a theorem by Alberti and Uhlmann, we provide simpler and more widely applicable conditions for certain extension problems on qubits, and by using a semidefinite programming formulation we exhibit counterexamples to seemingly reasonable but false generalizations of the Alberti-Uhlmann theorem.

The quantification of heating, ventilation, and air condition (HVAC) system flexibility is critical to the operations of both the grid and buildings in demand response (DR) programs. However, the flexibility quantification is challenging due to the non-linearity and non-convexity of thermal dynamics associated with HVAC components. This paper proposes a novel HVAC flexibility quantification method based on a semidefinite programming (SDP) formulation. The SDP is reformulated from the non-convex problem of HVAC power optimization, and can be solved efficiently in real-time. The physics-based HVAC model is incorporated to ensure the reliability and accuracy of solutions. The quantification results are organized into an HVAC flexibility table that can provide response strategies on adjusting HVAC setpoints in response to the grid signals received. The developed response strategies minimize occupant discomfort while satisfying grid requirements. A case study of a test building model is carried out to illustrate the flexibility quantification framework and compares the performance of two DR strategies.

Semidefinite programming solution of economic dispatch problem with non-smooth, non-convex cost functions

A gridless method for direction finding with sparse arrays in nonuniform noise

State estimation (SE) is an important task allowing power networks to monitor accurately the underlying system state, which is useful for security-constrained dispatch and power system control. For nonlinear AC power systems, SE amounts to minimizing a weighted least-squares cost that is inherently nonconvex, thus giving rise to many local optima. As a result, estimators used extensively in practice rely on iterative optimization methods, which are destined to return only locally optimal solutions, or even fail to converge. A semidefinite programming (SDP) formulation for SE has been advocated, which relies on the convex semidefinite relaxation (SDR) of the original problem and thereby renders it efficiently solvable. Theoretical analysis under simplified conditions is provided to shed light on the near-optimal performance of the SDR-based SE solution at polynomial complexity. The new approach is further pursued toward complementing traditional nonlinear measurements with linear synchrophasor measurements and reducing computational complexity through distributed implementations. Numerical tests on the standard IEEE 30- and 118-bus systems corroborate that the SE algorithms outperform existing alternatives, and approach near-optimal performance.

We offer a redesigned form of the classical multidimensional scaling (C-MDS) algorithm suitable to handle the localization of multiple sources under line-of-sight (LOS) and non-line-of-sight (NLOS) conditions. To do so we propose to modify the kernel matrix used in the MDS algorithm to allow for both distance and angle information to be processed algebraically (without iteration) and simultaneously. In so doing we also show that the new formulation overcomes two well known limitations of the C-MDS approach, namely the propagation error problem and the possibility to weight the dissimilarities used as measurement information, including, for the case of binary weights, the data erasure problem. Due to the increased size of the proposed edge kernel matrix KE used in the algorithm, the Nystrom approximation is applied to reduce the overall computational complexity to few matrix multiplications. Range only scenarios are also dealt with by approximating the matrix KE. Simulations in range-angle as well as range-only scenarios demonstrate the superiority of our solution under both LOS and NLOS conditions versus semidefinite programming (SDP) formulations of the problem specifically designed to exploit the heterogeneity of the information available.

We consider optimal designs for clinical trials when response variance depends on treatment and covariates are included in the response model. These designs are generalizations of Neyman allocation, and commonly employed in personalized medicine where external covariates linearly affect the response. Very often, these designs aim at maximizing the amount of information gathered but fail to assure ethical requirements. We analyze compound optimal designs that maximize a criterion weighting the amount of information and the reward of allocating the patients to the most effective/least risky treatment. We develop a general representation for static (a priori) allocation and propose a semidefinite programming (SDP) formulation to support their numerical computation. This setup is extended assuming the variance and the parameters of the response of all treatments are unknown and an adaptive sequential optimal design scheme is implemented and used for demonstration. Purely information theoretic designs for the same allocation have been addressed elsewhere, and we use them to support the techniques applied to compound designs.

We consider the problem of nonparametric estimation of unknown smooth functions in the presence of restrictions on the shape of the estimator and on its support using polynomial splines. We provide a general computational framework that treats these estimation problems in a unified manner, without the limitations of the existing methods. Applications of our approach include computing optimal spline estimators for regression, density estimation, and arrival rate estimation problems in the presence of various shape constraints. Our approach can also handle multiple simultaneous shape constraints. The approach is based on a characterization of nonnegative polynomials that leads to semidefinite programming (SDP) and second-order cone programming (SOCP) formulations of the problems. These formulations extend and generalize a number of previous approaches in the literature, including those with piecewise linear and B-spline estimators. We also consider a simpler approach in which nonnegative splines are approximated by splines whose pieces are polynomials with nonnegative coefficients in a nonnegative basis. A condition is presented to test whether a given nonnegative basis gives rise to a spline cone that is dense in the space of nonnegative continuous functions. The optimization models formulated in the article are solvable with minimal running time using off-the-shelf software. We provide numerical illustrations for density estimation and regression problems. These examples show that the proposed approach requires minimal computational time, and that the estimators obtained using our approach often match and frequently outperform kernel methods and spline smoothing without shape constraints. Supplementary materials for this article are provided online.

Contemporary practical methods for protein nuclear magnetic resonance (NMR) structure determination use molecular dynamics coupled with a simulated annealing schedule. The objective of these methods is to minimize the error of deviating from the nuclear overhauser effect (NOE) distance constraints. However, the corresponding objective function is highly nonconvex and, consequently, difficult to optimize. Euclidean distance matrix (EDM) methods based on semidefinite programming (SDP) provide a natural framework for these problems. However, the high complexity of SDP solvers and the often noisy distance constraints provide major challenges to this approach. The main contribution of this article is a new SDP formulation for the EDM approach that overcomes these two difficulties. We model the protein as a set of intersecting two- and three-dimensional cliques. Then, we adapt and extend a technique called semidefinite facial reduction to reduce the SDP problem size to approximately one quarter of the size of the original problem. The reduced SDP problem can be solved approximately 100 times faster, and it is also more resistant to numerical problems from erroneous and inexact distance bounds.

Solving optimization problems with sparse or low-rank optimal solutions has been an important topic since the recent emergence of compressed sensing and its matrix extensions such as the matrix rank minimization and robust principal component analysis problems. Compressed sensing enables one to recover a signal or image with fewer observations than the “length” of the signal or image, and thus provides potential breakthroughs in applications where data acquisition is costly. However, the potential impact of compressed sensing cannot be realized without efficient optimization algorithms that can handle extremely large-scale and dense data from real applications. Although the convex relaxations of these problems can be reformulated as either linear programming, second-order cone programming or semidefinite programming problems, the standard methods for solving these relaxations are not applicable because the problems are usually of huge size and contain dense data. In this dissertation, we give efficient algorithms for solving these “sparse” optimization problems and analyze the convergence and iteration complexity properties of these algorithms. Chapter 2 presents algorithms for solving the linearly constrained matrix rank minimization problem. The tightest convex relaxation of this problem is the linearly constrained nuclear norm minimization. Although the latter can be cast and solved as a semidefinite programming problem, such an approach is computationally expensive when the matrices are large. In Chapter 2, we propose fixed-point and Bregman iterative algorithms for solving the nuclear norm minimization problem and prove convergence of the first of these algorithms. By using a homotopy approach together with an approximate singular value decomposition procedure, we get a very fast, robust and powerful algorithm, which we call FPCA (Fixed Point Continuation with Approximate SVD), that can solve very large matrix rank minimization problems. Our numerical results on randomly generated and real matrix completion problems demonstrate that this algorithm is much faster and provides much better recoverability than semidefinite programming solvers such as SDPT3. For example, our algorithm can recover 1000 × 1000 matrices of rank 50 with a relative error of 10−5 in about 3 minutes by sampling only 20 percent of the elements. We know of no other method that achieves as good recoverability. Numerical experiments on online recommendation, DNA microarray data set and image inpainting problems demonstrate the effectiveness of our algorithms. In Chapter 3, we study the convergence/recoverability properties of the fixed-point continuation algorithm and its variants for matrix rank minimization. Heuristics for determining the rank of the matrix when its true rank is not known are also proposed. Some of these algorithms are closely related to greedy algorithms in compressed sensing. Numerical results for these algorithms for solving linearly constrained matrix rank minimization problems are reported. Chapters 4 and 5 considers alternating direction type methods for solving composite convex optimization problems. We present in Chapter 4 alternating linearization algorithms that are based on an alternating direction augmented Lagrangian approach for minimizing the sum of two convex functions. Our basic methods require at most O(1/e) iterations to obtain an e-optimal solution, while our accelerated (i.e., fast) versions require at most O(1/ 3 ) iterations, with little change in the computational effort required at each iteration. For more general problem, i.e., minimizing the sum of K convex functions, we propose multiple-splitting algorithms for solving them. We propose both basic and accelerated algorithms with O(1/e) and O(1/ 3 ) iteration complexity bounds for obtaining an e-optimal solution. To the best of our knowledge, the complexity results presented in these two chapters are the first ones of this type that have been given for splitting and alternating direction type methods. Numerical results on various applications in sparse and low-rank optimization, including compressed sensing, matrix completion, image deblurring, robust principal component analysis, are reported to demonstrate the efficiency of our methods.