Multi-task closed-loop inverse kinematics stability through semidefinite programming

In this paper, we consider a least square semidefinite programming problem under ellipsoidal data uncertainty. We show that the robustification of this uncertain problem can be reformulated as a semidefinite linear programming problem with an additional second-order cone constraint. We then provide an explicit quantitative sensitivity analysis on how the solution under the robustification depends on the size/shape of the ellipsoidal data uncertainty set. Next, we prove that, under suitable constraint qualifications, the reformulation has zero duality gap with its dual problem, even when the primal problem itself is infeasible. The dual problem is equivalent to minimizing a smooth objective function over the Cartesian product of second-order cones and the Euclidean space, which is easy to project onto. Thus, we propose a simple variant of the spectral projected gradient method (Birgin et al. in SIAM J. Optim. 10:1196---1211, 2000) to solve the dual problem. While it is well-known that any accumulation point of the sequence generated from the algorithm is a dual optimal solution, we show in addition that the dual objective value along the sequence generated converges to a finite value if and only if the primal problem is feasible, again under suitable constraint qualifications. This latter fact leads to a simple certificate for primal infeasibility in situations when the primal feasible set lies in a known compact set. As an application, we consider robust correlation stress testing where data uncertainty arises due to untimely recording of portfolio holdings. In our computational experiments on this particular application, our algorithm performs reasonably well on medium-sized problems for real data when finding the optimal solution (if exists) or identifying primal infeasibility, and usually outperforms the standard interior-point solver SDPT3 in terms of CPU time.

Semidefinite programming (SDP) covers a wide range of applications such as robust optimization, polynomial optimization, combinatorial optimization, system and control theory, financial engineering, machine learning, quantum information and quantum chemistry. In those applications, SDP problems can be large scale easily. Such large scale SDP problems often satisfy a certain sparsity characterized by a chordal graph structure. This sparsity is classified in two types. The one is the domain space sparsity (d-space sparsity) for positive semidefinite symmetric matrix variables involved in SDP problems, and the other the range space sparsity (r-space sparsity) for matrix-inequality constraints in SDP problems. In this short note, we survey how we exploit these two types of sparsities to solve large scale linear and nonlinear SDP problems. We refer to the paper [7] for more details.

This work studies the problem of analyzing and, subsequently, optimizing the stabilization capabilities of a class of controllers for input constrained nonlinear systems. The proposed techniques apply to continuous, state feedback controllers which are defined in a subset of the state space where the time derivative of a known, candidate Control Lyapunov Function (CLF) can be made negative definite under the input constraints. This set is associated with the domain of this class of controllers and depends on the CLF. The analysis problem concerns approximating the domain and is posed via appropriately formulated set containment relationships through the generalized S-procedure with sum of squares (SOS) constraints. The optimization problem is concerned with the adjustment or enlargement of the domain and constitutes a way of controller synthesis. These objectives are pursued via optimizing over the coefficients of polynomial CLFs, through a sequence of semidefinite programming (SDP) problems with SOS constraints. By partitioning the state space based on the structure of the input value set and building upon earlier results on SOS methods, the SDP problems are subject to only convex constraints, rendering thus the proposed techniques computationally viable. The capabilities of the proposed algorithms are demonstrated through numerical examples.

Antenna array pattern synthesis with mask constraints can be formulated as a convex optimization problem with semiinfinite trigonometric polynomial constraints. The current approach uses a Linear Matrix Inequality (LMI) characterization of the semi-infinite constraints to convert the original problem into a semidefinite programming (SDP) problem. However, an important drawback of this approach is the large number of additional variables incurred in the equivalent SDP representation, which in turn prohibits its use in the design of large antenna arrays that arise in many modern applications. This paper presents an efficient method for the synthesis of large antenna arrays via a novel LMI characterization of semi-infinite constraints that only involves a minimal number of additional variables. Subsequently, the design of patterns for arrays with hundreds of elements can be easily achieved on a standard personal computer using existing SDP solvers.

Several important problems in control theory can be reformulated as semidefinite programming problems, i.e., minimization of a linear objective subject to linear matrix inequality (LMI) constraints. From convex optimization duality theory, conditions for infeasibility of the LMIs, as well as dual optimization problems, can be formulated. These can in turn be reinterpreted in control or system theoretic terms, often yielding new results or new proofs for existing results from control theory. We explore such connections for a few problems associated with linear time-invariant systems.

This issue's SIGEST paper, from the SIAM Journal on Optimization (SIOPT), is an excellent example of deep mathematical analysis, involving a range of mathematical techniques, which is having significant practical algorithmic and computational consequences. The title of the paper, “A Sum of Squares Approximation of Nonnegative Polynomials,” by Jean B. Lasserre, combines two concepts that are known to all SIAM readers: sums of squares, the types of functions that arise in many data fitting and other applications; and nonnegative polynomials, polynomials whose values are greater than or equal to zero for all inputs. It is obvious that any polynomial that can be written as a sum of squares is nonnegative. It is far from obvious whether any nonnegative polynomial has a representation as, or arbitrarily close to, a sum of squares of polynomials. In this SIGEST paper, Lasserre shows that this is the case. The nomination from the editors of SIOPT captures the immediate and more general contributions of this paper eloquently, in the context of a series of three papers of which this is the second: “Jean Lasserre, in a series of papers (starting from the first paper in SIOPT Vol. 11, pp. 796–817, and concluding with the third paper in SIOPT Vol. 15, pp. 383–393) by exploiting deep results from real algebraic geometry and theory of moments for multivariate polynomials, showed that the optimal value of a nonconvex polynomial optimization problem (i.e., an optimization problem whose objective function and constraints are specified by multivariate polynomials) can asymptotically be approximated by an increasing sequence of optimal values from increasing tighter semidefinite programming relaxation problems. Even though the result is asymptotic in the most general case, fortuitously, practical experience has revealed that very often, the first few levels of semidefinite programming relaxations typically can achieve the global optimum value. The theoretical connection that Jean Lasserre (and Pablo Parrilo at about the same time from the viewpoint of sum of squares representations of multivariate polynomials) had established between polynomial optimizations with real algebraic geometry and theory of moments has led to exciting developments in combinatorial optimization. In particular, many hard combinatorial problems such as the maximum clique problem of a graph and the quadratic assignment problem are now actively revisited by researchers under the framework of polynomial optimization to derive increasingly tighter bounds.” That is, a consequence of this and related work is the ability to use a sequence of semidefinite programming problems to (nearly) solve much more difficult optimization problems. Semidefinite programming problems are problems with a linear objective function and linear constraints in a matrix variable X, plus the additional constraint that X be positive semidefinite. Since a rich array of efficient algorithms for solving semidefinite programming problems has been developed in recent years, reducing more general problems to a sequence of semidefinite programming problems is an important advance. We hope that readers of SIAM Review will enjoy this glimpse into the sophisticated mathematics behind an important algorithmic innovation in optimization.

Solving optimization problems with sparse or low-rank optimal solutions has been an important topic since the recent emergence of compressed sensing and its matrix extensions such as the matrix rank minimization and robust principal component analysis problems. Compressed sensing enables one to recover a signal or image with fewer observations than the “length” of the signal or image, and thus provides potential breakthroughs in applications where data acquisition is costly. However, the potential impact of compressed sensing cannot be realized without efficient optimization algorithms that can handle extremely large-scale and dense data from real applications. Although the convex relaxations of these problems can be reformulated as either linear programming, second-order cone programming or semidefinite programming problems, the standard methods for solving these relaxations are not applicable because the problems are usually of huge size and contain dense data. In this dissertation, we give efficient algorithms for solving these “sparse” optimization problems and analyze the convergence and iteration complexity properties of these algorithms. Chapter 2 presents algorithms for solving the linearly constrained matrix rank minimization problem. The tightest convex relaxation of this problem is the linearly constrained nuclear norm minimization. Although the latter can be cast and solved as a semidefinite programming problem, such an approach is computationally expensive when the matrices are large. In Chapter 2, we propose fixed-point and Bregman iterative algorithms for solving the nuclear norm minimization problem and prove convergence of the first of these algorithms. By using a homotopy approach together with an approximate singular value decomposition procedure, we get a very fast, robust and powerful algorithm, which we call FPCA (Fixed Point Continuation with Approximate SVD), that can solve very large matrix rank minimization problems. Our numerical results on randomly generated and real matrix completion problems demonstrate that this algorithm is much faster and provides much better recoverability than semidefinite programming solvers such as SDPT3. For example, our algorithm can recover 1000 × 1000 matrices of rank 50 with a relative error of 10−5 in about 3 minutes by sampling only 20 percent of the elements. We know of no other method that achieves as good recoverability. Numerical experiments on online recommendation, DNA microarray data set and image inpainting problems demonstrate the effectiveness of our algorithms. In Chapter 3, we study the convergence/recoverability properties of the fixed-point continuation algorithm and its variants for matrix rank minimization. Heuristics for determining the rank of the matrix when its true rank is not known are also proposed. Some of these algorithms are closely related to greedy algorithms in compressed sensing. Numerical results for these algorithms for solving linearly constrained matrix rank minimization problems are reported. Chapters 4 and 5 considers alternating direction type methods for solving composite convex optimization problems. We present in Chapter 4 alternating linearization algorithms that are based on an alternating direction augmented Lagrangian approach for minimizing the sum of two convex functions. Our basic methods require at most O(1/e) iterations to obtain an e-optimal solution, while our accelerated (i.e., fast) versions require at most O(1/ 3 ) iterations, with little change in the computational effort required at each iteration. For more general problem, i.e., minimizing the sum of K convex functions, we propose multiple-splitting algorithms for solving them. We propose both basic and accelerated algorithms with O(1/e) and O(1/ 3 ) iteration complexity bounds for obtaining an e-optimal solution. To the best of our knowledge, the complexity results presented in these two chapters are the first ones of this type that have been given for splitting and alternating direction type methods. Numerical results on various applications in sparse and low-rank optimization, including compressed sensing, matrix completion, image deblurring, robust principal component analysis, are reported to demonstrate the efficiency of our methods.

In this paper, we address the motion parameter estimation problem using time difference of arrival (TDOA) measurements, where a mobile source starts from an initial position with constant velocity. The problem is formulated as three non-convex constrained weighted least squares (CWLS) problems. Owing to their non-convex nature, local convergence may occur when solving them by an iterative algorithm, implying good estimates are not guaranteed. We propose to solve these CWLS problems by applying semidefinite relaxation (SDR) to generate three different semidefinite programming (SDP) problems, namely, the motion unconstrained semidefinite programming (MU-SDP), motion constrained semidefinite programming (MC-SDP), and motion parameter direct semidefinite programming (MPD-SDP). The MU-SDP, MC-SDP, and MPD-SDP methods are then extended to the scenario of unknown propagation speed (PS), which is common in underwater and underground localizations. Specifically, for this scenario, the two-step MU-SDP, two-step MC-SDP, and two-step MPD-SDP methods are designed to keep the relaxed SDP problems tight by introducing the penalty function. Simulation results show that MU-SDP performs worse than MC-SDP and MPD-SDP owing to the ignorance of the motion equation, and MC-SDP and MPD-SDP are able to reach the Cramér-Rao lower bound (CRLB) accuracy.

In this paper we consider the general problem of designing the optimal quantum detection for minimum-error quantum state discrimination. Since it is hard to get an explicit solution and we must resort to numerical methods for the general case. We show the problem can be cast into a semidefinite programming(SDP) problem, we use the theory of SDP to solve it. Based on the idea, we propose a method for minimum-error discrimination by formulating the design of optimal quantum detection as a standard SDP problem; furthermore, based on the optimality conditions for SDP, we derive a set of optimality conditions for quantum detection. It is exemplified that this method is simple to implement and can be used to design optimum detection operators for minimum-error discrimination efficiently.

Considered is the problem of the attraction domain estimation in the space “distance to the trajectory – orientation” for the problem of the planar motion control of a wheeled robot. This problem has received much attention in connection with precision farming applications [11]. The mathematical model of the robot takes into account kinematic relationships between velocity of a given target point, orientation of the platform, and control. It is supposed that the four wheels platform moves without slipping. The rear wheels are assumed to be driving while the front wheels are responsible for the rotation of the platform. The control goal is to drive the target point to the desired trajectory and to stabilize its motion. The case of the straight line trajectory is considered in the paper. The control was obtained using the feedback linearization approach [5] and is subject to the two-sided constraints. The system is then rewritten in the so called Lurie form [1, 12] and embedded in the class of systems with nonlinearities constrained by the sector condition. Based on this, the method of attraction domain estimation in the state space of the system is proposed. The negativity condition for the derivative of the Lyapunov function with respect to the system’s dynamics under sector conditions is formulated in terms of solvability of the linear matrix inequality (LMI) [2]. To take into account quadratic constraints the S-procedure [12] is applied. The Lyapunov function is supposed to be a quadratic form with addition of an integral over nonlinearity (so called Lurie-Postnikov function). Earlier, other classes of Lyapunov functions were used for this problem, see for example [8]. The optimization problem was formulated as a semidefinite programming (SDP) problem with LMI constraints. Numerical results are presented. The estimates achieved show less conservativeness in comparison with ellipsoidal ones.KeywordsWheeled robotAbsolute stabilityLurie systemsAttraction domain

Suboptimal LMI-based Solution of Minimum Time Control Problem

In this paper, we develop a discretisation algorithm with an adaptive scheme for solving a class of combined semi-infinite and semi-definite programming problems. We show that any sequence of points generated by the algorithm contains a convergent subsequence; and furthermore, each accumulation point is a local optimal solution of the combined semi-infinite and semi-definite programming problem. To illustrate the effectiveness of the algorithm, two specific classes of problems are solved. They are relaxations of quadratically constrained semi-infinite quadratic programming problems and semi-infinite eigenvalue problems.

In contrast to many other convex optimization classes, state-of-the-art semidefinite programming solvers are still unable to efficiently solve large-scale instances. This work aims to reduce this scalability gap by proposing a novel proximal algorithm for solving general semidefinite programming problems. The key characteristic of the proposed algorithm is to be able to exploit the low-rank property inherent to several semidefinite programming problems. Exploiting the low-rank structure provides a substantial speedup and allows the operator splitting method to efficiently scale to larger instances. As opposed to other low-rank based methods, the proposed algorithm has convergence guarantees for general semidefinite programming problems. Additionally, an open-source semidefinite programming solver called ProxSDP is made available and its implementation details are discussed. Case studies are presented in order to evaluate the performance of the proposed methodology.

In this paper, we propose to use a new optimization method, i.e., semidefinite programming (SDP), to solve large margin estimation (LME) problem of continuous density hidden Markov models (CDHMM’s) in speech recognition. First of all, we introduce a new constraint into the LME to guarantee the boundedness of the margin of CDHMM’s. Secondly, we show that the LME problem under this new constraint can be formulated as a semidefinite programming problem under some relaxation conditions and it can be solved very efficiently by using some fast optimization algorithms specially designed for SDP. The new method is evaluated in a speaker independent E-set speech recognition task on the OGI ISOLET database. Experimental results clearly demonstrate that the large margin estimation via the semidefinite programing (SDP) method can achieve significant word error rate (WER) reduction over the conventional HMM training methods, such as MLE and MCE. It is also shown that the new SDP-based method outperforms the previously proposed optimization methods using gradient descent search.

In this paper, we develop a systematic Lyapunov approach to the regional stability and performance analysis of saturated systems in a general feedback configuration. The only assumptions we make about the system are well-posedness of the algebraic loop and local stability. Problems to be considered include the estimation of the domain of attraction, the reachable set under a class of bounded energy disturbances and the nonlinear L2 gain. The regional analysis is established through an effective treatment of the algebraic loop and the saturation/deadzone function. This treatment yields two forms of differential inclusions, a polytopic differential inclusion (PDI) and a norm-bounded differential inclusion (NDI) that contain the original system. Adjustable parameters are incorporated into the differential inclusions to reflect the regional property. The main idea behind the regional analysis is to ensure that the state remain inside the level set of a certain Lyapunov function where the PDI or the NDI is valid. With quadratic Lyapunov functions, conditions for stability and performances are derived as linear matrix inequalities (LMIs). To obtain less conservative conditions, we use a pair of conjugate non-quadratic Lyapunov functions, the convex hull quadratic function and the max quadratic function. These functions yield bilinear matrix inequalities (BMIs) as conditions for stability and guaranteed performance level. The BMI conditions cover the corresponding LMI conditions as special cases, hence the BMI results are guaranteed to be as good as the LMI results. In most examples, the BMI results are significantly better than the LMI results