New Advances in Convex Optimization and Control Applications

Existing methods for nonlinear robust control often use scenario-based approaches to formulate the control problem as nonlinear optimization problems. Increasing the number of scenarios improves robustness, while increasing the size of the optimization problems. Mitigating the size of the problem by reducing the number of scenarios requires knowledge about how the uncertainty affects the system. This paper draws from local reduction methods used in semi-infinite optimization to solve robust optimal control problems with parametric uncertainty. We show that nonlinear robust optimal control problems are equivalent to semi-infinite optimization problems and can be solved by local reduction. By iteratively adding interim globally worst-case scenarios to the problem, methods based on local reduction provide a way to manage the total number of scenarios. In particular, we show that local reduction methods find worst case scenarios that are not on the boundary of the uncertainty set. The proposed approach is illustrated with a case study with both parametric and additive time-varying uncertainty. The number of scenarios obtained from local reduction is 101, smaller than in the case when all 214+3x192 boundary scenarios are considered. A validation with randomly drawn scenarios shows that our proposed approach reduces the number of scenarios and ensures robustness even if local solvers are used.

Solving optimization problems with sparse or low-rank optimal solutions has been an important topic since the recent emergence of compressed sensing and its matrix extensions such as the matrix rank minimization and robust principal component analysis problems. Compressed sensing enables one to recover a signal or image with fewer observations than the “length” of the signal or image, and thus provides potential breakthroughs in applications where data acquisition is costly. However, the potential impact of compressed sensing cannot be realized without efficient optimization algorithms that can handle extremely large-scale and dense data from real applications. Although the convex relaxations of these problems can be reformulated as either linear programming, second-order cone programming or semidefinite programming problems, the standard methods for solving these relaxations are not applicable because the problems are usually of huge size and contain dense data. In this dissertation, we give efficient algorithms for solving these “sparse” optimization problems and analyze the convergence and iteration complexity properties of these algorithms. Chapter 2 presents algorithms for solving the linearly constrained matrix rank minimization problem. The tightest convex relaxation of this problem is the linearly constrained nuclear norm minimization. Although the latter can be cast and solved as a semidefinite programming problem, such an approach is computationally expensive when the matrices are large. In Chapter 2, we propose fixed-point and Bregman iterative algorithms for solving the nuclear norm minimization problem and prove convergence of the first of these algorithms. By using a homotopy approach together with an approximate singular value decomposition procedure, we get a very fast, robust and powerful algorithm, which we call FPCA (Fixed Point Continuation with Approximate SVD), that can solve very large matrix rank minimization problems. Our numerical results on randomly generated and real matrix completion problems demonstrate that this algorithm is much faster and provides much better recoverability than semidefinite programming solvers such as SDPT3. For example, our algorithm can recover 1000 × 1000 matrices of rank 50 with a relative error of 10−5 in about 3 minutes by sampling only 20 percent of the elements. We know of no other method that achieves as good recoverability. Numerical experiments on online recommendation, DNA microarray data set and image inpainting problems demonstrate the effectiveness of our algorithms. In Chapter 3, we study the convergence/recoverability properties of the fixed-point continuation algorithm and its variants for matrix rank minimization. Heuristics for determining the rank of the matrix when its true rank is not known are also proposed. Some of these algorithms are closely related to greedy algorithms in compressed sensing. Numerical results for these algorithms for solving linearly constrained matrix rank minimization problems are reported. Chapters 4 and 5 considers alternating direction type methods for solving composite convex optimization problems. We present in Chapter 4 alternating linearization algorithms that are based on an alternating direction augmented Lagrangian approach for minimizing the sum of two convex functions. Our basic methods require at most O(1/e) iterations to obtain an e-optimal solution, while our accelerated (i.e., fast) versions require at most O(1/ 3 ) iterations, with little change in the computational effort required at each iteration. For more general problem, i.e., minimizing the sum of K convex functions, we propose multiple-splitting algorithms for solving them. We propose both basic and accelerated algorithms with O(1/e) and O(1/ 3 ) iteration complexity bounds for obtaining an e-optimal solution. To the best of our knowledge, the complexity results presented in these two chapters are the first ones of this type that have been given for splitting and alternating direction type methods. Numerical results on various applications in sparse and low-rank optimization, including compressed sensing, matrix completion, image deblurring, robust principal component analysis, are reported to demonstrate the efficiency of our methods.

This paper gives a new subspace correction algorithm for nonlinear unconstrained convex optimization problems based on the multigrid approach proposed by S. Nash in 2000 and the subspace correction algorithm proposed by X. Tai and J. Xu in 2001. Under some reasonable assumptions, we obtain the convergence as well as a convergence rate estimate for the algorithm. Numerical results show that the algorithm is effective.

Recent research shows that non-convex AC OPF problem can be recast as a convex Semidefinite (SD) problem or Second Order Cone Programming (SOCP) problem. This paper presents a stochastic SOCP OPF (SSOCP-OPF) model for power systems connected to the wind farms. This is performed by reformulating the original non-convex OPF problem using more practical parameters of the power system. Finally, we obtain a convex optimization problem through some well-known approximations and an exact relaxation incorporating the stochastic nature of wind power. One of the advantage of convex SOCP problems, which are a general form of linear problems, is that they can be efficiently solved through Interior Point Methods (IPMs). The proposed OPF model takes advantage of both DC-OPF models (solution efficiency) and full AC-OPF models (solution accuracy). As an application of SSOCP-OPF model, we study the impact of wind power uncertainty on the transmission loss in the power systems. To evaluate the proposed stochastic model modified IEEE 30-bus test system is used. The optimization problem is coded in GAMS platform and solved using its embedded interior point optimizer MOSEK.

Several types of relaxations for binary quadratic polynomial programs can be obtained using linear, second-order cone, or semidefinite techniques. In this paper, we propose a general framework to construct conic relaxations for binary quadratic polynomial programs based on polynomial programming. Using our framework, we re-derive previous relaxation schemes and provide new ones. In particular, we present three relaxations for binary quadratic polynomial programs. The first two relaxations, based on second-order cone and semidefinite programming, represent a significant improvement over previous practical relaxations for several classes of nonconvex binary quadratic polynomial problems. From a practical point of view, due to the computational cost, semidefinite-based relaxations for binary quadratic polynomial problems can be used only to solve small to midsize instances. To improve the computational efficiency for solving such problems, we propose a third relaxation based purely on second-order cone programming. Computational tests on different classes of nonconvex binary quadratic polynomial problems, including quadratic knapsack problems, show that the second-order-cone-based relaxation outperforms the semidefinite-based relaxations that are proposed in the literature in terms of computational efficiency, and it is comparable in terms of bounds.

The AC optimal power flow (OPF) problem is essential for the schedule and operation of power systems. Convex relaxation methods have been studied and used extensively to obtain an optimal solution to the OPF problem. When the exactness of convex relaxations is not guaranteed, it is important to recover a feasible solution for the convex relaxation methods. This paper presents an alternative convex optimisation (ACP) approach that can efficiently recover a feasible solution from the result of second‐order cone programming (SOCP) relaxed OPF in mesh networks. The OPF problem is first formulated as a difference‐of‐convex programming (DCP) problem, then efficiently solved by a penalty convex concave procedure (CCP). By using the solution of a tightened SOCP OPF as an initial point, the proposed algorithm is able to find a global or near‐global optimal solution to the AC OPF problem. Numerical tests show that the proposed method outperforms those semi‐definite programming (SDP) and quadratically constrained quadratic programming (QCQP)–based algorithms.

Existing methods for nonlinear robust control often use scenario‐based approaches to formulate the control problem as large nonlinear optimization problems. The optimization problems are challenging to solve due to their size, especially if the control problems include time‐varying uncertainty. This paper draws from local reduction methods used in semi‐infinite optimization to solve robust optimal control problems with parametric and time‐varying uncertainty. By iteratively adding interim worst‐case scenarios to the problem, methods based on local reduction provide a way to manage the total number of scenarios. We show that the local reduction method for optimal control problems consists of solving a series of simplified optimal control problems to find worst‐case constraint violations. In particular, we present examples where local reduction methods find worst‐case scenarios that are not on the boundary of the uncertainty set. We also provide bounds on the error if local solvers are used. The proposed approach is illustrated with two case studies with parametric and additive time‐varying uncertainty. In the first case study, the number of scenarios obtained from local reduction is 101, smaller than in the case when all extreme scenarios are considered. In the second case study, the number of scenarios obtained from the local reduction is two compared to 512 extreme scenarios. Our approach was able to satisfy the constraints both for parametric uncertainty and time‐varying disturbances, whereas approaches from literature either violated the constraints or became computationally expensive.

In general, optimal control problems rely on numerically rather than analytically solving methods, due to their nonlinearities. The direct method, one of the numerically solving methods, is mainly to transform the optimal control problem into a nonlinear optimization problem with finite dimensions, via discretizing the objective functional and the forced dynamical equations directly. However, in the procedure of the direct method, the classical discretizations of the forced equations will reduce or affect the accuracy of the resulting optimization problem as well as the discrete optimal control. In view of this fact, more accurate and efficient numerical algorithms should be employed to approximate the forced dynamical equations. As verified, the discrete variational difference schemes for forced Birkhoffian systems exhibit excellent numerical behaviors in terms of high accuracy, long-time stability and precise energy prediction. Thus, the forced dynamical equations in optimal control problems, after being represented as forced Birkhoffian equations, can be discretized according to the discrete variational difference schemes for forced Birkhoffian systems. Compared with the method of employing traditional difference schemes to discretize the forced dynamical equations, this way yields faithful nonlinear optimization problems and consequently gives accurate and efficient discrete optimal control. Subsequently, in the paper we are to apply the proposed method of numerically solving optimal control problems to the rendezvous and docking problem of spacecrafts. First, we make a reasonable simplification, i.e., the rendezvous and docking process of two spacecrafts is reduced to the problem of optimally transferring the chaser spacecraft with a continuously acting force from one circular orbit around the Earth to another one. During this transfer, the goal is to minimize the control effort. Second, the dynamical equations of the chaser spacecraft are represented as the form of the forced Birkhoffian equation. Then in this case, the discrete variational difference scheme for forced Birkhoffian system can be employed to discretize the chaser spacecraft's equations of motion. With further discretizing the control effort and the boundary conditions, the resulting nonlinear optimization problem is obtained. Finally, the optimization problem is solved directly by the nonlinear programming method and then the discrete optimal control is achieved. The obtained optimal control is efficient enough to realize the rendezvous and docking process, even though it is only an approximation of the continuous one. Simulation results fully verify the efficiency of the proposed method for numerically solving optimal control problems, if the fact that the time step is chosen to be very large to limit the dimension of the optimization problem is noted.

The paper reviews some recent advances in interior-point methods for linear programming and indicates directions in which future progress can be made. Most of the interior-point methods belong to any of three categories: affine-scaling methods, potential reduction methods and central path methods. These methods are discussed together with infeasible interior methods and homogeneous self-dual methods for linear programming. Also discussed are some theoretical issues in interior-point methods like dependence of complexity bounds on some non-traditional measures different from the input length L of the problem. Finally, the paper concludes with remarks on the comparison of interior-point methods with the simplex method based on their performance on NITLIB suite, a standard collection of test problems.

In this short paper, we study the problem of traversing a crossbar through a bent channel, which has been formulated as a nonlinear convex optimization problem. The result is a MATLAB code that we can use to compute the maximum length of the crossbar as a function of the width of the channel (its two parts) and the angle between them. In case they are perpendicular to each other, the result is expressed analytically and is closely related to the astroid curve (a hypocycloid with four cusps).

This paper proposed a hot-start based fast optimal speed planning method for intelligent vehicles through multiple intersections. The purpose is to improve the real-time performance of current methods. The optimal control problem is formulated into a nonlinear optimization problem. A hierarchical architecture is constructed, where the upper layer utilizes dynamic programming(DP) to provide constraints and an initial guess of the nonlinear optimization at the lower layer. At the upper layer, based on cruise speed economy analysis, a weighted orientation graph problem can be established. The optimal solution of the graph is obtained via DP, which determines the optimal passing green phase at each intersection. At the lower layer, the interior point method for nonlinear optimization is adopted. We use the DP solution as an initial guess of the interior point method, which is a hot-start mechanism for fast convergence in the process of optimization iteration. Simulation results demonstrate that the proposed hot-start mechanism reduces the computational time by up to 68.4% compared with a rule-based initial guess method. The proposed global speed planning method is able to be implemented in real-time while maintaining the same speed profile optimality.

We study support vector machines (SVM) for which the kernel matrix is not specified exactly and it is only known to belong to a given uncertainty set. We consider uncertainties that arise from two sources: (i) data measurement uncertainty, which stems from the statistical errors of input samples; (ii) kernel combination uncertainty, which stems from the weight of individual kernel that needs to be optimized in multiple kernel learning (MKL) problem. Much work has been studied, such as uncertainty sets that allow the corresponding SVMs to be reformulated as semi-definite programs (SDPs), which is very computationally expensive however. Our focus in this paper is to identify uncertainty sets that allow the corresponding SVMs to be reformulated as second-order cone programs (SOCPs), since both the worst case complexity and practical computational effort required to solve SOCPs is at least an order of magnitude less than that needed to solve SDPs of comparable size. In the main part of the paper we propose four uncertainty sets that meet this criterion. Experimental results are presented to confirm the validity of these SOCP reformulations.

Convex optimization is a well-established research area with applications in almost all fields. Over the decades, multiple approaches have been proposed to solve convex programs. The development of interior-point methods allowed solving a more general set of convex programs known as semi-definite programs and second-order cone programs. However, it has been established that these methods are excessively slow for high dimensions, i.e., they suffer from the curse of dimensionality. On the other hand, optimization algorithms on manifold have shown great ability in finding solutions to nonconvex problems in reasonable time. This paper is interested in solving a subset of convex optimization using a different approach. The main idea behind Riemannian optimization is to view the constrained optimization problem as an unconstrained one over a restricted search space. The paper introduces three manifolds to solve convex programs under particular box constraints. The manifolds, called the doubly stochastic, symmetric and the definite multinomial manifolds, generalize the simplex also known as the multinomial manifold. The proposed manifolds and algorithms are well-adapted to solving convex programs in which the variable of interest is a multidimensional probability distribution function. Theoretical analysis and simulation results testify the efficiency of the proposed method over state of the art methods. In particular, they reveal that the proposed framework outperforms conventional generic and specialized solvers, especially in high dimensions.

This paper presents a novel function of energy-optimal adaptive cruise control (EACC) for electric vehicles based on model predictive control (MPC), which optimally plans the host car's speed trajectory for higher energy efficiency through taking the surrounding traffic information and the road conditions ahead into account. To cope with the nonlinearity of the system dynamics, nonlinear MPC (NMPC) is designed in this paper. Afterwards, the nonlinear equality constraints are relaxed so that the optimization problem has a convex feasible set. Moreover, it is proven that the nonlinear problem can be recast as a second-order cone programming (SOCP) problem. The performance of NMPC is evaluated in the real driving condition and compared to both linear MPC and receding horizon dynamic programming (RH-DP). In order to enable real-time control, the interior point method is used to solve both linear and nonlinear MPC problems and the computation time is investigated in this paper.

This paper presents a method for rapidly generating controllability and reachability sets for constrained finite horizon Linear Time Varying (LTV) control systems by using convex optimization techniques. Set generation is accomplished by first solving a Semi-Definite Programming (SDP) problem and then solving a series of Second Order Cone Programming (SOCP) problems. Recent advances in convex optimization solvers have made it possible to find the solutions to these problems very quickly. From a geometric stand-point, we first find the largest volume symmetric simplex that fits within the constrained control problem, then grow new simplices out of the faces of the original simplex. This process is repeated until the growing polytope converges to the constraint boundaries of the actual set. Additionally, a method for incorporating stochastic constraints and uncertainties into the deterministic framework is developed by posing the stochastic constraints as chance-constrained constraints. Finally, the controllability set for a two-vehicle Low Earth Orbit (LEO) rendezvous problem with stochastic uncertainties is generated using the new algorithm.