Semi-Algebraic Structure and Central Paths in Nonlinear Optimization

We study the properties of the weighted central paths in linear programming. We consider each path as the function of the parameter 0 where the value at D 0 corresponds to the limit point at the boundary of the feasible set. We calculate the recursive formulas for the central path derivatives of all orders valid at each 0. We establish the geometric growth of the derivatives and, consequently, the analyticity of the weighted central path at 0 . This paper also provides the analysis of limiting behavior of those projection operators which often appear in the interior point methods. The central trajectory is one of the most important concepts in the interior point methods for linear programming. The central trajectory (or the central path) is a distinguished curve that is interior to the feasible set and tends to an optimal point. Most interior points algorithms follow, or are related to, the central trajectory. Although the central path concept is essential for the design and analysis of algorithms relatively few papers study properties of this curve itself. For a review on this topic we refer to (3). Recently Zhao and Zhu (16) have studied some analytical properties of the central trajectory in the interior of the feasible set. They have established an upper bound for the high order derivatives of the weighted central path and estimated the convergence radius of the corresponding Taylor series. They have also pointed out the relationship of these properties with the complexity of a path following algorithm where the number of iterations is estimated by a "curvature integral". (See also (15).) Further, Guler (3) has analysed some limiting properties of the central trajectory derivatives with respect to the parameter >0. He has established the existence of finite limits for derivatives of all orders as tends to infinity (that is, the central path stays in the interior) and to zero (the central path tends to the boundary). These results have applications to the construction of polynomial- time algorithms which are also locally fast. A surprising result of this study is that all derivatives of order > 1t end to 0 as!1, which helps to explain why interior point methods for linear programming require so few iterations (see (3)).

Real algebraic geometry deals with the solution set of (possibly quantified) systems of polynomial equations and/or inequalities over the real numbers, which arise frequently in science and engineering. Main concern in real algebraic geometry is to determine the properties of the solution sets such as non-emptiness, dimension and quantifier free description as a semi-algebraic set. Such tasks are carried out by symbolic and algebraic algorithms:cylindrical algebraic decomposition (CAD) or quantifier elimination (QE). Various algorithms and deep complexity results about CAD and QE have been studied during the last several decades. Moreover, practically efficient software systems of QE have been developed and also are applied to many nontrivial application problems. In this talk we explain several algorithms of CAD and QE together with their engineering applications.

We study the relationships between three concepts which arise in connection with variational inequality problems: central paths defined by arbitrary barriers, generalized proximal point methods (where a Bregman distance substitutes for the Euclidean one), and Cauchy trajectory in Riemannian manifolds. First we prove that under rather general hypotheses the central path defined by a general barrier for a monotone variational inequality problem is well defined, bounded, and continuous and converges to the analytic center of the solution set (with respect to the given barrier), thus generalizing results which deal only with complementarity problems and with the logarithmic barrier. Next we prove that a sequence generated by the proximal point method with the Bregman distance naturally induced by the barrier function converges precisely to the same point. Furthermore, for a certain class of problems (including linear programming), such a sequence is contained in the central path, making the concepts of central path and generalized proximal point sequence virtually equivalent. Finally we prove that for this class of problems the central path also coincides with the Cauchy trajectory in the Riemannian manifold defined on the positive orthant by a metric given by the Hessian of the barrier (i.e., a curve whose direction at each point is the negative gradient of the objective function at that point in the Riemannian metric).

Suppose we have a number of noisy measurements of an unknown real-valued function $f$ near point of interest $mathbf{x}_0 in mathbb{R}^d$. Suppose also that nothing can be assumed about the noise distribution, except for zero mean and bounded covariance matrix. We want to estimate $f$ at $mathbf{x=x}_0$ using a general linear parametric family $f(mathbf{x};mathbf{a}) = a_0 h_0 (mathbf{x}) ++ a_q h_q (mathbf{x})$, where $mathbf{a} in mathbb{R}^q$ and $h_i$'s are bounded functions on a neighborhood $B$ of $mathbf{x}_0$ which contains all points of measurement. Typically, $B$ is a Euclidean ball or cube in $mathbb{R}^d$ (more generally, a ball in an $l_p$-norm). In the case when the $h_i$'s are polynomial functions in $x_1,ldots,x_d$ the model is called locally-polynomial. In particular, if the $h_i$'s form a basis of the linear space of polynomials of degree at most two, the model is called locally-quadratic (if the degree is at most three, the model is locally-cubic, etc.). Often, there is information, which is called context, about the function $f$ (restricted to $B$ ) available, such as that it takes values in a known interval, or that it satisfies a Lipschitz condition. The theory of local minimax estimation with context for locally-polynomial models and approximately locally polynomial models has been recently initiated by Jones. In the case of local linearity and a bound on the change of $f$ on $B$, where $B$ is a ball, the solution for squared error loss is in the form of ridge regression, where the ridge parameter is identified; hence, minimax justification for ridge regression is given together with explicit best error bounds. The analysis of polynomial models of degree above 1 leads to interesting and difficult questions in real algebraic geometry and non-linear optimization. We show that in the case when $f$ is a probability function, the optimal (in the minimax sense) estimator is effectively computable (with any given precision), thanks to Tarski's elimination principle.

Nonlinear optimization is becoming challengeable in information sciences and various industrial applications, but nonlinear problems solved by the classical particle swarm-based methods are usually characterized with low efficiency, accuracy, and convergence speed in specific issues. To solve these problems and enhance the nonlinear optimization performance, an improved metaheuristic particle swarm optimization (PSO) model is proposed here. First, the optimization principles and model of the new method are introduced, and algorithms of the improved PSO are presented. Then, influence of the model parameters, input dimensions, and different nonlinear problems on the PSO optimization characterizations are studied by Pareto set solving and optimization performance comparison. The analysis regarding diverse nonlinear problems and optimization methods demonstrates the feasibility of the proposed method. Finally, the performance evaluation is exhibited by the case study of nonlinear parameter optimization, CEC benchmark problems, and rank sum test, which all verify its effectiveness and reliability, as well as the significance and great application promise. Additionally, the future work is discussed.

This issue's SIGEST paper, from the SIAM Journal on Optimization (SIOPT), is an excellent example of deep mathematical analysis, involving a range of mathematical techniques, which is having significant practical algorithmic and computational consequences. The title of the paper, “A Sum of Squares Approximation of Nonnegative Polynomials,” by Jean B. Lasserre, combines two concepts that are known to all SIAM readers: sums of squares, the types of functions that arise in many data fitting and other applications; and nonnegative polynomials, polynomials whose values are greater than or equal to zero for all inputs. It is obvious that any polynomial that can be written as a sum of squares is nonnegative. It is far from obvious whether any nonnegative polynomial has a representation as, or arbitrarily close to, a sum of squares of polynomials. In this SIGEST paper, Lasserre shows that this is the case. The nomination from the editors of SIOPT captures the immediate and more general contributions of this paper eloquently, in the context of a series of three papers of which this is the second: “Jean Lasserre, in a series of papers (starting from the first paper in SIOPT Vol. 11, pp. 796–817, and concluding with the third paper in SIOPT Vol. 15, pp. 383–393) by exploiting deep results from real algebraic geometry and theory of moments for multivariate polynomials, showed that the optimal value of a nonconvex polynomial optimization problem (i.e., an optimization problem whose objective function and constraints are specified by multivariate polynomials) can asymptotically be approximated by an increasing sequence of optimal values from increasing tighter semidefinite programming relaxation problems. Even though the result is asymptotic in the most general case, fortuitously, practical experience has revealed that very often, the first few levels of semidefinite programming relaxations typically can achieve the global optimum value. The theoretical connection that Jean Lasserre (and Pablo Parrilo at about the same time from the viewpoint of sum of squares representations of multivariate polynomials) had established between polynomial optimizations with real algebraic geometry and theory of moments has led to exciting developments in combinatorial optimization. In particular, many hard combinatorial problems such as the maximum clique problem of a graph and the quadratic assignment problem are now actively revisited by researchers under the framework of polynomial optimization to derive increasingly tighter bounds.” That is, a consequence of this and related work is the ability to use a sequence of semidefinite programming problems to (nearly) solve much more difficult optimization problems. Semidefinite programming problems are problems with a linear objective function and linear constraints in a matrix variable X, plus the additional constraint that X be positive semidefinite. Since a rich array of efficient algorithms for solving semidefinite programming problems has been developed in recent years, reducing more general problems to a sequence of semidefinite programming problems is an important advance. We hope that readers of SIAM Review will enjoy this glimpse into the sophisticated mathematics behind an important algorithmic innovation in optimization.

New interactions between real algebraic geometry, convex optimization and free non-commutative geometry have recently emerged, and have been the subject of numerous international meetings. The aim of the workshop was to bring together experts, as well as young researchers, to investigate current key questions at the interface of these fields, and to explore emerging interdisciplinary applications.

A nonlinear programming approach to kinematic shakedown analysis of frictional materials

Recently, in structural engineering, many main elements in structural systems such as columns or girders are designed with variable cross-section for certain purposes. In addition, in order to determine the appropriate sizes or suitable shapes of structural elements for optimal volume of material under the effect of certain loading or with the given sizes of elements the structures can be subjected maximum loading, it is required to studied the optimization problem. This paper presents the non-linear weight optimization problem of frames with variable section stiffness of columns according to the strength, stiffness and stability criteria with the help of finite element method. The non-linear optimization problem is solved by the Genetic Algorithm. The calculation procedure of non-linear optimization of plane frames with variable section stiffness of columns is established and implemented using Matlab calculation programming software. From the results of numerical examples, the conclusions and recommendations will be proposed.

B ackground. Non-linear optimization problems for the operation of large space or functionally distributed systems with independently functioning subsystems and restrictions on some of the common resources are considered. O bjective. The aim is to build an effective approach to solving nonlinear optimization problems of block structure with coupling constraints on the basis of the combination of approximating nonlinear optimization methods and decomposition techniques. M ethods. The three-step iterative scheme has been proposed. On the upper level the original problem is replaced by a sequence of approximating problems with additively-separable objective function and linear constraints. On the second level, coordinating problems, which are formed by the information received at the lowest level from local block problems, are solved. R esults. The algorithm, which is a combination of the linearization method of B.Pshenichniy and the dual decomposition, has been built. The structure of the dual coordination problem has been described. Estimation of the convergence rate has been carried out. Сonclusions. The proposed approach and the algorithm constructed based on it can be applied to a wide range of problems associated with optimal allocation of limited resources in the large block-structured systems.

In this dissertation we develop numerical algorithms and software tools to facilitate parallel solutions of nonlinear programming (NLP) problems. In particular, we address large-scale, block-structured problems with an intrinsic decomposable configuration. These problems arise in a great number of engineering applications, including parameter estimation, optimal control, network optimization, and stochastic programming. The structure of these problems can be leveraged by optimization solvers to accelerate solutions and overcome memory limitations, and we propose variants to two classes of optimization algorithms: augmented Lagrangian (AL) schemes and Schur-complement interior-point methods. The convergence properties of augmented Lagrangian decomposition schemes like the alternating direction method of multipliers (ADMM) and progressive hedging (PH) are well established for convex optimization but convergence guarantees in non-convex settings are still poorly understood. In practice, however, ADMM and PH often perform satisfactorily in complex non-convex NLPs. In this work, we study connections between the method of multipliers (MM), ADMM, and PH to derive benchmarking metrics that explain why PH and ADMM work in practice. We illustrate the concepts using challenging dynamic optimization problems. Our exposition seeks to establish more formalism in benchmarking ADMM, PH, and AL schemes and to motivate algorithmic improvements. The effectiveness of nonlinear interior-point solvers for solving large-scale problems relies quite heavily on the solution of the underlying linear algebra systems. The schur-complement decomposition is very effective for parallelizing the solution of linear systems with modest coupling. However, for systems with large number of coupling variables the schur-complement method does not scale favorably. We implement an approach that uses a Krylov solver (GMRES) preconditioned with ADMM to solve block-structured linear systems that arise in the interior-point method. We show that this ADMM-GMRES approach overcomes the well-known scalability issues of Schur decomposition. One important drawback of using decomposition approaches like ADMM and PH is their convergence rate. Unlike Schur-complement interior-point algorithms that have super-linear convergence, augmented Lagrangian approaches typically exhibit linear and sublinear rates. We exploit connections between ADMM and the Schur-complement decomposition to derive an accelerated version of ADMM. Specifically, we study the effectiveness of performing a Newton-Raphson algorithm to compute multiplier estimates for augmented Lagrangian methods. We demonstrate using two-stage stochastic programming problems that our multiplier update achieves convergence in fewer iterations for MM on general nonlinear problems. In the case of ADMM, the newton update significantly reduces the number of subproblem solves for convex quadratic programs (QPs). Moreover, we show that using newton multiplier updates makes the method robust to the selection of the penalty parameter. Traditionally, state-of-the-art optimization solvers are implemented in low-level programming languages. In our experience, the development of decomposition algorithms in these frameworks is challenging. They present a steep learning curve and can slow the development and testing of new numerical algorithms. To mitigate these challenges, we developed PyNumero, a new open source framework implemented in Python and C++. The package seeks to facilitate development of optimization algorithms for large-scale optimization within a high-level programming environment while at the same time minimizing the computational burden of using Python. The efficiency of PyNumero is illustrated by implementing algorithms for problems arising in stochastic programming and optimal control. Timing results are presented for both serial and parallel implementations. Our computational studies demonstrate that with the appropriate balance between compiled code and Python, efficient implementations of optimization algorithms are achievable in these high-level languages.

Optimal Operation of Integrated Heat and Electricity Systems: A Tightening McCormick Approach

In this study, a constructive criterion algorithm of the global controllability for a class of planar polynomial systems is obtained. This algorithm is imposed on the coefficients of polynomials only and can be realised easily by computer. The analysis is based on a classical result in real algebraic geometry – Sturm theorem and its modern progress. According to this algorithm, the global controllability of the system can be determined in finite steps arithmetic operations. Finally, the authors give some examples to show the application of the author's results.

The paper reviews and introduces the state-of-the-art nature-inspired metaheuristic algorithms in optimization, including Firefly algorithm, PSO algorithms and ABC algorithm. By implementing these algorithms in Matlab, we will use worked examples to show how each algorithm works. Firefly algorithm is one of the evolutionary optimization algorithms, and is inspired by the flashing behaviour of fireflies in nature. There are many noisy non-linear mathematical optimization problems that can be effectively solved by Metaheuristic Algorithms. Mathematical optimization or programming is the study of such planning and design problems using mathematical tools. Nowadays, computer simulations become an indispensable tool for solving such optimization problems with various efficient search algorithms. Nature-inspired algorithms are among the most powerful algorithms for optimization. Firefly algorithm is one of the new metaheuristic algorithms for optimization problems. The algorithm is inspired by the flashing behaviour of fireflies. A Firefly Algorithm (FA) is a recent nature inspired optimization algorithm, which simulates the flash pattern and characteristics of fireflies. It is a powerful swarm intelligence algorithm inspired by the flash phenomenon of the fireflies. In this context, three types of meta-heuristics called Artificial bee Colony algorithm, Particle Swarm Optimization (PSO) and Firefly algorithms were devised to find optimal solutions of noisy non-linear continuous mathematical models. A series of computational experiments using each algorithm were conducted. The stimulation result of this experiment were analyzed and compared to the best solutions found so The Firefly algorithm in each noisy non linear optimization function seems to perform better and efficient.

We consider the logarithmic and the volumetric barrier functions used in interior point methods. In the case of the logarithmic barrier function, the analytic center of a level set is the point at which the central path intersects that level set. We prove that this also holds for the volumetric path. For the central path, it is also true that the analytic center of the optimal level set is the limit point of the central path. The only known case where this last property for the logarithmic barrier function fails occurs in case of semidefinite optimization in the absence of strict complementarity. For the volumetric path, we show with an example that this property does not hold even for a linear optimization problem in canonical form.