On Nonconvex Optimization Problems with D.C. Equality and Inequality Constraints

This paper addresses the nonconvex optimization problem with the cost function and equality and inequality constraints given by d.c. functions. The original problem is reduced to a problem without constraints by means of the exact penalization techniques. Furthermore, the penalized problem is presented as a d.c. minimization problem. For the latter problem, we apply the global optimality conditions (GOCs), which possess the so-called constructive (algorithmic) property. These new GOCs are generalized for the minimizing sequences, and a theoretical method is developed. Based on this theoretical foundation, a new global search scheme is designed for the auxiliary (penalized) and original problems, the convergence of which is one of the new results of the work.

Trajectory optimization for asteroid landing with two-phase free final time

In this paper, the classical exact absolute value penalty function method is used for solving a new class of nonconvex nonsmooth optimization problems. Nonconvex nondifferentiable optimization problems with both inequality and equality constraints are considered here, in which not all functions constituting them have the fundamental property of convex functions and most classes of generalized convex functions—namely, a stationary point of such a function is its global minimum. It is proved for such nonconvex optimization problems that there exists a finite threshold of penalty parameters equal to the largest absolute value of a Lagrange multiplier such that, for every penalty parameter exceeding this lower bound, there is the equivalence between an optimal solution in the original constrained minimization problem with ‐invex functions and a minimizer in its associated penalized optimization problem with the exact l1 penalty function. Further, under nondifferentiable ‐invexity assumptions, a characterization of a saddle point of the Lagrange function, defined for the considered constrained optimization problem in terms of minimizers of its associated exact penalized optimization problem with the exact l1 penalty function, is presented.

Many engineering applications face the problem of bounding the expected value of a quantity of interest (performance, risk, cost, etc.) that depends on stochastic uncertainties whose probability distribution is not known exactly. Optimal uncertainty quantification (OUQ) is a framework that aims at obtaining the best bound in these situations by explicitly incorporating available information about the distribution. Unfortunately, this often leads to non-convex optimization problems that are numerically expensive to solve. This thesis emphasizes on efficient numerical algorithms for OUQ problems. It begins by investigating several classes of OUQ problems that can be reformulated as convex optimization problems. Conditions on the objective function and information constraints under which a convex formulation exists are presented. Since the size of the optimization problem can become quite large, solutions for scaling up are also discussed. Finally, the capability of analyzing a practical system through such convex formulations is demonstrated by a numerical example of energy storage placement in power grids. When an equivalent convex formulation is unavailable, it is possible to find a convex problem that provides a meaningful bound for the original problem, also known as a convex relaxation. As an example, the thesis investigates the setting used in Hoeffding's inequality. The naive formulation requires solving a collection of non-convex polynomial optimization problems whose number grows doubly exponentially. After structures such as symmetry are exploited, it is shown that both the number and the size of the polynomial optimization problems can be reduced significantly. Each polynomial optimization problem is then bounded by its convex relaxation using sums-of-squares. These bounds are found to be tight in all the numerical examples tested in the thesis and are significantly better than Hoeffding's bounds.

A solution algorithm for non-convex mixed integer optimization problems with only few continuous variables

A vast majority of machine learning algorithms train their models and perform inference by solving optimization problems. In order to capture the learning and prediction problems accurately, structural constraints such as sparsity or low rank are frequently imposed or else the objective itself is designed to be a non-convex function. This is especially true of algorithms that operate in high-dimensional spaces or that train non-linear models such as tensor models and deep networks. The freedom to express the learning problem as a non-convex optimization problem gives immense modeling power to the algorithm designer, but often such problems are NP-hard to solve. A popular workaround to this has been to relax non-convex problems to convex ones and use traditional methods to solve the (convex) relaxed optimization problems. However this approach may be lossy and nevertheless presents significant challenges for large scale optimization. On the other hand, direct approaches to non-convex optimization have met with resounding success in several domains and remain the methods of choice for the practitioner, as they frequently outperform relaxation-based techniques - popular heuristics include projected gradient descent and alternating minimization. However, these are often poorly understood in terms of their convergence and other properties. This monograph presents a selection of recent advances that bridge a long-standing gap in our understanding of these heuristics. We hope that an insight into the inner workings of these methods will allow the reader to appreciate the unique marriage of task structure and generative models that allow these heuristic techniques to (provably) succeed. The monograph will lead the reader through several widely used nonconvex optimization techniques, as well as applications thereof. The goal of this monograph is to both, introduce the rich literature in this area, as well as equip the reader with the tools and techniques needed to analyze these simple procedures for non-convex problems.

This paper addresses a bilevel optimization problem with a guaranteed (pessimistic) solution in a rather general form. The problem is reduced to a family of bilevel problems with optimistic solutions and perturbed follower's problems. For the quadratic-linear bilevel optimization problem, a reduction of the bilevel problem to a series of non-convex one-level optimization problems is proposed using the penalty method. Additionally, a solution method based on Global Search Theory (GST) is developed for the obtained problem. The GST consists of two stages: special local search and Global Search Scheme (GSS), which is based on Global Optimality Conditions (GOCs). The computational testing of the developed approach on specially constructed bilevel problems of high dimensions was successfully carried out. Finally, the computational testing on specially constructed bilevel problems of rather high dimensions was successfully carried out, that substantiate the developed approach.

A neurodynamic optimization technique based on overestimator and underestimator functions for solving a class of non-convex optimization problems

We develop new fast and efficient algorithms for designing single or multiple unimodular waveforms with good auto- and cross-correlation or weighted correlation properties, which are highly desired in radar and communication systems. The waveform design is based on the minimization of the integrated sidelobe level (ISL) and weighted ISL (WISL) of waveforms. As the corresponding problems can quickly grow to a large scale with increasing the code length and the number of waveforms, the main issue turns to be the development of fast large-scale optimization techniques. The difficulty is also that the corresponding optimization problems are nonconvex, but the required accuracy is high. Therefore, we formulate the ISL and WISL minimization problems as nonconvex quartic optimization problems in frequency domain, and then simplify them into quadratic problems via majorization-minimization technique, which is one of the basic techniques for addressing large-scale and/or nonconvex optimization problems. While designing our fast algorithms, we explore and use the inherent algebraic structures in objective functions to rewrite them into quartic forms, and in the case of WISL minimization, to derive additionally an alternative quartic form that allows us to apply the quartic-quadratic transformation. Our algorithms are applicable to large-scale unimodular waveform design problems as they are proved to have lower or comparable computational burden (analyzed theoretically) and faster convergence speed (confirmed by comprehensive simulations) than the state-of-the-art algorithms. In addition, the waveforms designed by our algorithms demonstrate better correlation properties compared to their counterparts.

Nonlinear and nonconvex optimization problems are vital and fundamental problems in science and engineering fields. In this article, a novel finite-time circadian rhythms learning network (called FT-CRLN) is proposed for solving nonlinear and nonconvex optimization problems with periodic noises. Different from the traditional recurrent neural networks, the proposed FT-CRLN can suppress the periodic noise notably and achieve excellent convergence performance in solving nonlinear and nonconvex problems. The theoretical analysis and rigorous mathematical proof verify the superior convergence, high accuracy, and strong robustness of the proposed FT-CRLN. The simulation results demonstrate the effectiveness and robustness of the proposed FT-CRLN in solving nonlinear and nonconvex problems compared with other state-of-art neural networks.

Convex relaxation methods are commonly used to solve nonconvex mathematical optimization problems. These methods transform the original nonconvex problem in such a way that effective methods of solving convex optimization problems become applicable. Thus, a convex problem giving the approximate solution of the original task can be solved instead of the original computationally complex problem. Presented is the application of semidefinite relaxation to the task of determining the optimal set of Global navigation satellite systems signals that are selected for processing while solving the positioning problem. The need for signals set optimization is due to large number of navigation satellites accessible for the customers on the ground level. This binary optimization problem is hard to solve in real time. Two approaches are proposed to reduce the initial problem to the convex problem allowing the effective solution.

Optimal insurance contract specification in the upstream sector of the oil and gas industry

In this paper, a new class of nonconvex nonsmooth multiobjective programming problems with both inequality and equality constraints is considered. Namely, the sufficient optimality conditions and Mond-Weir duality results are established for such nondifferentiable multicriteria optimization problems in which the involved functions are nondifferentiable (b,?,?,?)w-univex functions (not necessarly with respect to the same functions b and ? and ?). Then the aforesaid results developed here under (b,?,?, ?)w-uinvexity are applicable for a larger class of nonsmooth vector optimization problems than under other generalized convexity notions existing in the literature.

We consider minimizing a class of nonconvex composite stochastic optimization problems, and deterministic optimization problems whose objective function consists of an expectation function (or an average of finitely many smooth functions) and a weakly convex but potentially nonsmooth function. And in this paper, we focus on the theoretical properties of two types of stochastic splitting methods for solving these nonconvex optimization problems: stochastic alternating direction method of multipliers and stochastic proximal gradient descent. In particular, several inexact versions of these two types of splitting methods are studied. At each iteration, the proposed schemes inexactly solve their subproblems by using relative error criteria instead of exogenous and diminishing error rules, which allows our approaches to handle some complex regularized problems in statistics and machine learning. Under mild conditions, we obtain the convergence of the schemes and their computational complexity related to the evaluations on the component gradient of the smooth function, and find that some conclusions of their exact counterparts can be recovered.

In this paper, we design a Branch and Bound algorithm based on interval arithmetic to address nonconvex robust optimization problems. This algorithm provides the exact global solution of such difficult problems arising in many real life applications. A code was developed in MatLab and was used to solve some robust nonconvex problems with few variables. This first numerical study shows the interest of this approach providing the global solution of such difficult robust nonconvex optimization problems.