Analogues of Switching Subgradient Schemes for Relatively Lipschitz-Continuous Convex Programming Problems

In this paper, online convex optimization (OCO) problems with time-varying objective and constraint functions are studied from the perspective of an agent who takes actions in real time. Information about the current objective and constraint functions is revealed only after the corresponding action is already chosen. Inspired by a fast converging algorithm for time-invariant optimization in the very recent work [1], we develop a novel online algorithm based on virtual queues for constrained OCO. Optimal points of the dynamic optimization problems with full knowledge of the current objective and constraint functions are used as a dynamic benchmark sequence. Upper bounds on the regrets with respect to the dynamic benchmark and the constraint violations are derived for the presented algorithm in terms of the temporal variations of the underlying dynamic optimization problems. It is observed that the proposed algorithm possesses sublinear regret and sublinear constraint violations, as long as the temporal variations of the optimization problems are sublinear, i.e., the objective and constraint functions do not vary too drastically across time. The performance bounds of the proposed algorithm are superior to those of the state-of-the-art OCO method in most scenarios. Besides, different from the saddle point methods widely used in constrained OCO, the stepsize of the proposed algorithm does not rely on the total time horizon, which may be unknown in practice. Finally, the algorithm is applied to a dynamic resource allocation problem in data center networks. Numerical experiments are conducted to corroborate the merit of the developed algorithm and its advantage over the state-of-the-art.

The class of combinatorial optimization problems over polyhedral-spherical sets is considered. The results of theory of convex extensions are generalized to certain classes of functions defined on sphere-located and vertex-located sets. The original problem is equivalently formulated as a mathematical programming problem with convex both objective function and functional constraints. A numerical illustration and possible applications of the results to solving combinatorial optimization problems are given.

Many engineering applications face the problem of bounding the expected value of a quantity of interest (performance, risk, cost, etc.) that depends on stochastic uncertainties whose probability distribution is not known exactly. Optimal uncertainty quantification (OUQ) is a framework that aims at obtaining the best bound in these situations by explicitly incorporating available information about the distribution. Unfortunately, this often leads to non-convex optimization problems that are numerically expensive to solve. This thesis emphasizes on efficient numerical algorithms for OUQ problems. It begins by investigating several classes of OUQ problems that can be reformulated as convex optimization problems. Conditions on the objective function and information constraints under which a convex formulation exists are presented. Since the size of the optimization problem can become quite large, solutions for scaling up are also discussed. Finally, the capability of analyzing a practical system through such convex formulations is demonstrated by a numerical example of energy storage placement in power grids. When an equivalent convex formulation is unavailable, it is possible to find a convex problem that provides a meaningful bound for the original problem, also known as a convex relaxation. As an example, the thesis investigates the setting used in Hoeffding's inequality. The naive formulation requires solving a collection of non-convex polynomial optimization problems whose number grows doubly exponentially. After structures such as symmetry are exploited, it is shown that both the number and the size of the polynomial optimization problems can be reduced significantly. Each polynomial optimization problem is then bounded by its convex relaxation using sums-of-squares. These bounds are found to be tight in all the numerical examples tested in the thesis and are significantly better than Hoeffding's bounds.

In this paper, a concept of Euclidean combinatorial configuration as a mapping of a set of certain objects into a point of Euclidean space is introduced. A typology of Euclidean combinatorial configurations sets is offered based on their main structural and geometric characteristics. Optimization problems over vertex-located sets of Euclidean combinatorial configurations are studied, in particular, their equivalent formulation with convex both objective function and functional constraints is constructed.

본 논문은 사각 보강 패널에 사용되는 모자형 보강재의 크기 최적화에 대한 결과를 제시하였다. 보강 패널은 1500 ~ 2500 m/s 의 속도를 가지는 발사체와 충돌한다. 모자형 보강재의 크기를 결정하기 위해서 크기 최적화가 수행되었다. 크기 최적화를 수행하기 위해서는 3 개의 함수들로 구성되어 있다. 이 함수들은 목적함수, 제한함수 그리고 설계 함수로 이루어져 있다. 목적 함수는 보강 패널의 1 차 고유진동수의 최대화가 되도록 하는 것이다. 제함 함수는 보강재의 부피가 사각 패널 부피의 10 % 이내가 되도록 하는 것이며, 설계 변수로는 모자형 보강재의 치수가 된다. 최적화된 모자형 보강재를 이용하한 보강 패널을 사용하여 초고속 충돌에 대한 시뮬레이션을 수행하였으며, 최적화된 보강재에 대해서 속도와 운동 에너지 변화에 대한 결과를 얻었다. 보강 패널의 충돌 저감을 평가하기 위해서 발사체의 운동에너지와 속도를 무차원화 계산을 수행하여 비교 분석 하였다.

In the paper, we introduce the concepts of $$G$$ -type I and generalized $$G$$ -type I functions for a new class of nonconvex multiobjective variational control problems. For such nonconvex vector optimization problems, we prove sufficient optimality conditions for weakly efficiency, efficiency and properly efficiency under assumptions that the functions constituting them are $$G$$ -type I and/or generalized $$G$$ -type I objective and constraint functions. Further, for the considered multiobjective variational control problem, its dual multiobjective variational control problem is given and several duality results are established under (generalized) $$G$$ -type I objective and constraint functions.

A class of nonsmooth vector optimization problems are considered, where the feasible set defined by cone constraint and the objective and constraint functions are locally Lipschitz. The concept of the Clarke's generalized directional derivative for a locally Lipschitz function is introduced. By using this concept, necessary optimality condition for the unconstrained optimization problem is established. Furthermore, a Slater-type constraint qualification is given in such a way that they generalize the classical one, when the constraint functions are differentiable. Then, Kuhn–Tucker necessary optimality condition in terms of the Clarke subdifferentials is obtained.

This paper presents a subgradient-based algorithm for constrained nonsmooth convex optimization that does not require projections onto the feasible set. While the well-established Frank–Wolfe algorithm and its variants already avoid projections, they are primarily designed for smooth objective functions. In contrast, our proposed algorithm can handle nonsmooth problems with general convex functional inequality constraints. It achieves an ϵ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\epsilon $$\\end{document}-suboptimal solution in O(ϵ-2)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathcal {O}(\\epsilon ^{-2})$$\\end{document} iterations, with each iteration requiring only a single (potentially inexact) Linear Minimization Oracle call and a (possibly inexact) subgradient computation. This performance is consistent with existing lower bounds. Similar performance is observed when deterministic subgradients are replaced with stochastic subgradients. In the special case where there are no functional inequality constraints, our algorithm competes favorably with a recent nonsmooth projection-free method designed for constraint-free problems. Our approach utilizes a simple separation scheme in conjunction with a new Lagrange multiplier update rule.

This paper introduces a systematic approach for designing standing wave oscillators (SWOs) optimized through geometric programming. A geometric program is a type of mathematical optimization problem characterized by objective and constraint functions, where all functions are of posynomial or monomial forms. The design of the SWO is formulated as a geometric program problem, which is then solved efficiently as a convex optimization problem. Owing to the efficiency of the proposed method, quantitative tradeoffs among different performance specifications can be analyzed very quickly. As for a SWO operating at 30GHz fabricated using 0.18um 1P6M CMOS technology, simulation results show that the phase noise is around -100dBc/Hz at 1MHz offset, and the power consumption is 27.45mW with a supply voltage of 1.8V.

We revisit and adapt the extended sequential quadratic method (ESQM) in Auslender (J Optim Theory Appl 156:183–212, 2013) for solving a class of difference-of-convex optimization problems whose constraints are defined as the intersection of level sets of Lipschitz differentiable functions and a simple compact convex set. Particularly, for this class of problems, we develop a variant of ESQM, called ESQM with extrapolation (ESQMe), which incorporates Nesterov’s extrapolation techniques for empirical acceleration. Under standard constraint qualifications, we show that the sequence generated by ESQMe clusters at a critical point if the extrapolation parameters are uniformly bounded above by a certain threshold. Convergence of the whole sequence and the convergence rate are established by assuming Kurdyka-Łojasiewicz (KL) property of a suitable potential function and imposing additional differentiability assumptions on the objective and constraint functions. In addition, when the objective and constraint functions are all convex, we show that linear convergence can be established if a certain exact penalty function is known to be a KL function with exponent 12; we also discuss how the KL exponent of such an exact penalty function can be deduced from that of the original extended objective (i.e., sum of the objective and the indicator function of the constraint set). Finally, we perform numerical experiments to demonstrate the empirical acceleration of ESQMe over a basic version of ESQM, and illustrate its effectiveness by comparing with the natural competing algorithm SCPls from Yu et al. (SIAM J Optim 31:2024–2054, 2021).

A surrogate-assisted evolution strategy for constrained multi-objective optimization

Optimization of complex mechanics simulations with object-oriented software designs

This book is on modern continuous nonlinear optimization. Continuous nonlinear optimization problems have a simple mathematical model and always refer to a real physical system, the running of which we want to optimize. Firstly, a nonlinear optimization problem contains an objective function which measures the performances or requirements of the system. Often, this function represents a profit, a time interval, a level, a sort of energy or combination of different quantities which have a physical significance for the modeler. The objective function depends on some characteristics of the system, called variables or unknowns. The purpose of any optimization problem is to find the values of these variables that minimize (or maximize) the objective function subject to some constraints which the variables must satisfy. Constraints of an optimization problem may have different algebraic expressions. There are static and dynamic constraints called functional constraints. The difference between these types of constraints comes from the structure of their Jacobian. The dynamic constraints always involve a Jacobian with a block-diagonal structure. Besides, these functional constraints are nonlinear functions which may be equalities or inequalities, or even range constraints. Another very important type of constraints is the simple bounds on variables. The real applications of optimization include simple bounds on variables which express the constructive engineering conditions and limits of the system to be optimized. The constraints of an optimization problem define the so-called feasible domain of the problem. Both the objective function and the constraints may depend on some parameters with known values which characterize the system under optimization. The process of identifying the variables, parameters, simple bounds on variables, the objective functions, and constraints is known as modeling, one of the finest intellectual activities. In this book we assume that the variables can take real values and the objective function and the constraints are smooth enough (at least twice continuously differentiable) with known first-order derivatives. When the number of variables and the number of constraints is large, then the optimization problem is quite challenging.

The CoMirror algorithm, by Beck et al. (Oper Res Lett 38(6):493–498, 2010), is designed to solve convex optimization problems with one functional constraint. At each iteration, it performs a mirror-descent update using either the subgradient of the objective function or the subgradient of the constraint function, depending on whether or not the constraint violation is below some tolerance. In this paper, we combine the CoMirror algorithm with inexact cut generation to create the SIP-CoM algorithm for solving semi-infinite programming (SIP) problems. First, we provide general error bounds for SIP-CoM. Then, we propose two specific random constraint sampling schemes to approximately solve the cut generation problem for generic SIP. When the objective and constraint functions are generally convex, randomized SIP-CoM achieves an $${\mathcal {O}}(1/\sqrt{N})$$ convergence rate in expectation (in terms of the optimality gap and SIP constraint violation). When the objective and constraint functions are all strongly convex, this rate can be improved to $${\mathcal {O}}(1/N)$$ .

This paper establishes the relations between the changes in the values of the augmented multipliers and the changes in the values of the constraint and objective functions for a non-linear, non-convex programme based on the augmented Lagrangian which was introduced by one of the present authors. If the penalty parameters are sufficiently large, the monotonic relations hold between the augmented multipliers and the constraint functions near the optimal solution. Illustrative examples show that the classical Lagrangian fails to give rise to the relations for a. non-convex programme.