Duality Gap Estimates for Weak Chebyshev Greedy Algorithms in Banach Spaces

Theoretical greedy type algorithms are studied: a Weak Greedy Algorithm, a Weak Orthogonal Greedy Algorithm, and a Weak Relaxed Greedy Algorithm. These algorithms are defined by weaker assumptions than their analogs the Pure Greedy Algorithm, an Orthogonal Greedy Algorithm, and a Relaxed Greedy Algorithm. The weaker assumptions make these new algorithms more ready for practical implementation. We prove the convergence theorems and also give estimates for the rate of approximation by means of these algorithms. The convergence and the estimates apply to approximation from an arbitrary dictionary in a Hilbert space.

In this paper we study local behavior of optimal solutions of parametric programs in Banach spaces. A general approach to approximation of the original problem by a simpler one is outlined. Under various regularity conditions, Lipschitz continuity and directional differentiability properties of the optimal solutions are derived.

Function spaces and its various applications are subjects of investigation in many research centers and we are glad that this special issue has received the remarkable attention by researchers in these areas. We have received papers from wide range of topics of function spaces such as interpolation methods for stochastic processes spaces, vector optimization problems in Banach spaces, Marcinkiewicz integral operator in Morrey spaces, Hardy’s operator in the variable exponent Lebesgue spaces, and differential equations of weakly parabolic type in Banach spaces. A number of papers are devoted to investigating different kinds of problems in Hardy spaces or Hardy subspaces, for example, Calderon-Zygumnd singular integral operators in Hardy spaces, weighted Hardy spaces related to the Schrodinger operator, and composition operator in hyperbolic Bloch-type spaces. Geometric problems such as modulus of convexity in some Banach sequence spaces, p-uniform convexity, and quniform smoothness in C spaces are also covered by this issue.

In this paper, the convergence of the fundamental alternating minimization is established for non-smooth non-strongly convex optimization problems in Banach spaces, and novel rates of convergence are provided. As objective function a composition of a smooth, and a block-separable, non-smooth part is considered, covering a large range of applications. For the former, three different relaxations of strong convexity are considered: (i) quasi-strong convexity; (ii) quadratic functional growth; and (iii) plain convexity. With new and improved rates benefiting from both separate steps of the scheme, linear convergence is proved for (i) and (ii), whereas sublinear convergence is showed for (iii).

This paper studies constrained optimization problems in Banach spaces without usual differentiability and convexity assumptions on the functionals involved in the data. The aim is to give optimality conditions for the problems from which one can derive the characterization of best approximations. The objective and the inequality constraint functionals are assumed to have one-sided directional derivatives. First-order necessary conditions are given in terms of subdifferentials of the directional derivatives. The notion of “max-pseudoconvexity” weaker than pseudoconvexity is introduced for sufficiency. The optimality conditions are applied to linear and nonlinear Tchebycheff approximation problems to derive the characterization of best approximations.

This paper is concerned with generalized differentiation of the efficient point multifunctions of parametric vector optimization problems in Banach spaces. We give formulae for computing and/or estimating the Fréchet coderivative (precoderivative) of this multifunction in a broad class of conventional vector optimization problems with the presence of geometric, operator, (finite and infinite) functional constraints. Examples are given to illustrate the obtained results.

This study is devoted to constrained optimization problems in Banach spaces. We present different properties of tangent cones associated with an arbitrary subset of a Banach space and establish correlations with some of the existing results. In absence of both differentiability and convexity assumptions on the functions involved in the optimization problem, the consideration of these tangent cones and their polars leads us to introduce new concepts in nondifferentiable programming. Necessary optimality conditions are first developed in a general abstract form; then these conditions are made more precise in the presence of equality constraints by introducing the concept of normal subcone.

We analyze a convex stochastic optimization problem where the state is\nassumed to belong to the Bochner space of essentially bounded random variables\nwith images in a reflexive and separable Banach space. For this problem, we\nobtain optimality conditions that are, with an appropriate model, necessary and\nsufficient. Additionally, the Lagrange multipliers associated with optimality\nconditions are integrable vector-valued functions and not only measures. A\nmodel problem is given demonstrating the application to PDE-constrained\noptimization under uncertainty with an outlook for further applications.\n

Families of perturbed, (non-convex) optimization problems in Banach spaces have been shown to be densely solvable ; this paper tries to present a unified approach of such density results and highlights their intimate connection with the geometry of Banach spaces.

This paper considers a class of nonlinear differentiable optimization problems depending on a parameter. We show that, if constraint regularity, a second-order sufficient optimality condition, and a stability condition for the Lagrange multipliers hold, then for sufficiently smooth perturbations of the constraints and the objective function the optimal solutions locally obey a type of Lipschitz condition. The results are applied to finite-dimensional problems, equality constrained problems, and optimal control problems.

A unified way of analyzing some greedy algorithms

The set of feasible solutions of a discrete optimization problem can, in many cases, be represented as an independence system, or hereditary system of sets. It is often the case that this system has some structure which may be exploited in the solution of the problem. For example, in the case of a matroid a “greedy algorithm” can be used; this situation, however, occurs rarely in practice. We investigate here the abstract structure of independence systems using concepts from matroid theory, in particular analogues of matroid closure. Two functions are defined which coincide with the ususal closure operator if the independence system is a matroid, and for each of these functions an axiomatic characterization is given. In each case this consists of a weakened form of one of the matroid closure axioms with the other three remaining unchanged. Finally, duality is considered and this leads to characterizations, in terms of closure operators and paintings, of those independence systems where all bases have the same cardinality: these include the “Travelling Salesman” and “Acyclic Subgraph” independence systems.

In this paper some theorems which characterize the global optimal solutions of nonlinear programming problems are proved. Two algorithms are derived using these results. The first is a path following algorithm to approximate the Karush Kuhn Tucker points of linearly constrained optimization problems and the second is an algorithm to solve linearly constrained global optimization problems. The convergence of these algorithms is proved under suitable assumptions. Numerical results obtained on several test problems are shown.

Model order reduction (MOR) is usually equipped with two stages: the offline stage and the online stage.For many intrusive MOR methods that are based on projection, the offline stage is usually realized via a greedy algorithm.The greedy algorithm is used to properly select important parameter samples that contribute most to the solution space.The offline computational time is basically the runtime of the greedy algorithm.For large-scale systems, the offline computation is expensive and the runtime is longer than several hours even if it is run on a high-performance server.Sometimes, the system is not very large, for example, the number of degrees of freedom is only O(10 5 ), but the system structure is complicated such that the greedy algorithm still takes long time to converge.Once the reduced-order model is obtained, the online stage of querying the reduced-order model for simulation is very fast and in real-time.This work concerns a strategy of significantly accelerating the greedy algorithm.We propose multifidelity error estimators and replace the high-fidelity error estimator in the greedy algorithm.In fact, a bi-fidelity error estimation has been used in the adaptive greedy algorithm proposed in [1] without being formally defined.To further improve the convergence speed of the greedy algorithm, we propose multi-fidelity error estimation built upon the bi-fidelity error estimation.Here, we use a more efficient high-fidelity error estimator than the two different high-fidelity error estimators used in [1].Although the proposed multi-fidelity error estimation is dependent on the original high-fidelity error estimator, the concept of using multi-fidelity error estimation is generic and can be extended to develop multi-fidelity error estimation associated with other high-fidelity error estimators.The multi-fidelity error estimator is then used to adaptively update the training set of the greedy algorithm, keeping the offline computational costs on a possibly lowest level.Consequently, the computational complexity at each iteration of the greedy algorithm is reduced and the algorithm converges more than 3 times faster without incurring noticeable accuracy loss.

It is well known that among the current methods for unconstrained optimization problems the quasi-Newton methods with global strategy may be the most efficient methods, which have local superlinear convergence. However, when the iterative point is far away from the solution of the problem, quasi-Newton method may proceed slowly for the general unconstrained optimization problems. In this article an adaptive conic trust-region method for unconstrained optimization is presented. Not only the gradient information but also the values of the objective function are used to construct the local model at the current iterative point. Moreover, we define a concept of super steepest descent direction and embed its information into the local model. The amount of computation in each iteration of this adaptive algorithm is the same as that of the standard quasi-Newton method with trust region. Some numerical results show that the modified method requires fewer iterations than the standard methods to reach the solution of the optimization problem. Global and local convergence of the method is also analyzed.