A class of nonmonotone Armijo-type line search method for unconstrained optimization

In this paper, we present two nonmonotone versions of adaptive cubic regularized (ARC) method for unconstrained optimization problems. The proposed methods are a combination of the ARC algorithm with the nonmonotone line search methods introduced by Zhang and Hager [A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim. 14 (2004), pp. 1043–1056] and Ahookhosh et al. [A nonmonotone trust-region line search method for large-scale unconstrained optimization, Appl. Math. Model. 36 (2012), pp. 478–487]. The global convergence analysis for these iterative algorithms is established under suitable conditions. Several numerical examples are given to illustrate the efficiency and robustness of the newly suggested methods. The obtained results show the satisfactory performance of the proposed algorithms when compared to the basic ARC algorithm.

In this paper, a new nonmonotone adaptive trust region algorithm is proposed for unconstrained optimization by combining a multidimensional filter and the Goldstein-type line search technique. A modified trust region ratio is presented which results in more reasonable consistency between the accurate model and the approximate model. When a trial step is rejected, we use a multidimensional filter to increase the likelihood that the trial step is accepted. If the trial step is still not successful with the filter, a nonmonotone Goldstein-type line search is used in the direction of the rejected trial step. The approximation of the Hessian matrix is updated by the modified Quasi-Newton formula (CBFGS). Under appropriate conditions, the proposed algorithm is globally convergent and superlinearly convergent. The new algorithm shows better performance in terms of the Dolan–Moré performance profile. Numerical results demonstrate the efficiency and robustness of the proposed algorithm for solving unconstrained optimization problems.

This paper describes a class of frame-based direct search methods for unconstrained optimization without derivatives. A template for convergent direct search methods is developed, some requiring only the relative ordering of function values. At each iteration, the template considers a number of search steps which form a positive basis and conducts a ray search along a step giving adequate decrease. Various ray search strategies are possible, including discrete equivalents of the Goldstein–Armijo and one-sided Wolfe–Powell ray searches. Convergence is shown under mild conditions which allow successive frames to be rotated, translated, and scaled relative to one another.

Techniques for obtaining safely positive definite Hessian approximations with self-scaling and modified quasi-Newton updates are combined to obtain `better' curvature approximations in line search methods for unconstrained optimization. It is shown that this class of methods, like the BFGS method, has the global and superlinear convergence for convex functions. Numerical experiments with this class, using the well-known quasi-Newton BFGS, DFP and a modified SR1 updates, are presented to illustrate some advantages of the new techniques. These experiments show that the performance of several combined methods are substantially better than that of the standard BFGS method. Similar improvements are also obtained if the simple sufficient function reduction condition on the steplength is used instead of the strong Wolfe conditions.

Homotopy with 2-Step Predictor-corrector Method (HSPM) is a global search method for unconstrained optimization. This method is based on a trusted interval technique to locate a subinterval which contain at least one minimizer and it is capable of locating all the minima in one dimensional functions. In this paper, the time complexity of the method is improved by modifying its internal loop to produce a better algorithm. This modification will not affect its original capability but increase the efficiency of the method. Here, an improved method from HSPM called Kerk and Rohanin’s Trusted Interval (KRTI) will be introduced. The convexity of the trusted interval is shown. Time complexity and numerical results of KRTI will be discussed.

Though quasi-Newton methods have been extensively studied in the literature, they either suffer from local convergence or use a series of line searches for global convergence. In this work, we propose a line search free greedy quasi-Newton method with adaptive steps and establish explicit nonasymptotic bounds for both the global convergence rate and local superlinear rate. Our novel idea lies in the design of multiple greedy quasi-Newton updates to control the Hessian approximation error and a simple mechanism to adjust stepsizes to ensure function improvement per iteration. The global superlinear convergence <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> of our method is validated via numerical experiments.

It is well known that among the current methods for unconstrained optimization problems the quasi-Newton methods with global strategy may be the most efficient methods, which have local superlinear convergence. However, when the iterative point is far away from the solution of the problem, quasi-Newton method may proceed slowly for the general unconstrained optimization problems. In this article an adaptive conic trust-region method for unconstrained optimization is presented. Not only the gradient information but also the values of the objective function are used to construct the local model at the current iterative point. Moreover, we define a concept of super steepest descent direction and embed its information into the local model. The amount of computation in each iteration of this adaptive algorithm is the same as that of the standard quasi-Newton method with trust region. Some numerical results show that the modified method requires fewer iterations than the standard methods to reach the solution of the optimization problem. Global and local convergence of the method is also analyzed.

A new super-memory gradient method with curve search rule

Armijo method is a kind of line search method that usually be used when look for the step size in nonlinear optimization. This paper makes the summary of its modified forms, and then the nonmonotone Armijo-type line search methods are proposed. The numerical results will show that some line search methods with the novel nonmonotone line search are available and efficient in practical computation.

In this paper, by using a forcing function and the reverse modulus of continuity of gradient, we propose a generalization and development of the nonmonotone line search method for unconstrained optimization problems. The global convergence property of the new method is established under some reasonable conditions that are weaker than those of the existing nonmonotone line search methods. Furthermore, the proof of convergence is simpler than other methods.

Adaptive scaling damped BFGS method without gradient Lipschitz continuity

&lt;p style='text-indent:20px;'&gt;Motivated by recent derivative-free projection methods proposed in the literature for solving nonlinear constrained equations, in this paper we propose a unified derivative-free projection method model for large-scale nonlinear equations with convex constraints. Under mild conditions, the global convergence and convergence rate of the proposed method are established. In order to verify the feasibility and effectiveness of the model, a practical algorithm is devised and the corresponding numerical experiments are reported, which show that the proposed practical method is efficient and can be applied to solve large-scale nonsmooth equations. Moreover, the proposed practical algorithm is also extended to solve the obstacle problem.&lt;/p&gt;

A superlinearly convergent SQP algorithm for mathematical programs with linear complementarity constraints

In this article, we investigate the possibilities of accelerating the double smoothing (DS) technique when solving unconstrained nondifferentiable convex optimization problems. This approach relies on the regularization in two steps of the Fenchel dual problem associated with the problem to be solved into an optimization problem having a differentiable strongly convex objective function with Lipschitz continuous gradient. The doubly regularized dual problem is then solved via a fast gradient method. The aim of this article is to show how the properties of the functions in the objective of the primal problem influence the implementation of the DS approach and its rate of convergence. The theoretical results are applied to linear inverse problems by making use of different regularization functionals.

Since optimization problems are getting more complicated, new ways to solve them must be thought of, or existing methods must be improved. In this research, we expand the different parameters of the three-term conjugate gradient method to work out unconstrained optimization problems. Our new CG approach meets the conditions of sufficient descent, and global convergence. In addition, we describe some numerical results that imply comparisons to relevant methodologies in the existing research literature.