An alternative three-dimensional subspace method based on conic model for unconstrained optimization

In this paper, a three-dimensional subspace method is proposed, in which the search direction is generated by minimizing the approximation model of the objective function in a three-dimensional subspace. The approximation model of the objective function is not unique, and alternatives can be chosen between a symmetric quadratic model and a conic model by specific criteria. Moreover, the idea of a WLY conjugate gradient method is applied to characterize the change of gradient direction between adjacent iteration points. The strategy of initial stepsize and nonmonotone line search are adopted, and the global convergence of the presented algorithm is established under mild assumptions. In numerical experiments, we use a collection of 80 unconstrained optimization test problems to show the competitive performance of the presented method.

In this paper, we present two nonmonotone versions of adaptive cubic regularized (ARC) method for unconstrained optimization problems. The proposed methods are a combination of the ARC algorithm with the nonmonotone line search methods introduced by Zhang and Hager [A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim. 14 (2004), pp. 1043–1056] and Ahookhosh et al. [A nonmonotone trust-region line search method for large-scale unconstrained optimization, Appl. Math. Model. 36 (2012), pp. 478–487]. The global convergence analysis for these iterative algorithms is established under suitable conditions. Several numerical examples are given to illustrate the efficiency and robustness of the newly suggested methods. The obtained results show the satisfactory performance of the proposed algorithms when compared to the basic ARC algorithm.

In this paper, we introduce a new concept of approximate optimal stepsize for gradient method, use it to interpret the Barzilai-Borwein (BB) method, and present an efficient gradient method with approximate optimal stepsize for large unconstrained optimization. If the objective function f is not close to a quadratic on a line segment between the current iterate x k and the latest iterate x k−1, we construct a conic model to generate the approximate optimal stepsize for gradient method if the conic model is suitable to be used. Otherwise, we construct a new quadratic model or two other new approximation models to generate the approximate optimal stepsize for gradient method. We analyze the convergence of the proposed method under some suitable conditions. Numerical results show the proposed method is very promising.

In this paper, a new nonmonotone adaptive trust region algorithm is proposed for unconstrained optimization by combining a multidimensional filter and the Goldstein-type line search technique. A modified trust region ratio is presented which results in more reasonable consistency between the accurate model and the approximate model. When a trial step is rejected, we use a multidimensional filter to increase the likelihood that the trial step is accepted. If the trial step is still not successful with the filter, a nonmonotone Goldstein-type line search is used in the direction of the rejected trial step. The approximation of the Hessian matrix is updated by the modified Quasi-Newton formula (CBFGS). Under appropriate conditions, the proposed algorithm is globally convergent and superlinearly convergent. The new algorithm shows better performance in terms of the Dolan–Moré performance profile. Numerical results demonstrate the efficiency and robustness of the proposed algorithm for solving unconstrained optimization problems.

A modified Polak–Ribière–Polyak conjugate gradient algorithm for nonsmooth convex programs

A variant of trust-region methods for unconstrained optimization

An adaptive conjugate gradient algorithm for large-scale unconstrained optimization

We propose a new truncated Newton method for large scale unconstrained optimization, where a Conjugate Gradient (CG)-based technique is adopted to solve Newton’s equation. In the current iteration, the Krylov method computes a pair of search directions: the first approximates the Newton step of the quadratic convex model, while the second is a suitable negative curvature direction. A test based on the quadratic model of the objective function is used to select the most promising between the two search directions. Both the latter selection rule and the CG stopping criterion for approximately solving Newton’s equation, strongly rely on conjugacy conditions. An appropriate linesearch technique is adopted for each search direction: a nonmonotone stabilization is used with the approximate Newton step, while an Armijo type linesearch is used for the negative curvature direction. The proposed algorithm is both globally and superlinearly convergent to stationary points satisfying second order necessary conditions. We carry out a significant numerical experience in order to test our proposal.

In this paper, some quadratic interpolation models are constructed by developing new trust region sub-problems. These sub-problems are derived by modifying search directions which combine the information progressively obtained during the iterations of the algorithm to form new search subspaces. The quadratic models are solved in the new subspaces. The motive is to use the information disclosed by the previous steps to construct probably more promising directions. The effectiveness is proved in that the number of function evaluations are reduced obviously for most tested problems. At the end of this paper, we test the models on some variable dimension problems.

Hybridizing monotone and nonmonotone approaches, we propose a modified trust region ratio in which more reasonable information is provided about consistency between the exact and the approximate models. Also, we employ an Armijo-type line search strategy to avoid resolving the trust region subproblem whenever a trial step is rejected. We show that the proposed algorithm can preserve global convergence of the traditional trust region algorithm as well as the superlinear convergence property. Numerical experiments are done on a set of unconstrained optimization test problems of the CUTEr library. They demonstrate practical efficiency of the proposed algorithm in the sense of the Dolan–More performance profile.

When solving ill-conditioned nonlinear programs by descent algorithms, the descent requirement may induce the step lengths to become very. small. thus resulting in very poor performances. Recently, suggestions have been made to circumvent this problem. among which is a class of approaches in which the objective value may be allowed to increase temporarily. Grippo et al. [GLL91] introduce nonmonotone line searches in the class of deflected gradient methods in unconstrained differentiable optimization; this technique allows for longer steps (typically of unit length) to be taken, and is successfully applied to some ill-conditioned problems. This paper extends their nonmonotone approach and convergence results to the large class of cost approximation algorithms of Patriksson [Pat93b]. and to optimization problems with both convex constraints and nondifferentiable objective functions

In this article, we propose a new line search method for solving unconstrained optimization problems in that we combine a nonmonotone strategy into a modified Armijo rule and design a new algorithm that possibly chooses a larger steplength. This can decrease the number of iterations and function evaluations and can improve the efficiency of the algorithm. The global convergence and convergence rate are analysed under some suitable conditions. Preliminary numerical experiments establish that the new approach is robust and efficient for unconstrained optimization problems.

Steepest Descent is known as the simplest gradient method. Recently, many researches are done to obtain the appropriate step size in order to reduce the objective function value progressively. In this paper, the properties of steepest descent method from literatures are reviewed together with advantages and disadvantages of each step size procedure. The development of steepest descent method due to its step size procedure is discussed. In order to test the performance of each step size, we run a steepest descent procedure in C++ program. We implemented it to unconstrained optimization test problem with two variables, then we compare the numerical results of each step size procedure. Based on the numerical experiment, we conclude the general computational features and weaknesses of each procedure in each case of problem.

An accelerated descent CG algorithm with clustering the eigenvalues for large-scale nonconvex unconstrained optimization and its application in image restoration problems

This paper proposes a novel nonmonotone proximal gradient method for solving nondifferentiable composite multiobjective optimization problems. The method integrates a nonmonotone line search technique into the proximal gradient framework, offering enhanced flexibility and efficiency for handling nonsmooth or nonconvex objective functions. We analyze the convergence properties of the proposed algorithm, demonstrate that all accumulation points of the generated sequence satisfy Pareto stationarity when the problem is nonconvex. Furthermore, under convexity and mild conditions, we establish that the sequence converges to a weakly Pareto optimal solution of the problem. Numerical experiments on robust multiobjective optimization problems and Lasso multiobjective optimization problems are provided to illustrate the computational efficiency and the practical utility of the proposed method.