A Modified Hestenes-Stiefel Conjugate Gradient Algorithm for Large-Scale Optimization

Recently, similar to Hager and Zhang (SIAM J Optim 16:170–192, 2005), Yu (Nonlinear self-scaling conjugate gradient methods for large-scale optimization problems. Thesis of Doctors Degree, Sun Yat-Sen University, 2007) and Yuan (Optim Lett 3:11–21, 2009) proposed modified PRP conjugate gradient methods which generate sufficient descent directions without any line searches. In order to obtain the global convergence of their algorithms, they need the assumption that the stepsize is bounded away from zero. In this paper, we take a little modification to these methods such that the modified methods retain sufficient descent property. Without requirement of the positive lower bound of the stepsize, we prove that the proposed methods are globally convergent. Some numerical results are also reported.

At present, the conjugate gradient (CG) method of Hager and Zhang (Hager and Zhang, SIAM Journal on Optimization, 16(2005)) is regarded as one of the most effective CG methods for optimization problems. In order to further study the CG method, we develop the Hager and Zhang's CG method and present two modified CG formulas, where the given formulas possess the value information of not only the gradient but also the function. Moreover, the sufficient descent condition will be holden without any line search. The global convergence is established for nonconvex function under suitable conditions. Numerical results show that the proposed methods are competitive to the normal conjugate gradient method.

A modified Polak–Ribière–Polyak conjugate gradient algorithm for nonsmooth convex programs

The convergence of conjugate gradient method with nonmonotone line search

A Liu-Storey-type conjugate gradient method for unconstrained minimization problem with application in motion control

In this work, we modify a conjugate gradient (CG) method recently proposed in the literature, where a PRP conjugate gradient method is modified using trust region. Particularly, we propose a hybrid CG method that incorporates the parameters $β^{PRP},$ $β^{FR}$ and $β^{CD},$ and this new search direction satisfies both the trust region feature and the sufficient descent conditions. Furthermore, under suitable conditions the developed method is proved to be globally convergent. The method is tested on some benchmark problems from the literature and numerical results show that it is quite efficient in solving large scale problems.

Hybrid conjugate gradient (CG) techniques are one of the most prominent procedure for obtaining the solution of large-scale unconstrained optimization problems. This is due to its simplicity, global convergence, and low memory requirement. Numerous modifications have been done recently to improve the performance of these methods. In this paper, we proposed new class of hybrid CG coefficients with guaranteed descent under exact line search. Numerical results are presented to illustrate the efficiency of the proposed methodscompared to other classical CG coefficients.

This study introduces an improved three-term conjugate gradient (CG) algorithm which fulfills the descent conditions and exhibits good global convergence properties. The development of this new algorithm is based on the findings of recently introduced generalized RMIL CG technique. The algorithm has been modified so that the search direction would always meet the sufficient descent condition no matter the line-search techniques employed. Under some given assumptions, the algorithm’s global convergence result for the general nonconvex functions is established. The numerical efficacy of the suggested three-term CG algorithm is assessed through comparisons with four other CG algorithms. The assessment is performed using an array of unconstrained optimization benchmark functions. The acquired findings indicate that all the considered algorithms exhibit similar performance, particularly the variants of the RMIL method. One algorithm is more robust and slightly faster than the rest. However, it is essential to highlight that the newly developed CG formula outperforms all the other algorithms, including a recently presented three-term PRP algorithm. The study further expanded the proposed method to address an image restoration problem. The computational experiments yielded promising results, proving that the proposed CG algorithm surpasses others. It produces higher-quality output images, requires less CPU time (sec), and achieves higher PSNR values.

In this paper, a family of three-term conjugate gradient methods is proposed to solve a large-scale unconstrained optimization problem. With the help of suitable features of the new family (like sufficient descent directions) a strong global convergence theorem for uniformly convex functions under weak Wolfe-Powell line search technique is established. Furthermore, a new well-defined modification of weak Wolfe-Powell line search technique is presented and a strong global convergence theorem for general smooth functions is obtained. In two competitions contained two line search techniques, five well behaved conjugate gradient methods and 200 standard problems the efficiency of these new methods in numerical experience is indicated.

The conjugate gradient methods are numerously used for solving nonlinear unconstrained optimization problems, especially of large scale. Their wide applications are due to their simplicity and low memory requirement. To analyze conjugate gradient methods, two types of line searches are used; exact and inexact. In this paper, we present a new method of nonlinear conjugate gradient methods under the exact line search. The theoretical analysis shows that the new method generates a descent direction in each iteration and globally convergent under the exact line search. Moreover, numerical experiments based on comparing the new method with other well known conjugate gradient methods show that the new is efficient for some unconstrained optimization problems.

In this paper, we present two nonmonotone versions of adaptive cubic regularized (ARC) method for unconstrained optimization problems. The proposed methods are a combination of the ARC algorithm with the nonmonotone line search methods introduced by Zhang and Hager [A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim. 14 (2004), pp. 1043–1056] and Ahookhosh et al. [A nonmonotone trust-region line search method for large-scale unconstrained optimization, Appl. Math. Model. 36 (2012), pp. 478–487]. The global convergence analysis for these iterative algorithms is established under suitable conditions. Several numerical examples are given to illustrate the efficiency and robustness of the newly suggested methods. The obtained results show the satisfactory performance of the proposed algorithms when compared to the basic ARC algorithm.

In this paper, we present two families of modified three-term conjugate gradient methods for solving unconstrained large-scale smooth optimization problems. We show that our new families satisfy the Dai-Liao conjugacy condition and the sufficient descent condition under any line search technique which guarantees the positiveness of ${y_{k}^{T}} s_{k}$. For uniformly convex functions, we indicate that our families are globally convergent under weak-Wolfe-Powell line search technique and standard conditions on the objective function. We also establish a weaker global convergence theorem for general smooth functions under similar assumptions. Our numerical experiments for 260 standard problems and seven other recently developed conjugate gradient methods illustrate that the members of our families are numerically efficient and effective.

To find a solution of unconstrained optimization problems, we normally use a conjugate gradient (CG) method since it does not cost memory or storage of second derivative like Newton’s method or Broyden–Fletcher–Goldfarb–Shanno (BFGS) method. Recently, a new modification of Polak and Ribiere method was proposed with new restart condition to give a so-call AZPRP method. In this paper, we propose a new modification of AZPRP CG method to solve large-scale unconstrained optimization problems based on a modification of restart condition. The new parameter satisfies the descent property and the global convergence analysis with the strong Wolfe-Powell line search. The numerical results prove that the new CG method is strongly aggressive compared with CG_Descent method. The comparisons are made under a set of more than 140 standard functions from the CUTEst library. The comparison includes number of iterations and CPU time.

The Barzilai-Borwein (BB) method is a popular and efficient tool for solving large-scale unconstrained optimization problems.Its search direction is the same as for the steepest descent (Cauchy) method, but its stepsize rule is different.Owing to this, it converges much faster than the Cauchy method.A feature of the BB method is that it may generate too long steps, which throw the iterates too far away from the solution.Moreover, it may not converge, even when the objective function is strongly convex.In this paper, a stabilization technique is introduced.It consists in bounding the distance between each pair of successive iterates, which often allows for decreasing the number of BB iterations.When the BB method does not converge, our simple modification of this method makes it convergent.For strongly convex functions with Lipschits gradients, we prove its global convergence, despite the fact that no line search is involved, and only gradient values are used.Since the number of stabilization steps is proved to be finite, the stabilized version inherits the fast local convergence of the BB method.The presented results of extensive numerical experiments show that our stabilization technique often allows the BB method to solve problems in a fewer iterations, or even to solve problems where the latter fails.

In this paper, we present a new hybrid conjugate gradient algorithm for unconstrained optimization. This method is a convex combination of Liu-Storey conjugate gradient method and Fletcher-Reeves conjugate gradient method. We also prove that the search direction of any hybrid conjugate gradient method, which is a convex combination of two conjugate gradient methods, satisfies the famous D-L conjugacy condition and in the same time accords with the Newton direction with the suitable condition. Furthermore, this property doesn’t depend on any line search. Next, we also prove that, moduling the value of the parameter t, the Newton direction condition is equivalent to Dai-Liao conjugacy condition. The strong Wolfe line search conditions are used. The global convergence of this new method is proved. Numerical comparisons show that the present hybrid conjugate gradient algorithm is the efficient one.