Two-versions of descent conjugate gradient methods for large-scale unconstrained optimization

Conjugate gradient methods are a class of very effective iterative methods for large-scale unconstrained optimization. In this paper, a new Dai-Liao conjugate gradient method for solving large-scale unconstrained optimization problem is proposed. Based on the approximately optimal stepsize for the gradient method, we derive three new choices for the important parameters tk in Dai-Liao conjugate gradient method. The search direction satisfies the sufficient descent condition, and the global convergences of the proposed method for uniformly convex and general functions are proved under some mild conditions. Numerical experiments on a set of test problems from the CUTEst library show that the proposed method is superior to some well-known conjugate gradient methods.

This paper presents a hybrid spectral conjugate gradient method for large-scale unconstrained optimization, which possesses a self-adjusting property. Under the standard Wolfe conditions, its global convergence result is established. Preliminary numerical results are reported on a set of large-scale problems in CUTEr to show the convergence and efficiency of the proposed method.

A descent nonlinear conjugate gradient method for large-scale unconstrained optimization

This paper reports two proposals of possible preconditioners for the Nonlinear Conjugate Gradient (NCG) method, in large scale unconstrained optimization. On one hand, the common idea of our preconditioners is inspired to L-BFGS quasi–Newton updates, on the other hand we aim at explicitly approximating in some sense the inverse of the Hessian matrix. Since we deal with large scale optimization problems, we propose matrix–free approaches where the preconditioners are built using symmetric low–rank updating formulae. Our distinctive new contributions rely on using information on the objective function collected as by-product of the NCG, at previous iterations. Broadly speaking, our first approach exploits the secant equation, in order to impose interpolation conditions on the objective function. In the second proposal we adopt and ad hoc modified–secant approach, in order to possibly guarantee some additional theoretical properties.

To find a solution of unconstrained optimization problems, we normally use a conjugate gradient (CG) method since it does not cost memory or storage of second derivative like Newton’s method or Broyden–Fletcher–Goldfarb–Shanno (BFGS) method. Recently, a new modification of Polak and Ribiere method was proposed with new restart condition to give a so-call AZPRP method. In this paper, we propose a new modification of AZPRP CG method to solve large-scale unconstrained optimization problems based on a modification of restart condition. The new parameter satisfies the descent property and the global convergence analysis with the strong Wolfe-Powell line search. The numerical results prove that the new CG method is strongly aggressive compared with CG_Descent method. The comparisons are made under a set of more than 140 standard functions from the CUTEst library. The comparison includes number of iterations and CPU time.

Nonlinear conjugate gradient (CG) methods are widely used for solving large scale unconstrained optimization problems. Many studies have been devoted to modified and improve this method. In this paper, a new parameter of CG method that possesses global convergence properties using exact line search is proposed. Numerical results show that the new formula is best and more efficient when compared with the other classical CG methods

In this paper, we present two nonmonotone versions of adaptive cubic regularized (ARC) method for unconstrained optimization problems. The proposed methods are a combination of the ARC algorithm with the nonmonotone line search methods introduced by Zhang and Hager [A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim. 14 (2004), pp. 1043–1056] and Ahookhosh et al. [A nonmonotone trust-region line search method for large-scale unconstrained optimization, Appl. Math. Model. 36 (2012), pp. 478–487]. The global convergence analysis for these iterative algorithms is established under suitable conditions. Several numerical examples are given to illustrate the efficiency and robustness of the newly suggested methods. The obtained results show the satisfactory performance of the proposed algorithms when compared to the basic ARC algorithm.

ABSTRACTThis paper proposes a nonmonotone scaled conjugate gradient algorithm for solving large-scale unconstrained optimization problems, which combines the idea of scaled memoryless Broyden–Fletcher–Goldfarb–Shanno preconditioned conjugate gradient method with the nonmonotone technique. An attractive property of the proposed method is that the search direction always provides sufficient descent step at each iteration. This property is independent of the line search used. Under appropriate assumptions, the method is proven to possess global convergence for nonconvex smooth functions, and R-linear convergence for strongly convex functions. Preliminary numerical results and related comparisons show the efficiency of the proposed method in practical computation.

Although there are many studies on large-scale unconstrained optimization (e.g., with 100 to 1000 variables) and small-scale constrained optimization (e.g., with 10 to 30 variables) using nature-inspired algorithms (e.g., evolutionary algorithms and swarm intelligence algorithms), no publicly available nature-inspired algorithm is developed for large-scale constrained optimization. In this paper, we combine a cooperative coevolutionary particle swarm optimization (CCPSO) algorithm with the e constrained method to solve large-scale real-valued constrained optimization problems. The eCCPSO framework is proposed, and three different algorithms based on the framework, i.e., eCCPSOd, eCCPSOw and eCCPSOw2, are developed. The proposed algorithms compare favorably to the state-of-the-art constrained optimization algorithm eDEag on large-scale problems. The experimental results further suggest that eCCPSOw2 with adaptive improvement detection technique is highly competitive compared with the other algorithms considered in this work for solving large-scale real-valued constrained optimization problems.

The limited memory BFGS method (L-BFGS) is an adaptation of the BFGS method for large-scale unconstrained optimization. However, The L-BFGS method need not converge for nonconvex objective functions and it is inefficient on highly ill-conditioned problems. In this paper, we proposed a regularization strategy on the L-BFGS method, where the used regularization parameter may play a compensation role in some sense when the condition number of Hessian approximation tends to become ill-conditioned. Then we proposed a regularized L-BFGS method and established its global convergence even when the objective function is nonconvex. Numerical results show that the proposed method is efficient.

This paper deals with a new nonlinear conjugate gradient method for solving large-scale unconstrained optimization problems. We prove that the new conjugate gradient coefficients  with exact line search is globally convergent. Preliminary numerical results show that  is very efficient when compared to the other classical conjugate gradient coefficients FR,PRP, NPRP, and RMILThis paper deals with a new nonlinear conjugate gradient method for solving large-scale unconstrained optimization problems. We prove that the new conjugate gradient coefficients  with exact line search is globally convergent. Preliminary numerical results show that  is very efficient when compared to the other classical conjugate gradient coefficients FR,PRP, NPRP, and RMIL

A Liu-Storey-type conjugate gradient method for unconstrained minimization problem with application in motion control

In this work, we modify a conjugate gradient (CG) method recently proposed in the literature, where a PRP conjugate gradient method is modified using trust region. Particularly, we propose a hybrid CG method that incorporates the parameters $β^{PRP},$ $β^{FR}$ and $β^{CD},$ and this new search direction satisfies both the trust region feature and the sufficient descent conditions. Furthermore, under suitable conditions the developed method is proved to be globally convergent. The method is tested on some benchmark problems from the literature and numerical results show that it is quite efficient in solving large scale problems.

Energy minimization plays an important role in structure determination and analysis of proteins, peptides, and other organic molecules; therefore, development of efficient minimization algorithms is important. Recently, Morales and Nocedal developed hybrid methods for large-scale unconstrained optimization that interlace iterations of the limited-memory BFGS method (L-BFGS) and the Hessian-free Newton method (Computat Opt Appl 2002, 21, 143-154). We test the performance of this approach as compared to those of the L-BFGS algorithm of Liu and Nocedal and the truncated Newton (TN) with automatic preconditioner of Nash, as applied to the protein bovine pancreatic trypsin inhibitor (BPTI) and a loop of the protein ribonuclease A. These systems are described by the all-atom AMBER force field with a dielectric constant epsilon = 1 and a distance-dependent dielectric function epsilon = 2r, where r is the distance between two atoms. It is shown that for the optimal parameters the hybrid approach is typically two times more efficient in terms of CPU time and function/gradient calculations than the two other methods. The advantage of the hybrid approach increases as the electrostatic interactions become stronger, that is, in going from epsilon = 2r to epsilon = 1, which leads to a more rugged and probably more nonlinear potential energy surface. However, no general rule that defines the optimal parameters has been found and their determination requires a relatively large number of trial-and-error calculations for each problem.

Conjugate gradient (CG) methods have been practically used to solve large-scale unconstrained optimization problems due to their simplicity and low memory storage. In this paper, we proposed a new type of CG coefficients