Global Convergence of a Spectral Conjugate Gradient Method for Unconstrained Optimization

The Spectral conjugate gradient (SCG) methods are among the efficient variants of CG algorithms which are obtained by combining the spectral gradient parameter and CG parameter. The success of SCG methods relies on effective choices of the step-size $\alpha _{k}$ and the search direction $d_{k}$ . This paper presents an SCG method for unconstrained optimization models. The search directions generated by the new method possess sufficient descent property without the restart condition and independent of the line search procedure used. The global convergence of the new method is proved under the weak Wolfe line search. Preliminary numerical results are presented which show that the method is efficient and promising, particularly for large-scale problems. Also, the method was applied to solve the robotic motion control problem and portfolio selection problem.

The best spectral conjugate gradient algorithm by (Birgin, E. and Martínez, J.M., 2001, A spectral conjugate gradient method for unconstrained optimization. Applied Mathematics and Optimization, 43, 117–128). which is mainly a scaled variant of (Perry, J.M., 1977, A class of Conjugate gradient algorithms with a two step varaiable metric memory, Discussion Paper 269, Center for Mathematical Studies in Economics and Management Science, Northwestern University), is modified in such a way as to overcome the lack of positive definiteness of the matrix defining the search direction. This modification is based on the quasi-Newton BFGS updating formula. The computational scheme is embedded into the restart philosophy of Beale–Powell. The parameter scaling the gradient is selected as spectral gradient or in an anticipative way by means of a formula using the function values in two successive points. In very mild conditions it is shown that, for strongly convex functions, the algorithm is global convergent. Computational results and performance profiles for a set consisting of 700 unconstrained optimization problems show that this new scaled nonlinear conjugate gradient algorithm substantially outperforms known conjugate gradient methods including: the spectral conjugate gradient SCG by Birgin and Martínez, the scaled Fletcher and Reeves, the Polak and Ribière algorithms and the CONMIN by (Shanno, D.F. and Phua, K.H., 1976, Algorithm 500, Minimization of unconstrained multivariate functions. ACM Transactions on Mathematical Software, 2, 87–94).

In this paper, we present a new hybrid method to solve a nonlinear unconstrained optimization problem by using conjugate gradient, which is a convex combination of Liu–Storey (LS) conjugate gradient method and Hager–Zhang (HZ) conjugate gradient method. This method possesses the sufficient descent property with Strong Wolfe line search and the global convergence with the strong Wolfe line search. In the end of this paper, we illustrate our method by giving some numerical examples.

Conjugate gradient methods are a popular class of iterative methods for solving linear systems of equations and nonlinear optimization problems as they do not require the storage of any matrices. In order to obtain a theoretically effective and numerically efficient method, two modified conjugate gradient methods (called the MCB1 and MCB2 methods) are proposed. In which the coefficientβkin the two proposed methods is inspired by the structure of the conjugate gradient parameters in some existing conjugate gradient methods. Under the strong Wolfe line search, the sufficient descent property and global convergence of the MCB1 method are proved. Moreover, the MCB2 method generates a descent direction independently of any line search and produces good convergence properties when the strong Wolfe line search is employed. Preliminary numerical results show that the MCB1 and MCB2 methods are effective and robust in minimizing some unconstrained optimization problems and each of these modifications outperforms the four famous conjugate gradient methods. Furthermore, the proposed algorithms were extended to solve the problem of mode function.

In this paper, we propose a three-parameter family of conjugate gradient methods for unconstrained optimization. The three-parameter family of methods not only includes the already existing six practical nonlinear conjugate gradient methods, but subsumes some other families of nonlinear conjugate gradient methods as its subfamilies. With Powell’s restart criterion, the three-parameter family of methods with the strong Wolfe line search is shown to ensure the descent property of each search direction. Some general convergence results are also established for the three-parameter family of methods. This paper can also be regarded as a brief review on nonlinear conjugate gradient methods.

This paper presents a hybrid spectral conjugate gradient method for large-scale unconstrained optimization, which possesses a self-adjusting property. Under the standard Wolfe conditions, its global convergence result is established. Preliminary numerical results are reported on a set of large-scale problems in CUTEr to show the convergence and efficiency of the proposed method.

A spectral conjugate gradient method for solving large-scale unconstrained optimization

In this paper, we are concerned with the conjugate gradient methods for solving unconstrained optimization problems. we propose a modified Fletcher–Reeves (abbreviated FR) [Function minimization by conjugate gradients, Comput. J. 7 (1964) 149–154] conjugate gradient algorithm satisfying a parametrized sufficient descent condition with a parameter [Formula: see text] is proposed. The parameter [Formula: see text] is computed by means of the conjugacy condition, thus an algorithm which is a positive multiplicative modification of the Hestenes and Stiefel (abbreviated HS) [Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Standards Sec. B 48 (1952) 409–436] algorithm is obtained, which produces a descent search direction at every iteration that the line search satisfies the Wolfe conditions. Under appropriate conditions, we show that the modified FR method with the strong Wolfe line search is globally convergent of uniformly convex functions. We also present extensive preliminary numerical experiments to show the efficiency of the proposed method.

In this paper, a three-dimensional spatial-temporal decomposition modelling method is proposed to build the alkali-surfactant-polymer (ASP) flooding model, in which a new dynamic recurrent wavelet neural network (DRWNN) is presented to identify the temporal coefficients. At first, the detailed mathematical model of ASP flooding is described which is a complex distributed parameter system. Then a three-dimensional spatial-temporal modelling method is inferred based on Karhunen-Loeve (K-L) decomposition to decompose the water saturation of reservoir into a series of spatial basis functions and corresponding temporal coefficients. Furthermore, the recurrent wavelet neural network is used to acquire the identification model, in which the injection concentrations of ASP flooding and temporal coefficients are taken as the input and output information. In order to improve the capability of dynamic modelling, DRWNN is proposed through adding feedback layers and setting the different weights with time to achieve dynamic memory of the past information. Considering the gradient descent method for the neural networks training easily leads to local minimum and slow convergence speed, the spectral conjugate gradient method is introduced to optimize the weights of DRWNN. At last, DRWNN is used to build the relation between the moisture content of production wells and the water saturation of the corresponding grids. Thus, the final approximate model of ASP flooding is finished. The accuracy is proved by model with four injection wells and nine production wells through data from the mechanism model.

The spectral conjugate gradient (SCG) method was one of the most commomly used method to solve nonlinear unconstrained optimization problems. In this paper, an improved HS spectral conjugate gradient method is proposed. Under common assumptions, the descent and global convergence of this method are proved.

ABSTRACTIn this paper, we propose a family of effective spectral conjugate gradient methods with an adaptive restart scheme for solving unconstrained optimization problems. First, we construct a new composite conjugate parameter with two parameters by employing a convex combination of classical conjugate parameters and their variants. Then, we use the spectral technique to guarantee that the search direction possesses the sufficient descent property independent of any line search. Additionally, we incorporate a new spectral gradient‐based adaptive restart scheme to ensure the global convergence of the family under weak Wolfe line search and obtain iteration complexity results under Armijo line search. Finally, we conduct numerical experiments on unconstrained optimization problems and image restoration applications to demonstrate the effectiveness and practicality of the proposed family.

This paper presents two new spectral conjugate gradient methods which are designed for solving nonlinear unconstrained optimization problems. These methods are based on the idea of the Hideaki – Yasushi method. Which produce sufficient descent search direction at every iteration. Experimental results indicate that the new proposed methods more efficient than the Hideaki – Yasush method

Recently, numerous studies have been concerned in conjugate gradient methods for solving large-scale unconstrained optimization method. In this paper, a three-term conjugate gradient method is proposed for unconstrained optimization which always satisfies sufficient descent direction and namely as Three-Term Rivaie-Mustafa-Ismail-Leong (TTRMIL). Under standard conditions, TTRMIL method is proved to be globally convergent under strong-Wolfe line search. Finally, numerical results are provided for the purpose of comparison.

Conjugate gradient methods are an important class of methods for solving unconstrained optimization problems, especially for large-scale problems. Recently, they have been much studied. In this paper, we further study the conjugate gradient method for unconstrained optimization. We focus our attention to the descent conjugate gradient method. This paper presents a modified conjugate gradient method. An interesting feature of the presented method is that the direction is always a descent direction for the objective function. Moreover, the property is independent of the line search used. Under mild conditions, we prove that the modified conjugate gradient method with Armijo-type line search is globally convergent. We also present some numerical results to show the efficiency of the proposed method. DOI: http://dx.doi.org/10.11591/telkomnika.v11i11.2894

Conjugate gradient (CG) methods are renowned for their efficiency in optimizing problems due to their speed and memory use. However, certain formulations of CG methods frequently change between different parameters, potentially omitting valuable values. Additionally, proving adequate descent properties in these methods often relies on a strategy that assumes the ability to exclude specific function values. This study introduces a novel spectral CG (SCG) algorithm that integrates both the spectral gradient parameter and conjugate gradient coefficient. This method stands out for its applicability to large-scale unconstrained optimization problems. Initial numerical findings employing a robust Wolfe line search are provided to showcase the method's efficiency. The spectral coefficient in the field of optimization is a concept used to enhance the performance of search algorithms in unconstrained optimization. It is typically employed in the context of line search methods to improve the convergence rate of algorithms. In optimization algorithms, the spectral coefficient helps adjust the step size to achieve faster convergence towards the optimal solution. The coefficient is calculated based on information from previous iterations of the algorithm. A common application of this concept is in the Spectral Gradient method, where the spectral coefficient is used to modify the gradient direction to accelerate the search process. In this method, the spectral coefficient is used to determine the step length in each iteration, aiming to achieve an optimal balance between speed and stability in finding the optimal solution