An Improved HS Type Spectral Conjugate Gradient Method

The Spectral conjugate gradient (SCG) methods are among the efficient variants of CG algorithms which are obtained by combining the spectral gradient parameter and CG parameter. The success of SCG methods relies on effective choices of the step-size $\alpha _{k}$ and the search direction $d_{k}$ . This paper presents an SCG method for unconstrained optimization models. The search directions generated by the new method possess sufficient descent property without the restart condition and independent of the line search procedure used. The global convergence of the new method is proved under the weak Wolfe line search. Preliminary numerical results are presented which show that the method is efficient and promising, particularly for large-scale problems. Also, the method was applied to solve the robotic motion control problem and portfolio selection problem.

The best spectral conjugate gradient algorithm by (Birgin, E. and Martínez, J.M., 2001, A spectral conjugate gradient method for unconstrained optimization. Applied Mathematics and Optimization, 43, 117–128). which is mainly a scaled variant of (Perry, J.M., 1977, A class of Conjugate gradient algorithms with a two step varaiable metric memory, Discussion Paper 269, Center for Mathematical Studies in Economics and Management Science, Northwestern University), is modified in such a way as to overcome the lack of positive definiteness of the matrix defining the search direction. This modification is based on the quasi-Newton BFGS updating formula. The computational scheme is embedded into the restart philosophy of Beale–Powell. The parameter scaling the gradient is selected as spectral gradient or in an anticipative way by means of a formula using the function values in two successive points. In very mild conditions it is shown that, for strongly convex functions, the algorithm is global convergent. Computational results and performance profiles for a set consisting of 700 unconstrained optimization problems show that this new scaled nonlinear conjugate gradient algorithm substantially outperforms known conjugate gradient methods including: the spectral conjugate gradient SCG by Birgin and Martínez, the scaled Fletcher and Reeves, the Polak and Ribière algorithms and the CONMIN by (Shanno, D.F. and Phua, K.H., 1976, Algorithm 500, Minimization of unconstrained multivariate functions. ACM Transactions on Mathematical Software, 2, 87–94).

Conjugate gradient (CG) methods are renowned for their efficiency in optimizing problems due to their speed and memory use. However, certain formulations of CG methods frequently change between different parameters, potentially omitting valuable values. Additionally, proving adequate descent properties in these methods often relies on a strategy that assumes the ability to exclude specific function values. This study introduces a novel spectral CG (SCG) algorithm that integrates both the spectral gradient parameter and conjugate gradient coefficient. This method stands out for its applicability to large-scale unconstrained optimization problems. Initial numerical findings employing a robust Wolfe line search are provided to showcase the method's efficiency. The spectral coefficient in the field of optimization is a concept used to enhance the performance of search algorithms in unconstrained optimization. It is typically employed in the context of line search methods to improve the convergence rate of algorithms. In optimization algorithms, the spectral coefficient helps adjust the step size to achieve faster convergence towards the optimal solution. The coefficient is calculated based on information from previous iterations of the algorithm. A common application of this concept is in the Spectral Gradient method, where the spectral coefficient is used to modify the gradient direction to accelerate the search process. In this method, the spectral coefficient is used to determine the step length in each iteration, aiming to achieve an optimal balance between speed and stability in finding the optimal solution

A new nonlinear spectral conjugate descent method for solving unconstrained optimization problems is proposed on the basis of the CD method and the spectral conjugate gradient method. For any line search, the new method satisfies the sufficient descent condition . Moreover, we prove that the new method is globally convergent under the strong Wolfe line search. The numerical results show that the new method is more effective for the given test problems from the CUTE test problem library (Bongartz et al., 1995) in contrast to the famous CD method, FR method, and PRP method.

Metaheuristic algorithms are used to solve many optimization problems. Firefly algorithm, particle swarm improvement, harmonic search, and bat algorithm are used as search algorithms to find the optimal solution to the problem field. In this paper, we have investigated and analyzed a new scaled conjugate gradient algorithm and its implementation, based on the exact Wolfe line search conditions and the restart Powell criterion. The new spectral conjugate gradient algorithm is a modification of the Birgin and Martínez method, a manner to overcome the lack of positive definiteness of the matrix defining the search direction. The preliminary computational results for a set of 30 unconstrained optimization test problems show that this new spectral conjugate gradient outperforms a standard conjugate gradient in this field and we have applied the newly proposed spectral conjugate gradient algorithm in bat algorithm to reach the lowest possible goal of bat algorithm. The newly proposed approach, namely, the directional bat algorithm (CG-BAT), has been then tested using several standard and nonstandard benchmarks from the CEC’2005 benchmark suite with five other algorithms and has been then tested using nonparametric statistical tests and the statistical test results show the superiority of the directional bat algorithm, and also we have adopted the performance profiles given by Dolan and More which show the superiority of the new algorithm (CG-BAT).

This paper presents two new spectral conjugate gradient methods which are designed for solving nonlinear unconstrained optimization problems. These methods are based on the idea of the Hideaki – Yasushi method. Which produce sufficient descent search direction at every iteration. Experimental results indicate that the new proposed methods more efficient than the Hideaki – Yasush method

In this paper, a three-dimensional spatial-temporal decomposition modelling method is proposed to build the alkali-surfactant-polymer (ASP) flooding model, in which a new dynamic recurrent wavelet neural network (DRWNN) is presented to identify the temporal coefficients. At first, the detailed mathematical model of ASP flooding is described which is a complex distributed parameter system. Then a three-dimensional spatial-temporal modelling method is inferred based on Karhunen-Loeve (K-L) decomposition to decompose the water saturation of reservoir into a series of spatial basis functions and corresponding temporal coefficients. Furthermore, the recurrent wavelet neural network is used to acquire the identification model, in which the injection concentrations of ASP flooding and temporal coefficients are taken as the input and output information. In order to improve the capability of dynamic modelling, DRWNN is proposed through adding feedback layers and setting the different weights with time to achieve dynamic memory of the past information. Considering the gradient descent method for the neural networks training easily leads to local minimum and slow convergence speed, the spectral conjugate gradient method is introduced to optimize the weights of DRWNN. At last, DRWNN is used to build the relation between the moisture content of production wells and the water saturation of the corresponding grids. Thus, the final approximate model of ASP flooding is finished. The accuracy is proved by model with four injection wells and nine production wells through data from the mechanism model.

Theconjugate gradient method is widely used to solve large scale unconstrainedoptimization problems. However, the rate of convergence conjugate gradient method is linearunless it restarted. In this study, we present a new spectral conjugategradient modification formula with restart property obtains the globalconvergence and descent properties.In addition, we proposed a new restart condition for Fletcher-Reeves conjugate gradient formula. The numerical resultsdemonstrated that the modified Fletcher-Reeves parameter and the new CG formulawith their restart conditions are more efficient and robustness than otherconventional methods.

The prominent computational features of the Hestenes–Stiefel parameter as one of the fundamental members of conjugate gradient methods have attracted the attention of many researchers. Yet, as a weak stop, it lacks global convergence for general functions. To overcome this defect, a family of spectral version of Hestenes–Stiefel conjugate gradient methods is introduced. To compute the spectral parameter, in the account of worthy properties of quasi-Newton methods, we minimize the distance between the search direction matrix of the spectral conjugate gradient method and the BFGS (Broyden–Fletcher–Goldfarb–Shanno) update. To achieve sufficient descent property, the search direction is projected in the orthogonal subspace to the gradient of the objective function. The convergence analysis of the proposed method is carried out under standard assumptions for general functions. Finally, the practical merits of the suggested method are investigated by numerical experiments on a set of CUTEr test functions using the Dolan–Moré performance profile. The results show the computational efficiency of the proposed method.

Spectral modified Polak–Ribiére–Polyak projection conjugate gradient method for solving monotone systems of nonlinear equations

This paper introduces a hybrid spectral-conjugate gradient (SCG) method to solve nonlinear monotone operator equations efficiently. The proposed method incorporates a hybrid parameter that encom- passes the Polak–Ribi`ere–Polyak (PRP), Liu-Storey (LS), Fletcher-Reeves (FR), and conjugate descent (CD) meth- ods as particular instances. Additionally, we derive the spectral parameter to ensure that the search direction adheres to the sufficient descent condition. The search direction is also designed to be bounded, and under spe- cific conditions, we demonstrate that the sequence produced by our hybrid SCG algorithm converges toward a solution. Furthermore, to underscore the effectiveness of our proposed method, we conducted extensive numer- ical experiments comparing its performance against that of existing algorithms. These experiments were based on a selection of benchmark nonlinear monotone operator equations, highlighting our proposed algorithm’s superior efficiency and potential in practical applications.

A spectral conjugate gradient method for solving large-scale unconstrained optimization

At present, the conjugate gradient (CG) method of Hager and Zhang (Hager and Zhang, SIAM Journal on Optimization, 16(2005)) is regarded as one of the most effective CG methods for optimization problems. In order to further study the CG method, we develop the Hager and Zhang's CG method and present two modified CG formulas, where the given formulas possess the value information of not only the gradient but also the function. Moreover, the sufficient descent condition will be holden without any line search. The global convergence is established for nonconvex function under suitable conditions. Numerical results show that the proposed methods are competitive to the normal conjugate gradient method.

This paper presents a hybrid spectral conjugate gradient method for large-scale unconstrained optimization, which possesses a self-adjusting property. Under the standard Wolfe conditions, its global convergence result is established. Preliminary numerical results are reported on a set of large-scale problems in CUTEr to show the convergence and efficiency of the proposed method.

The left-preconditioned communication avoiding conjugate gradient (LP-CA-CG) method is applied to the pressure Poisson equation in the multiphase CFD code JUPITER. The arithmetic intensity of the LP-CA-CG method is analyzed, and is dramatically improved by loop splitting for inner product operations and for three term recurrence operations. Two LPCA-CG solvers with block Jacobi preconditioning and with underlap preconditioning are developed. The former is developed based on a hybrid CA approach, in which CA is applied only to global collective communications for inner product operations. The latter is a full CA approach, in which CA is applied also to local point-to-point communications in sparse matrix-vector (SpMV) operations and preconditioning. CA-SpMV requires additional computation for overlapping regions. CA-preconditiong is enabled by underlap preconditioning, which approximates preconditioning for overlapping regions by point Jacobi preconditioning. It is shown that on the K computer, the former is faster, because the performance of local point-to-point communications scales well, and the convergence property becomes worse with underlap preconditioning. The LP-CA-CG solver shows good strong scaling up to 30,000 nodes, where the LP-CA-CG solver achieved higher performance than the original CG solver by reducing the cost of global collective communications by 69 percent.