A nonmonotone line search method and its convergence for unconstrained optimization

In this paper, we present two nonmonotone versions of adaptive cubic regularized (ARC) method for unconstrained optimization problems. The proposed methods are a combination of the ARC algorithm with the nonmonotone line search methods introduced by Zhang and Hager [A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim. 14 (2004), pp. 1043–1056] and Ahookhosh et al. [A nonmonotone trust-region line search method for large-scale unconstrained optimization, Appl. Math. Model. 36 (2012), pp. 478–487]. The global convergence analysis for these iterative algorithms is established under suitable conditions. Several numerical examples are given to illustrate the efficiency and robustness of the newly suggested methods. The obtained results show the satisfactory performance of the proposed algorithms when compared to the basic ARC algorithm.

This paper presents a hybrid ODE-based method for unconstrained optimization problems, which combines the idea of IMPBOT with the subspace technique and a fixed step-length. The main characteristic of this method is that at each iteration, a lower dimensional system of linear equations is solved only once to obtain a trial step. Another is that when a trial step is not accepted, this proposed method uses minimization of a convex overestimation, thus avoiding performing a line search to compute a step-length. Under some reasonable assumptions, the method is proven to be globally convergent. Numerical results show the efficiency of this proposed method in practical computations, especially for solving small scale unconstrained optimization problems.

This paper presents a new supermemory gradient method for unconstrained optimization problems. It can be regarded as a combination of ODE-based methods, line search and subspace techniques. The main characteristic of this method is that, at each iteration, a lower dimensional system of linear equations is solved only once to obtain a trial step, thus avoiding solving a quadratic trust region subproblem. Another is that when a trial step is not accepted, this proposed method generates an iterative point whose step-length satisfies Armijo line search rule, thus avoiding resolving linear system of equations. Under some reasonable assumptions, the method is proven to be globally convergent. Numerical results show the efficiency of this proposed method in practical computation.

In this article, we propose a new line search method for solving unconstrained optimization problems in that we combine a nonmonotone strategy into a modified Armijo rule and design a new algorithm that possibly chooses a larger steplength. This can decrease the number of iterations and function evaluations and can improve the efficiency of the algorithm. The global convergence and convergence rate are analysed under some suitable conditions. Preliminary numerical experiments establish that the new approach is robust and efficient for unconstrained optimization problems.

Armijo method is a kind of line search method that usually be used when look for the step size in nonlinear optimization. This paper makes the summary of its modified forms, and then the nonmonotone Armijo-type line search methods are proposed. The numerical results will show that some line search methods with the novel nonmonotone line search are available and efficient in practical computation.

In matrix completion based MIMO radars, the data matrix coherence, and consequently the performance of matrix completion depend on the transmit waveforms. It was recently shown that for uniform linear arrays and orthogonal waveforms, the optimal choice for waveforms are white-noise type functions. This paper deals with the design of optimal transmit waveforms. The design is formulated as an optimization problem on the complex Stiefel manifold, which is a non-convex set, and the optimal waveforms are found via a steepest descent algorithm with nonmonotone line search methods. Numerical results show that for large dimensional data matrices and for a large number of antennas, the objective function converges to its global minimum and the matrix coherence corresponding to the optimal waveforms is asymptotically optimal, thus resulting in very good target estimation performance.

It is well known that among the current methods for unconstrained optimization problems the quasi-Newton methods with global strategy may be the most efficient methods, which have local superlinear convergence. However, when the iterative point is far away from the solution of the problem, quasi-Newton method may proceed slowly for the general unconstrained optimization problems. In this article an adaptive conic trust-region method for unconstrained optimization is presented. Not only the gradient information but also the values of the objective function are used to construct the local model at the current iterative point. Moreover, we define a concept of super steepest descent direction and embed its information into the local model. The amount of computation in each iteration of this adaptive algorithm is the same as that of the standard quasi-Newton method with trust region. Some numerical results show that the modified method requires fewer iterations than the standard methods to reach the solution of the optimization problem. Global and local convergence of the method is also analyzed.

Nonlinear conjugate gradient methods for unconstrained optimization problems are used in many aspects of theoretical and applied sciences. They are iterative methods, so at any iteration a step length is computed using a method called line search. In most cases, the sufficient descent condition plays an important role to prove the global convergence of a conjugate gradient method. Due to its outperformance in practical computation, the Polak-Ribière-Polyak (PRP) conjugate gradient method is widely used for solving nonlinear unconstrained optimization problems. However, the sufficient descent condition of PRP has not established without line search yet. In this study, we established the sufficient descent condition without line search based on the conditions 0 < bkPRP £ x bkFR and 1 < bkPRP £mbkFR, where 0 < x < 1 and m > 1. As a result, we found that under certain conditions, the sufficient descent condition is satisfied when the PRP implemented without line search

Recently, Zhang (2006) proposed a three‐term modified HS (TTHS) method for unconstrained optimization problems. An attractive property of the TTHS method is that the direction generated by the method is always descent. This property is independent of the line search used. In order to obtain the global convergence of the TTHS method, Zhang proposed a truncated TTHS method. A drawback is that the numerical performance of the truncated TTHS method is not ideal. In this paper, we prove that the TTHS method with standard Armijo line search is globally convergent for uniformly convex problems. Moreover, we propose a new truncated TTHS method. Under suitable conditions, global convergence is obtained for the proposed method. Extensive numerical experiment show that the proposed method is very efficient for the test problems from the CUTE Library.

Global convergence theorems for a class of descent methods for unconstrained optimization problems in normed spaces, using multidirectional search, are proved. Exact and inexact search are considered and the results allow to define a globally convergent algorithm for an unconstrained optimal control problem which operates, at each step, on discrete approximations of the original continuous problem.

Chapter 3 - Numerical search methods for unconstrained optimization problems

This paper introduces a matrix-free variant of the Davidon-Fletcher-Powell (DFP) method for unconstrained optimization problems with applications in compressive sensing and image restoration. The main contribution lies in the new search direction incorporating a scaling parameter that ensures the satisfaction of the sufficient descent condition, independent of the line search conditions. A rigorous convergence analysis guarantees the boundedness and theoretical validity of the proposed method. Comprehensive numerical experiments on benchmark unconstrained optimization test problems and compressive sensing problems demonstrate the efficiency and robustness of the algorithm. Specifically, in image restoration tasks, our method outperforms CG-DESCENT, MDL, and NSMA, achieving a 100% success rate compared to 95.8%, 84.5%, and 53.5%, respectively. Additionally, results on computational time, relative error, and PSNR confirm the superior performance of the proposed approach. These findings establish the proposed method as a competitive alternative for large-scale optimization problems.

Quasi-Newton methods for unconstrained optimization problems are considered for solving a system of linear equations Ax = b where A ∈ R n×n , Rank(A )= n, b ∈ R n , and x ∈ R n is the vector of unknowns. This problem can be converted into an equivalent quadratic optimization problem. Based on the observation that if H ≈ (A T A) −1 = A −1 (A T ) −1 then ¯ x = HA T b can be taken as an approximate solution of the problem, we propose a modification to the Quasi-Newton method. The modified algorithm incorporates the above observation. Global convergence is ensured by adding the steepest descent direction into the combination. Numerical experiments are also reported.

In this paper, we develop some three term nonlinear conjugate gradient methods based on the Hestenes–Stiefel (HS), the Polak–Ribière–Polyak (PRP) and the Liu–Storey (LS) methods. The proposed algorithms always generate sufficient descent directions which satisfy [Formula: see text]. When the Wolfe or the Armijo line search is used, we establish the global convergence of the proposed methods in a concise way. Moreover, the linear convergence rate of the methods is discussed as well. The extensive numerical results show the efficiency of the proposed methods.

Two modified HS type conjugate gradient methods for unconstrained optimization problems