Shape Optimization with Nonlinear Conjugate Gradient Methods

The gradient-based optimization methods are preferable for the large-scale three-dimensional (3D) magnetotelluric (MT) inverse problem. Compared with the popular nonlinear conjugate gradient (NLCG) method, however, the limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) method is less adopted. This paper aims to implement a L-BFGS-based inversion algorithm for the 3D MT problem. And we develop our code on top of the ModEM package, which is highly extensible and popular among the MT community. To accelerate the convergence speed, the preconditioning technique by the affine linear transformation of the original model parameters is used. Two modifications of the conventional L-BFGS algorithm are also made to get a comparable convergence rate with the NLCG method. The impacts of the preconditioner parameters, the regularization parameters, the starting model, etc., on the inversion are evaluated by synthetic examples for both L-BFGS and NLCG methods. And the real MT Kayabe dataset is also inverted by the inversion algorithms. The synthetic tests show that through our L-BFGS inversion algorithm the similar resistivity models can be obtained with that from the NLCG method. For the real data inversion, the L-BFGS method performs more efficiently and reasonable results could be obtained by less iterations of the inversion process than the NLCG method. Thus, we suggest the common usage of the L-BFGS method for the 3D MT inverse problem.

High Quality 4-dimensional Cone Beam CT by Deforming Prior Images

In this paper, we propose a three-parameter family of conjugate gradient methods for unconstrained optimization. The three-parameter family of methods not only includes the already existing six practical nonlinear conjugate gradient methods, but subsumes some other families of nonlinear conjugate gradient methods as its subfamilies. With Powell’s restart criterion, the three-parameter family of methods with the strong Wolfe line search is shown to ensure the descent property of each search direction. Some general convergence results are also established for the three-parameter family of methods. This paper can also be regarded as a brief review on nonlinear conjugate gradient methods.

Both fully adaptive and partially adaptive processing algorithms can be implemented using linear and nonlinear conjugate gradient (CG) methods. This approach can be computationally attractive when a family of continuous time-varying adaptive problems must be solved. An iterative approach that starts with the previous solution can converge quickly to the new solution. This paper considers linear and nonlinear CG methods for adaptive processing. The example used is space-time adaptive processing (STAP) and clutter interference subspace tracking within the context of an airborne radar with a rotating array. It is seen that the linear CG method for adaptive filtering in this context can be several times less costly than sample matrix inversion if the interference subspace has sufficient eigenvalue structure, and that a nonlinear CG algorithm is capable of tracking the principal clutter interference subspace.

Purpose– Generally, iterative methods which have some instability solutions in complex structural and non-linear mechanical problems are used to compute reliability index. The purpose of this paper is to establish a non-linear conjugate gradient (NCG) optimization algorithm to overcome instability solution of the Hasofer-Lind and Rackwitz-Fiessler (HL-RF) method in first-order reliability analysis. The NCG algorithms such as the Conjugate-Descent (CD) and the Liu-Storey (LS) are used for determining the safety index. An algorithm is found based on the new line search in the reliability analysis.Design/methodology/approach– In the proposed line search for calculating the safety index, search direction is computed by using the conjugate gradient approach and the HL-RF method based on the new and pervious gradient vector of the reliability function. A simple step size is presented for the line search in the proposed algorithm, which is formulated by the Wolfe conditions based on the new and previous safety index results in the reliability analysis.Findings– From the current work, it is concluded that the proposed NCG algorithm has more efficient, robust and appropriate convergence in comparison with the HL-RF method. The proposed methods can eliminate numerical instabilities of the HL-RF iterative algorithm in highly non-linear performance function and complicated structural limit state function. The NGC optimization is applicable to reliability analysis and it is correctly converged on the reliability index. In the NCG method, the CD algorithm is slightly more efficient than the LS algorithm.Originality/value– This paper usefully shows how the HL-RF algorithm and the NCG scheme are formulated in first-order reliability analysis. The proposed algorithm is validated from six numerical and structural examples taken from the literature. The HL-RF method is not converged on several non-linear mathematic and complex structural examples, while the two proposed conjugate gradient methods are appropriately converged for all examples. The CD algorithm is converged about twice faster than the LS algorithm in most of the problems. Therefore, application of the NCG method is possible in reliability analysis.

We study the development of nonlinear conjugate gradient methods, Fletcher Reeves (FR) and Polak Ribiere (PR). FR extends the linear conjugate gradient method to nonlinear functions by incorporating two changes, for the step length αk a line search is performed and replacing the residual, rk (rk=b-Axk) by the gradient of the nonlinear objective function. The PR method is equivalent to FR method for exact line searches and when the underlying quadratic function is strongly convex. The PR method is basically a variant of FR and primarily differs from it in the choice of the parameter βk. On applying the nonlinear Rosenbrock function to the MATLAB code for the FR and the PR algorithms we observe that the performance of PR method (k=29) is far better than the FR method (k=42). But, we observe that when the MATLAB codes are applied to general nonlinear functions, specifically functions whose minimum is a large negative number not close to zero and the iterates too are large values far off from zero the PR algorithm does not perform well. This problem with the PR method persists even if we run the PR algorithm for more iterations or with an initial guess closer to the actual minimum. To improve the PR algorithm we suggest finding a better weighing parameter βk, using better line search method and/or using specific line search for certain functions and identifying specific restart criteria based on the function to be optimized.

PreviousNext No AccessSEG Technical Program Expanded Abstracts 20013‐D magnetotelluric inversion for resource explorationAuthors: Randall L. MackieWilliam RodiM. Donald WattsRandall L. MackieGSY‐USA, Inc., William RodiMassachusetts Institute of Technology, and M. Donald WattsGeosystem srlhttps://doi.org/10.1190/1.1816392 SectionsAboutPDF/ePub ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinked InReddit Permalink: https://doi.org/10.1190/1.1816392FiguresReferencesRelatedDetailsCited by3D Step-by-step inversion strategy for audio magnetotellurics data based on unstructured mesh17 February 2022 | Applied Geophysics, Vol. 18, No. 3Three‐dimensional electrical structure of the northwestern margin of the Karamay region, China, revealed by magnetotelluric data14 September 2020 | Geophysical Prospecting, Vol. 68, No. 9Three-dimensional electrical conductivity in the mantle beneath the Payún Matrú Volcanic Field in the Andean backarc of Argentina near 36.5°S: evidence for decapitation of a mantle plume by resurgent upper mantle shear during slab steepening25 June 2014 | Geophysical Journal International, Vol. 198, No. 2NLCG and L-BFGS optimization for 3D HEM inversionYunhe Liu and Changchun Yin19 August 2013Methods and algorithms for reconstructing three-dimensional distributions of electric conductivity and polarization in the medium by finite-element 3D modeling using the data of electromagnetic sounding8 May 2013 | Izvestiya, Physics of the Solid Earth, Vol. 49, No. 3Multiple-domain, simultaneous joint inversion of geophysical data with application to subsalt imagingMichele De Stefano, Federico Golfré Andreasi, Simone Re, Massimo Virgilio, and Fred F. Snyder5 May 2011 | GEOPHYSICS, Vol. 76, No. 3A compressed implicit Jacobian scheme for 3D electromagnetic data inversionMaokun Li, Aria Abubakar, Jianguo Liu, Guangdong Pan, and Tarek M. Habashy28 April 2011 | GEOPHYSICS, Vol. 76, No. 3Electrical resistivity structure at the northern margin of the Tibetan Plateau and tectonic implications7 December 2011 | Journal of Geophysical Research, Vol. 116, No. B12Application of the 3D magnetotelluric inversion code in a geologically complex area9 June 2010 | Geophysical Prospecting, Vol. 58, No. 6Three‐dimensional regularized Gauss‐Newton inversion algorithm using a compressed implicit Jacobian calculation for electromagnetic applicationsMaokun Li, Aria Abubakar, Jianguo Liu, Guangdong Pan, and Tarek M. Habashy21 October 2010Three-dimensional inversion of magnetotelluric data for mineral exploration: An example from the McArthur River uranium deposit, Saskatchewan, CanadaJournal of Applied Geophysics, Vol. 68, No. 43D magnetotelluric inversion using a limited-memory quasi-Newton optimizationDmitry Avdeev and Anna Avdeeva27 April 2009 | GEOPHYSICS, Vol. 74, No. 3Advancing process‐based watershed hydrological research using near‐surface geophysics: a vision for, and review of, electrical and magnetic geophysical methods11 March 2008 | Hydrological Processes, Vol. 22, No. 18A survey of current trends in near-surface electrical and electromagnetic methodsEsben Auken, Louise Pellerin, Niels B. Christensen, and Kurt Sørensen7 September 2006 | GEOPHYSICS, Vol. 71, No. 5Three-Dimensional Electromagnetic Modelling and Inversion from Theory to ApplicationSurveys in Geophysics, Vol. 26, No. 6Mining, Environmental, Petroleum, and Engineering Industry Applications of Electromagnetic Techniques in GeophysicsSurveys in Geophysics, Vol. 26, No. 5Multi-dimensional electromagnetic modeling and inversion with application to near-surface earth investigationsComputers and Electronics in Agriculture, Vol. 46, No. 1-3Accelerated integral equation inversion of 3‐D magnetotelluric data in models with inhomogeneous backgroundNikolay Golubev and Michael S. Zhdanov7 December 20053D inversion of a scalar radio magnetotelluric field data setGregory A. Newman, Stephan Recher, Bülent Tezkan, and Fritz M. Neubauer29 May 2003 | GEOPHYSICS, Vol. 68, No. 32-D and 3-D interpretation of magnetotelluric data in the Bajawa geothermal field, central Flores, IndonesiaBULLETIN OF THE GEOLOGICAL SURVEY OF JAPAN, Vol. 53, No. 2-3 SEG Technical Program Expanded Abstracts 2001ISSN (print):1052-3812 ISSN (online):1949-4645Copyright: 2001 Pages: 2135 publication data© 2001 Copyright © 2001 Society of Exploration GeophysicistsPublisher:Society of Exploration Geophysicists HistoryPublished Online: 03 Jan 2005 CITATION INFORMATION Randall L. Mackie, William Rodi, and M. Donald Watts, (2001), "3‐D magnetotelluric inversion for resource exploration," SEG Technical Program Expanded Abstracts : 1501-1504. https://doi.org/10.1190/1.1816392 Plain-Language Summary PDF DownloadLoading ...

Nonlinear conjugate gradient methods are a class of techniques that are used for solving nonlinear optimisation problems frequently arising in many engineering applications such as machine learning, computer vision, least-square optimisations, to name a few. With so many surveys on the nonlinear conjugate gradient method (NLCG) available around, this paper addresses the current updates and sheds a new light on the evolution of hybrid conjugate gradient parameters with their global convergence properties.

Nonlinear conjugate gradient (CG) methods are widely used for solving large scale unconstrained optimization problems. Many studies have been devoted to modified and improve this method. In this paper, a new parameter of CG method that possesses global convergence properties using exact line search is proposed. Numerical results show that the new formula is best and more efficient when compared with the other classical CG methods

This paper reports two proposals of possible preconditioners for the Nonlinear Conjugate Gradient (NCG) method, in large scale unconstrained optimization. On one hand, the common idea of our preconditioners is inspired to L-BFGS quasi–Newton updates, on the other hand we aim at explicitly approximating in some sense the inverse of the Hessian matrix. Since we deal with large scale optimization problems, we propose matrix–free approaches where the preconditioners are built using symmetric low–rank updating formulae. Our distinctive new contributions rely on using information on the objective function collected as by-product of the NCG, at previous iterations. Broadly speaking, our first approach exploits the secant equation, in order to impose interpolation conditions on the objective function. In the second proposal we adopt and ad hoc modified–secant approach, in order to possibly guarantee some additional theoretical properties.

Recently, similar to Hager and Zhang (SIAM J Optim 16:170–192, 2005), Yu (Nonlinear self-scaling conjugate gradient methods for large-scale optimization problems. Thesis of Doctors Degree, Sun Yat-Sen University, 2007) and Yuan (Optim Lett 3:11–21, 2009) proposed modified PRP conjugate gradient methods which generate sufficient descent directions without any line searches. In order to obtain the global convergence of their algorithms, they need the assumption that the stepsize is bounded away from zero. In this paper, we take a little modification to these methods such that the modified methods retain sufficient descent property. Without requirement of the positive lower bound of the stepsize, we prove that the proposed methods are globally convergent. Some numerical results are also reported.

SummaryAcceleration schemes can dramatically improve existing optimization procedures. In most of the work on these schemes, such as nonlinear generalized minimal residual (N‐GMRES), acceleration is based on minimizing the ℓ2 norm of some target on subspaces of . There are many numerical examples that show how accelerating general‐purpose and domain‐specific optimizers with N‐GMRES results in large improvements. We propose a natural modification to N‐GMRES, which significantly improves the performance in a testing environment originally used to advocate N‐GMRES. Our proposed approach, which we refer to as O‐ACCEL (objective acceleration), is novel in that it minimizes an approximation to the objective function on subspaces of . We prove that O‐ACCEL reduces to the full orthogonalization method for linear systems when the objective is quadratic, which differentiates our proposed approach from existing acceleration methods. Comparisons with the limited‐memory Broyden–Fletcher–Goldfarb–Shanno and nonlinear conjugate gradient methods indicate the competitiveness of O‐ACCEL. As it can be combined with domain‐specific optimizers, it may also be beneficial in areas where limited‐memory Broyden–Fletcher–Goldfarb–Shanno and nonlinear conjugate gradient methods are not suitable.

The nonlinear conjugate gradient method (CGM) is a very effective iterative method for solving large-scale optimal problems. In this paper, based on a variant of Polak–Ribiere–Polyak method, two modified CGMs with disturbance factors are proposed. By the disturbance factors, the two proposed methods not only generate sufficient descent direction at each iteration but also converge globally for nonconvex minimization if the strong Wolfe line search is used. Finally, elementary numerical experiment results are reported, which show that the proposed methods are promising.

A study of Liu-Storey conjugate gradient methods for vector optimization

In this paper, we present an inversion estimation method of thin film parameters based on thermal effects induced by laser irradiation. Firstly, the theoretical model of classical Fourier heat conduction of thin film irradiated by laser is established, and the analytical solutions of temperature fields are obtained by using Laplace transform. Then, the inversion model and the iteration algorithm are established based on the nonlinear conjugate gradient method on condition that the thermal conductivities of the film and the substrate are selected as inversion parameters and the temperature fields of the thin film surface in different irradiation times are selected as measured data. In view of the fact that the sensitivity coefficient plays a decisive role in determining the accuracy and efficiency of the nonlinear conjugate gradient iteration inversion algorithm, we derive the closed form expressions of the sensitivity coefficients for the thermal conductivities of the film and the substrate based on the above analytical solutions of the temperature fields, and this closed form expressions can improve the accuracy and efficiency of the thin film parameter inversion significantly. Taking four kinds of metal films (aluminum, silver, copper and gold) with glass substrate for example, the accuracies of the analytical solutions of temperature fields are verified by comparing with the numerical results from the finite element method in the existing literature, and it can ensure the accuracies of the sensitivity coefficients in the process of iteration inversion. Finally, the thermal conductivities of the above four kinds of thin films are estimated by using the presented iteration inversion method. The accuracy and efficiency of the parameter inversion are verified by investigating and analyzing the inversion results of the parameters for different random noises and different iterative initial values. The inversion results show that this method has a high accuracy and efficiency, and it only needs less than 20 iteration times to convergence when the iteration stop error is 10-7. The smaller random noise is added in the measured data, and the less iteration times to convergence are needed. It can achieve higher convergence efficiency even in the iterative initial values from the inversion results that differ greatly for the case of 5% random noise. This inversion method of thin film parameters is not only applicable to the inversion of the thermal conductivity, but can also be used to inverse the parameters such as the reflection coefficient or the absorption coefficient. The presented method has a certain guiding significance for the parameters inversion and the parameters optimization in the process of the laser processing or the laser damage.