Linear and nonlinear conjugate gradient methods for adaptive processing

We study the development of nonlinear conjugate gradient methods, Fletcher Reeves (FR) and Polak Ribiere (PR). FR extends the linear conjugate gradient method to nonlinear functions by incorporating two changes, for the step length αk a line search is performed and replacing the residual, rk (rk=b-Axk) by the gradient of the nonlinear objective function. The PR method is equivalent to FR method for exact line searches and when the underlying quadratic function is strongly convex. The PR method is basically a variant of FR and primarily differs from it in the choice of the parameter βk. On applying the nonlinear Rosenbrock function to the MATLAB code for the FR and the PR algorithms we observe that the performance of PR method (k=29) is far better than the FR method (k=42). But, we observe that when the MATLAB codes are applied to general nonlinear functions, specifically functions whose minimum is a large negative number not close to zero and the iterates too are large values far off from zero the PR algorithm does not perform well. This problem with the PR method persists even if we run the PR algorithm for more iterations or with an initial guess closer to the actual minimum. To improve the PR algorithm we suggest finding a better weighing parameter βk, using better line search method and/or using specific line search for certain functions and identifying specific restart criteria based on the function to be optimized.

This paper presents a new approach based on a Two-Stage Linear Discriminant Analysis (Two-Stage LDA) and Conjugate Gradient Algorithms (CGAs) for face recognition. A Two-Stage LDA technique is proposed that utilises the null space of the sample covariance matrix as well as using the range space of the between-class scatter matrix to extract discriminant information. Classic Back Propagation (BP) is a widely used Neural Network (NN) training algorithm in many detectors and classifiers. However, it is both too slow for many practical problems and its performance is not satisfactory in many application areas, including face recognition. To overcome these problems, four CGA algorithms (Fletcher-Reeves CGA, Polak-Ribiere CGA, Powell-Beale CGA, scaled CGA) have been proposed, the utility of which we investigate here in combination with Two-Stage LDA features. To further improve the accuracy, a modified AdaBoost.M1 approach was employed, which combines results of several NN classifiers as a single strong classifier. Experiments are performed on the ORL, FERET and AR face databases. The results show that all of the proposed methods lead to increased recognition rates and shorter training times compared to the classic BP.

The gradient-based optimization methods are preferable for the large-scale three-dimensional (3D) magnetotelluric (MT) inverse problem. Compared with the popular nonlinear conjugate gradient (NLCG) method, however, the limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) method is less adopted. This paper aims to implement a L-BFGS-based inversion algorithm for the 3D MT problem. And we develop our code on top of the ModEM package, which is highly extensible and popular among the MT community. To accelerate the convergence speed, the preconditioning technique by the affine linear transformation of the original model parameters is used. Two modifications of the conventional L-BFGS algorithm are also made to get a comparable convergence rate with the NLCG method. The impacts of the preconditioner parameters, the regularization parameters, the starting model, etc., on the inversion are evaluated by synthetic examples for both L-BFGS and NLCG methods. And the real MT Kayabe dataset is also inverted by the inversion algorithms. The synthetic tests show that through our L-BFGS inversion algorithm the similar resistivity models can be obtained with that from the NLCG method. For the real data inversion, the L-BFGS method performs more efficiently and reasonable results could be obtained by less iterations of the inversion process than the NLCG method. Thus, we suggest the common usage of the L-BFGS method for the 3D MT inverse problem.

A major issue in space-time adaptive processing (STAP) for airborne moving target indicator (MTI) radar is the so-called sample support problem. Often, the available sample support for estimating the interference covariance matrix leads to severe rank deficiency, thereby precluding STAP beamforming based on the direct sample matrix inversion (SMI) method. The intrinsic interference subspace removal (ISR) technique, which is a computationally useful form of diagonally loaded SMI method, can handle this case, although the performance is poor in low sample situations. In this context, new subarray-subpulse schemes using forward and backward data vectors are introduced to overcome the data deficiency problem. It is shown that a multiplicative improvement in data samples can be obtained at the expense of negligible loss in space-time aperture of the steering vector.

In this chapter, we investigate recently proposed nonlinear conjugate gradient (NCG) methods for shape optimization problems. We briefly introduce the methods as well as the corresponding theoretical background and investigate their performance numerically. The obtained results confirm that the NCG methods are efficient and attractive solution algorithms for shape optimization problems.

Traditional space-time (ST) adaptive processing (STAP) theory is based on the assumption of narrowband or “zero-bandwidth,” where the decorrelation within the ST snapshot is ignored. However, with radar bandwidths increasing, this assumption becomes invalid due to the deteriorated decorrelation of the received signals within the ST snapshot. The decorrelation directly causes the dispersion of the received signals in both spatial and temporal domains, leading to the spreading of the clutter spectrum in the 2-D frequency (Doppler-spatial frequency) domain. With the spreading of the clutter spectrum, the clutter suppression notch in the traditional STAP filters is widened, resulting in a relative poor ability to detect slow-moving targets. In this article, we focus on the clutter suppression for wideband radar STAP. A generalized signal model of the ground clutter is first established for the wideband array radar. Using this outcome, we analyze the influence of bandwidth on the characteristics of the ground clutter and quantitatively describe the 2-D spreading of the ground clutter on the Doppler-spatial frequency plane. Moreover, the model of clutter covariance matrix for wideband STAP (W-STAP) is established. Finally, a 2-D keystone transform (KT) algorithm, referred to as ST KT (ST-KT), is proposed to eliminate the spreading of the ground clutter in the 2-D frequency domain caused by increasing bandwidths. Simulation results are employed to validate the theoretical analysis and verify the overperformance of the ST-KT based W-STAP method in terms of the output signal-to-clutter-plus-noise ratio (SCNR) of moving targets.

In this paper, a robust statistical method called projection statistics (PS) is developed into an TV-dimensional complex form required to detect strong nonhomogeneities, or TV-dimensional outlier vectors (i.e., outliers), in space-time adaptive processing (STAP) radar training data, where N is the number of degrees of freedom (DOF) associated with the STAP scenario. The PS technique does not require the estimation of a covariance matrix, as is the case with the generalized inner product (GIP) test. Rather, it uses robust estimators of location (the median) and scale (the median absolute deviation from the median (MAD)) to identify and then excise strong nonhomogeneities in STAP training data that are capable of rendering traditional nonhomogeneity detection (NHD) techniques, such as the GIP, ineffective. Shown in the results section of this paper is sample matrix inversion (SMI) STAP performance using three separate NHD techniques: 1) a power test for high power outlier snapshots, 2) a GIP test, and 3) a PS-based test. The performance when not using any form of NHD is shown for comparison. It is clear from the results that the PS NHD method provides an SMI-STAP processor the best training data selection in the multi channel airborne radar measurements (MCARM) [B.N.S Babu et al., 1996] based radar data scenario studied here, and does so across all range cells of the scenario.

Purpose– Generally, iterative methods which have some instability solutions in complex structural and non-linear mechanical problems are used to compute reliability index. The purpose of this paper is to establish a non-linear conjugate gradient (NCG) optimization algorithm to overcome instability solution of the Hasofer-Lind and Rackwitz-Fiessler (HL-RF) method in first-order reliability analysis. The NCG algorithms such as the Conjugate-Descent (CD) and the Liu-Storey (LS) are used for determining the safety index. An algorithm is found based on the new line search in the reliability analysis.Design/methodology/approach– In the proposed line search for calculating the safety index, search direction is computed by using the conjugate gradient approach and the HL-RF method based on the new and pervious gradient vector of the reliability function. A simple step size is presented for the line search in the proposed algorithm, which is formulated by the Wolfe conditions based on the new and previous safety index results in the reliability analysis.Findings– From the current work, it is concluded that the proposed NCG algorithm has more efficient, robust and appropriate convergence in comparison with the HL-RF method. The proposed methods can eliminate numerical instabilities of the HL-RF iterative algorithm in highly non-linear performance function and complicated structural limit state function. The NGC optimization is applicable to reliability analysis and it is correctly converged on the reliability index. In the NCG method, the CD algorithm is slightly more efficient than the LS algorithm.Originality/value– This paper usefully shows how the HL-RF algorithm and the NCG scheme are formulated in first-order reliability analysis. The proposed algorithm is validated from six numerical and structural examples taken from the literature. The HL-RF method is not converged on several non-linear mathematic and complex structural examples, while the two proposed conjugate gradient methods are appropriately converged for all examples. The CD algorithm is converged about twice faster than the LS algorithm in most of the problems. Therefore, application of the NCG method is possible in reliability analysis.

ABSTRACTComputational homogenization may be used to evaluate effective properties of microstructured materials. As digital imaging techniques generally generate microstructure information on a regular grid, a common approach to compute the effective properties of the microstructured materials is to use fast Fourier transform (FFT)‐based methods. Traditionally, this approach involves periodic boundary conditions. However, for certain microstructure types, nonperiodic boundary conditions are of interest. In this work, we show how to efficiently impose Dirichlet boundary conditions for FFT‐based micromechanics on stochastic bicontinuous microstructures in combinations with a linear conjugate gradient (CG) solver. Consequently, we compute the effective properties of bicontinuous stochastic microstructures and compare the results to values reported in the literature computed by a finite element (FE) code.

High Quality 4-dimensional Cone Beam CT by Deforming Prior Images

Nonlinear conjugate gradient (CG) methods are widely used for solving large scale unconstrained optimization problems. Many studies have been devoted to modified and improve this method. In this paper, a new parameter of CG method that possesses global convergence properties using exact line search is proposed. Numerical results show that the new formula is best and more efficient when compared with the other classical CG methods

The range-dependence of clutter spectrum for forward-looking airborne radar strongly affects the accuracy of the estimation of clutter covariance matrix at the range under test, which results in poor clutter suppression performance if the conventional space-time adaptive processing (STAP) algorithms were applied, especially in the short range cells. Therefore, a new STAP algorithm with clutter spectrum compensation by utilizing knowledge-aided subspace projection is proposed to suppress clutter for forward-looking airborne radar in this paper. In the proposed method, the clutter covariance matrix of the range under test is firstly constructed based on the prior knowledge of antenna array configuration, and then by decomposing the corresponding space-time covariance matrix to calculate the clutter subspace projection matrix which is applied to transform the secondary range samples so that the compensation of clutter spectrum for forward-looking airborne radar is accomplished. After that the conventional STAP algorithm can be applied to suppress clutter in the range under test. The proposed method is compared with the sample matrix inversion (SMI) and the Doppler Warping (DW) methods. The simulation results show that the proposed STAP method can effectively compensate the clutter spectrum and mitigate the range-dependence significantly.

Space-time adaptive monopulse processing for airborne radar in non-homogeneous environments

We propose an effective conjugate gradient method belonging to the class of Dai–Liao methods for solving unconstrained optimization problems. We employ a variant of the modified secant condition and introduce a new conjugate gradient parameter by solving an optimization problem. The optimization problem combines the well-known features of the linear conjugate gradient method using some penalty functions. This new parameter takes advantage of function information as well as the gradient information to provide the iterations. Our proposed method is globally convergent under mild assumptions. We examine the ability of the method for solving some real-world problems from image processing field. Numerical results show that the proposed method is efficient in the sense of the PSNR test. We also compare our proposed method with some well-known existing algorithms using a collection of CUTEr problems to show its efficiency.

We introduce a new efficient nonlinear conjugate gradient method for unconstrained optimization, based on minimizing a penalty function. Our penalty function combines the good properties of the linear conjugate gradient method using some penalty parameters. We show that the new method is a member of Dai---Liao family and, more importantly, propose an efficient Dai---Liao parameter by closely analyzing the penalty function. Numerical experiments show that the proposed parameter is promising.