Estimation of Open Boundary Conditions for an Internal Tidal Model with Adjoint Method: A Comparative Study on Optimization Methods

An optimization method of adjoint assimilation based on the cumulative gradient: An application for correction of the bottom friction coefficient in the tidal wave model

The gradient-based optimization methods are preferable for the large-scale three-dimensional (3D) magnetotelluric (MT) inverse problem. Compared with the popular nonlinear conjugate gradient (NLCG) method, however, the limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) method is less adopted. This paper aims to implement a L-BFGS-based inversion algorithm for the 3D MT problem. And we develop our code on top of the ModEM package, which is highly extensible and popular among the MT community. To accelerate the convergence speed, the preconditioning technique by the affine linear transformation of the original model parameters is used. Two modifications of the conventional L-BFGS algorithm are also made to get a comparable convergence rate with the NLCG method. The impacts of the preconditioner parameters, the regularization parameters, the starting model, etc., on the inversion are evaluated by synthetic examples for both L-BFGS and NLCG methods. And the real MT Kayabe dataset is also inverted by the inversion algorithms. The synthetic tests show that through our L-BFGS inversion algorithm the similar resistivity models can be obtained with that from the NLCG method. For the real data inversion, the L-BFGS method performs more efficiently and reasonable results could be obtained by less iterations of the inversion process than the NLCG method. Thus, we suggest the common usage of the L-BFGS method for the 3D MT inverse problem.

This report presents a header-only C++ class implementation of the Limited-memory BroydenFletcher-Goldfarb-Shanno (L-BFGS) algorithm. The L-BFGS method is a general purpose quasi-Netwon optimization method that builds an approximation of the descent direction from consecutive iterate and gradient vectors. The limited-memory aspect stems from the replacement of the N × N approximation matrix of the original BFGS method with M vectors of length N. An example usage of the class is included along with the reference source code.

SummarySubsurface development involves well-placement decisions considering the highly uncertain understanding of the reservoir in the subsurface. The simultaneous optimization of a large number of well locations is a challenging problem. Traditional gradient-based methods can be adapted for well location optimization (WLO) when these problems are converted into real-valued representations and equipped with protocols to handle noisy objective functions. However, their application to large-scale scenarios often remains impractical. This impracticality arises because computing gradients of the objective function can be prohibitively expensive in realistic settings, particularly without using the adjoint method. In this paper, we explore the application of a novel quasi-Newton trust-region (TR) method that employs the stochastic simplex approximate gradient (StoSAG).We have implemented the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton updating algorithm together with line-search (LS) and TR optimization strategies. The StoSAG-based optimization methods have been applied to a realistic synthetic reservoir featuring 26 wells considering two distinct cases: Each includes 20 realizations of porosity and permeability. The first case exhibits mild heterogeneity, while the second exhibits significant heterogeneity with a large correlation length. We have conducted a series of runs to evaluate the performance of these algorithms in addition to comparisons to the finite-difference (FD) and particle-swarm-optimization (PSO) algorithms. We introduce a novel approach to enhance the accuracy of StoSAG gradients by proposing modified StoSAG formulations. These formulations are tailored to exploit the structure of the objective function and to capture the relationships between its components and the individual optimization parameters. This approach involves using a correction matrix W informed by problem-specific knowledge. The entries of W vary from 0 to 1 and are proportional to the interference effects the neighboring wells have on each other. We determine those entries (or weights) based on the radii of investigation around the wells and the distance between the well pairs.Results indicate that the steepest-descent (SD) algorithm coupled with StoSAG has superior performance to PSO and FD. Although the objective function is prone to numerical noise and not continuously differentiable with respect to well locations, StoSAG overcomes this challenge because it acts as a smooth approximation. Comparative tests further confirm that the TR-BFGS is more effective than the LS-BFGS. Moreover, we show that using the proposed gradient correction procedure results in a significant acceleration in convergence, indicating an enhancement in the StoSAG gradient approximation quality. This enhancement allows the TR-BFGS algorithm to achieve considerably higher performance than SD, illustrating that the accuracy of the BFGS approximation benefits from improved gradient quality.

The inverse problem of interest is the reconstruction of an astronomical objectand wavefront aberrations due to atmospheric turbulence from a sequence of shorttime exposure (speckle) phase diversity images obtained from a ground-basedtelescope. A regularized least squares approach is taken, and two numericaltechniques are applied to the resulting unconstrained optimization problem. Thefirst technique is the limited memory BFGS method with line search globalization.The second is a Newton/trust region iteration in which the trust regionsubproblem is solved using a truncated conjugate gradient method. Anumerical comparison based on real data is presented, and the roles ofpreconditioning and cost functional reduction (elimination of the object) areexamined.

The limited memory BFGS method (L-BFGS) is an adaptation of the BFGS method for large-scale unconstrained optimization. However, The L-BFGS method need not converge for nonconvex objective functions and it is inefficient on highly ill-conditioned problems. In this paper, we proposed a regularization strategy on the L-BFGS method, where the used regularization parameter may play a compensation role in some sense when the condition number of Hessian approximation tends to become ill-conditioned. Then we proposed a regularized L-BFGS method and established its global convergence even when the objective function is nonconvex. Numerical results show that the proposed method is efficient.

The yearly mode-1 M2 internal tide model in 2019 is constructed using sea surface height measurements made by six concurrent satellite altimetry missions: Jason-3, Sentinel-3A, Sentinel-3B, CryoSat-2, Haiyang-2A, and SARAL/AltiKa. The model is developed following a three-step procedure consisting of two rounds of plane wave analysis with a spatial bandpass filter in between. Prior mesoscale correction is made on the altimeter data using AVISO gridded mesoscale fields. The model is labeled Y2019, because it represents the 1-yr-coherent internal tide field in 2019. In contrast, the model developed using altimeter data from 1992 to 2017 is labeled MY25, because it represents the multiyear-coherent internal tide field in 25 years. Thanks to the new mapping technique, model errors in Y2019 are as low as those in MY25. Evaluation using independent altimeter data confirms that Y2019 reduces slightly less variance (∼6%) than MY25. Further analysis reveals that the altimeter data from five missions (without Jason-3) can yield an internal tide model of almost the same quality. Comparing Y2019 and MY25 shows that mode-1 M2 internal tides are subject to significant interannual variability in both amplitude and phase, and their interannual variations are a function of location. Along southward internal tides from Amukta Pass, the energy flux in Y2019 is 2 times larger and the phase speed is about 1.1% faster. This mapping technique has been applied successfully to 2017 and 2018. This work demonstrates that yearly internal tides can be observed by concurrent altimetry missions and their interannual variations can be determined. Significance Statement This work is motivated to study the interannual variations of internal tides using observation-based yearly internal tide models from satellite altimetry. Previous satellite observations of internal tides are usually based on 25 years of altimeter data from 1993 to 2017. The yearly subsetted altimeter data are short, so that the resultant yearly models are overwhelmed by noise. A new mapping technique is developed and demonstrated in this paper. It paves a path to study the interannual and decadal variations of internal tides on a global scale and monitor the global ocean changes by tracking long-range internal tides.

Application of gradient descent method to the sedimentary grain-size distribution fitting

Gradient descent (GD) method is used to fit the the laser grain-size distribution of the sediments with a sum of four weighted lognormal functions. The numerical results indicate that the GD method is not only easy to operate but also could effectively optimize the parameters of the fitting function with the error decreasing steadily. Meanwhile the overall fitting results are satisfactory. As a new way of data fitting, the GD method could also be used to solve other optimization problems.

A numerical study for estimating the tidal open boundary conditions of a shelf current modrl from tb coastal tidal observations is presented. The method is based on the optimal control/adjoint method. A lrast square fitting of the model state to simulated data is used. Two ideal domains and coastlines are considered. Using the IAP shallow. water model and its adjoint model, some identical twin experiments are carried out to test efficiency and lirnilsd of the method. The results show that the adjoint method can efficiently estimate the open boundary conditions well for gulf/bay like domains. The adjoint method seems to have great potential to improve the accuracy of tide and shelf current modeling in coastal regions.

Summary Least-squares reverse time migration (LSRTM) has an excellent imaging effect, but suffers from a great challenge on the significant computational cost. AA has become popular in the recent decades for accelerating convergence speed and reducing computational cost. According to the Broyden’s method, AA is divided into type-I AA (AA-I) and type-II AA (AA-II). Most AA methods are based on AA-II and few researchers set foot in AA-I to solve the instability problem due to the residual vibration, an oscillation of the objective misfit function. In order to deal with the residual vibration problem and apply AA-I to LSRTM to improve its computational efficiency, we propose a safe AA-I method by combining the Powell-type regularization, re-start checking, and safe-guarding steps with AA-I. We find that the appropriate step length for the safe AA-Ⅰ is close to 10 times 1st iteration steepest descent (SD) step. By using the Marmousi model to conduct LSRTM, we compare the safe AA-Ⅰ with SD, the conjugate gradient method, the limited-memory Broyden-Fletcher-Goldfarb-Shanno (LBFGS), and AA-II. The results show that the safe AA-I has a faster convergence speed and a better imaging quality than other methods. The safe AA-I has twice computational efficiency than LBFGS approximately.

Two strategies for estimating open boundary conditions (OBCs) with adjoint method are compared by carrying out semi-idealized numerical experiments. In the first strategy, the OBC is assumed to be partly space varying and generated by linearly interpolating the values at selected feature points. The advantage is that the values at feature points are taken as control variables so that the variations of the curves can be reproduced by the minimum number of points. In the second strategy, the OBC is assumed to be fully space varying and the values at every open boundary points are taken as control variables. A series of semi-idealized experiments are carried out to compare the effectiveness of two inversion strategies. The results demonstrate that the inversion effect is in inverse proportion to the number of feature points which characterize the spatial complexity of open boundary forcing. The effect of ill-posedness of inverse problem will be amplified if the observations contain noises. The parameter estimation problems with more control variables will be much more sensitive to data noises, and the negative effects of noises can be restricted by reducing the number of control variables. This work provides a concrete evidence that ill-posedness of inverse problem can generate wrong parameter inversion results and produce an unreal “good data fitting.”

COVID-19 is a type of an infectious disease that is caused by the new coronavirus. The spread of COVID-19 needs to be suppressed because COVID-19 can cause death, especially for sufferers with congenital diseases and a weak immune system. COVID-19 spreads through direct contact, wherein the infected individual spreads the COVID-19 virus through cough, sneeze, or close contacts. Predicting the number of COVID-19 sufferers becomes an important task in the effort to curb the spread of COVID-19. Artificial neural network (ANN) is the prediction method that delivers effective results in doing this job. Backpropagation, a type of ANN algorithm, offers predictive problem solving with good performance. However, its performance depends on the optimization method applied during the training process. In general, the optimization method in ANN is the gradient descent method, which is known to have a slow convergence rate. Meanwhile, the Fletcher–Reeves method has a faster convergence rate than the gradient descent method. Based on this hypothesis, this paper proposes a prediction model for the number of COVID-19 sufferers in Malang using the Backpropagation neural network with the Fletcher–Reeves method. The experimental results show that the Backpropagation neural network with the Fletcher–Reeves method has a better performance than the Backpropagation neural network with the gradient descent method. This is shown by the Means Square Error (MSE) resulting from the proposed method which is smaller than the MSE resulting from the Backpropagation neural network with the gradient descent method.

Nowadays, the influence of machine learning algorithms on our lives is booming gradually, and gradient descent (GD) method is a typical big data analysis method that can be applied to various machine learning algorithms. Traditional GD method has its advantages in dealing with the optimal parameter finding issue for partial loss functions, but the optimal parameter finding process may be trapped into local minimum sometimes if the loss function with certain convexity; moreover, traditional GD cannot meet the requirement of high accuracy. Fractional order GD, which is the core of fixed memory step GD, has better performance by comparing with traditional GD, but still have some problems in the aspect of convergence. On the other hand, t-distributed machine learning algorithm for dimensionality reduction, can reduce the dimensionality of data in high-dimensional space to two-dimensional or three-dimensional space, and can convert the original data into more comprehensible images. In order to solve the problem mentioned above reasonably, in this paper, a fixed-memory step GD method combined with t-distributed stochastic neighbor embedding (t-SNE) is employed to replace the traditional GD method, so as to keep the consistency of low-dimensional data being mapped from high-dimensional data, and to achieve efficient data visualization. The effectiveness of proposed t-SNE algorithm based on fixed memory step gradient descent method has been verified finally via numerical experiment.

SUMMARYFull-waveform inversion has become an essential technique for mapping geophysical subsurface structures. However, proper uncertainty quantification is often lacking in current applications. In theory, uncertainty quantification is related to the inverse Hessian (or the posterior covariance matrix). Even for common geophysical inverse problems its calculation is beyond the computational and storage capacities of the largest high-performance computing systems. In this study, we amend the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm to perform uncertainty quantification for large-scale applications. For seismic inverse problems, the limited-memory BFGS (L-BFGS) method prevails as the most efficient quasi-Newton method. We aim to augment it further to obtain an approximate inverse Hessian for uncertainty quantification in FWI. To facilitate retrieval of the inverse Hessian, we combine BFGS (essentially a full-history L-BFGS) with randomized singular value decomposition to determine a low-rank approximation of the inverse Hessian. Setting the rank number equal to the number of iterations makes this solution efficient and memory-affordable even for large-scale problems. Furthermore, based on the Gauss–Newton method, we formulate different initial, diagonal Hessian matrices as pre-conditioners for the inverse scheme and compare their performances in elastic FWI applications. We highlight our approach with the elastic Marmousi benchmark model, demonstrating the applicability of pre-conditioned BFGS for large-scale FWI and uncertainty quantification.