The Third Problem of Algebraic Regression

Summary In this study, we show significant improvements over conventional pressure-transient analysis on the basis of nonlinear regression by using total least squares (TLS), which minimizes errors in both pressure and time simultaneously. To our knowledge, TLS has not been applied to pressure-transient analysis before in this sense. TLS regression is not an easy problem to solve mathematically, especially for nonlinear pressure-transient-model functions. In this work, we compare four different versions of TLS. We formulate a robust approximation of the TLS solution, which can handle a variety of tests and reservoir models yet does not compromise the performance of the analysis. We show that our technique reduces ambiguity in the estimation of parameters to a large extent, especially in the presence of noise in time. Using our TLS algorithm, we obtain much narrower confidence intervals on the parameter estimates of a variety of real data sets, compared to the conventional least squares (LS) approach. For synthetic data sets, we observe that the TLS estimates are often closer to the true values than estimates made with LS, especially for poorly determined problems. When the deviation is in pressure only, TLS and LS results are comparable. However, in the presence of deviations in time in addition to pressure, the performance of TLS algorithms is substantially better. We, therefore, expect that our technique will provide more accurate estimation of reservoir parameters, allowing for better forecasting of reservoir performance.

The Total Least Squares (TLS) method is a generalized least square technique to solve an overdetermined system of equations Ax\simeqb . The TLS solution differs from the usual Least Square (LS) in that it tries to compensate for arbitrary present in both A and b . In certain problems the perturbations of A and b are linear functions of a common noise source vector. In this case we obtain a generalization of the TLS criterion called the Constrained Total Least Squares (CTLS) method by taking into account the linear dependence of the terms in A and b . If the columns of A and b are linearly related then the CTLS solution is obtained in terms of the largest eigenvalue and corresponding eigenvector of a certain matrix. The CTLS technique can be applied to problems like Maximum Likelihood Signal Parameter Estimation, Frequency Estimation of Sinusoids in white or colored by Linear Prediction and others.

Iterative methods for best approximate solutions of linear integral equations of the first and second kinds

Linear inverse problems with uncertain measurement matrices appear in many different applications. One of the standard techniques for solving such problems is the total least squares (TLS) method. Recently, an alternative approach has been suggested, based on maximizing an appropriate likelihood function assuming that the measurement matrix consists of random Gaussian variables. We refer to this technique as the total maximum likelihood (TML) method. Here we extend this strategy to the case in which the measurement matrix is structured so that the perturbations are not arbitrary but rather follow a fixed pattern. The resulting estimate is referred to as the structured TML (STML). As we show, the STML can be viewed as a regularized version of the structured TLS (STLS) approach in which the regularization consists of a logarithmic penalty. In contrast to the STLS solution, the STML always exists. Furthermore, its performance in practice tends to be superior to that of the STLS and competitive to other regularized solvers, as we illustrate via several examples. We also consider a few interesting special cases in which the STML can be computed efficiently either by reducing it into a one-dimensional problem regardless of the problem size or by a decomposition via a discrete Fourier transform.

Numerical weather prediction (NWP) models produce a quantitative precipitation forecast (QPF), which is vital for a wide range of applications, especially for accurate flash flood forecasting. The under- and over-estimation of forecast uncertainty pose operational risks and often encourage overly conservative decisions to be made. Since NWP models are subject to many uncertainties, the QPFs need to be post-processed. The NWP biases should be corrected prior to their use as a reliable data source in hydrological models. In recent years, several post-processing techniques have been proposed. However, there is a lack of research on post-processing the real-time forecast of NWP models considering bias lead-time dependency for short- to medium-range forecasts. The main objective of this study is to use the total least squares (TLS) method and the lead-time dependent bias correction method—known as dynamic weighting (DW)—to post-process forecast real-time data. The findings show improved bias scores, a decrease in the normalized error and an improvement in the scatter index (SI). A comparison between the real-time precipitation and flood forecast relative bias error shows that applying the TLS and DW methods reduced the biases of real-time forecast precipitation. The results for real-time flood forecasts for the events of 2002, 2007 and 2011 show error reductions and accuracy improvements of 78.58%, 81.26% and 62.33%, respectively.

In this paper a new fast motion estimation method useful for video encoding in a noisy environment is presented. Motion estimation methods can broadly categorized as Block Matching (BM) methods, Pel Recursive (PR) methods, and Frequency Domain (FD) methods. Majority of the current fast motion estimation methods belong to block matching category. These methods typically achieve speedup by either reducing the number of search locations or by reducing the number of computations at each search location. Our proposed method is based on the pel recursive formulation. However, in order to achieve fast estimation, we operate on a block of pixels. To provide a robust behavior in a noisy environment we use a Total Least Squares (TLS) based estimation scheme which aims at estimating the true motion vectors. The simulation results indicate the speedup and robustness of the algorithm in noisy environment.

The reduced-rank method exploits the distortion-variance tradeoff to yield superior solutions for classic problems in statistical signal processing such as parameter estimation and filtering. The central idea is to reduce the variance of the solution at the expense of introducing a little distortion. In the context of parameter estimation, this yields an estimator whose sum of distortion plus variance is smaller than the variance of its undistorted counterpart. The method intrinsically results in an ordering mechanism for the singular vectors of the system matrix in the measurement model used for estimating the parameter of interest. According to this ordering rule, only a few dominant singular vectors need to be selected to construct the reduced-rank solution while the rest can be discarded. The reduced-rank estimator is less sensitive to measurement errors. In this paper, we attempt to derive the reduced-rank estimator for the total least squares (TLS) problem, including the order selection rule. It will be shown that, due to the inherent structure of the problem, it is not possible to exploit the distortion-variance tradeoff in TLS formulations using existing techniques, except in some special cases. This result has not been reported previously and warrants a new viewpoint to achieve rank reduction for the TLS estimation problem. The work is motivated by the problems arising in practical applications such as channel estimation in wireless communication systems and time synchronization in wireless sensor networks.

The total least squares (TLS) method is an appropriate approach for linear systems when not only the right-hand side but also the system matrix is contaminated by some noise. For ill-posed problems regularization is necessary to stabilize the computed solutions. In this presentation we discuss two approaches for regularizing large scale TLS problems. One which is based on adding a quadratic constraint and a Tikhonov type regularization concept.

This paper deals with the problem of finding the minimum norm least-squares solution of a quite general class of coupled linear matrix equations defined over field of complex numbers. To this end, we examine a gradient-based approach and present the convergence properties of the algorithm. The highlight of the elaborated results in the current work is using a new sight of view for construction of the gradient-based algorithm which turns out that we can ignore some of the limitations assumed by the authors in the recently published works for the application of the algorithm to obtain the solution of the referred problems. To the best of our knowledge, so far, computing the optimal convergence factor of the algorithm to determine the (least-squares) solution of general complex linear matrix equations has left as a project to be investigated. In the current work, we determine the optimal convergence factor of the algorithm. Some numerical experiments are reported to illustrate the validity of the presented results.

The restricted total least squares (RTLS) problem, presented in this paper, is devised for solving overdetermined sets of linear equations $AX \approx B$ in which the data $[ A;B ]$ are perturbed by errors of the form $E^ * = DEC$. D and C are known matrices and E is an arbitrary but bounded matrix. By choosing D and C appropriately, the RTLS problem formulation can handle any weighted least squares (LS), generalized LS, total LS, and generalized total LS problem. Also, equality constraints can be imposed. In order to solve these problems, a computationally efficient and numerically reliable restricted TLS algorithm, based on the restricted singular value decomposition (RSVD), of the matrix triplet $( [ A;B ],D,C )$, is developed. This RSVD is a generalization of the ordinary SVD for triple matrix products. The matrices involved may be rank-deficient and the explicit formation of matrix inverses and products is avoided. Using the RSVD, some properties of the RTLS problem are proven.

A subspace decomposition based algorithm for joint frequency of arrival (FOA) and direction of arrival (DOA) estimation is presented in this paper. The proposed method termed as QR-TLS ESPRIT, estimates the signal subspace by decomposing the received signal in a specially formed Toeplitz matrix. The structure of the matrix favors the application of QR factorization, which yields the signal subspace at the expense of less number of computations. A Total Least Squares (TLS) method is applied to obtain the frequency and angle estimates from the signal space. The proposed QR-TLS ESPRIT method provides several benefits: 1) It avoids the computations of applying computationally complex Eigen Value Decomposition (EVD) or Singular Value Decomposition (SVD) methods to obtain signal subspace alike conventional methods, 2) QR decomposition method, which is intrinsically applicable to numerical rank deficient problems, is used to extract signal subspace from the observation data matrix, 3) the TLS method is integrated with the QR which enhances the estimation accuracy in the presence of noisy measurements, 4) the proposed method can estimate the FOA and DOA from a small snapshot length of data which further reduces the computational time, and 5) the method works for both coherent as well as noncoherent sources without employing pre-whitening or spatial smoothing techniques. The performance of the proposed method is evaluated through simulations and the results are illustrated with respect to various received signal strength.

By extending the idea of LSMR method, we present an iterative method to solve the general coupled matrix equations∑k=1qAikXkBik=Ci,i=1,2,…,p, (including the generalized (coupled) Lyapunov and Sylvester matrix equations as special cases) over some constrained matrix groups(X1,X2,…,Xq), such as symmetric, generalized bisymmetric, and(R,S)-symmetric matrix groups. By this iterative method, for any initial matrix group(X1(0),X2(0),…,Xq(0)), a solution group(X1*,X2*,…,Xq*)can be obtained within finite iteration steps in absence of round-off errors, and the minimum Frobenius norm solution or the minimum Frobenius norm least-squares solution group can be derived when an appropriate initial iterative matrix group is chosen. In addition, the optimal approximation solution group to a given matrix group(X¯1,X¯2,…,X¯q)in the Frobenius norm can be obtained by finding the least Frobenius norm solution group of new general coupled matrix equations. Finally, numerical examples are given to illustrate the effectiveness of the presented method.

In this article, we use standard extreme temperature indices to detect and attribute external forcing in Madagascar. These indices are calculated from observations and multi‐model ensemble mean responses on anthropogenic‐plus‐natural (ALL), greenhouse gases (GHG), natural (NAT) and anthropogenic (ANT) which are subtracted from ALL and NAT forcings over 1950–2018. Correlation analysis emphasizes that the observed changes are more influenced by ENSO events, especially in minimum temperature. The observed changes are regressed or combined with model simulations from the sixth phase of the Coupled Model Inter‐comparison Project (CMIP6) to assess human impacts in indices. CMIP6 models with ALL, GHG and ANT forcings correspond well with the observations for the frequency indices than the intensity indices. Moreover, decadal trends indicate the existence of anthropogenic warming according to observations and multi‐model ensembles with ALL, GHG and ANT forcings. Detection and attribution parties identify and justify the causes of the observed changes. We do this by performing the single‐signal and two‐signal analysis using the Regularized Optimal Fingerprinting (ROF) method with Total Least Square (TLS) regression. We estimate internal climate variability by means of control model simulations. As a result, we note an inconsistency in the warming trend with the NAT forcing. The influence of ALL, GHG and ANT forcings is detectable for standard extreme temperature indices between 1950 and 2018. Nearly, observed changes are attributed to GHG and ANT forcings except for coldest night and warm nights in Madagascar.

State-of-charge (SOC) estimation of lithium-ion batteries (LIBs) is the basis of other state estimations. However, its accuracy can be affected by many factors, such as temperature and ageing. To handle this bottleneck issue, we here propose a joint SOC-SOH estimation method considering the influence of the temperature. It combines the Forgetting Factor Recursive Least Squares (FFRLS) algorithm, Total Least Squares (TLS) algorithm, and Unscented Kalman Filter (UKF) algorithm. First, the FFRLS algorithm is used to identify and update the parameters of the equivalent circuit model in real time under different battery ageing degrees. Then, the TLS algorithm is used to estimate the battery SOH to improve the prior estimation accuracy of SOC. Next, the SOC is calculated by the UKF algorithm, and finally, a more accurate SOH can be obtained according to the UKF-based SOC trajectory. The battery-in-the-loop experiments are utilized to verify the proposed algorithm. For the cases of temperature change up to 35 °C and capacity decay up to 10%, our joint estimator can achieve ultra-low errors, bounded by 2%, respectively, for SOH and SOC. The proposed method paves the way for the advancement of battery use in applications, such as electric vehicles and microgrid applications.

This paper presents a hybrid approach for estimating the admittance matrix of open-loop distribution networks by integrating Total Least Squares (TLS) regression with deep learning (DL) and graph neural networks (GNNs). The proposed model refines initial TLS estimates by leveraging deep learning to handle nonlinearities and graph neural networks to incorporate network topology, thereby addressing the challenges posed by noisy and incomplete data. Physical constraints-including Kirchhoff's current and voltage laws-are embedded within the optimization process to ensure that the estimated admittance matrix remains physically consistent. Extensive testing on the IEEE 33-bus and 123-bus systems demonstrates significant improvements in estimation accuracy, convergence speed, and adherence to physical principles, underscoring the model's potential for real-world applications in modern power system analysis and operation.