Constrained total least squares

Modified Yule-Walker (MYW) equations are often used to estimate autoregressive parameters of an autoregressive moving average (ARMA) model. Commonly used algorithms, i.e., the least square (LS) algorithm, the total least square (TLS) algorithm, cannot give an optimal estimate because they do not exploit the Toeplitz property and covariance of the perturbation matrix. In this paper, a constrained total least squares (CTLS) algorithm is applied to solve modified Yule-Walker equations. The perturbation covariance matrix of the autocorrelation functions required by the CTLS algorithm is estimated by introducing the bootstrap method. By utilizing the Toeplitz property and the covariance of perturbation matrix, a Newton method based CTLS algorithm is shown to outperform TLS and LS solutions.

Summary In this study, we show significant improvements over conventional pressure-transient analysis on the basis of nonlinear regression by using total least squares (TLS), which minimizes errors in both pressure and time simultaneously. To our knowledge, TLS has not been applied to pressure-transient analysis before in this sense. TLS regression is not an easy problem to solve mathematically, especially for nonlinear pressure-transient-model functions. In this work, we compare four different versions of TLS. We formulate a robust approximation of the TLS solution, which can handle a variety of tests and reservoir models yet does not compromise the performance of the analysis. We show that our technique reduces ambiguity in the estimation of parameters to a large extent, especially in the presence of noise in time. Using our TLS algorithm, we obtain much narrower confidence intervals on the parameter estimates of a variety of real data sets, compared to the conventional least squares (LS) approach. For synthetic data sets, we observe that the TLS estimates are often closer to the true values than estimates made with LS, especially for poorly determined problems. When the deviation is in pressure only, TLS and LS results are comparable. However, in the presence of deviations in time in addition to pressure, the performance of TLS algorithms is substantially better. We, therefore, expect that our technique will provide more accurate estimation of reservoir parameters, allowing for better forecasting of reservoir performance.

A unified treatment of several least squares (LS) algorithms is presented for bearings-only tracking of a target moving at constant acceleration. The close link between the maximum likelihood (ML) estimator and other nonlinear and "linearized" LS algorithms is explored under the assumption of Gaussian bearing noise. In this context, a new asymptotically unbiased closed-form instrumental variables (IV) algorithm is derived. Reduced-bias total least squares (TLS) and constrained TLS (CTLS) algorithms are developed. The equivalence of the ML algorithm to the structured TLS (STLS) algorithm is established. Simulation examples are provided to demonstrate the improved performance of the IV and TLS estimators vis-a-vis the pseudolinear estimator.

Approximate model equations often relate given measurements to unknown parameters whose estimate is sought. The least-squares (LS) estimation criterion assumes the measured data to be exact, and seeks parameters which minimize the model errors. Existing extensions of LS, such as the total LS (TLS) and constrained TLS (CTLS) take the opposite approach, namely assume the model equations to be exact, and attribute all errors to measurement inaccuracies. We introduce the extended LS (XLS) criterion, which accommodates both error sources. We define 'pseudo-linear' models, with which we provide an iterative algorithm for minimization of the XLS criterion. Under certain statistical assumptions, we show that XLS coincides with a statistical criterion, which we term the 'joint maximum-a-posteriori-maximum-likelihood' (JMAP-ML) criterion. We identify the differences between the JMAP-ML and ML criteria, and explain the observed superiority of JMAP-ML over ML under non-asymptotic conditions.

The restricted total least squares (RTLS) problem, presented in this paper, is devised for solving overdetermined sets of linear equations $AX \approx B$ in which the data $[ A;B ]$ are perturbed by errors of the form $E^ * = DEC$. D and C are known matrices and E is an arbitrary but bounded matrix. By choosing D and C appropriately, the RTLS problem formulation can handle any weighted least squares (LS), generalized LS, total LS, and generalized total LS problem. Also, equality constraints can be imposed. In order to solve these problems, a computationally efficient and numerically reliable restricted TLS algorithm, based on the restricted singular value decomposition (RSVD), of the matrix triplet $( [ A;B ],D,C )$, is developed. This RSVD is a generalization of the ordinary SVD for triple matrix products. The matrices involved may be rank-deficient and the explicit formation of matrix inverses and products is avoided. Using the RSVD, some properties of the RTLS problem are proven.

On the accuracy of total least squares and least squares techniques in the presence of errors on all data

The inconsistent system of linear observational equations with datum defect is treated. The system also called underdetermined—overdetermined is solved by means of an optimization problem. The main notions are reviewed as minimum norm least squares solution (MINOLESS) based on the trinity: (i) algebraic partitioning (rank partitioning), (ii) geometric (slicing of the observation space) and (iii) set theoretical (fibering). The Total Least Squares and the celebrated Tykhonov-Phillips regularization, also called “Ridge Estimation” or “α–HAPS” (Hybrid Approximative Solution) are presented.KeywordsMinimum Norm Least Squares Solution (MINOLESS)Total Least Squares (TLS)Tykhonov regularizationMinimum Norm Solution (MINOS)Generalized inverseRAO's Pandora BoxGeneralized inverseEigenvalue decomposition of MINOLESS

Linear inverse problems with uncertain measurement matrices appear in many different applications. One of the standard techniques for solving such problems is the total least squares (TLS) method. Recently, an alternative approach has been suggested, based on maximizing an appropriate likelihood function assuming that the measurement matrix consists of random Gaussian variables. We refer to this technique as the total maximum likelihood (TML) method. Here we extend this strategy to the case in which the measurement matrix is structured so that the perturbations are not arbitrary but rather follow a fixed pattern. The resulting estimate is referred to as the structured TML (STML). As we show, the STML can be viewed as a regularized version of the structured TLS (STLS) approach in which the regularization consists of a logarithmic penalty. In contrast to the STLS solution, the STML always exists. Furthermore, its performance in practice tends to be superior to that of the STLS and competitive to other regularized solvers, as we illustrate via several examples. We also consider a few interesting special cases in which the STML can be computed efficiently either by reducing it into a one-dimensional problem regardless of the problem size or by a decomposition via a discrete Fourier transform.

Numerical weather prediction (NWP) models produce a quantitative precipitation forecast (QPF), which is vital for a wide range of applications, especially for accurate flash flood forecasting. The under- and over-estimation of forecast uncertainty pose operational risks and often encourage overly conservative decisions to be made. Since NWP models are subject to many uncertainties, the QPFs need to be post-processed. The NWP biases should be corrected prior to their use as a reliable data source in hydrological models. In recent years, several post-processing techniques have been proposed. However, there is a lack of research on post-processing the real-time forecast of NWP models considering bias lead-time dependency for short- to medium-range forecasts. The main objective of this study is to use the total least squares (TLS) method and the lead-time dependent bias correction method—known as dynamic weighting (DW)—to post-process forecast real-time data. The findings show improved bias scores, a decrease in the normalized error and an improvement in the scatter index (SI). A comparison between the real-time precipitation and flood forecast relative bias error shows that applying the TLS and DW methods reduced the biases of real-time forecast precipitation. The results for real-time flood forecasts for the events of 2002, 2007 and 2011 show error reductions and accuracy improvements of 78.58%, 81.26% and 62.33%, respectively.

In this paper a new fast motion estimation method useful for video encoding in a noisy environment is presented. Motion estimation methods can broadly categorized as Block Matching (BM) methods, Pel Recursive (PR) methods, and Frequency Domain (FD) methods. Majority of the current fast motion estimation methods belong to block matching category. These methods typically achieve speedup by either reducing the number of search locations or by reducing the number of computations at each search location. Our proposed method is based on the pel recursive formulation. However, in order to achieve fast estimation, we operate on a block of pixels. To provide a robust behavior in a noisy environment we use a Total Least Squares (TLS) based estimation scheme which aims at estimating the true motion vectors. The simulation results indicate the speedup and robustness of the algorithm in noisy environment.

The reduced-rank method exploits the distortion-variance tradeoff to yield superior solutions for classic problems in statistical signal processing such as parameter estimation and filtering. The central idea is to reduce the variance of the solution at the expense of introducing a little distortion. In the context of parameter estimation, this yields an estimator whose sum of distortion plus variance is smaller than the variance of its undistorted counterpart. The method intrinsically results in an ordering mechanism for the singular vectors of the system matrix in the measurement model used for estimating the parameter of interest. According to this ordering rule, only a few dominant singular vectors need to be selected to construct the reduced-rank solution while the rest can be discarded. The reduced-rank estimator is less sensitive to measurement errors. In this paper, we attempt to derive the reduced-rank estimator for the total least squares (TLS) problem, including the order selection rule. It will be shown that, due to the inherent structure of the problem, it is not possible to exploit the distortion-variance tradeoff in TLS formulations using existing techniques, except in some special cases. This result has not been reported previously and warrants a new viewpoint to achieve rank reduction for the TLS estimation problem. The work is motivated by the problems arising in practical applications such as channel estimation in wireless communication systems and time synchronization in wireless sensor networks.

The total least squares (TLS) method is an appropriate approach for linear systems when not only the right-hand side but also the system matrix is contaminated by some noise. For ill-posed problems regularization is necessary to stabilize the computed solutions. In this presentation we discuss two approaches for regularizing large scale TLS problems. One which is based on adding a quadratic constraint and a Tikhonov type regularization concept.

Cardiovascular time series are often affected by noise and contain components generated by different physiological sources. Least squares (LS) modelling is a powerful tool to identify these components but it provides biased estimates when the input measurements are noisy. The total least squares (TLS) model is appropriate when noise is present in all the measures. If the noise is additive, zero-mean, independent and white, TLS method results in asymptotically unbiased, consistent estimates. The authors applied the TLS method to the following applications obtaining good results: impulsive response estimation of the transfer function which relates the respiratory sinus arrhythmia with the respiratory activity; estimation of oscillatory components in RR-interval time series, cancelling of the respiratory component in the RR-interval time series. A comparison with the LS method, performed on simulated signals, showed a better impulsive response estimation and a higher resolution frequency estimate also in a highly nonstationary environment.

A subspace decomposition based algorithm for joint frequency of arrival (FOA) and direction of arrival (DOA) estimation is presented in this paper. The proposed method termed as QR-TLS ESPRIT, estimates the signal subspace by decomposing the received signal in a specially formed Toeplitz matrix. The structure of the matrix favors the application of QR factorization, which yields the signal subspace at the expense of less number of computations. A Total Least Squares (TLS) method is applied to obtain the frequency and angle estimates from the signal space. The proposed QR-TLS ESPRIT method provides several benefits: 1) It avoids the computations of applying computationally complex Eigen Value Decomposition (EVD) or Singular Value Decomposition (SVD) methods to obtain signal subspace alike conventional methods, 2) QR decomposition method, which is intrinsically applicable to numerical rank deficient problems, is used to extract signal subspace from the observation data matrix, 3) the TLS method is integrated with the QR which enhances the estimation accuracy in the presence of noisy measurements, 4) the proposed method can estimate the FOA and DOA from a small snapshot length of data which further reduces the computational time, and 5) the method works for both coherent as well as noncoherent sources without employing pre-whitening or spatial smoothing techniques. The performance of the proposed method is evaluated through simulations and the results are illustrated with respect to various received signal strength.

BackgroundThe response of many biomedical systems can be modelled using a linear combination of damped exponential functions. The approximation parameters, based on equally spaced samples, can be obtained using Prony’s method and its variants (e.g. the matrix pencil method). This paper provides a tutorial on the main polynomial Prony and matrix pencil methods and their implementation in MATLAB and analyses how they perform with synthetic and multifocal visual-evoked potential (mfVEP) signals.This paper briefly describes the theoretical basis of four polynomial Prony approximation methods: classic, least squares (LS), total least squares (TLS) and matrix pencil method (MPM). In each of these cases, implementation uses general MATLAB functions. The features of the various options are tested by approximating a set of synthetic mathematical functions and evaluating filtering performance in the Prony domain when applied to mfVEP signals to improve diagnosis of patients with multiple sclerosis (MS).ResultsThe code implemented does not achieve 100%-correct signal approximation and, of the methods tested, LS and MPM perform best. When filtering mfVEP records in the Prony domain, the value of the area under the receiver-operating-characteristic (ROC) curve is 0.7055 compared with 0.6538 obtained with the usual filtering method used for this type of signal (discrete Fourier transform low-pass filter with a cut-off frequency of 35 Hz).ConclusionsThis paper reviews Prony’s method in relation to signal filtering and approximation, provides the MATLAB code needed to implement the classic, LS, TLS and MPM methods, and tests their performance in biomedical signal filtering and function approximation. It emphasizes the importance of improving the computational methods used to implement the various methods described above.