Linear observation based total least squares

Observation systems known as errors-in-variables (EIV) models with model parameters estimated by total least squares (TLS) have been discussed for more than a century, though the terms EIV and TLS were coined much more recently. So far, it has only been shown that the inequality-constrained TLS (ICTLS) solution can be obtained by the combinatorial methods, assuming that the weight matrices of observations involved in the data vector and the data matrix are identity matrices. Although the previous works test all combinations of active sets or solution schemes in a clear way, some aspects have received little or no attention such as admissible weights, solution characteristics and numerical efficiency. Therefore, the aim of this study was to adjust the EIV model, subject to linear inequality constraints. In particular, (1) This work deals with a symmetrical positive-definite cofactor matrix that could otherwise be quite arbitrary. It also considers cross-correlations between cofactor matrices for the random coefficient matrix and the random observation vector. (2) From a theoretical perspective, we present first-order Karush–Kuhn–Tucker (KKT) necessary conditions and the second-order sufficient conditions of the inequality-constrained weighted TLS (ICWTLS) solution by analytical formulation. (3) From a numerical perspective, an active set method without combinatorial tests as well as a method based on sequential quadratic programming (SQP) is established. By way of applications, computational costs of the proposed algorithms are shown to be significantly lower than the currently existing ICTLS methods. It is also shown that the proposed methods can treat the ICWTLS problem in the case of more general weight matrices. Finally, we study the ICWTLS solution in terms of non-convex weighted TLS contours from a geometrical perspective.

The errors-in-variables (EIV) model is a nonlinear model, the parameters of which can be solved by singular value decomposition (SVD) method or the general iterative algorithm. The existing formulae for covariance matrix of total least squares (TLS) parameter estimates don't fully consider the randomness of quantities in iterative algorithm and the biases of parameter estimates and residuals. In order to reflect more reasonable precision information for TLS adjustment, the derivative-free unscented transformation with scaled symmetric sampling strategy, i.e. scaled unscented transformation (SUT), is introduced and implemented. In this contribution, we firstly discuss the existing various solutions of TLS adjustment and covariance matrices of TLS parameter estimates and derive the general first-order approximate cofactor matrices of random quantities in TLS adjustment. Secondly, based on the combination of TLS iterative algorithm and calculation process of SUT, we design the two SUT algorithms to calculate the biases and the second-order approximate covariance matrices. Finally, the straight line fitting model and plane coordinate transformation model are used to demonstrate that applying SUT for precision estimation of TLS adjustment is feasible and effective.

Bilinear equation method for unbiased identification of linear FIR systems in the presence of input and output noises

TLS Method for Pulse Transfer Function Estimation in the Presence of Input and Output Noise

Total least squares (TLS) is a technique that solves the traditional least squares (LS) problem for an errors-in-variables (EIV) model, in which both the observation vector and the design matrix are contaminated by random errors. Four- and seven-parameter models of coordinate transformation are typical EIV model. To determine which one of TLS and LS is more effective, taking the four- and seven-parameter models of Global Navigation Satellite System (GNSS) coordinate transformation with different coincidence pointsas examples, the relative effectiveness of the two methods was compared through simulation experiments. The results showed that in the EIV model, the errors-in-variables-only (EIVO) model and the errors-in-observations-only (EIOO) model, TLS is slightly inferior to LS in the four-parameter model coordinate transformation, and TLS is equivalent to LS in the seven-parameter model coordinate transformation. Consequently, in the four- and seven-parameter model coordinate transformation, TLS has no obvious advantage over LS.

In recent years, there has been a growing interest within the potential user community in applying GPS to a variety of high-precision navigation problems such as aircraft nonprecision approach, river and harbour navigation, real-time or kinematic surveying, etc. In view of more and more GPS applications, the reliability of GPS is at issue. This promotes the need for several proposed receiver autonomous integrity monitoring (RAIM), GPS integrity channel (GIC), and wide area augmentation system (WAAS) schemes. The authors propose an approach for autonomous GPS satellite failure detection. The method exploits total least squares (TLS) optimisation to achieve fault detection. The traditional least squares approach fails to address the change in the data matrix associated with GPS satellite constellation. This drawback is shown to be remedied by the total least squares method. The proposed method can further be extended to account for failure isolation in which failed GPS satellites are identified and hence excluded in positioning.

The registration of 3-D point clouds is an important procedure during the terrestrial laser scanning data processing. Recently, due to their high flexibility and the powerful mathematical model, a large amount of least-squares-based (LSs-based) methods are proposed and widely applied to estimate the transformation parameters of 3-D point clouds registration. In these LSs-based methods some based on the generalized Gauss–Markov model do not correct the influence of random errors on source 3-D point clouds. Although there are other methods based on the errors-in-variables (EIV) model, they are inapplicable for transformation problems with large rotation angles and arbitrary scale ratio. In addition, the gross errors are usually ignored in previous studies on 3-D point clouds registration, which, however, exists commonly and could distort the registration severely. Aiming to avoid the influence of gross errors and extend its application, an advanced outlier detected total least-squares (OD-TLS) method is proposed in this paper. Based on the generalized EIV model OD-TLS performs a seven-parameter 3-D similarity transformation with large rotation angles and arbitrary scale ratio. The random errors of both source and target 3-D point clouds are considered. Furthermore, outliers are detected and removed automatically by combining the data snooping method with total least-squares (TLS) estimation. In order to indicate the benefits of OD-TLS, comparative experiments with the LS3D and weighted total least squares (WTLS) on synthetic and real-world scanned 3-D point clouds were performed. The experimental results show OD-TLS not only enhances the registration accuracy but also increases its robustness.

The classical Least-Squares (LS) adjustment has been widely used in processing and analysing observations from Global Satellite Navigation System (GNSS). However, in detecting temporal correlations of GNSS observations, which can be described by means of autoregressive (AR) process, the LS method may not provide reliable estimates of process coefficients, since the Yule-Walker (YW) equations refer to structured Errors-In-Variables (EIV) equations. In this contribution, we proposed a Total Least-Squares (TLS) solution with the singular cofactor matrix to solve the YW equations. The proposed TLS solution is obtained based on the fact that random errors belong to column space of its cofactor matrix. In addition the proposed solution does not need any substitution of the squared true parameter vector as done by the current publications. Finally, we simulate the AR process to prove that our solution is more reliable than the existing methods.

Evaluating methods for estimating process and observation error variances in statistical catch-at-age analysis

The resolution of the closely spaced sinusoids becomes poor as the SNR of the received signal reduces. This resolution can be improved by using the Total Least Squares (TLS) method in solving the linear prediction (LP) equation. This approach makes use of the singular value decomposition (SVD) of the augmented matrix to reduce the noise effect from both the observation vector and the LP data matrix. At low SNR, the performance of this frequency estimator is found to be better than the frequency estimator using the Principal Eigenvector (PE) method which is proposed by Tufts and Kumaresan. They used the SVD of the LP data matrix to reduce the noise effect from the data matrix only.

Coordinate transformation is the process of converting spatial data from a source coordinate to a target coordinate system. A set of control points, measured in the two coordinate systems, is used to estimate the transformation parameters. In general, more control points are measured, and the over-determined system is adjusted using the least squares method. However, the standard least squares method assumes that errors exist only in the measurements made at one coordinate system, or at the observation vector (y). This is not the case in many physical systems where errors exist in all the measurements made in both the source coordinate and the target coordinate systems. The Structured Total Least Norm (STLN) method is a relatively new mathematical concept developed to solve estimation problems of so-called Error-In-Variables (EIV) models. The method is specifically suitable for dealing with transformation problems, since it can handle the special structure of the data matrix (A). The STLN method is uniquely used to compute the parameters of common linear coordinate transformations (affine and similarity). A numerical example is presented to demonstrate the superiority of this technique in terms of accuracy and to compare the standard LS method, the generalized LS algorithm, and the STLN approach.

A common task in the analysis of multi‐environmental trials (MET) by linear mixed models (LMM) is the estimation of variance components (VCs). Most often, MET data are imbalanced (e.g., due to selection). The imbalance mechanism can be missing completely at random (MCAR), missing at random (MAR), or missing not at random. If the missing‐data pattern in MET was caused by selection, it is usually MAR. In this case, likelihood‐based methods are the preferred methods for analysis as they can account for a MAR data pattern. Likelihood‐based methods used to estimate VCs in LMM have the property that all VC estimates are constrained to be non‐negative; thus, the estimators are generally biased. Therefore, there are two potential causes of bias in MET analysis: a data pattern not being MCAR, and the bias of likelihood‐based VC estimators. The current study tries to dissect and quantify both possible sources of bias. A simulation study with MET data typical of cultivar evaluation trials was conducted in which the missing data pattern and the size of VCs were varied. The results showed that for the simulated MET, bias in VC estimates was similar under MCAR and MAR. Thus, the bias is solely due to the likelihood‐based estimation. Bias increases when increasing the ratio of genotype variance to error variance is small. Bias was similar for MAR and MCAR data patterns. Thus, it may be concluded that selection does not increase bias in VC estimation.

A graph subspace approach to system identification based on errors-in-variables system models

The traditional method for estimating the linear function of fixed parameters in mixed linear model is a two‐stage procedure. In the first stage of this procedure the variance components estimators are calculated and next in the second stage these estimators are taken as true values of variance components to estimating the linear function of fixed parameters according to generalized least squares method.In this paper the general mixed linear model is considered in which a matrix related to fixed parameters and or/a dispersion matrix of observation vector may be deficient in rank. It is shown that the estimators of a set of functions of fixed parameters obtained in second stage are unbiased if only the observation vector is symmetrically distributed about its expected value and the estimators of variance components from first stage are translation‐invariant and are even functions of the observation vector.

Estimation of density dependence from time series data on population abundance is hampered in the presence of observation or measurement errors. Fitting state-space models has been proposed as a solution that reduces the bias in estimates of density dependence caused by ignoring observation errors. While this is often true, I show that, for specific parameter values, there are identifiability issues in the linear state-space model when the strength of density dependence and the observation and process error variances are all unknown. Using simulation to explore properties of the estimators, I illustrate that, unless assumptions are imposed on the process or observation error variances, the variance of the estimator of density dependence varies critically with the strength of the density dependence. Under compensatory dynamics, the stronger the density dependence the more difficult it is to estimate in the presence of observation errors. The identifiability issues disappear when density dependence is estimated from the state-space model with the observation error variance known to the correct value. Direct estimates of observation variance in abundance censuses could therefore prove helpful in estimating density dependence but care needs to be taken to assess the uncertainty in variance estimates.