Structured Total Least Squares Method Based on Variable Projection

Linear inverse problems with uncertain measurement matrices appear in many different applications. One of the standard techniques for solving such problems is the total least squares (TLS) method. Recently, an alternative approach has been suggested, based on maximizing an appropriate likelihood function assuming that the measurement matrix consists of random Gaussian variables. We refer to this technique as the total maximum likelihood (TML) method. Here we extend this strategy to the case in which the measurement matrix is structured so that the perturbations are not arbitrary but rather follow a fixed pattern. The resulting estimate is referred to as the structured TML (STML). As we show, the STML can be viewed as a regularized version of the structured TLS (STLS) approach in which the regularization consists of a logarithmic penalty. In contrast to the STLS solution, the STML always exists. Furthermore, its performance in practice tends to be superior to that of the STLS and competitive to other regularized solvers, as we illustrate via several examples. We also consider a few interesting special cases in which the STML can be computed efficiently either by reducing it into a one-dimensional problem regardless of the problem size or by a decomposition via a discrete Fourier transform.

Multiple delays estimation for chirp signals using structured total least squares

The theory of Baarda’s data snooping — normal and F tests respectively based on the known and unknown posteriori variance — is applied to detect blunders in errors-invariables (EIV) models, in which gross errors are in the vector of observations and/or in the coefficient matrix. This work is a follow-up to an earlier work in which we presented the formulation of the weighted total least squares (WTLS) based on the standard least squares theory. This method allows one to directly apply the existing body of knowledge of the least squares theory to the errors-in-variables models. Among those applications, data snooping methods in an EIV model are of particular interest, which is the subject of discussion in the present contribution. This paper generalizes the Baarda’s data snooping procedure of the standard least squares theory to an EIV model. Two empirical examples, a linear regression model and a 2-D affine transformation, using simulated and real data are presented to show the efficacy of the presented formulation. It is highlighted that the method presented is capable of detecting outlying equations (rather than outlying observations) in a straightforward manner. Further, the WTLS method can be used to handle different TLS problems. For example, the WTLS problem for the conditions and mixed models, the WTLS problem subject to constraints and variance component estimation for an EIV model can easily be established. These issues are in progress for future publications.

Coordinate transformation is the process of converting spatial data from a source coordinate to a target coordinate system. A set of control points, measured in the two coordinate systems, is used to estimate the transformation parameters. In general, more control points are measured, and the over-determined system is adjusted using the least squares method. However, the standard least squares method assumes that errors exist only in the measurements made at one coordinate system, or at the observation vector (y). This is not the case in many physical systems where errors exist in all the measurements made in both the source coordinate and the target coordinate systems. The Structured Total Least Norm (STLN) method is a relatively new mathematical concept developed to solve estimation problems of so-called Error-In-Variables (EIV) models. The method is specifically suitable for dealing with transformation problems, since it can handle the special structure of the data matrix (A). The STLN method is uniquely used to compute the parameters of common linear coordinate transformations (affine and similarity). A numerical example is presented to demonstrate the superiority of this technique in terms of accuracy and to compare the standard LS method, the generalized LS algorithm, and the STLN approach.

The weighted total least-squares (WTLS) estimate for the partial EIV (PEIV) model is very susceptible to outliers. In view of outliers in observations and coefficient matrix, a robust estimate of WTLS which makes using of iteratively reweighted technique (IRTLS) for the PEIV model is developed by combining a new two-step iterated algorithm of the WTLS estimate with the sensitivity-analysis based on the robust M-estimation. The uniformly most powerful test statistics are constructed to determine the down-weighting factor and the variance of unit weight is estimated by least median squares (LMS) method possessing high break-down point. Depended on the PEIV model, two different down-weighting schemes are presented. In the first scheme down-weighting is only implemented for the coefficient matrix and not for observations when the elements of the coefficient matrix are estimated, and the second scheme is contrary. The most attractive aspect is that the suggested computational formulae are the same with the traditional robust least-squares (LS) methods. The two-dimensional affine transformation and liner fitting example are analyzed, and some comparisons are performed with different available weight functions. The proposed approach (Scheme 1) proves to be a powerful tool in detecting outliers for the PEIV model.

In the image reconstruction of the electrical capacitance tomography (ECT) system, the application of the total least squares theory transforms the ill-posed problem into a nonlinear unconstrained minimization problem, which avoids calculating the matrix inversion. But in the iterative process of the coefficient matrix, the ill-posed problem is also produced. For the effect on the final image reconstruction accuracy of this problem, combined with the principle of the ECT system, the coefficient matrix is targeted and updated in the overall least squares iteration process. The new coefficient matrix is calculated, and then, the regularization matrix is corrected according to the adaptive targeting singular value, which can reduce the ill-posed effect. In this study, the total least squares iterative method is improved by introducing the mathematical model of EIV to deal with the errors in the measured capacitance data and coefficient matrix. The effect of noise interference on the measurement capacitance data is reduced, and finally, the high-quality reconstructed images are calculated iteratively.

Total least squares (TLS) is a technique that solves the traditional least squares (LS) problem for an errors-in-variables (EIV) model, in which both the observation vector and the design matrix are contaminated by random errors. Four- and seven-parameter models of coordinate transformation are typical EIV model. To determine which one of TLS and LS is more effective, taking the four- and seven-parameter models of Global Navigation Satellite System (GNSS) coordinate transformation with different coincidence pointsas examples, the relative effectiveness of the two methods was compared through simulation experiments. The results showed that in the EIV model, the errors-in-variables-only (EIVO) model and the errors-in-observations-only (EIOO) model, TLS is slightly inferior to LS in the four-parameter model coordinate transformation, and TLS is equivalent to LS in the seven-parameter model coordinate transformation. Consequently, in the four- and seven-parameter model coordinate transformation, TLS has no obvious advantage over LS.

Fast deconvolution with approximated PSF by RSTLS with antireflective boundary conditions

On the basis of the stream function field computed using a diagnostic model in the South China Sea in the summer 1998, the acoustic ray travel time tomography algorithm for reconstruction of two-dimensional flow field was tested by the numerical simulation in the region east to Vietnam. Error or noise contaminates the coefficients of both transfer matrix and measure data of the inverse equation of ocean acoustic tomography (OAT). Previous work on ill-posed problem in OAT commonly used Tikhonov regularization technology considering error only in observation data. Here, the truncated total least squares method (TTLS), which simultaneously considers error and noise on both sides, is applied for inverse solutions and its performances is compared with conventional Tikhonov method. The investigation suggests that the truncated total least squares method can substantially improve reconstruction of stream function field and is a promising alternative method for the solution of ocean acoustic tomography inverse problem.

In this paper, a new global camera pose estimation algorithm WTLS-IRLS is proposed, which can effectively solve the global rotation when there are outliers. Firstly, according to the relationship between the rotation vector and the rotation matrix, we simplify the product operation of the rotation matrix into the subtraction operation of the rotation vector, which reduces the complexity of the algorithm. Secondly, the weighted total least squares (WTLS) and the iteratively reweighted least squares (IRLS) are used to average relative rotations. As the initialization of IRLS, WTLS provides a good initial guess by correcting the linearization equation and adding weight information to the relative rotations. IRLS continues to add weight information to the relative rotation matrices to optimize the global rotations. We demonstrate the performance of our approach by a number of large-scale data sets, the results show that our method has been greatly improved in efficiency, accuracy and iteration. In order to verify the correctness of our proposed method, we completed the complete reconstruction process, the experimental results show that our proposed WTLS-IRLS rotation averaging algorithm can obtain dense point clouds with more three-dimensional points.

The paper addresses the identification of time-invariant bilinear system (BS) models in the errors-in-variables (EIV) framework. The proposed scheme is based on the structured total least norm (STLN) technique extended here to handle BS. The performance of the presented approach, i.e. the bilinear STLN (BSTLN) with its further extension incorporating the Tikhonov regularization is compared to several other EIV identification techniques via an extensive Monte-Carlo simulation study. The results obtained demonstrate a considerable noise robustness and therefore the applicability of the BSTLN algorithm in the EIV framework.

A novel interference alignment (IA) scheme based on the errors-in-variables (EIV) mathematic model has been proposed to overcome the channel state information (CSI) estimation error for the MIMO interference channels. By solving an equivalently unconstrained optimization problem, the proposed IA scheme employing a weighted total least squares (WTLS) algorithm can obtain the solution to a constrained optimization problem for transmit precoding (TPC) matrices and minimizes the distortion caused by imperfect CSI according to the EIV model. It is shown that the design of TPC matrices can be realized through an efficient iterative algorithm. The convergence of the proposed scheme is presented as well. Simulation results show that the proposed IA scheme is robust over MIMO interference channels with imperfect CSI, which yields significantly better sum rate performance than the existing IA schemes such as distributed iterative IA, maximum signal-to-interference-plus-noise ratio (Max SINR), and minimum mean square error (MMSE) schemes.

Although the analytical solutions for total least-squares with multiple linear and single quadratic constraints were developed quite recently in different geodetic publications, these methods are restricted in number and type of constraints, and currently their computational efficiency and applications are mostly unknown. In this contribution, it is shown how the weighted total least-squares (WTLS) problem with arbitrary applicable constraints can be solved based on a Newton type methodology. This iterative process with quadratic convergence is expanded upon to become a compact solution for the WTLS with or without constraints. This compact solution is then further interpreted as a universal formula for the symmetrical adjustment of the errors-in-variables model which represents affine, similarity and rigid transformations in two- and three-dimensional space. Furthermore, statistical analysis of the constrained WTLS including the first-order approximation of precision and the bias was investigated. In order to substantiate our proposed method’s applicability, it was used to solve the affine, similarity and rigid transformation problem in two- and three-dimensional cases, where the structure of the coefficient matrix and multiple constraints were taken into account simultaneously.

The problem of inconsistency between constraints often arises in practice as the result, among others, of the complexity of real models or due to unrealistic requirements and preferences. To overcome such inconsistency two major actions may be taken: removal of constraints or changes in the coefficients of the model. This last approach, that can be generically described as "model correction" is the problem we address in this paper in the context of linear constraints over the reals. The correction of the right hand side alone, which is very close to a fuzzy constraints approach, was one of the first proposals to deal with inconsistency, as it may be mapped into a linear problem. The correction of both the matrix of coefficients and the right hand side introduces non linearity in the constraints. The degree of difficulty in solving the problem of the optimal correction depends on the objective function, whose purpose is to measure the closeness between the original and corrected model. Contrary to other norms, that provide corrections with quite rigid patterns, the optimization of the important Frobenius norm was still an open problem. We have analyzed the problem using the KKT conditions and derived necessary and sufficient conditions which enabled us to unequivocally characterize local optima, in terms of the solution of the Total Least Squares and the set of active constraints. These conditions justify a set of pruning rules, which proved, in preliminary experimental results, quite successful in a tree search procedure for determining the global minimizer.

The registration of 3-D point clouds is an important procedure during the terrestrial laser scanning data processing. Recently, due to their high flexibility and the powerful mathematical model, a large amount of least-squares-based (LSs-based) methods are proposed and widely applied to estimate the transformation parameters of 3-D point clouds registration. In these LSs-based methods some based on the generalized Gauss–Markov model do not correct the influence of random errors on source 3-D point clouds. Although there are other methods based on the errors-in-variables (EIV) model, they are inapplicable for transformation problems with large rotation angles and arbitrary scale ratio. In addition, the gross errors are usually ignored in previous studies on 3-D point clouds registration, which, however, exists commonly and could distort the registration severely. Aiming to avoid the influence of gross errors and extend its application, an advanced outlier detected total least-squares (OD-TLS) method is proposed in this paper. Based on the generalized EIV model OD-TLS performs a seven-parameter 3-D similarity transformation with large rotation angles and arbitrary scale ratio. The random errors of both source and target 3-D point clouds are considered. Furthermore, outliers are detected and removed automatically by combining the data snooping method with total least-squares (TLS) estimation. In order to indicate the benefits of OD-TLS, comparative experiments with the LS3D and weighted total least squares (WTLS) on synthetic and real-world scanned 3-D point clouds were performed. The experimental results show OD-TLS not only enhances the registration accuracy but also increases its robustness.