Maximum likelihood estimation in Gaussian models under total positivity

We pose the estimation of the parameters of multiple superimposed exponential signals in additive Gaussian noise problem as a maximum likelihood (ML) estimation problem. The ML problem is non linear and hard to solve. Some previous works focused on finding alternative estimation procedures, for example by denoising. In contrast, we tackle the ML estimation problem directly. First, we use the same transformation as the first step of iterative quadratic maximum likelihood (IQML) and transform the ML problem into another optimization problem that gets rid of the amplitude coefficients. Second, we solve the remaining optimization problem with a gradient descent approach ("pseudo-quadratic maximum likelihood"). We also use this algorithm for ultra-wideband channel estimation and estimate ranging in non-line of sight environment.

The problem of distributed parameter estimation from binary quantized observations is studied when the unquantized observations are corrupted by combined multiplicative and additive Gaussian noise. These results are applicable to sensor networks where the sensors observe a parameter in combined additive and nonadditive noise and to a case where dispersed receivers are employed with analog communication over fading channels where the receivers employ binary quantization before noise-free digital communications to a fusion center. We first discuss the case in which all the quantizers use an identical threshold. The parameter identifiability condition is given, and, surprisingly, it is shown that unless the common threshold is chosen properly and the parameter lies in an open interval, the parameter will not generally be identifiable, in contrast to the additive noise case. The best achievable mean square error (MSE) performance is characterized by deriving the corresponding Cramer-Rao Lower Bound (CRLB). A closed-form expression describing the corresponding maximum likelihood (ML) estimator is presented. The stability of the performance of the ML estimators is improved when a nonidentical threshold strategy is utilized to estimate the unknown parameter. The thresholds are designed by maximizing the minimum asymptotic relative efficiency (ARE) between quantized and unquantized ML estimators. Although the ML estimation problem is nonconvex, it is shown that one can relax the optimization to make it convex. The solution to the relaxed problem is used as an initial solution in a gradient algorithm to solve the original problem. Next, the case where both the variances of the additive noise and multiplicative noise are unknown is studied. The corresponding CRLB is obtained, and the ML estimation problem is transformed to a convex optimization, which can be solved efficiently. Finally, numerical simulations are performed to substantiate the theoretical analysis.

We consider the problem of single source localization using time-difference-of-arrival (TDOA) measurements. By analyzing the maximum-likelihood (ML) formulation of the problem, we show that under certain mild assumptions on the measurement noise, the estimation errors of both the closed-form least-squares estimate proposed in [1] and the ML estimate, as measured by their distances to the true source location, are of the same order. We then use this to establish the curious result that the objective function of the ML estimation problem is actually locally strongly convex at an optimal solution. This implies that some lightweight solution methods, such as the gradient descent (GD) and Levenberg-Marquardt (LM) methods, will converge to an optimal solution to the ML estimation problem when properly initialized, and the convergence rates can be determined by standard arguments. To the best of our knowledge, these results are new and contribute to the growing literature on the effectiveness of lightweight solution methods for structured non-convex optimization problems. Lastly, we demonstrate via simulations that the GD and LM methods can indeed produce more accurate estimates of the source location than some existing methods, including the widely used semidefinite relaxation-based methods.

The oscillator phase hit (PH) results in a temporary link loss in the communication system and bears an expensive cost on the operator companies. In this work, we propose a double-stage solution to mitigate the PH. In the first stage, we use Neyman-Pearson binary hypothesis testing to develop a low-cost PH detection algorithm, in which a likelihood ratio test is designed, and the optimal detection threshold is analytically calculated. The proposed PH detection scheme entails small real-time computations while using the existing pilot symbols in the system. Hence, no extra pilot overhead is required. In the second stage, the maximum likelihood (ML) estimation is used to develop a PH correction scheme. In particular, the phase noise and the PH are jointly mitigated by solving an ML estimation problem. By applying the proposed correction scheme, the number of remaining erroneous symbols due to PH (if any) is within the error correction capability of modern forward error-correcting codes. Numerical results verify the effectiveness of the proposed scheme in detecting and correcting the PH.

We consider the problem of maximum likelihood estimation of logistic sinusoidal regression models and develop some asymptotic theory including the consistency and joint rates of convergence for the maximum likelihood estimators. The key techniques build upon a synthesis of the results of Walker and Song and Li for the widely studied sinusoidal regression model and on making a connection to a result of Radchenko. Monte Carlo simulations are also presented to demonstrate the finite-sample performance of the estimators

A rectangular matrix is called {\it totally positive} if all its minors are positive. A point of a real Grassmanian manifold $G_{l,m}$ of $l$-dimensional subspaces in $\mathbb R^m$ is called {\it strictly totally positive} if one can normalize its Pl\ucker coordinates to make all of them positive. Clearly if a $k\times m$-matrix, $k<m$, is totally positive, then each collection of its $l\leq k$ raws generates an $l$-subspace represented by a strictly totally positive point of the Grassmanian manifold $G_{l,m}$. The totally positive matrices and the subsets of strictly totally positive points in Grassmanian manifolds arise in many domains of mathematics, mechanics and physics. F.R.Gantmacher and M.G.Krein considered totally positive matrices in the context of classical mechanics. As it was shown in a joint paper by M.Boiti, F.Pemperini and A.Pogrebkov, each matrix of appropriate dimension with positive minors of higher dimension generates a multisoliton solution of the Kadomtsev-Petviashvili (KP) partial differential equation. There exist several approaches of construction of totally positive matrices due to F.R. Gantmacher, M.G.Krein, A.E.Postnikov and ourselves. In our previous paper we have proved that certain determinants formed by modified Bessel functions of the first type are positive on the positive semi-axis. This yields a one-dimensional family of totally positive points in all the Grassmanian manifolds. In the present paper we provide a construction of multidimensional families of totally positive points in all the Grassmanian manifolds, again using modified Bessel functions of the first type but different from the above-mentioned construction. These families represent images of explicit injective mappings of the convex open subset $\{ x=(x_1,\dots,x_l)\in\mathbb R_+^l | x_1<\dots<x_l\}\subset\mathbb R^l$ to the Grassmanian manifolds $G_{l,m}$, $l<m$.

The focus of this paper is to develop a framework for distributed estimation via convex optimization. We deal with a network of complex sensor subsystems with local estimation and signal processing. More specifically, the sensor subsystems locally solve a maximum likelihood (or maximum a posteriori probability) estimation problem by maximizing a (strictly) concave log-likelihood function subject to convex constraints. These local implementations are not revealed outside the subsystem. The subsystems interact with one another via convex coupling constraints. We discuss a distributed estimation scheme to fuse the local subsystem estimates into a globally optimal estimate that satisfies the coupling constraints. The approach uses dual decomposition techniques in combination with the subgradient method to develop a simple distributed estimation algorithm. Many existing methods of data fusion are suboptimal, i.e., they do not maximize the log-likelihood exactly but rather `fuse' partial results from many processors. For linear gaussian formulation, least mean square (LMS) consensus provides optimal (maximum likelihood) solution. The main contribution of this work is to provide a new approach for data fusion which is based on distributed convex optimization. It applies to a class of problems, described by concave log-likelihood functions, which is much broader than the LMS consensus setup.

The problem of maximum likelihood estimation of the parameters of Mallows’ ranking model based on partial rankings (with a fixed number of tie groups) has been approached by Beckett (1992), applying the EM algorithm to estimate both the center and the scale parameter. This paper offers an alternative procedure for maximum likelihood estimation without relying on the EM algorithm:the center is estimated by minimizing the sum of the distances between the center and the observed partial rankings, and the scale parameter is estimated by solving a relatively simple equation.

This study describes the basic framework of EM algorithm and gives how to apply EM algorithm to solve the problem of maximum-likelihood parameters estimation combining with the models of HMM and PCFG. In the process of statistics natural language, one kind of problem is often encountered that is how to solve the parameter's maximum-likelihood estimation when observation data is incomplete. EM algorithm is the classical method to solve this problem. Finally, the advantages and disadvantages of EM algorithm are discussed.

Recent work has shown that the resonate-and-fire neuronal model is both computationally efficient and suitable for large network simulations. In this paper, we examine the estimation problem of a resonate-and-fire neuronal model with stochastic firing threshold. The model parameters are divided into two sets. The first set is associated with the subthreshold behaviour and can be estimated by a least squares algorithm, while the second set includes parameters associated with the firing threshold and its identification is formulated as a maximum likelihood estimation problem. The latter is in turn solved by a simulated annealing approach that avoids local optima. The proposed identification approach is evaluated using both simulated and in-vitro data, which shows a good match between prediction by identified model and the actual data, concluding the efficiency and accuracy of the proposed approach.

This paper proposes a method for obtaining nonlinear models of distribution grid based on available measurements from the power grid. We formulate a maximum likelihood estimation (MLE) problem that estimates unknown line parameters—specifically, the impedance between nodes—using measured voltage magnitudes and net active and reactive power injections at each node. The nonlinear model for the distribution grid uses a nonlinear approximation of the DistFlow model, which includes line losses and is parameterized by the unknown line impedances. We solve the resulting MLE problem using an expectation maximization (EM) algorithm, tailored for the nonlinear model, and provide a numerically robust implementation. The proposed method is demonstrated on the IEEE 37-node test network, and we compare it with the state-of-the-art methods. The proposed method achieves a 70% reduction in voltage error and an error for state variables that is more than 10000 times smaller. A final comparison uses data from a real network, and the proposed method achieves parameter estimates with errors 100 times smaller than competing methods.

Total positivity and Toda flow

This paper considers a massive random access problem in which a large number of sporadically active devices wish to communicate with a base station (BS) equipped with massive multiple-input multiple-output (MIMO) antennas. Each device is preassigned a unique signature sequence, and the BS identifies the active devices by detecting which sequences are transmitted. This device activity detection problem can be formulated as a maximum likelihood estimation (MLE) problem for which the sample covariance matrix of the received signal is a sufficient statistic. The goal of this paper is to characterize the feasible set of problem parameters under which this covariance based approach is able to successfully recover the device activities in the massive MIMO regime. Through an analysis of the asymptotic behaviors of MLE via its associated Fisher information matrix, this paper derives a necessary and sufficient condition on the Fisher information matrix to ensure a vanishing probability of detection error as the number of antennas goes to infinity, based on which a numerical phase transition analysis is obtained. This condition is also examined from a perspective of covariance matching, which relates the phase transition analysis to a recently derived scaling law. Further, we provide a characterization of the distribution of the estimation error in MLE, based on which the error probabilities in device activity detection can be accurately predicted. Finally, this paper studies a random access scheme with joint device activity and data detection and analyzes its performance in a similar way.

Acoustic source localization using multiple microphones can be formulated as a maximum likelihood estimation problem. The estimator is implicitly defined as the minimum of a certain objective function. As a result, we cannot get explicit expressions for the mean and the covariance of the estimator. We derive approximate expressions for the mean vector and covariance matrix of the estimator using Taylor's series expansion of the implicitly defined estimator. The validity of our expressions is verified by Monte-Carlo simulations. We also study the performance of the estimator for different microphone array configurations.

In an effort to develop topic modeling methods that can be quickly applied to large data sets, we revisit the problem of maximum-likelihood estimation in topic models. It is known, at least informally, that maximum-likelihood estimation in topic models is closely related to non-negative matrix factorization (NMF). Yet, to our knowledge, this relationship has not been exploited previously to fit topic models. We show that recent advances in NMF optimization methods can be leveraged to fit topic models very efficiently, often resulting in much better fits and in less time than existing algorithms for topic models. We also formally make the connection between the NMF optimization problem and maximum-likelihood estimation for the topic model, and using this result we show that the expectation maximization (EM) algorithm for the topic model is essentially the same as the classic multiplicative updates for NMF. Our methods are implemented in the R package "fastTopics".