Maximum Likelihood Estimation of Logistic Sinusoidal Regression Models

We pose the estimation of the parameters of multiple superimposed exponential signals in additive Gaussian noise problem as a maximum likelihood (ML) estimation problem. The ML problem is non linear and hard to solve. Some previous works focused on finding alternative estimation procedures, for example by denoising. In contrast, we tackle the ML estimation problem directly. First, we use the same transformation as the first step of iterative quadratic maximum likelihood (IQML) and transform the ML problem into another optimization problem that gets rid of the amplitude coefficients. Second, we solve the remaining optimization problem with a gradient descent approach ("pseudo-quadratic maximum likelihood"). We also use this algorithm for ultra-wideband channel estimation and estimate ranging in non-line of sight environment.

The problem of distributed parameter estimation from binary quantized observations is studied when the unquantized observations are corrupted by combined multiplicative and additive Gaussian noise. These results are applicable to sensor networks where the sensors observe a parameter in combined additive and nonadditive noise and to a case where dispersed receivers are employed with analog communication over fading channels where the receivers employ binary quantization before noise-free digital communications to a fusion center. We first discuss the case in which all the quantizers use an identical threshold. The parameter identifiability condition is given, and, surprisingly, it is shown that unless the common threshold is chosen properly and the parameter lies in an open interval, the parameter will not generally be identifiable, in contrast to the additive noise case. The best achievable mean square error (MSE) performance is characterized by deriving the corresponding Cramer-Rao Lower Bound (CRLB). A closed-form expression describing the corresponding maximum likelihood (ML) estimator is presented. The stability of the performance of the ML estimators is improved when a nonidentical threshold strategy is utilized to estimate the unknown parameter. The thresholds are designed by maximizing the minimum asymptotic relative efficiency (ARE) between quantized and unquantized ML estimators. Although the ML estimation problem is nonconvex, it is shown that one can relax the optimization to make it convex. The solution to the relaxed problem is used as an initial solution in a gradient algorithm to solve the original problem. Next, the case where both the variances of the additive noise and multiplicative noise are unknown is studied. The corresponding CRLB is obtained, and the ML estimation problem is transformed to a convex optimization, which can be solved efficiently. Finally, numerical simulations are performed to substantiate the theoretical analysis.

We analyze the problem of maximum likelihood estimation for Gaussian distributions that are multivariate totally positive of order two ($\mathrm{MTP}_{2}$). By exploiting connections to phylogenetics and single-linkage clustering, we give a simple proof that the maximum likelihood estimator (MLE) for such distributions exists based on $n\geq2$ observations, irrespective of the underlying dimension. Slawski and Hein [Linear Algebra Appl. 473 (2015) 145–179], who first proved this result, also provided empirical evidence showing that the $\mathrm{MTP}_{2}$ constraint serves as an implicit regularizer and leads to sparsity in the estimated inverse covariance matrix, determining what we name the ML graph. We show that we can find an upper bound for the ML graph by adding edges corresponding to correlations in excess of those explained by the maximum weight spanning forest of the correlation matrix. Moreover, we provide globally convergent coordinate descent algorithms for calculating the MLE under the $\mathrm{MTP}_{2}$ constraint which are structurally similar to iterative proportional scaling. We conclude the paper with a discussion of signed $\mathrm{MTP}_{2}$ distributions.

We consider the problem of single source localization using time-difference-of-arrival (TDOA) measurements. By analyzing the maximum-likelihood (ML) formulation of the problem, we show that under certain mild assumptions on the measurement noise, the estimation errors of both the closed-form least-squares estimate proposed in [1] and the ML estimate, as measured by their distances to the true source location, are of the same order. We then use this to establish the curious result that the objective function of the ML estimation problem is actually locally strongly convex at an optimal solution. This implies that some lightweight solution methods, such as the gradient descent (GD) and Levenberg-Marquardt (LM) methods, will converge to an optimal solution to the ML estimation problem when properly initialized, and the convergence rates can be determined by standard arguments. To the best of our knowledge, these results are new and contribute to the growing literature on the effectiveness of lightweight solution methods for structured non-convex optimization problems. Lastly, we demonstrate via simulations that the GD and LM methods can indeed produce more accurate estimates of the source location than some existing methods, including the widely used semidefinite relaxation-based methods.

Multicollinearity in logistic semiparametric regression models inflates the variance of maximum likelihood estimators, leading to unreliable parameter estimates. While these models offer flexibility by combining parametric and nonparametric components, they are sensitive to multicollinearity. To address this, we propose a restricted ridge estimator for logistic semiparametric regression models with exact linear restrictions, extending the method introduced by Asar (2017) for standard logistic regression. This approach effectively reduces variance and provides more stable estimates. Additionally, we use generalized cross-validation (GCV) to select the optimal ridge penalty and kernel bandwidth, ensuring a balanced bias-variance tradeoff for both components. We validate the proposed method through simulation and real-data applications. The simulation results show that the restricted ridge estimator outperforms traditional maximum likelihood estimation (MLE) in terms of stability and accuracy under multicollinearity. The real-data application demonstrates its practical advantages in producing reliable and interpretable estimates.

The binary logistic regression model (LRM) is practical in situations when the response variable (RV) is dichotomous. The maximum likelihood estimator (MLE) is generally considered to estimate the LRM parameters. However, in the presence of multicollinearity (MC), the MLE is not the correct choice due to its inflated standard deviation (SD) and standard errors (SE) of the estimates. To combat MC, commonly used biased estimators, i.e., the Ridge estimators (RE) and Liu estimators (LEs), are preferred. However, most of the time, the traditional LE attains a negative value for its Liu parameter (LP), which is considered to be a major drawback. Therefore, to overcome this issue, we proposed a new adjusted LE for the binary LRM. Owing to numerical evaluation purposes, Monte Carlo simulation (MCS) study is performed under different conditions where bias and mean squared error are the performance criteria. Findings showed the superiority of our proposed estimator in comparison with the other estimation methods due to the existence of high but imperfect multicollinearity, which clearly means that it is consistent when the regressors are multicollinear. Furthermore, the findings demonstrated that whenever there is MC, the MLE is not the best choice. Finally, a real application is being considered to be evidence for the advantage of the intended estimator. The MCS and the application findings pointed out that the considered adjusted LE for the binary logistic regression model is a more efficient estimation method whenever the regressors are highly multicollinear.

The oscillator phase hit (PH) results in a temporary link loss in the communication system and bears an expensive cost on the operator companies. In this work, we propose a double-stage solution to mitigate the PH. In the first stage, we use Neyman-Pearson binary hypothesis testing to develop a low-cost PH detection algorithm, in which a likelihood ratio test is designed, and the optimal detection threshold is analytically calculated. The proposed PH detection scheme entails small real-time computations while using the existing pilot symbols in the system. Hence, no extra pilot overhead is required. In the second stage, the maximum likelihood (ML) estimation is used to develop a PH correction scheme. In particular, the phase noise and the PH are jointly mitigated by solving an ML estimation problem. By applying the proposed correction scheme, the number of remaining erroneous symbols due to PH (if any) is within the error correction capability of modern forward error-correcting codes. Numerical results verify the effectiveness of the proposed scheme in detecting and correcting the PH.

The logistic regression model is very commonly used in real life problems as a classifier for binary data. However, it is known the lack of robustness of the maximum likelihood estimator for the model, and therefore robust estimators must be developed so as to deal with contaminated data. In this paper, we consider the modified median estimator for the logistic regression model and we prove it consistency under general conditions.

Binary logistic semiparametric regression analysis is a commonly used statistical technique when the dependent variable is dichotomous or binary. In this analysis, the relationship between the success probability and certain explanatory variables is assumed to have a linear form, while the relationship to other variables is unknown. Multicollinearity is a serious problem that arises when explanatory variables in logistic semiparametric regression are highly correlated. It is well known that the variance of the maximum likelihood estimator is inflated due to multicollinearity in the semiparametric logistic regression model. Therefore, a novel stochastic restricted iterative weighted ridge estimator for logistic semiparametric regression is introduced, and its statistical properties are extracted asymptotically. Moreover, an extension of the generalized cross validation (GCV) function is introduced and applied for choosing the best values of the ridge parameter and the bandwidth of the kernel smoother. Additionally, some theorems are developed to illustrate the convergence of the GCV mean. Ultimately, the Monte-Carlo simulation studies and an actual real-life data set are conducted to support our theoretical discussion, and the findings indicated that the new estimator outperformed the other estimators under consideration.

A cost-minimizing diagnostic methodology for discrimination between neoplastic and non-neoplastic brain lesions: utilizing a genetic algorithm 1 scientific reports

In this work, we addressed parameter estimation and prediction in the high-dimensional sparse logistic regression model through both Monte Carlo simulations and application to real data. We applied two well-known penalized maximum likelihood (ML) methods (LASSO and aLASSO) for variable screening. There may exist overfitting from LASSO or underfitting from aLASSO, making ML estimators based on these models inefficient. Hence, after performing variable selection, we proposed post-selection improved estimation based on linear shrinkage, pretest, and James-Stein shrinkage strategies, which efficiently combine overfitted and underfitted ML estimators. Regardless of the correctness in the variable selection stage, the proposed estimators were shown to be more efficient than the classical ML estimators, which were severely affected by inappropriate variable selection.

Acoustic source localization using multiple microphones can be formulated as a maximum likelihood estimation problem. The estimator is implicitly defined as the minimum of a certain objective function. As a result, we cannot get explicit expressions for the mean and the covariance of the estimator. We derive approximate expressions for the mean vector and covariance matrix of the estimator using Taylor's series expansion of the implicitly defined estimator. The validity of our expressions is verified by Monte-Carlo simulations. We also study the performance of the estimator for different microphone array configurations.

The breakdown behavior of the maximum likelihood estimator in the logistic regression model

Distribution mixtures are used as models to analyze grouped data. The estimation of parameters is an important step for mixture distributions. The latent class model is generally used as the analysis of mixture distributions for discrete data. In this paper, we consider the parameter estimation for a mixture of logistic regression models. We know that the expectation maximization (EM) algorithm was most used for estimating the parameters of logistic regression mixture models. In this paper, we propose a new type of fuzzy class model and then derive an algorithm for the parameter estimation of a fuzzy class logistic regression model. The effects of the explanatory variables on the response variables are described. The focus is on binary responses for the logistic regression mixture analysis with a fuzzy class model. An algorithm, called a fuzzy classification maximum likelihood (FCML), is then created. The mean squared error (MSE) based accuracy criterion for the FCML and EM algorithms to the parameter estimation of logistic regression mixture models are compared using the samples drawn from logistic regression mixtures of two classes. Numerical results show that the proposed FCML algorithm presents good accuracy and is recommended as a new tool for the parameter estimation of the logistic regression mixture models.

The beta regression is a widely known statistical model when the response (or the dependent) variable has the form of fractions or percentages. In most of the situations in beta regression, the explanatory variables are related to each other which is commonly known as the multicollinearity problem. It is well-known that the multicollinearity problem affects severely the variance of maximum likelihood (ML) estimates. In this article, we developed a new biased estimator (called a two-parameter estimator) for the beta regression model to handle this problem and decrease the variance of the estimation. The properties of the proposed estimator are derived. Furthermore, the performance of the proposed estimator is compared with the ML estimator and other common biased (ridge, Liu, and Liu-type) estimators depending on the mean squared error criterion by making a Monte Carlo simulation study and through two real data applications. The results of the simulation and applications indicated that the proposed estimator outperformed ML, ridge, Liu, and Liu-type estimators.