Finite Mixture Model: A Comparison of Maximum Likelihood Estimation and Bayesian Analysis

We show that the identification results of finite mixture and misclassification models are equivalent in a widely used scenario except for an extra ordering assumption. In the misclassification model, an ordering condition is imposed to pin down the precise values of the latent variable, which are also of interest to researchers and need to be identified. In contrast, finite mixture models are usually identified up to permutations of a latent index, which results in local identification. This local identification is satisfactory because the latent index does not convey any economic meaning. However, reaching global identification is important for estimation, especially when researchers use bootstrap to estimate standard errors. This is because standard errors approximated by bootstrap may be incorrect without a global estimator. We demonstrate that games with multiple equilibria fit in our framework and the global estimator with ordering conditions provides more reliable estimates.

WINNER OF THE 2007 DEGROOT PRIZE! The prominence of finite mixture modelling is greater than ever. Many important statistical topics like clustering data, outlier treatment, or dealing with unobserved heterogeneity involve finite mixture models in some way or other. The area of potential applications goes beyond simple data analysis and extends to regression analysis and to non-linear time series analysis using Markov switching models. For more than the hundred years since Karl Pearson showed in 1894 how to estimate the five parameters of a mixture of two normal distributions using the method of moments, statistical inference for finite mixture models has been a challenge to everybody who deals with them. In the past ten years, very powerful computational tools emerged for dealing with these models which combine a Bayesian approach with recent Monte simulation techniques based on Markov chains. This book reviews these techniques and covers the most recent advances in the field, among them bridge sampling techniques and reversible jump Markov chain Monte Carlo methods. It is the first time that the Bayesian perspective of finite mixture modelling is systematically presented in book form. It is argued that the Bayesian approach provides much insight in this context and is easily implemented in practice. Although the main focus is on Bayesian inference, the author reviews several frequentist techniques, especially selecting the number of components of a finite mixture model, and discusses some of their shortcomings compared to the Bayesian approach. The aim of this book is to impart the finite mixture and Markov switching approach to statistical modelling to a wide-ranging community. This includes not only statisticians, but also biologists, economists, engineers, financial agents, market researcher, medical researchers or any other frequent user of statistical models. This book should help newcomers to the field to understand how finite mixture and Markov switching models are formulated, what structures they imply on the data, what they could be used for, and how they are estimated. Researchers familiar with the subject also will profit from reading this book. The presentation is rather informal without abandoning mathematical correctness. Previous notions of Bayesian inference and Monte Carlo simulation are useful but not needed.

Technological advances led to the generation of large scale complex data. Thus, extraction and retrieval of information to automatically discover latent pattern have been largely studied in the various domains of science and technology. Consequently, machine learning experienced tremendous development and various statistical approaches have been suggested. In particular, data clustering has received a lot of attention. Finite mixture models have been revealed to be one of the flexible and popular approaches in data clustering. Considering mixture models, three crucial aspects should be addressed. The first issue is choosing a distribution which is flexible enough to fit the data. In this paper, a model based on multivariate Beta distributions is proposed. The two other challenges in mixture models are estimation of model's parameters and model complexity. To tackle these challenges, variational inference techniques demonstrated considerable robustness. In this paper, two methods are studied, namely, batch and online variational inferences and the models are evaluated on four medical applications including image segmentation of colorectal cancer, multi‐class colon tissue analysis, digital imaging in skin lesion diagnosis and computer aid detection of Malaria.

Pre-analysis of Multi-batch Bioprocesses Data with Finite Mixture Models in the Reduced Feature Subspace

In the statistical modeling framework, the form of the income distribution can be approaching based on certain statistical distributions. The use of the finite mixture model is relatively flexible in the modeling of the income distribution that has a multimodal pattern. The multimodal pattern can be indicated as the existence of different cluster on the data. The different clusters which can reflect the economic homogeneity of income are represented by the mixture components of the finite mixture model. In this paper, the finite mixture model is implemented for modeling the distribution of household income per capita in Indonesia based on The Fifth Wave of the Indonesia Family Life Survey (IFLS5) 2014-2015. The mixture components of the finite mixture model have been build based on the heavy-tailed statistical distributions, i.e., gamma, lognormal, and Weibull distributions. The estimation of the fitting finite mixture model was conducted using the maximum-likelihood estimation method through the expectation-maximization (EM) algorithm. The suitable finite mixture models were verified with the bootstrap likelihood ratio statistics test, Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC). Based on the results, the distribution of household income per capita in Indonesia can be modeled by the four components-lognormal mixture model.

Finite mixture modeling is a popular statistical technique capable of accounting for various shapes in data. One popular application of mixture models is model-based clustering. This paper considers the problem of clustering regression autoregressive moving average time series. Two novel estimation procedures for the considered framework are developed. The first one yields the conditional maximum likelihood estimates which can be used in cases when the length of times series is substantial. Simple analytical expressions make fast parameter estimation possible. The second method incorporates the Kalman filter and yields the exact maximum likelihood estimates. The procedure for assessing variability in obtained estimates is discussed. We also show that the Bayesian information criterion can be successfully used to choose the optimal number of mixture components and correctly assess time series orders. The performance of the developed methodology is evaluated on simulation studies. An application to the analysis of tree ring data is thoroughly considered. The results are very promising as the proposed approach overcomes the limitations of other methods developed so far.

The foundations of -analysis date back to the 1740s, when Euler introduced the theory of partitions, also referred to as additive analytic number theory. Over the years, the discovery of -calculus applications in fields such as operator theory, combinatory, probability theory, and many others has sparked tremendous interest in this mathematical framework. Mixture distributions are probabilistic models in which a data set is assumed to originate from multiple underlying distributions, each contributing with a certain probability. These distributions are commonly used to model complex data structures more accurately. This paper introduces -finite mixture models as a novel extension of the classical finite mixture family, motivated by recent progress in -calculus and generalized probability distributions. By incorporating a deformation parameter , the proposed mixture models offer enhanced modeling flexibility for a variety of stochastic phenomena. The fundamental distributional and statistical properties of the suggested -mixture models are systematically are explored.

A finite mixture model is used to describe the diameter distributions of mixed-species forest stands. A three-parameter Weibull function is assumed as the component probability density function in the finite mixture model. Four example plots, each with two species, are selected to demonstrate model fitting and comparison. It appears that the finite mixture model is flexible enough to fit irregular, multimodal, or highly skewed diameter distributions. Compared with traditional methods in which a single Weibull function is fit to either the whole plot or each species component separately, the finite mixture model produces much smaller root mean square error and bias, and fits the entire distribution of the plots with extreme peaks, bimodality, or heavy-tails well. In some cases, a single Weibull function fitted to individual species separately may produce more accurate estimations for the component distributions of the two species than the finite mixture model. The summation of the two independent species results, however, may not produce a better prediction for the entire plot. This study shows that the finite mixture model is a promising alternative method for modeling the diameter distribution of multispecies mixed forest stands. For. Sci. 48(4):653–661.

Mixture modeling is a popular technique for identifying unobserved subpopulations (e.g., components) within a data set, with Gaussian (normal) mixture modeling being the form most widely used. Generally, the parameters of these Gaussian mixtures cannot be estimated in closed form, so estimates are typically obtained via an iterative process. The most common estimation procedure is maximum likelihood via the expectation-maximization (EM) algorithm. Like many approaches for identifying subpopulations, finite mixture modeling can suffer from locally optimal solutions, and the final parameter estimates are dependent on the initial starting values of the EM algorithm. Initial values have been shown to significantly impact the quality of the solution, and researchers have proposed several approaches for selecting the set of starting values. Five techniques for obtaining starting values that are implemented in popular software packages are compared. Their performances are assessed in terms of the following four measures: (1) the ability to find the best observed solution, (2) settling on a solution that classifies observations correctly, (3) the number of local solutions found by each technique, and (4) the speed at which the start values are obtained. On the basis of these results, a set of recommendations is provided to the user.

Chapter Preview. Actuaries have been studying loss distributions since the emergence of the profession. Numerous studies have found that the widely used distributions, such as lognormal, Pareto, and gamma, do not fit insurance data well. Mixture distributions have gained popularity in recent years because of their flexibility in representing insurance losses from various sizes of claims, especially on the right tail. To incorporate the mixture distributions into the framework of popular generalized linear models (GLMs), the authors propose to use finite mixture models (FMMs) to analyze insurance loss data. The regression approach enhances the traditional whole-book distribution analysis by capturing the impact of individual explanatory variables. FMM improves the standard GLM by addressing distribution-related problems, such as heteroskedasticity, over- and underdispersion, unobserved heterogeneity, and fat tails. A case study with applications on claims triage and on high-deductible pricing using workers’ compensation data illustrates those benefits. Introduction Conventional Large Loss Distribution Analysis Large loss distributions have been extensively studied because of their importance in actuarial applications such as increased limit factor and excess loss pricing (Miccolis, 1977), reinsurance retention and layer analysis (Clark, 1996), high deductible pricing (Teng, 1994), and enterprise risk management (Wang, 2002). Klugman et al. (1998) discussed the frequency, severity, and aggregate loss distributions in detail in their book, which has been on the Casualty Actuarial Society syllabus of exam Construction and Evaluation of Actuarial Models for many years. Keatinge (1999) demonstrated that popular single distributions, including those in Klugman et al. (1998), are not adequate to represent the insurance loss well and suggested using mixture exponential distributions to improve the goodness of fit. Beirlant et al. (2001) proposed a flexible generalized Burr-gamma distribution to address the heavy tail of loss and validated the effectiveness of this parametric distribution by comparing its implied excess-of-loss reinsurance premium with other nonparametric and semi-parametric distributions. Matthys et al. (2004) presented an extreme quantile estimator to deal with extreme insurance losses. Fleming (2008) showed that the sample average of any small data from a skewed population is most likely below the true mean and warned the danger of insurance pricing decisions without considering extreme events. Henry and Hsieh (2009) stressed the importance of understanding the heavy tail behavior of a loss distribution and developed a tail index estimator assuming that the insurance loss possess Pareto-type tails.

Finite mixture model is a useful probabilistic model for representing the probability distributions of observations. Gaussian mixture model (GMM) is a widely used one whose parameters are always estimated by the famous EM algorithm. But when probability distributions contain asymmetric modes, GMM requires much more constituent components to achieve a satisfied accuracy. Therefore, a generalized mixture finite model is proposed. First, it adopts the derivative [Formula: see text]-PDF which can well represent the asymmetric probability distributions as the probability density. However, the differential operation and solving the likelihood equations are scarcely possible in the M step of EM algorithm for the complex expression of the derivative [Formula: see text]-PDF. Second, a pseudo EM method is proposed to avoid the abovementioned difficulty which finds the pseudo maximum likelihood estimates of the parameters by utilizing the moment matching principle. It more easily estimates the parameters by solving a series of moment matching equations. Finally, four examples are presented to verify that the proposed generalized finite mixture model has advantage on representing the asymmetric probability distributions observations.

Finite mixture models, that is, weighted averages of parametric distributions, provide a powerful way to extend parametric families of distributions to fit data sets not adequately fit by a single parametric distribution. First-order finite mixture models have been widely used in the physical, chemical, biological, and social sciences for over 100 years. Using maximum likelihood estimation, we demonstrate how a first-order finite mixture model can represent the large variability in data collected by the U.S. Environmental Protection Agency for the concentration of Radon 222 in drinking water supplied from ground water, even when 28% of the data fall at or below the minimum reporting level. Extending the use of maximum likelihood, we also illustrate how a second-order finite mixture model can separate and represent both the variability and the uncertainty in the data set.

On model-based clustering of skewed matrix data

A multivariate linear regression analysis using finite mixtures of t distributions

Using incremental general regression neural network for learning mixture models from incomplete data