Model-based clustering using a new multivariate skew distribution

Model-based time-varying clustering of multivariate longitudinal data with covariates and outliers

Finite mixtures of multivariate skew t (MST) distributions have proven to be useful in modelling heterogeneous data with asymmetric and heavy tail behaviour. Recently, they have been exploited as an effective tool for modelling flow cytometric data. A number of algorithms for the computation of the maximum likelihood (ML) estimates for the model parameters of mixtures of MST distributions have been put forward in recent years. These implementations use various characterizations of the MST distribution, which are similar but not identical. While exact implementation of the expectation-maximization (EM) algorithm can be achieved for `restricted' characterizations of the component skew t-distributions, Monte Carlo (MC) methods have been used to fit the `unrestricted' models. In this paper, we review several recent fitting algorithms for finite mixtures of multivariate skew t-distributions, at the same time clarifying some of the connections between the various existing proposals. In particular, recent results have shown that the EM algorithm can be implemented exactly for faster computation of ML estimates for mixtures with unrestricted MST components. The gain in computational time is effected by noting that the semi-infinite integrals on the E-step of the EM algorithm can be put in the form of moments of the truncated multivariate non-central t-distribution, similar to the restricted case, which subsequently can be expressed in terms of the non-truncated form of the central t-distribution function for which fast algorithms are available. We present comparisons to illustrate the relative performance of the restricted and unrestricted models, and demonstrate the usefulness of the recently proposed methodology for the unrestricted MST mixture, by some applications to three real datasets.

This paper introduces structural identification using expectation maximization (STRIDE), a novel application of the expectation maximization (EM) algorithm and approach for output-only modal identification. The EM algorithm can be used to estimate the maximum likelihood parameters of a state-space model. In this context, the state-space model represents the equation of motion for a linear dynamic system. STRIDE is an iterative procedure that uses Kalman filtering and Rauch-Tung-Striebel (RTS) smoothing equations to produce estimates of the unobserved states; these calculations are based on the observed data and prior estimates of the state-space parameters. With this information, the conditional likelihood of the model is maximized and the state-space parameters are updated at each iteration. Once an iteration meets user-prescribed convergence criterion, the algorithm ends—yielding maximum likelihood estimates (MLE) for the state-space model parameters. The modal properties of the structure are th...

This paper proposes a method for obtaining nonlinear models of distribution grid based on available measurements from the power grid. We formulate a maximum likelihood estimation (MLE) problem that estimates unknown line parameters—specifically, the impedance between nodes—using measured voltage magnitudes and net active and reactive power injections at each node. The nonlinear model for the distribution grid uses a nonlinear approximation of the DistFlow model, which includes line losses and is parameterized by the unknown line impedances. We solve the resulting MLE problem using an expectation maximization (EM) algorithm, tailored for the nonlinear model, and provide a numerically robust implementation. The proposed method is demonstrated on the IEEE 37-node test network, and we compare it with the state-of-the-art methods. The proposed method achieves a 70% reduction in voltage error and an error for state variables that is more than 10000 times smaller. A final comparison uses data from a real network, and the proposed method achieves parameter estimates with errors 100 times smaller than competing methods.

Many diseases show dichotomous phenotypic variation but do not follow a simple Mendelian pattern of inheritance. Variances of these binary diseases are presumably controlled by multiple loci and environmental variants. A least-squares method has been developed for mapping such complex disease loci by treating the binary phenotypes (0 and 1) as if they were continuous. However, the least-squares method is not recommended because of its ad hoc nature. Maximum Likelihood (ML) and Bayesian methods have also been developed for binary disease mapping by incorporating the discrete nature of the phenotypic distribution. In the ML analysis, the likelihood function is usually maximized using some complicated maximization algorithms (e.g. the Newton-Raphson or the simplex algorithm). Under the threshold model of binary disease, we develop an Expectation Maximization (EM) algorithm to solve for the maximum likelihood estimates (MLEs). The new EM algorithm is developed by treating both the unobserved genotype and the disease liability as missing values. As a result, the EM iteration equations have the same form as the normal equation system in linear regression. The EM algorithm is further modified to take into account sexual dimorphism in the linkage maps. Applying the EM-implemented ML method to a four-way-cross mouse family, we detected two regions on the fourth chromosome that have evidence of QTLs controlling the segregation of fibrosarcoma, a form of connective tissue cancer. The two QTLs explain 50-60% of the variance in the disease liability. We also applied a Bayesian method previously developed (modified to take into account sex-specific maps) to this data set and detected one additional QTL on chromosome 13 that explains another 26% of the variance of the disease liability. All the QTLs detected primarily show dominance effects.

The EM algorithm is a widely used technique for finding maximum likelihood (ML) estimates when the data are not fully observed. Despite its popularity for computing ML estimates in unrestricted problems and the need for simplified computations for problems with equality and inequality restrictions, there have been few applications of the algorithm to restricted ML estimation. The EM algorithm is presented for restricted ML estimation and provides its applications to the probit model under equality and inequality restrictions using two small data sets.

In this paper a quantitative analysis of the ruin probability in finite time of discrete risk process with proportional reinsurance and investment of finance surplus is focused on. It is assumed that the total loss on a unit interval has a light-tailed distribution -- exponential distribution and a heavy-tailed distribution -- Pareto distribution. The ruin probability for finite-horizon 5 and 10 was determined from recurrence equations. Moreover for exponential distribution the upper bound of ruin probability by Lundberg adjustment coefficient is given. For Pareto distribution the adjustment coefficient does not exist, hence an asymptotic approximation of the ruin probability if an initial capital tends to infinity is given. Obtained numerical results are given as tables and they are illustrated as graphs.

Computation: Expectation-Maximization Algorithm

In this paper, a computationally efficient algorithm is presented for joint maximum likelihood (ML) timing and carrier phase estimation of OFDM systems employing M-PSK modulation scheme with additive Gaussian noise, based on the expectation-maximization (EM) algorithm. A nondata-aided (NDA) scheme is considered for the joint timing and phase synchronizer which maximizes the low SNR limit of the likelihood function averaged over the M-PSK signal constellation. For this, an EM algorithm is derived which estimates the timing offset and the phase rotations of each subcarrier iteratively and which converges to the true ML estimation of the unknown timing and phase. It is shown that the algorithm becomes independent of the signal-to-noise ratio for both low and high SNR cases. The algorithm is applied to the QPSK modulated OFDM systems and it is concluded that for SNR values greater than 10 dB the convergence is achieved in first iteration and for SNR values less than 10 dB, at most in three iterations. It is also concluded that the convergence is independent of the initial starting points.

There is a problem arising when Lindley distribution is used to model right–skewed and unimodal data, which lies in its inability to model data with its peak farther from 0. A modification is required to increase the flexibility of this distribution, with “transformed–transformer” being one of the proposed method. This method is done by making a composition of two random variables through their respective distribution functions, with random variable T being the “transformed”, and random variable X being the “transformer”. In this paper, Lindley distribution was chosen to be the “transformed” and exponential distribution was chosen to be the “transformer”, constructing the Lindley–exponential distribution. However, there is a trouble using distribution if the data has a heavier tail. An alternative distribution is required while maintaining the properties of the Lindley-Exponential Distribution. Through transformation of variables method, a new distribution, Lindley-Exponential Slash Distribution is introduced, which is unimodal, right–skewed, heavier tailed. This paper covered some of its statistical characteristics, such as pdf, cdf, survival function, hazard rate, k th moment, mean, variance, skewness, and kurtosis. Parameter estimation was carried out with maximum likelihood estimation through numerical method. An application of distribution was illustrated on maximum annual precipitation data of Durham City.

Time series modeling of surface EMG based hand manipulation identification via expectation maximization algorithm

Examining the robustness properties of maximum likelihood (ML) estimators of parameters in exponential power and generalized t distributions has been considered together. The well-known asymptotic properties of ML estimators of location, scale and added skewness parameters in these distributions are studied. The ML estimators for location, scale and scale variant (skewness) parameters are represented as an iterative reweighting algorithm (IRA) to compute the estimates of these parameters simultaneously. The artificial data are generated to examine performance of IRA for ML estimators of parameters simultaneously. We make a comparison between these two distributions to test the fitting performance on real data sets. The goodness of fit test and information criteria approve that robustness and fitting performance should be considered together as a key for modeling issue to have the best information from real data sets.

In this research, the statistical inference of unknown lifetime parameters is proposed in the presence of independent competing risks using a progressive Type-II censored dataset. The lifetime distribution associated with a failure mode is assumed to follow the new Pareto distribution, with consideration given to two distinct competing failure reasons. Maximum likelihood estimators (MLEs) for the unknown model parameters, as well as reliability and hazard functions, are derived, noting that they are not expressible in closed form. The Newton–Raphson, expectation maximization (EM), and stochastic expectation maximization (SEM) methods are employed to generate maximum likelihood (ML) estimations. Approximate confidence intervals for the unknown parameters, reliability, and hazard rate functions are constructed using the normal approximation of the MLEs and the normal approximation of the log-transformed MLEs. Additionally, the missing information principle is utilized to derive the closed form of the Fisher information matrix, which, in turn, is used with the delta approach to calculate confidence intervals for reliability and hazards. Bayes estimators are derived under both symmetric and asymmetric loss functions, with informative and non-informative priors considered, including independent gamma distributions for informative priors. The Monte Carlo Markov Chain sampling approach is employed to obtain the highest posterior density credible intervals and Bayesian point estimates for unknown parameters and reliability characteristics. A Monte Carlo simulation is conducted to assess the effectiveness of the proposed techniques, with the performances of the Bayes and maximum likelihood estimations examined using average values and mean squared errors as benchmarks. Interval estimations are compared in terms of average lengths and coverage probabilities. Real datasets are considered and examined for each topic to provide illustrative examples.

The pharmaceutical industry is currently interested in the population approach and population models, also known as mixed effects models and random effects models depending on the precise form. Population models are useful in that they can account for both withinand between-individual sources of variability and serial correlation within individual observations when analyzing unbalanced repeated measures data. The modelling of population pharmacodynamic or pharmacokinetic profiles typically involves nonlinear random effects models. Each individual's observations are modelled by identical (up to unknown parameter values) nonlinear regression models, with the distribution of the observations, or a transformation of the observations, about expected responses taken to be normal, with the degree of variability described by a variance model. Between-individual variability is modelled by a population distribution for the individual regression parameter values (random effects). In a parametric analysis the population distribution is taken to be normal, the parameters of which, along with the parameters of the variance model, are known as the population parameters. Maximum likelihood estimation of the population parameters for nonlinear random effects models was pioneered by Beal and Sheiner (1979), and since then a number of algorithms have appeared for approximate maximum likelihood, including Steimer et al. (1984), Lindstrom and Bates (1990), Beal and Sheiner (1992), and Mentre and Gomeni (1995). All of these algorithms are approximate in some way. For a summary see Beal and Sheiner (1992), Wolfinger (1993), Pinheiro and Bates (1994), and Davidian and Giltinan (1995). In this paper an EM algorithm for exact maximum likelihood estimation is introduced. An EM algorithm obtaining maximum likelihood estimates for linear random effects models was introduced by Dempster, Laird, and Rubin (1977). Laird and Ware (1982), Lindstrom and Bates (1988), Jennrich and Schluchter (1986), and Liu and Rubin (1994) all describe hybrid EM algorithms for the linear random effects model. A true EM algorithm for the linear model is described by Jamshidian and Jennrich (1993). Mentre and Gomeni (1995) describe an approximate EM algorithm for nonlinear random effects models and, from the algorithm given in this paper, it can be seen clearly how their approximations arise. The present algorithm uses Monte Carlo methods to perform the E step, a strategy previously adopted in an altogether different model by Guo and Thompson (1994). Guo and Thompson require a Gibbs sampler, that is, a Markov chain Monte Carlo method for their E step, but the present algorithm uses independent samples. In Section 2 of this paper the nonlinear random effects model is described. Section 3 gives the EM algorithm without random effect covariates, while Section 4 gives the modified algorithm in the

The mixture model is known as model-based clustering that is used to model a mixture of unknown distributions. The clustering of mixture model is based on four important criteria, including the number of components in the mixture model, clustering kernel (such as Gaussian mixture models, Dirichlet, etc.), estimation methods, and dimensionality (Lai et al., 2019). Finite mixture model is a finite dimensional of a hierarchical model. It is useful in modeling the data with outliers, non-normal distributed or heavy tails. Furthermore, finite mixture model is flexible when fitted with the models that have multiple modes or skewed distribution. The flexibility depends on the increasing number of parameters with the existence of a number of components. The finite mixture model is a flexible model family and widely applied for large heterogeneous datasets. In addition, the finite mixture model is a probabilistic model that is used to examine the presence of unobserved situations or groups and to measure the distinct parameters or distribution. The situations, such as trend, seasoning, crisis time, normal situation, etc., might affect the number of components that exist for a probabilistic distribution. Furthermore, the finite mixture model is essential for time series data because these data exhibit nonlinearity properties and may have missing data or a jump-diffusion situation (Gensler, 2017; McLachlan and Lee, 2019). Keywords: Bayesian method; Finite Mixture Model; Maximum Likelihood Estimation; Prior distribution; Likelihood Function.