Robust processes through latent variable modeling and optimization

Latent variable (LV) models are increasingly being used in environmental epidemiology as a way to summarize multiple environmental exposures and thus minimize statistical concerns that arise in multiple regression. LV models may be especially useful when multivariate exposures are collected repeatedly over time. LV models can accommodate a variety of assumptions but, at the same time, present the user with many choices for model specification particularly in the case of exposure data collected repeatedly over time. For instance, the user could assume conditional independence of observed exposure biomarkers given the latent exposure and, in the case of longitudinal latent exposure variables, time invariance of the measurement model. Choosing which assumptions to relax is not always straightforward. We were motivated by a study of prenatal lead exposure and mental development, where assumptions of the measurement model for the time-changing longitudinal exposure have appreciable impact on (maximum-likelihood) inferences about the health effects of lead exposure. Although we were not particularly interested in characterizing the change of the LV itself, imposing a longitudinal LV structure on the repeated multivariate exposure measures could result in high efficiency gains for the exposure-disease association. We examine the biases of maximum likelihood estimators when assumptions about the measurement model for the longitudinal latent exposure variable are violated. We adapt existing instrumental variable estimators to the case of longitudinal exposures and propose them as an alternative to estimate the health effects of a time-changing latent predictor. We show that instrumental variable estimators remain unbiased for a wide range of data generating models and have advantages in terms of mean squared error. Copyright © 2017 John Wiley & Sons, Ltd.

Fisher information matrix based time-series segmentation of process data

The high dimensionality of global gene expression profiles, where number of variables (genes) is very large compared to the number of observations (samples), presents challenges that affect generalizability and applicability of microarray analysis. Latent variable modeling offers a promising approach to deal with high-dimensional microarray data. The latent variable model is based on a few latent variables that capture most of the gene expression information. Here, we describe how to accomplish a reduction in dimension by a latent variable methodology, which can greatly reduce the number of features used to characterize microarray data. We propose a general latent variable framework for prediction of predefined classes of samples using gene expression profiles from microarray experiments. The framework consists of (i) selection of smaller number of genes that are most differentially expressed between samples, (ii) dimension reduction using hierarchical clustering, where each cluster partition is identified as latent variable, (iii) discretization of gene expression matrix, (iv) fitting the Rasch item response model for genes in each cluster partition to estimate the expression of latent variable, and (v) construction of prediction model with latent variables as covariates to study the relationship between latent variables and phenotype. Two different microarray data sets are used to illustrate a general framework of the approach. We show that the predictive performance of our method is comparable to the current best approach based on an all-gene space. The method is general and can be applied to the other high-dimensional data problems.

In this chapter, we address distributed learning algorithms for statistical latent variable models, with a focus on topic models. Many high-dimensional datasets, such as text corpora and image databases, are too large to allow one to learn topic models on a single computer. Moreover, a growing number of applications require that inference be fast or in real time, motivating the exploration of parallel and distributed learning algorithms. We begin by reviewing topic models such as Latent Dirichlet Allocation and Hierarchical Dirichlet Processes. We discuss parallel and distributed algorithms for learning these models and show that these algorithms can achieve substantial speedups without sacrificing model quality. Next we discuss practical guidelines for running our algorithms within various parallel computing frameworks and highlight complementary speedup techniques. Finally, we generalize our distributed approach to handle Bayesian networks. Several of the results in this chapter have appeared in previous papers in the specific context of topic modeling. The goal of this chapter is to present a comprehensive overview of distributed inference algorithms and to extend the general ideas to a broader class of Bayesian networks. Latent Variable Models Latent variable models are a class of statistical models that explain observed data with latent (or hidden) variables. Topic models and hidden Markov models are two examples of such models, where the latent variables are the topic assignment variables and the hidden states, respectively. Given observed data, the goal is to perform Bayesian inference over the latent variables and use the learned model to make inferences or predictions.

Data-driven process monitoring based on latent variable models are widely employed in industry. This article proposes a novel monitoring framework for latent variable models using hierarchical feature extraction, Bayesian inference, and weighting strategy. We first establish a deep structure to implement hierarchical latent variables extraction, the extracted features are used to construct diverse monitoring statistics. Then, we utilize Bayesian inference and proper weighting strategy to fuse various useful information. In line with the different characteristics of principal component analysis (PCA) and independent component analysis (ICA), we construct a deep PCA-ICA model for process monitoring according to the proposed framework. The deep PCA-ICA model performs hierarchical feature extraction, which can simultaneously extract deep Gaussian information and deep non-Gaussian information. The features extracted by different layers are then transformed to posterior probabilities through Bayesian inference. After that, different posterior probabilities are combined through appropriate weighting strategy to build new probabilistic statistic, which can give more synthetic monitoring results. Moreover, the Bayesian inference and weighting strategy are further used to integrate the advantages of different models by transforming various probabilistic statistics into an overall monitoring index, which can comprehensively indicate the process status. The Tennessee Eastman process is used to validate the superiority of the proposed model over the existing methods. Besides, the extracted features are further analyzed to show the effectiveness and benefits of the deep hierarchical feature extraction structure.

Extending the traditional discrete choice model by incorporating latent psychological factors can help to better understand the individual’s decision-making process and therefore to yield more reliable part-worth estimates and market share predictions. Several integrated choice and latent variable (ICLV) models which merge the conditional logit model with a structural equation model exist in the literature. They assume homogeneity in the part-worths and use latent variables to model the heterogeneity among the respondents. This paper starts from the mixed logit model that describes the heterogeneity in the part-worths and uses the latent variables to decrease the unexplained part of the heterogeneity. The empirical study presented here shows these ICLV models perform very well with respect to model fit and prediction.

Product and Mixture Design in Latent Variable Space by Chemometric Techniques

Measurement of non-random attrition effects on mobility rates using trip diaries data

This doctoral thesis presents a framework for machine based interpretation of Malaysian Sign Language (MSL). The goal is to explore and develop necessary core algorithms that are capable of recognizing hand signs, based on the monocular/2D visual input of a low-cost hardware (Digital Webcam), without the assistance of visual-markers on hands. Successful hand sign interpretation with the aforementioned constrained visual system is challenging. Economical hardware devices often provide noisy/cluttered raw data with limited useful information. Non-involvement of visual markers imposes the requirement of additional strategies to disambiguate occlusions and background complexities. Natural variations in hand posture/gesture performances further convolute the recognition task. Addressing these challenges require computationally heavy algorithms in image processing, machine learning and pattern recognition. Thus vision-based sign language translator frameworks have to be multifaceted and versatile, in order to address innumerable practical concerns. In this research, hand postures interpretation is achieved by statistical projection methods. Analysis of high dimensional hand shape data is a complex task. Thus reducing dimensions via known projection methods aids in alleviating some of these complexities. Locality Preserving Projection (LPP) has been employed to accomplish this goal, while Principal Component Analysis (PCA) has been used as the benchmark method. Both methods were extensively tested against a hand posture database of 24 alphabet signs from MSL. As for the hand gesture interpretation, parameter-less Gaussian process based machine learning methods are proposed, as an alternative to current status quo. A variation of the Gaussian latent variable model – Gaussian Process Dynamic Model (GPDM) – has been employed for hand gesture interpretation. This has been comparatively studied against the established Hidden Markov Model (HMM). These machine learning methods were thoroughly tested against a hand gesture database of 64 signs from MSL. Grammatical sentence construction offers both intelligible sentences, and a mean of identifying erroneous sign classifications. In this thesis, a proof-of-concept on: use of Context Free Grammar (CFG) in successful parsing of meaningful sentences based on recognised gesture words, and how it can be extended in identifying (and correcting) erroneous sign classifications, is demonstrated with 92 sentences and the output of GPDM gesture classification. The novelty and contribution of this research: this is the first undertaking of markerless vision-based translation for MSL. Both LPP and GPDM are introduced to the sign language research field as an outcome of this investigation. The preliminary study and demonstration of grammatical sentence construction for translated gesture signs, is the first of its kind to the best of author's knowledge.

The Process Chemometrics Approach to Process Monitoring and Fault Detection

Often quantities of interest in psychology cannot be observed directly. These unobservable quantities, known as latent variables, tend to be complex, often multidimensional, constructs. In many cases these constructs are categorical, such that individuals belong to mutually exclusive and exhaustive unobservable subgroups. Latent class analysis (LCA) is a statistical approach to modeling a discrete latent variable using multiple, discrete observed variables as indicators. Examples of latent class variables that appear in the psychology literature include temperament type, substance use behavior, teaching style, stages of change in the transtheoretical model, and latent classes of risk. The first section of this chapter provides a conceptual introduction to the concept of a latent class followed by a technical introduction to the mathematical model, including multiple-groups LCA and LCA with covariates. This is followed by a discussion of parameter restrictions, model selection, and goodness-of-fit. The second section demonstrates LCA using the empirical example of depression subtypes in adolescence. Five latent classes were identified based on responses to eight questionnaire items assessing depression symptoms: Non-depressed (characterized by a low probability of reporting all eight depression symptoms), sad, disliked, sad + disliked, and depressed (characterized by a high probability of reporting all eight depression symptoms). The third section presents longitudinal extensions of the model, including repeated-measures LCA and latent transition analysis (LTA). The empirical example is extended to examine change in depression subtypes over time. The final sections describe recent extensions to the latent class model and areas that merit additional research in the future. Keywords: latent class analysis; latent transition analysis; latent variable model; categorical data

The selection of calibration method is one of the most important factors affecting the measurement accuracy with near infrared spectroscopy. In this research, the performance of two general calibration methods, namely, linear partial least squares regression and nonlinear back propagation artificial neural networks for firmness predicting of ‘Mazafati’ date fruit was investigated. A total of 175 date samples harvested during fruit ripening were selected as the data set. Optical scanning was performed with a fiber optic near infrared spectrometer with a range of 900–1700 nm. The inputs of back propagation artificial neural networks were the first five principal components resulting from the principal component analysis, namely, principal component analysis artificial neural network modeling or the first six latent variables obtained from partial least squares regression, namely, latent variable artificial neural network modeling. Both the leave-one-out cross validation and test set validation showed the priority of latent variable artificial neural network model with respect to partial least squares and also principal component analysis artificial neural network models. The best latent variable artificial neural network model could predict the firmness of Mazafati date fruits with Rp 2 of 0.90, RMSEP of 1.30 N, and SDRp of 3.28. The results also recommended the adoption of nonlinear latent variable artificial neural network modeling for applying light scattering properties of fruits in order to predict indirect property of firmness with near infrared spectroscopy.

In the past years, latent variable models have played an important role in various industrial AI systems, among which quality prediction is one of the most representative applications. Inspired by the idea of deep learning, those basic latent variable models have been extended to deep forms, based on which the quality prediction performance has been significantly improved. However, different latent variable models have their own strengths and weaknesses, a model works well under one scenario might not provide satisfactory performance under another. The motivation of this article is based on the viewpoint of information fusion and ensemble learning for heterogeneous latent variable models. Particularly, a collaborative deep learning and model fusion framework is formulated for the purpose of industrial quality prediction. In the first stage of the framework, collaborative layer-by-layer feature extractions are implemented among different latent variable models, through which different patterns of latent variables are identified in different layers of the deep model. Then, in the second stage, an ensemble regression modeling strategy is proposed to fuse the quality prediction results from different latent variable models, which is based on a well-designed data description method. Two real industrial examples are used for performance evaluation of the proposed method, based on which we can observe that information fusions in terms of both collaborative layer-by-layer feature extraction and heterogeneous model ensemble have positive effects in improving prediction accuracy and stability.

The biasing effects of measurement error in path analysis models can be overcome by the use of latent variable models. In cases where path analysis is used in practice, it is often possible to use parcels as indicators of a latent variable. The purpose of the current study was to compare latent variable models in which parcels were used as indicators of the latent variables, path analysis models of the aggregated variables, and models in which reliability estimates were used to correct for measurement error in path analysis models. Results showed that point estimates of path coefficients were smallest for the path analysis models and largest for the latent variable models. It is concluded that, whenever possible, it is better to use a latent variable model in which parcels are used as indicators than a path analysis model using total scale scores.

Latent trait models have long been used in the social science literature for studying variables that can only be measured indirectly through multiple items. However, such models are also very useful in accounting for correlation in multivariate and longitudinal data, particularly when outcomes have mixed measurement scales. Bayesian methods implemented with Markov chain Monte Carlo provide a flexible framework for routine fitting of a broad class of latent variable (LV) models, including very general structural equation models. However, in considering LV models, a number of challenging issues arise, including identifiability, confounding between the mean and variance, uncertainty in different aspects of the model, and difficulty in computation. Motivated by the problem of modelling multidimensional longitudinal data, this article reviews the recent literature, provides some recommendations and highlights areas in need of additional research, focusing on methods for model uncertainty.