A convergence diagnostic for Bayesian clustering

ABSTRACTInformation borrowing has been a general tool for improving estimation, testing, and prediction while learning from data distributed over several siloes (devices, trial groups, architecture, or sites), with or without preserving the privacy of the data, depending on the objectives. Federated learning is a more recent tool in this regard, which focuses on privacy preservation: this follows the concept of sharing the model parameters for a learning task rather than sharing the data, running algorithm constituents locally, and then globally optimizing parameters. There are other versions of privacy preserving information borrowing, such as personalized federated learning; the interest in such tasks can be evinced by the recent formation of a UN group to study computation measures enabling information borrowing under privacy guarantees. Heterogeneity is a bottleneck that significantly reduces the utility of information borrowing. In this paper, we identify the progress and gaps in current research regarding heterogeneity in information borrowing for federated learning. In addition to that, we also show using carefully constructed examples the limits of the federated learning paradigm when heterogeneity may be “washed out” when enough data are present. We show the limitations of current research in statistical learning and inference when heterogeneity is present.This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods Statistical Learning and Exploratory Methods of the Data Sciences > Deep Learning Statistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms

ABSTRACTVisual statistical inference determines the significance of patterns found in data exploration through graphics. It involves human observers inspecting a lineup of plots, with one real data plot randomly placed among decoys. Each observer's cognitive skills and judiciousness can influence results. The effectiveness of this method, measured by power, depends on combining evaluations from multiple observers. Human factors influencing power, as computed by the number of detections or identifications of an observed data plot in a lineup, include observer demographics, individual skills, and experience. This paper examines these factors through studies using Amazon's Mechanical Turk, finding individual skills vary but demographics have little impact. Learning increases speed but not accuracy.This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Exploratory Data Analysis Statistical and Graphical Methods of Data Analysis > Statistical Graphics and Visualization Statistical and Graphical Methods of Data Analysis > Nonparametric Methods

Bin smoothers, or regressograms, are piecewise constant regression function estimators whose values are averages of the response variable over the sets of a partition of the space of the explanatory variables. First, we review results about bin smoothers whose partition is regular, giving conditions for consistency and for achieving the optimal rate of convergence. Second, we review representative results about bin smoothers whose partition is irregular, again giving conditions for consistency and for achieving the optimal rate of convergence. Third, we give an exposition of recursive partitioning, main examples being greedy partitions and the classification and regression tress (CART) methodology. WIREs Comput Stat 2012 doi: 10.1002/wics.1214This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Classification and Regression Trees (CART) Statistical and Graphical Methods of Data Analysis > Density Estimation Statistical and Graphical Methods of Data Analysis > Nonparametric Methods

Measures of association have been widely used for describing statistical relationships between two sets of variables. Traditionally, such association measures focus on specialized settings. Based on an in‐depth summary of existing common measures, we present a general framework for association measures that unifies existing methods and novel extensions based on kernels, including practical solutions to computational challenges. Specifically, we introduce association screening and variable selection via maximizing kernel‐based association measures. We also develop a backward dropping procedure for feature selection when there are a large number of candidate variables. The proposed framework was evaluated by independence tests and feature selection using kernel association measures on a diversified set of simulated association patterns with different dimensions and variable types. The results show the superiority of the generalized association measures over existing ones. We also apply our framework to a real‐world problem of gender prediction from handwritten texts. We demonstrate, through this application, the data‐driven adaptation of kernels, and how kernel‐based association measures can naturally be applied to data structures including functional input spaces. This suggests that the proposed framework can guide derivation of appropriate association measures in a wide range of real‐world problems and work well in practice. WIREs Comput Stat 2018, 10:e1422. doi: 10.1002/wics.1422This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Pattern Recognition Statistical Learning and Exploratory Methods of the Data Sciences > Knowledge Discovery Statistical and Graphical Methods of Data Analysis > Multivariate Analysis

A Markov Chain Monte Carlo technique for parameter estimation and inference in pesticide fate and transport modeling

The rapid development of modeling techniques has brought many opportunities for data‐driven discovery and prediction. However, this also leads to the challenge of selecting the most appropriate model for any particular data task. Information criteria, such as the Akaike information criterion (AIC) and Bayesian information criterion (BIC), have been developed as a general class of model selection methods with profound connections with foundational thoughts in statistics and information theory. Many perspectives and theoretical justifications have been developed to understand when and how to use information criteria, which often depend on particular data circumstances. This review article will revisit information criteria by summarizing their key concepts, evaluation metrics, fundamental properties, interconnections, recent advancements, and common misconceptions to enrich the understanding of model selection in general.This article is categorized under:Data: Types and Structure > Traditional Statistical DataStatistical Learning and Exploratory Methods of the Data Sciences > Modeling MethodsStatistical and Graphical Methods of Data Analysis > Information Theoretic MethodsStatistical Models > Model Selection

The Ideal Observer (IO) performance has been advocated when optimizing medical imaging systems for signal detection tasks. However, analytical computation of the IO test statistic is generally intractable. To approximate the IO test statistic, sampling-based methods that employ Markov-Chain Monte Carlo (MCMC) techniques have been developed. However, current applications of MCMC techniques have been limited to several object models such as a lumpy object model and a binary texture model, and it remains unclear how MCMC methods can be implemented with other more sophisticated object models. Deep learning methods that employ generative adversarial networks (GANs) hold great promise to learn stochastic object models (SOMs) from image data. In this study, we described a method to approximate the IO by applying MCMC techniques to SOMs learned by use of GANs. The proposed method can be employed with arbitrary object models that can be learned by use of GANs, thereby the domain of applicability of MCMC techniques for approximating the IO performance is extended. In this study, both signal-known-exactly (SKE) and signal-known-statistically (SKS) binary signal detection tasks are considered. The IO performance computed by the proposed method is compared to that computed by the conventional MCMC method. The advantages of the proposed method are discussed.

This article is a review article on the optimal Markov chain Monte Carlo (MCMC) sampling. The focus is on homogeneous Markov chains. This article first reviews the problem of finding the optimal transition matrix, which is defined to minimize the asymptotic variance of MCMC estimators. The article later reviews the locally optimal sampler (LOS), an MCMC sampling that performs local updates based on the optimal transition matrix. We conducted a simulation study to compare the LOS with the Metropolis–Hastings and the Gibbs Sampler. The LOS was shown to provide an improved rate of convergence over these two most popular sampling schemes. The implementation of the LOS requires only minor modifications in existing Gibbs sampling code. WIREs Comput Stat 2013. doi: 10.1002/wics.1265This article is categorized under: Statistical and Graphical Methods of Data Analysis > Markov Chain Monte Carlo (MCMC)

Airport air traffic is one of the most important and hardest one among all the airport data forecasts. In this paper, the Markov Chain Monte Carlo (MCMC) method of applied theory of statistics has been introduced into the aviation sector, and the discussion on airport air traffic forecast has been conducted taking Shanghai airport as the application background. MCMC method is designed to solve the problem parameters expectations where some baysian inference can not be directly calculated. This paper conducts the iterative computation using WinBUGS, EViews is used to estimate the initial iteration parameter of the regression model, and it is on this basis that the regression equation under the MCMC method is obtained.

Group Importance Sampling for particle filtering and MCMC

Appropriate models in biostatistics are often quite complicated. Such models are typically most easily fit using Bayesian methods, which can often be implemented using simulation techniques. Markov chain Monte Carlo (MCMC) methods are an important set of tools for such simulations. We give an overview and references of this rapidly emerging technology along with a relatively simple example. MCMC techniques can be viewed as extensions of iterative maximization techniques, but with random jumps rather than maximizations at each step. Special care is needed when implementing iterative maximization procedures rather than closed-form methods, and even more care is needed with iterative simulation procedures: it is substantially more difficult to monitor convergence to a distribution than to a point. The most reliable implementations of MCMC build upon results from simpler models fit using combinations of maximization algorithms and noniterative simulations, so that the user has a rough idea of the location and scale of the posterior distribution of the quantities of interest under the more complicated model. These concerns with implementation, however, should not deter the biostatistician from using MCMC methods, but rather help to ensure wise use of these powerful techniques.

Clustering is a fundamental problem in statistics and machine learning, whose solution is commonly computed by the Expectation-Maximization (EM) method, which finds a locally optimal solution for an objective function called log-likelihood. Since the surface of the log-likelihood function is non convex, a stochastic search with Markov Chain Monte Carlo (MCMC) methods can help escaping locally optimal solutions. In this article, we tackle two fundamental conflicting goals: Finding higher quality solutions and achieving faster convergence. With that motivation in mind, we introduce an efficient algorithm that combines elements of the EM and MCMC methods to find clustering solutions that are qualitatively better than those found by the standard EM method. Moreover, our hybrid algorithm allows tuning model parameters and understanding the uncertainty in their estimation. The main issue with MCMC methods is that they generally require a very large number of iterations to explore the posterior of each model parameter. Convergence is accelerated by several algorithmic improvements which include sufficient statistics, simplified model parameter priors, fixing covariance matrices and iterative sampling from small blocks of the data set. A brief experimental evaluation shows promising results.

This article estimates technical and allocative inefficiencies and increase in costs therefrom of individual firms using a translog cost system consisting of the cost function and the cost share equations. We call it a nonlinear random-effects system because technical and allocative inefficiencies are random (hence the term random effects) and are separated from the random noise terms appearing in each equation of the system, and because the inefficiency terms appear in the system in a highly nonlinear fashion, which helps in separating them from random errors. We use Bayesian inference procedures based on Markov chain Monte Carlo (MCMC) techniques to estimate the proposed system. Inferences on firm-specific technical inefficiency and both input-specific and firm-specific allocative inefficiencies are developed using MCMC techniques. We apply the new methods to a sample of 500 U.S. commercial banks. We focus on input allocation problem based on the assumption that banks minimize cost. Empirical results show that cost for the top (bottom) 10% of banks is increased by at least 3% (11%) due to technical inefficiency. In contrast, very few banks are found to be efficient in allocating all the inputs. Costs are increased by 13% on average due to input misallocation. Increase in costs due to both technical and allocative inefficiencies for the top (bottom) 10% of the banks is at least 11% (29%). When translated into dollar figures, this result indicates that elimination of technical and allocative inefficiencies would save the top (bottom) 10% of the banks more than $$.20 ($$3.56) million. We also find that none of the banks in our sample exceeded the efficient scale size, although most of them were operating near their optimum scale (unitary returns to scale).

Using the standard one-zone synchrotron self-Compton model and the Markov chain Monte Carlo (MCMC) technique, we systematically analyse the quasi-simultaneous multiwavelength spectral energy distributions (SEDs) of Mrk 421 and Mrk 501 during states of relatively low activity. With this model in place, a semi-analytical method is developed to examine the uncertainty in jet power estimation caused by degeneracy of the radiative models. The semi-analytical method, in combination with the MCMC technique, allows us to explore the jet properties over a wide range of the variability time-scale. Our results seem to support the following. (i) In both Mrk 421 and Mrk 501, the jets are powered by rapidly rotating black holes. The black hole spin in Mrk 501 could be lower than that in Mrk 421 or possibly they are equal under the assumption of the Blandford–Znajek mechanism. (ii) The energy losses, which could be used to form the large-scale radio structure, are important for reconciling the differences of the kinetic power derived from the observations of the large-scale structure and the SED fitting results. Moreover, the jet energy losses in the propagation are more significant for Mrk 501 than for Mrk 421.

This paper presents a methodology for evaluating the time-dependent system reliability of a pressurised gas pipeline segment containing multiple active metal-loss corrosion defects. The methodology incorporates three distinctive failure modes of the pipe segment due to corrosion, namely small leak, large leak and rupture. The growth of the depth of individual corrosion defect is assumed to follow a power-law function of time. The Bayesian updating and Markov Chain Monte Carlo (MCMC) simulation techniques are used to quantify the parameters of the power-law growth model based on data obtained from multiple inspections carried out at different times. The simple Monte Carlo and MCMC techniques are combined to evaluate the system reliability. A numerical example involving an in-service gas pipeline located in Alberta, Canada, is used to illustrate the proposed methodology. Results of the sensitivity analysis suggest that the use of a defect-specific or segment-specific growth model for the defect depth has a marked impact on the evaluated system reliability. The proposed methodology can be incorporated in reliability-based pipeline corrosion management programmes to assist integrity engineers in making informed decisions about defect repair and mitigation.