Sampling constrained continuous probability distributions: A review

Level set trees provide a tool for analyzing multivariate functions. Level set trees are particularly efficient in visualizing and presenting properties related to local maxima and local minima of functions. Level set trees can help statistical inference related to estimating probability density functions and regression functions, and they can be used in cluster analysis and function optimization, among other things. Level set trees open a new way to look at multivariate functions, which makes the detection and analysis of multivariate phenomena feasible, going beyond one‐ and two‐dimensional analysis.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Analysis of High Dimensional Data Statistical and Graphical Methods of Data Analysis > Statistical Graphics and Visualization Statistical and Graphical Methods of Data Analysis > Nonparametric Methods

The Gibbs sampler is a simple but very powerful algorithm used to simulate from a complex high‐dimensional distribution. It is particularly useful in Bayesian analysis when a complex Bayesian model involves a number of model parameters and the conditional posterior distribution of each component given the others can be derived as a standard distribution. In the presence of a strong correlation structure among components, however, the Gibbs sampler can be criticized for its slow convergence. Here we discuss several algorithmic strategies such as blocking, collapsing, and partial collapsing that are available for improving the convergence characteristics of the Gibbs sampler.This article is categorized under:Statistical and Graphical Methods of Data Analysis > Markov Chain Monte CarloStatistical and Graphical Methods of Data Analysis > Bayesian Methods and TheoryStatistical and Graphical Methods of Data Analysis > Sampling

Over the last four decades, the bootstrap method has been considered so as to define data‐driven bandwidth selectors for nonparametric curve estimation. An extensive and updated review of bootstrap methods used to select the smoothing parameter for the nonparametric estimation of several curves has been carried out. Different data generating processes have been profoundly reviewed, such as the classical independent and identically distributed setup as well as dependent, censored, length‐biased, grouped, missing or directional data, among others. Several curves have also been considered, such as the density, regression, hazard rate, intensity, latency, incidence, or distribution functions, among many others. It is worth mentioning that there exist situations where Monte Carlo methods are not needed when using the bootstrap for bandwidth selections. This idea has also been exploited to find closed expressions for the bootstrap version of criterion functions for some proxy of the real curve estimator.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Bootstrap and Resampling Statistical and Graphical Methods of Data Analysis > Nonparametric Methods Statistical and Graphical Methods of Data Analysis > Density Estimation

This article is a review article on the optimal Markov chain Monte Carlo (MCMC) sampling. The focus is on homogeneous Markov chains. This article first reviews the problem of finding the optimal transition matrix, which is defined to minimize the asymptotic variance of MCMC estimators. The article later reviews the locally optimal sampler (LOS), an MCMC sampling that performs local updates based on the optimal transition matrix. We conducted a simulation study to compare the LOS with the Metropolis–Hastings and the Gibbs Sampler. The LOS was shown to provide an improved rate of convergence over these two most popular sampling schemes. The implementation of the LOS requires only minor modifications in existing Gibbs sampling code. WIREs Comput Stat 2013. doi: 10.1002/wics.1265This article is categorized under: Statistical and Graphical Methods of Data Analysis > Markov Chain Monte Carlo (MCMC)

Energy distance is a metric that measures the distance between the distributions of random vectors. Energy distance is zero if and only if the distributions are identical, thus it characterizes equality of distributions and provides a theoretical foundation for statistical inference and analysis. Energy statistics are functions of distances between observations in metric spaces. As a statistic, energy distance can be applied to measure the difference between a sample and a hypothesized distribution or the difference between two or more samples in arbitrary, not necessarily equal dimensions. The name energy is inspired by the close analogy with Newton's gravitational potential energy. Applications include testing independence by distance covariance, goodness‐of‐fit, nonparametric tests for equality of distributions and extension of analysis of variance, generalizations of clustering algorithms, change point analysis, feature selection, and more. WIREs Comput Stat 2016, 8:27–38. doi: 10.1002/wics.1375This article is categorized under: Statistical and Graphical Methods of Data Analysis > Multivariate Analysis Statistical and Graphical Methods of Data Analysis > Nonparametric Methods

While permutation tests and bootstraps have very wide‐ranging application, both share a common potential drawback: as data‐intensive resampling methods, both can be runtime prohibitive when applied to large‐ or even medium‐sized data samples drawn from large datasets. The data explosion over the past few decades has made this a common occurrence, and it highlights the increasing need for faster, and more efficient and scalable, permutation test and bootstrap algorithms. Seven bootstrap and six permutation test algorithms coded in SAS (the largest privately owned software firm globally) are compared herein. The fastest algorithms (‘OPDY’ for the bootstrap, ‘OPDN’ for permutation tests) are new, use no modules beyond Base SAS, and achieve speed increases orders of magnitude faster than the relevant ‘built‐in’ SAS procedures (OPDY is over 200× faster than Proc SurveySelect; OPDN is over 240× faster than Proc SurveySelect, over 350× faster than NPAR1WAY (which crashes on datasets less than a 10th the size OPDN can handle), and over 720× faster than Proc Multtest). OPDY also is much faster than hashing, which crashes on datasets smaller—sometimes by orders of magnitude—than OPDY can handle. OPDY is easily generalizable to multivariate regression models, and OPDN, which uses an extremely efficient draw‐by‐draw random‐sampling‐without‐replacement algorithm, can use virtually any permutation statistic, so both have a very wide range of application. And the time complexity of both OPDY and OPDN is sublinear, making them not only the fastest but also the only truly scalable bootstrap and permutation test algorithms, respectively, in SAS. WIREs Comput Stat 2013. doi: 10.1002/wics.1266This article is categorized under: Statistical and Graphical Methods of Data Analysis > Bootstrap and Resampling Statistical and Graphical Methods of Data Analysis > Nonparametric Methods Software for Computational Statistics > Software/Statistical Software

Bin smoothers, or regressograms, are piecewise constant regression function estimators whose values are averages of the response variable over the sets of a partition of the space of the explanatory variables. First, we review results about bin smoothers whose partition is regular, giving conditions for consistency and for achieving the optimal rate of convergence. Second, we review representative results about bin smoothers whose partition is irregular, again giving conditions for consistency and for achieving the optimal rate of convergence. Third, we give an exposition of recursive partitioning, main examples being greedy partitions and the classification and regression tress (CART) methodology. WIREs Comput Stat 2012 doi: 10.1002/wics.1214This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Classification and Regression Trees (CART) Statistical and Graphical Methods of Data Analysis > Density Estimation Statistical and Graphical Methods of Data Analysis > Nonparametric Methods

ABSTRACTVisual statistical inference determines the significance of patterns found in data exploration through graphics. It involves human observers inspecting a lineup of plots, with one real data plot randomly placed among decoys. Each observer's cognitive skills and judiciousness can influence results. The effectiveness of this method, measured by power, depends on combining evaluations from multiple observers. Human factors influencing power, as computed by the number of detections or identifications of an observed data plot in a lineup, include observer demographics, individual skills, and experience. This paper examines these factors through studies using Amazon's Mechanical Turk, finding individual skills vary but demographics have little impact. Learning increases speed but not accuracy.This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Exploratory Data Analysis Statistical and Graphical Methods of Data Analysis > Statistical Graphics and Visualization Statistical and Graphical Methods of Data Analysis > Nonparametric Methods

Information security technology is an essential component for protecting public and private computing infrastructures. With the widespread utilization of information technology applications, organizations are becoming more aware of the security threats to their resources. No matter how strict the security policies and mechanisms are, more organizations are becoming susceptible to a wide range of security breaches against their electronic resources. Network‐intrusion detection is an essential defense mechanism against security threats, which have been increasing in rate lately. It is defined as a special form of cyber threat analysis to identify malicious actions that could affect the integrity, confidentiality, and availability of information resources. Data mining‐based intrusion‐detection mechanisms are extremely useful in discovering security breaches. This article will provide an overview of the applications of data mining techniques in the information security domain. The focus will be on applying data mining to intrusion detection and intrusion prevention. This article will present a critical overview of the research directions in these fields, which will help researchers identify the key practical and research issues essential for building a successful network‐protection system. WIREs Comp Stat 2011 3 221–229 DOI: 10.1002/wics.161This article is categorized under: Statistical and Graphical Methods of Data Analysis > Data Reduction, Smoothing, and Filtering Algorithms and Computational Methods > Networks and Security Statistical Learning and Exploratory Methods of the Data Sciences > Text Mining

Because of its many practical applications, classifying functional data has received considerable attention over the last decades. Most classification approaches for functional data are extended from those for multivariate data. During the extension, two strategies, namely filtering and regularization, have commonly been employed to tackle the issues raised by the fact that functional data are intrinsically infinite‐dimensional. Because of space limitations, we focus on the filtering methods in this review.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Analysis of High Dimensional Data Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification

Correspondence analysis (CA) is a method of data visualization that is applicable to cross‐tabular data such as counts, compositions, or any ratio‐scale data where relative values are of interest. All the data should be on the same scale and the row and column margins of the table must make sense as weighting factors because the analysis gives varying importance to the respective rows and columns according to these margins. This method is one of a large class of methods based on the singular value decomposition and can be considered as the equivalent of principal component analysis for categorical and ratio‐scale data or as a pair of classical scalings of the rows and columns based on their interpoint χ2 distances, using the margins as weights. For categorical data, this method generalizes to multiple CA, a popular method for analyzing questionnaire data. A linearly constrained form of CA, canonical CA, is extensively used in ecological research where species abundance data at various sampling points are visualized subject to being linearly related to environmental variables measured at the same locations. When certain parameters are introduced into its definition, CA has been shown to have limiting cases of unweighted and weighted log‐ratio analysis (the latter also known as the spectral map), as well as classical multidimensional scaling. Copyright © 2010 John Wiley & Sons, Inc.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Multivariate Analysis

Gene set analysis (GSA), a popular approach for analyzing high‐throughput gene expression data, aims to identify sets of related genes that show significantly enriched or depleted expression patterns between different conditions. In the last years, a multitude of methods have been developed for this task. However, clear guidance is lacking: choosing the right method is the first hurdle a researcher is confronted with. No less challenging than overcoming this so‐called method uncertainty is the procedure of preprocessing, from knowing which steps are required to selecting a corresponding approach from the plethora of valid options to create the accepted input object (data preprocessing uncertainty), with clear guidance again being scarce. Here, we provide a practical guide through all steps required to conduct GSA, beginning with a concise overview of a selection of established methods, including Gene Set Enrichment Analysis and Database for Annotation, Visualization, and Integrated Discovery (DAVID). We thereby lay a special focus on reviewing and explaining the necessary preprocessing steps for each method under consideration (e.g., the necessity of a transformation of the RNA sequencing data)—an essential aspect that is typically paid only limited attention to in both existing reviews and applications. To raise awareness of the spectrum of uncertainties, our review is accompanied by an extensive overview of the literature on valid approaches for each step and illustrative R code demonstrating the complex analysis pipelines. It ends with a discussion and recommendations to both users and developers to ensure that the results of GSA are, despite the above‐mentioned uncertainties, replicable and transparent.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Analysis of High Dimensional Data Data: Types and Structure > Data Preparation and Processing Applications of Computational Statistics > Genomics/Proteomics/Genetics

The minimum covariance determinant (MCD) estimator is a highly robust estimator of multivariate location and scatter. It can be computed efficiently with the FAST‐MCD algorithm of Rousseeuw and Van Driessen. Since estimating the covariance matrix is the cornerstone of many multivariate statistical methods, the MCD has also been used to develop robust and computationally efficient multivariate techniques.In this paper, we review the MCD estimator, along with its main properties such as affine equivariance, breakdown value, and influence function. We discuss its computation, and list applications and extensions of the MCD in theoretical and applied multivariate statistics. Copyright © 2009 John Wiley & Sons, Inc.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Robust Methods

This is the second part of a three part article examining methods for Bayesian estimation and tracking. In the first part we presented the general theory of Bayesian estimation where we showed that Bayesian estimation methods can be divided into two very general classes: a class where the observation conditioned posterior densities are propagated in time through a predictor/corrector method; and a second class where the first two moments are propagated in time, with state and observation moment prediction steps followed by state moment update steps that use the latest observations. In this second part, we make the assumption that all densities are Gaussian and, after applying an affine transformation and approximating all nonlinear functions by interpolating polynomials, we recover the sigma‐point class of Kalman filters, including the unscented, spherical simplex, and Gauss‐Hermite Kalman filters. In part 3, we will show that approximating a density by a set of Monte Carlo samples leads to particle filter methods, where the posterior density is propagated in time and moment integrals are approximated by sample moments. WIREs Comput Stat 2012 doi: 10.1002/wics.1215This article is categorized under: Statistical and Graphical Methods of Data Analysis > Bayesian Methods and Theory

Achieving robustness in product and process designs is of importance to various stakeholders such as manufacturers, suppliers, and consumers. As variability exists in all operations, it is desirable to create products and processes that are not very sensitive to factors that are not controllable. The Taguchi method is an approach to robust design. Inherent in the Taguchi method is the definition of a loss function. This loss function formulation is influenced by the type of quality characteristic under consideration, that is, smaller‐is‐better, larger‐is‐better, or target‐is‐best. Furthermore, based on the selected type of quality characteristic, a performance measure is defined. Such performance measures, usually called signal‐to‐noise (S/N) ratios, are used to determine optimal settings of the controllable factors. Typically, a two‐step procedure is adopted in the Taguchi method. In the first step, the S/N ratio is maximized, whereas in the second step, using an adjustment factor that does not affect the S/N ratio, the mean response is adjusted to meet the target value, where appropriate. Experimental designs make use of orthogonal arrays to determine factor settings for obtaining data for subsequent analysis. The number of experimental runs is very modest in relation to the number of factors being investigated. WIREs Comp Stat 2011 3 472–480 DOI: 10.1002/wics.169This article is categorized under: Statistical and Graphical Methods of Data Analysis > Robust Methods Data: Types and Structure > Time Series, Stochastic Processes, and Functional Data