Mapper and Ball Mapper are Topological Data Analysis tools used for exploring high dimensional point clouds and visualizing scalar–valued functions on those point clouds. Inspired by open questions in knot theory, new features are added to Ball Mapper that enable encoding of the structure, internal relations and symmetries of the point cloud. Moreover, the strengths of Mapper and Ball Mapper constructions are combined to create a tool for comparing high dimensional data descriptors of a single dataset. This new hybrid algorithm, Mapper on Ball Mapper, is applicable to high dimensional lens functions. As a proof of concept we include applications to knot and game theory.

Quantile regression and conditional density estimation can reveal structure that is missed by mean regression, such as multimodality and skewness. In this article, we introduce a deep learning generative model for joint quantile estimation called Penalized Generative Quantile Regression (PGQR). Our approach simultaneously generates samples from many random quantile levels, allowing us to infer the conditional distribution of a response variable given a set of covariates. Our method employs a novel variability penalty to avoid the problem of vanishing variability, or memorization, in deep generative models. Further, we introduce a new family of partial monotonic neural networks (PMNN) to circumvent the problem of crossing quantile curves. A major benefit of PGQR is that it can be fit using a single optimization, thus, bypassing the need to repeatedly train the model at multiple quantile levels or use computationally expensive cross-validation to tune the penalty parameter. We illustrate the efficacy of PGQR through extensive simulation studies and analysis of real datasets. Code to implement our method is available at https://github.com/shijiew97/PGQR.

The simultaneous clustering of observations and features of datasets (known as co-clustering) has recently emerged as a central machine learning application to summarize massive datasets. However, most existing models focus on continuous and dense data in stationary scenarios, where cluster assignments do not evolve over time. This work introduces a novel latent block model for the dynamic co-clustering of data matrices with high sparsity. To properly model this type of data, we assume that the observations follow a time and block dependent mixture of zero-inflated distributions, thus, combining stochastic processes with the time-varying sparsity modeling. To detect abrupt changes in the dynamics of both cluster memberships and data sparsity, the mixing and sparsity proportions are modeled through systems of ordinary differential equations. The inference relies on an original variational procedure whose maximization step trains fully connected neural networks in order to solve the dynamical systems. Numerical experiments on simulated and real world datasets demonstrate the effectiveness of the proposed methodology in the context of count data.

This study proposes an interpretable neural network-based nonproportional odds model (N3POM) for ordinal regression. N3POM is different from conventional approaches to ordinal regression with nonproportional models in several ways: (a) N3POM is defined for both continuous and discrete responses, whereas standard methods typically treat the continuous variables as if they were discrete, (b) instead of estimating response-dependent finite-dimensional coefficients of linear models from discrete responses as is done in conventional approaches, we train a nonlinear neural network to serve as a coefficient function. Thanks to the neural network, N3POM offers flexibility while preserving the interpretability of conventional ordinal regression. We establish a sufficient condition under which the predicted conditional cumulative probability locally satisfies the monotonicity constraint over a user-specified region in the covariate space. Additionally, we provide a monotonicity-preserving stochastic (MPS) algorithm for effectively training the neural network. We apply N3POM to several real-world datasets. Supplementary materials for this article are available online.

Inferring the parameters of ordinary differential equations (ODEs) from noisy observations is an important problem in many scientific fields. Currently, most parameter estimation methods that bypass numerical integration tend to rely on basis functions or Gaussian processes to approximate the ODE solution and its derivatives. Due to the sensitivity of the ODE solution to its derivatives, these methods can be hindered by estimation error, especially when only sparse time-course observations are available. We present a Bayesian collocation framework that operates on the integrated form of the ODEs and also avoids the expensive use of numerical solvers. Our methodology has the capability to handle general nonlinear ODE systems. We demonstrate the accuracy of the proposed method through simulation studies, where the estimated parameters and recovered system trajectories are compared with other recent methods. A real data example is also provided. Supplementary materials for this article are available online.

Matrix factorization exploits the idea that, in complex high-dimensional data, the actual signal typically lies in lower-dimensional structures. These lower dimensional objects provide useful insights, with interpretation favored by sparse structures. Sparsity, in addition, is beneficial in terms of regularization and, thus, to avoid over-fitting. By exploiting Bayesian shrinkage priors, we devise a computationally convenient approach for high-dimensional matrix factorization. The dependence between row and column entities is modeled by inducing flexible sparse patterns within factors. The availability of external information is accounted for in such a way that structures are allowed while not imposed. Inspired by boosting algorithms, we pair the proposed approach with a numerical strategy relying on a sequential inclusion and estimation of low-rank contributions, with a data-driven stopping rule. Practical advantages of the proposed approach are demonstrated by means of a simulation study and the analysis of soccer heatmaps obtained from new generation tracking data. Supplementary materials for this article are available online.

High-dimensional multivariate nonstationary time series, that is, data whose second order properties vary over time, are common in many scientific and industrial applications. In this article we propose a novel wavelet domain dimension reduction technique for nonstationary time series. By constructing a time-scale adaptive principal component analysis of the data, our proposed method is able to capture the salient dynamic features of the multivariate time series. We also introduce a new time and scale dependent cross-coherence measure to quantify the extent of association between a multivariate nonstationary time series and its proposed wavelet domain principal component representation. Theoretical results establish that our associated estimation scheme enjoys good bias and consistency properties when determining wavelet domain principal components of input data. The proposed method is illustrated using extensive simulations and we demonstrate its applicability on a real-world dataset arising in a neuroscience study. Supplementary materials, with proofs of theoretical results, additional simulations and code, are available online.

Estimating free energy differences, an important problem in computational drug discovery and in a wide range of other application areas, commonly involves a computationally intensive process of sampling a family of high-dimensional probability distributions and a procedure for computing estimates based on those samples. The variance of the free energy estimate of interest typically depends strongly on how the total computational resources available for sampling are divided among the distributions, but determining an efficient allocation is difficult without sampling the distributions. Here we introduce the Times Square sampling algorithm, a novel on-the-fly estimation method that dynamically allocates resources in such a way as to significantly accelerate the estimation of free energies and other observables, while providing rigorous convergence guarantees for the estimators. We also show that it is possible, surprisingly, for on-the-fly free energy estimation to achieve lower asymptotic variance than the maximum-likelihood estimator MBAR, raising the prospect that on-the-fly estimation could reduce variance in a variety of other statistical applications. Supplementary materials for this article are available online.

There is increasing recognition that the order of administration of drugs in drug combination studies can markedly affect the outcome. Similarly, manufactured products are often sequentially produced and the final quality frequently depends on the order of assembly. Order-of-addition designs account for the order of administration of the components, and they are quite prevalent, yet research in this area is quite limited. Because of the large dimension of such optimization problems, analytical approaches are invariably very limited and apply to simple setups only. Numerical approaches are also seriously underdeveloped. To this end, we employ two exemplary nature-inspired metaheuristic algorithms, Differential Evolution (DE) and Particle Swarm Optimization (PSO), to search for efficient order-of-addition designs for two classes of important inferential problems: (a) estimating parameters in an imprecisely specified model, and (b) constructing space-filling designs without specifying a model. We evaluate the capability of DE and PSO to solve the two classes of order-of-addition design problems and compare their performance with other algorithms that have been used to tackle somewhat similar problems. Using different criteria, we demonstrate that DE and PSO clearly outperform current algorithms by a wide margin. Supplementary materials containing codes to generate all results in this article are available online.

The equality-constrained lasso problem augments the standard lasso by imposing additional structure on regression coefficients. Despite the broad utilities of the equality-constrained lasso, existing algorithms are typically computationally inefficient and only applicable to linear and logistic models. In this article, we devise a fast solution to the equality-constrained lasso problem with a two-stage algorithm: first obtaining candidate covariate subsets of increasing size from unconstrained lasso problems and then leveraging an efficient combined alternating direction method of multipliers/Newton-Raphson algorithm. Our proposed algorithm leads to substantial speedups in getting the solution path of the constrained lasso and can be easily adapted to generalized linear models and Cox proportional hazards models. We conduct extensive simulation studies to demonstrate the computational advantage of the proposed method over existing solvers. To further show the unique utility of our method, we consider two real-world data examples: a microbiome regression analysis and a myeloma survival analysis; neither example could be solved by naively fitting the constrained lasso problem on the full predictor set. Supplementary materials for this article are available online.