Cutting-plane algorithm for estimation of sparse Cox proportional hazards models

The present paper is about estimation and prediction in high-dimensional additive models under a sparsity assumption ($p\\gg n$ paradigm). A PAC-Bayesian strategy is investigated, delivering oracle inequalities in probability. The implementation is performed through recent outcomes in high-dimensional MCMC algorithms, and the performance of our method is assessed on simulated data.

A Bayesian network interpretation of the Cox's proportional hazard model

ABSTRACTIn this paper, we provide a full Bayesian analysis for Cox's proportional hazards model under different hazard rate shape assumptions. To this end, we select the modified Weibull distribution family to model failure rates. A novel Markov chain Monte Carlo method allows one to tackle both exact and right-censored failure time data. Both simulated and real data are used to illustrate the methods.

As a useful alternative to Cox's proportional hazard model, the additive risk model assumes that the hazard function is the sum of the baseline hazard function and the regression function of covariates. This article is concerned with estimation and prediction for the additive risk models with right censored survival data, especially when the dimension of the covariates is comparable to or larger than the sample size. Principal component regression is proposed to give unique and numerically stable estimators. Asymptotic properties of the proposed estimators, component selection based on the weighted bootstrap, and model evaluation techniques are discussed. This approach is illustrated with analysis of the primary biliary cirrhosis clinical data and the diffuse large B-cell lymphoma genomic data. It is shown that this methodology is numerically stable and effective in dimension reduction, while still being able to provide satisfactory prediction and classification results.

In this work, we addressed parameter estimation and prediction in the high-dimensional sparse logistic regression model through both Monte Carlo simulations and application to real data. We applied two well-known penalized maximum likelihood (ML) methods (LASSO and aLASSO) for variable screening. There may exist overfitting from LASSO or underfitting from aLASSO, making ML estimators based on these models inefficient. Hence, after performing variable selection, we proposed post-selection improved estimation based on linear shrinkage, pretest, and James-Stein shrinkage strategies, which efficiently combine overfitted and underfitted ML estimators. Regardless of the correctness in the variable selection stage, the proposed estimators were shown to be more efficient than the classical ML estimators, which were severely affected by inappropriate variable selection.

Threshold autoregressive models have a wide range of applications in the field of econometrics. This paper considers the Bayesian empirical likelihood (BEL) shrinkage estimation and order selection for a class of sparse threshold autoregressive models. With the help of Taylor explanation, a “normally distributed” nonparametric likelihood is obtained. Based on this likelihood, the BEL estimation is well addressed. By using the Markov Chain Monte Carlo (MCMC) techniques under a Bayesian hierarchical model, the order and the non-zero autoregressive coefficients of the model can be accurately determined. Some simulation studies are conducted to show the performances of the proposed methods. Finally, an application to the US gross national product data set is provided.

Identification of sparse high dimensional linear\nsystems pose sever challenges to off-the-shelf techniques for\nsystem identification. This is particularly so when relatively\nsmall data sets, as compared to the number of inputs and\noutputs, have to be used.\nWhile input/output selection could be performed via standard\nselection techniques, computational complexity may however\nbe a critical issue, being combinatorial in the number of\ninputs and outputs.\nParametric estimation techniques which result in sparse\nmodels have nowadays become very popular and include,\namong others, the well known Lasso, LAR and their “grouped”\nversions Group Lasso and Group LAR.\nIn this paper we introduce a new nonparametric technique\nwhich borrows ideas from a recently introduced Kernel estimator\ncalled “stable-spline” as well as from sparsity inducing\npriors which use l1 penalty. We compare the new method with\na group LAR-type of algorithm applied to estimation of sparse\nVector Autoregressive models and to standard PEM methods.

A sparse precision matrix can be directly translated into a sparse Gaussian graphical model under the assumption that the data follow a joint normal distribution. This neat property makes high-dimensional precision matrix estimation very appealing in many applications. However, in practice we often face nonnormal data, and variable transformation is often used to achieve normality. In this paper we consider the nonparanormal model that assumes that the variables follow a joint normal distribution after a set of unknown monotone transformations. The nonparanormal model is much more flexible than the normal model while retaining the good interpretability of the latter in that each zero entry in the sparse precision matrix of the nonparanormal model corresponds to a pair of conditionally independent variables. In this paper we show that the nonparanormal graphical model can be efficiently estimated by using a rank-based estimation scheme which does not require estimating these unknown transformation functions. In particular, we study the rank-based graphical lasso, the rank-based neighborhood Dantzig selector and the rank-based CLIME. We establish their theoretical properties in the setting where the dimension is nearly exponentially large relative to the sample size. It is shown that the proposed rank-based estimators work as well as their oracle counterparts defined with the oracle data. Furthermore, the theory motivates us to consider the adaptive version of the rank-based neighborhood Dantzig selector and the rank-based CLIME that are shown to enjoy graphical model selection consistency without assuming the irrepresentable condition for the oracle and rank-based graphical lasso. Simulated and real data are used to demonstrate the finite performance of the rank-based estimators.

Robust selection and estimation for sparse multivariate functional nonparametric additive models via regularized Huber regression

Motivated by the many applications associated with estimation of sparse multivariate models, the estimation of sparse directional connectivity between the imperfectly measured nodes of a network is studied. Node dynamics and interactions are assumed to follow a multivariate autoregressive model driven by noise, and the observations are a noisy linear combination of the underlying node activities. The corresponding maximum a posteriori (MAP) problem is derived to estimate system parameters. Due to the intractability of the MAP problem, the expectation maximization (EM) framework is used to iteratively implement the MAP estimation. To impose sparsity, the EM algorithm is augmented with an $\ell _1$ regularization of the connectivity matrix. Multiple techniques have been used to lower the computational complexity. Importantly, an efficient coordinate descent algorithm utilizing a closed-form solution is designed to solve the $\ell _1$ -regularized sub-problem of the EM. Cholesky factors of the unknown covariance matrices are used directly in the optimization process in order to impose positive definiteness and guarantee the functionality of the $\ell _1$ optimization. The algorithm is first applied to synthetic data to evaluate the estimation accuracy. A comparison with the previous work over an extensive set of configurations shows that our method is superior under moderate to high sparsity. Finally, the algorithm is applied to real temperature data obtained from 98 stations across the U.S. mainland to identify the predictive interactions between the time series.

High-dimensional time series data appear in many scientific areas in the current data-rich environment. Analysis of such data poses new challenges to data analysts because of not only the complicated dynamic dependence between the series, but also the existence of aberrant observations, such as missing values, contaminated observations, and heavy-tailed distributions. For high-dimensional vector autoregressive (VAR) models, we introduce a unified estimation procedure that is robust to model misspecification, heavy-tailed noise contamination, and conditional heteroscedasticity. The proposed methodology enjoys both statistical optimality and computational efficiency, and can handle many popular high-dimensional models, such as sparse, reduced-rank, banded, and network-structured VAR models. With proper regularization and data truncation, the estimation convergence rates are shown to be almost optimal in the minimax sense under a bounded (2+2ϵ)th moment condition. When ϵ≥1, the rates of convergence match those obtained under the sub-Gaussian assumption. Consistency of the proposed estimators is also established for some ϵ∈(0,1), with minimax optimal convergence rates associated with ϵ. The efficacy of the proposed estimation methods is demonstrated by simulation and a U.S. macroeconomic example.

We propose Bayesian methods for Gaussian graphical models that lead to sparse and adaptively shrunk estimators of the precision (inverse covariance) matrix. Our methods are based on lasso-type regularization priors leading to parsimonious parameterization of the precision matrix, which is essential in several applications involving learning relationships among the variables. In this context, we introduce a novel type of selection prior that develops a sparse structure on the precision matrix by making most of the elements exactly zero, in addition to ensuring positive definiteness - thus conducting model selection and estimation simultaneously. More importantly, we extend these methods to analyze clustered data using finite mixtures of Gaussian graphical model and infinite mixtures of Gaussian graphical models. We discuss appropriate posterior simulation schemes to implement posterior inference in the proposed models, including the evaluation of normalizing constants that are functions of parameters of interest, which result from the restriction of positive definiteness on the correlation matrix. We evaluate the operating characteristics of our method via several simulations and demonstrate the application to real data examples in genomics.

Current sensor-less state-of-charge estimation algorithm for lithium-ion batteries utilizing filtered terminal voltage

Methods are provided for regression parameter and cumulative baseline hazard estimation in the Cox proportional hazards model when the cohort is sampled according to a predictable sampling probability law. The key to the methodology is to define counting processes which count joint failure and sampled risk sets occurrences and to derive the appropriate intensities for these counting processes, leading to estimation methods parallel to those for full cohort data. These techniques are illustrated for a number of sampling designs, including three novel designs: counter-matching with additional randomly sampled controls; quota sampling of controls; and nested case-control sampling with number of controls dependent on the failure's exposure status. General asymptotic theory is developed for the maximum partial likelihood estimator and cumulative baseline hazard estimator and is used to derive the asymptotic distributions for estimators from a large class of designs.

Online adaptive lasso estimation in vector autoregressive models for high dimensional wind power forecasting