Orthogonalized smoothing for rescaled spike and slab models

Prediction of soil organic carbon content by diffuse reflectance spectroscopy using a local partial least square regression approach

Summary We need to determine the best use of soil vis– NIR spectral libraries that are being developed at regional, national and global scales to predict soil properties from new spectral readings. To reduce the complexity of a calibration dataset derived from the C hinese vis– NIR soil spectral library ( CSSL ), we tested a local regression method that combined geographical sub‐setting with a local partial least squares regression (local‐ PLSR ) that uses a limited number of similar vis– NIR spectra ( k ‐nearest neighbours). The central idea of the local regression, and of other local statistical approaches, is to derive a local prediction model by identifying samples in the calibration dataset that are similar, in spectral variable space, to the samples used for prediction. Here, to derive our local regressions we used E uclidean distance in spectral space between the calibration dataset and prediction samples, and we also used soil geographical zoning to account for similarities in soil‐forming conditions. We tested this approach with the CSSL , which comprised 2732 soil samples collected from 20 provinces in the P eople's R epublic of C hina to predict soil organic matter ( SOM ). Results showed that the prediction accuracy of our spatially constrained local‐ PLSR method ( R 2 = 0.74, RPIQ = 2.6) was better than that from local‐ PLSR ( R 2 = 0.69, RPIQ = 2.3) and PLSR alone ( R 2 = 0.50, RPIQ = 1.5). The coupling of a local‐ PLSR regression with soil geographical zoning can improve the accuracy of local SOM predictions using large, complex soil spectral libraries. The approach might be embedded into vis– NIR sensors for laboratory analysis or field estimation.

Locally weighted regression with different kernel smoothers for software effort estimation

Time-varying covariance is an important metric to measure the statistical dependence between non-stationary biological processes. Time-varying covariance is conventionally estimated from short-time data segments within a window having a certain bandwidth, but it is difficult to choose an appropriate bandwidth to estimate covariance with different degrees of non-stationarity. This paper introduces a local polynomial regression (LPR) method to estimate time-varying covariance and performs an asymptotic analysis of the LPR covariance estimator to show that both the estimation bias and variance are functions of the bandwidth and there exists an optimal bandwidth to minimize the mean square error (MSE) locally. A data-driven variable bandwidth selection method, namely the intersection of confidence intervals (ICI), is adopted in LPR for adaptively determining the local optimal bandwidth that minimizes the MSE. Experimental results on simulated signals show that the LPR-ICI method can achieve robust and reliable performance in estimating time-varying covariance with different degrees of variations and under different noise scenarios, making it a powerful tool to study the dynamic relationship between non-stationary biomedical signals. Further, we apply the LPR-ICI method to estimate time-varying covariance of functional magnetic resonance imaging (fMRI) signals in a visual task for the inference of dynamic functional brain connectivity. The results show that the LPR-ICI method can effectively capture the transient connectivity patterns from fMRI.

Tests for Gaussian graphical models

Optimal bandwidths for local polynomial regression usually involve functionals of the derivatives of the unknown regression function. In the multivariate case, estimates of these functionals are not readily available, primarily because estimating multivariate derivatives is complicated. In this paper, an estimator of multivariate second derivative is obtained via local quadratic regression with cross terms left out. This estimator has the optimal rate of convergence but is simpler and uses a lot less computing time than the full local quadratic estimator. Using this as a pilot estimator, an estimator of the integrated squared Laplacian of a multivariate regression function is obtained which leads to a plug-in formula of the optimal bandwidth for multivariate local linear regression. This bandwidth has good theoretical properties as well as satisfactory performance in our simulation study. It is also recommended for variable selection methods.

We have developed a tool for model space exploration and variable selec tion in linear regression models based on a simple spike and slab model (Dey, 2012). The model chosen is the best model with minimum final prediction error (FPE) values among all other models. This is implemented via the R package modelSampler. However, model selection based on FPE criteria is dubious and question able as FPE criteria can be sensitive to perturbations in the data. This R package can be used for empirical assessment of the stability of FPE criteria. A stable model selection is accomplished by using a bootstrap wrapper that calls the primary function of the package several times on the bootstrapped data. The heart of the method is the notion of model averaging for sta ble variable selection and to study the behavior of variables over the entire model space, a concept invaluable in high dimensional situations.

Covariance estimation for multiple groups is a key feature for drawing inference from a heterogeneous population. One should seek to share information about common features in the dependence structures across the various groups. In this paper, we introduce a novel approach for estimating the covariance matrices for multiple groups using a hierarchical latent factor model that shrinks the factor loadings across groups toward a global value. Using a sparse spike and slab model on these loading coefficients allows for a sparse formulation of our model. Parameter estimation is accomplished through a Markov chain Monte Carlo scheme, and a model selection approach is used to select the number of factors to use. We validate our model through extensive simulation studies. Finally, we apply our methodology to the NICHD Consecutive Pregnancies Study to estimate the correlations between birth weights and gestational ages of three consecutive birth within four different subgroups (underweight, normal, overweight, and obese) of women.

There are many studies on nondestructive evaluation using Near Infra-Red (NIR) Spectroscopy to determine the quality parameters of mangoes. The predictor variables used for predicting the mango quality parameters in the form of spectra values are generated from NIR spectroscopy. One parameter of the qualities of fruits is the fruit acidity. In general, the lower acidity of mangoes is the riper of the mangoes. Prediction of the acidity level of avomango by using one predictor still produces a high MAPE value of more than 40 percent, so that multi-predictor variables are needed to improve the prediction performance. This study aims to predict the acidity of Avomango (Gadung Klonal 21) by using the multi-predictor local polynomial regression approach and compare it with multiple polynomial regression. This study uses 100 divided samples into two parts, 80 in-sample data, and 20 out-sample data. The results showed that the multi-predictor local polynomial regression model gave highly prediction results for predicting acidity level of Avomango (Gadung Klonal 21) with MAPE value of 6.23 percent that is better than the MAPE value of MPR parametric approach of 10.79 percent.

Fitting statistical models is a widely employed technique for the segmentation of medical images. While this approach gives impressive results for simple structures, shape models are often not flexible enough to accurately represent complex shapes. We present a fitting approach, which increases the model fitting accuracy without requiring a larger training data-set. Inspired by a local regression approach known from statistics, our method fits the full model to a neighborhood around each point of the domain. This increases the model's flexibility considerably without the need to introduce an artificial segmentation of the structure. By adapting the size of the neighborhood from small to large, we can smoothly interpolate between localized fits, which accurately map the data but are more prone to noise, and global fits, which are less flexible but constrained to valid shapes only. We applied our method for the segmentation of teeth from 3D cone-beam ct-scans. Our experiments confirm that our method consistently increases the precision of the segmentation result compared to a standard global fitting approach.

In high dimensional setting, componentwise L 2boosting method has been used to construct sparse model of high prediction, but it tends to select many ineffective variables. Several sparse boosting methods, such as, SparseL 2Boosting and Twin Boosting, have been proposed to improve the variable selection of L 2boosting algorithm. In this paper, we propose a new general sparse boosting method (GSBoosting). The relations are established between GSBoosting and other well known regularized variable selection methods in orthogonal linear model, such as adaptive Lasso, hard thresholds etc. Simulations results show that GSBoosting has good performance in both prediction and variable selection.

Many clinical or prevention studies involve missing or censored outcomes. Maximum likelihood (ML) methods provide a conceptually straightforward approach to estimation when the outcome is partially missing. Methods of implementing ML methods range from the simple to the complex, depending on the type of data and the missing-data mechanism. Simple ML methods for ignorable missing-data mechanisms (when data are missing at random) include complete-case analysis, complete-case analysis with covariate adjustment, survival analysis with covariate adjustment, and analysis via propensity-to-be-missing scores. More complex ML methods for ignorable missing-data mechanisms include the analysis of longitudinal dropouts via a marginal model for continuous data or a conditional model for categorical data. A moderately complex ML method for categorical data with a saturated model and either ignorable or nonignorable missing-data mechanisms is a perfect fit analysis, an algebraic method involving closed-form estimates and variances. A complex and flexible ML method with categorical data and either ignorable or nonignorable missing-data mechanisms is the method of composite linear models, a matrix method requiring specialized software. Except for the method of composite linear models, which can involve challenging matrix specifications, the implementation of these ML methods ranges in difficulty from easy to moderate.

We propose a communication-efficient distributed learning algorithm for high-dimensional sparse linear regression models in the scenario that the data are stored across multiple machines. Our approach is a distributed version of the SDAR [Huang J, Jiao Y, Liu Y, et al. A constructive approach to l0 penalized regression. J Mach Learn Res. 2018;19(1):403–439] method for solving the KKT system of the regularized least squares. At each step of the proposed method, the reduced least squares are solved by the steepest descent method, which only needs to calculate the gradient vectors on each node machine and communicate them instead of the data. We refer to this as SD-SDAR for brevity. Under some regular conditions, we obtain the sharp and error bounds for the solution sequences generated by SD-SDAR algorithm. We investigate the computational complexity and show that the number of rounds of communications are bounded by and , respectively, where J is the number of important predictors, R is the relative magnitude of the non-zero target coefficients, N is the total sample size and p is the dimension of covariates. Simulation studies illustrate that SD-SDAR outperforms some existing distributed methods in accuracy, efficiency and support recovery.

We consider the problem of variable selection using Bayesian method in high-dimensional linear models, where the number of explanatory variables K n is possibly much larger than the sample size n. In particular, the true regression coefficients vector β* is required sparsity in the sense of , which describes a general situation when all explanatory variables are relevant, but most of them have very small effects. According to Bayesian inference of Wasserman (1998), given a proper prior to propose joint densities f(y, x) of response y and explanatory variables vector x, the posterior-proposed densities are often close to the true density f 0(y, x), for large n. It's the so-called “density consistency”. We aim to prove that in linear models, density consistency can induce regression function consistency, which ensures good performance of variable selection. Especially in the special case of , when some regression coefficients are bounded away from zero while the rest are exactly zero, our method would identify the underlying true model by giving consistent estimate of true regression coefficients vector. We also provide simulation studies and a real data example to demonstrate satisfactory finite-sample performance of our method.

Semiparametric accelerated failure time (AFT) models are a useful alternative to Cox proportional hazards models, especially when the assumption of constant hazard ratios is untenable. However, rank-based criteria for fitting AFT models are often nondifferentiable, which poses a computational challenge in high-dimensional settings. In this article, we propose a new alternating direction method of multipliers algorithm for fitting semiparametric AFT models by minimizing a penalized rank-based loss function. Our algorithm scales well in both the number of subjects and number of predictors, and can easily accommodate a wide range of popular penalties. To improve the selection of tuning parameters, we propose a new criterion which avoids some common problems in cross-validation with censored responses. Through extensive simulation studies, we show that our algorithm and software is much faster than existing methods (which can only be applied to special cases), and we show that estimators which minimize a penalized rank-based criterion often outperform alternative estimators which minimize penalized weighted least squares criteria. Application to nine cancer datasets further demonstrates that rank-based estimators of semiparametric AFT models are competitive with estimators assuming proportional hazards in high-dimensional settings, whereas weighted least squares estimators are often not. A software package implementing the algorithm, along with a set of auxiliary functions, is available for download at github.com/ajmolstad/penAFT.