Empirical Bayes estimators for high-dimensional sparse vectors

Non-local means (NLM) provides a powerful framework for denoising. However, there are a few parameters of the algorithm-most notably, the width of the smoothing kernel-that are data-dependent and difficult to tune. Here, we propose to use Stein's unbiased risk estimate (SURE) to monitor the mean square error (MSE) of the NLM algorithm for restoration of an image corrupted by additive white Gaussian noise. The SURE principle allows to assess the MSE without knowledge of the noise-free signal. We derive an explicit analytical expression for SURE in the setting of NLM that can be incorporated in the implementation at low computational cost. Finally, we present experimental results that confirm the optimality of the proposed parameter selection.

Purpose: Parallel imaging methods in MRI have resulted in faster acquisition times and improved noise performance. ESPIRiT is one such technique that estimates coil sensitivity maps from the auto-calibration region using an eigenvalue-based method. This method requires choosing several parameters for the the map estimation. Even though ESPIRiT is fairly robust to these parameter choices, occasionally, poor selection can result in reduced performance. The purpose of this work is to automatically select parameters in ESPIRiT for more robust and consistent performance across a variety of exams. Theory and Methods: Stein's unbiased risk estimate (SURE) is a method of calculating an unbiased estimate of the mean squared error of an estimator under certain assumptions. We show that this can be used to estimate the performance of ESPIRiT. We derive and demonstrate the use of SURE to optimize ESPIRiT parameter selection. Results: Simulations show SURE to be an accurate estimator of the mean squared error. SURE is then used to optimize ESPIRiT parameters to yield maps that are optimal in a denoising/data-consistency sense. This improves g-factor performance without causing undesirable attenuation. In-vivo experiments verify the reliability of this method. Conclusion: Simulation experiments demonstrate that SURE is an accurate estimate of expected mean squared error. Using SURE to determine ESPIRiT parameters allows for automatic parameter selections.In-vivo results are consistent with simulation and theoretical results.

Recently, total variation (TV)-based regularization has become a standard technique for signal denoising. The reconstruction quality is generally sensitive to the value of regularization parameter. In this work, based on Cham-bolle's algorithm, we develop two data-driven optimization schemes based on minimization of Stein's unbiased risk estimate (SURE)-statistically equivalent to mean squared error (MSE). First, we propose a recursive evaluation of SURE to monitor the estimation error during Chambolle's iteration; the optimal value is then identified by the minimum SURE. Second, for fast optimization, we perform alternating update between regularization parameter and solution within Chambolle's iteration. We exemplify the proposed methods with both 1-D and 2-D signal denoising. Numerical experiments show that the proposed methods lead to highly accurate estimate of regularization parameter and nearly optimal denoising performance.

Ultrasound (US) image quality often suffers from random and speckle noise. Especially in the case of difficult patients, increased presence of random noise leads to compromised signal to noise ratio (SNR). The proposed technique removes random or speckle noise by promoting the low rank component of temporal US image sequences comprising of RF-data or envelope-detected data, respectively. This approach leverages the fact that due to tissue connectedness, tissue motion tends to exhibit limited temporal profiles. An approach to approximate the low rank component of the image sequence involves truncating its singular value decomposition. In the proposed algorithm, this is performed in a block-wise manner. Additionally, an adaptive singular value threshold is proposed based on Stein's unbiased risk estimator. This paper presents applications of this methodology on sequences of RF-data and envelope-detected data.

Regularized iterative reconstruction algorithms for imaging inverse problems require selection of appropriate regularization parameter values. We focus on the challenging problem of tuning regularization parameters for nonlinear algorithms for the case of additive (possibly complex) Gaussian noise. Generalized cross-validation (GCV) and (weighted) mean-squared error (MSE) approaches (based on Steinfs Unbiased Risk Estimate. SURE) need the Jacobian matrix of the nonlinear reconstruction operator (representative of the iterative algorithm) with respect to the data. We derive the desired Jacobian matrix for two types of nonlinear iterative algorithms: a fast variant of the standard iterative reweighted least-squares method and the contemporary split-Bregman algorithm, both of which can accommodate a wide variety of analysis- and synthesis-type regularizers. The proposed approach iteratively computes two weighted SURE-type measures: Predicted-SURE and Projected-SURE (that require knowledge of noise variance Ð2), and GCV (that does not need Ð2) for these algorithms. We apply the methods to image restoration and to magnetic resonance image (MRI) reconstruction using total variation (TV) and an analysis-type .1-regularization. We demonstrate through simulations and experiments with real data that minimizing Predicted-SURE and Projected-SURE consistently lead to near-MSE-optimal reconstructions. We also observed that minimizing GCV yields reconstruction results that are near-MSE-optimal for image restoration and slightly suboptimal for MRI. Theoretical derivations in this work related to Jacobian matrix evaluations can be extended, in principle, to other types of regularizers and reconstruction algorithms.

We consider geostatistical regression models to predict spatial variables of interest, where likelihood‐based methods are used to estimate model parameters. It is known that parameters in the Matérn covariogram cannot be estimated well, even when increasing amounts of data are collected densely in a fixed domain. Although a best linear unbiased predictor has been proposed when model parameters are known, a predictor with estimated parameters is nonlinear and may be not the best in practice. Therefore, we propose an adjusted procedure for the likelihood‐based estimates to improve the predicted ability of the nonlinear spatial predictor. The adjusted parameter estimators based on minimizing a corrected Stein's unbiased risk estimator tend to have less bias than the conventional likelihood‐based estimators, and the resulting spatial predictor is more accurate and more stable. Statistical inference for the proposed method is justified both theoretically and numerically. To verify the practicability of the proposed method, a groundwater data set in Bangladesh is analyzed.

We propose a new orthonormal wavelet thresholding algorithm for denoising color images that are assumed to be corrupted by additive Gaussian white noise of known intercolor covariance matrix. The proposed wavelet denoiser consists of a linear expansion of thresholding (LET) functions, integrating both the interscale and intercolor dependencies. The linear parameters of the combination are then solved for by minimizing Stein's unbiased risk estimate (SURE), which is nothing but a robust unbiased estimate of the mean squared error (MSE) between the (unknown) noise-free data and the denoised one. Thanks to the quadratic form of this MSE estimate, the parameters optimization simply amounts to solve a linear system of equations. The experimentations we made over a wide range of noise levels and for a representative set of standard color images have shown that our algorithm yields even slightly better peak signal-to-noise ratios than most state-of-the-art wavelet thresholding procedures, even when the latters are executed in an undecimated wavelet representation.

The spectrum of surface plasmon resonance shows different amplitudes and peak widths with theeffects of resonant intensity and position, so it's necessary to develop the self-adaptive methods to guarantee the accurate positioning of SPR formant. Based on the optimized Savitzky-Golay filter, this paper presents a method for SPR spectrum, which can optimize the parameters (polynomial degree and window size) according to the characteristics of the real time SPR spectrum, and minimizes the Stein's unbiased risk estimate of the mean squared error, and it can be quickly solved by Monte-Carlo methods. It is confirmed that SURE can accurately reflect the trends and the results of the true MSE. The experiment measures the SPR signals of sucrose under different concentrations and integration time through the SPR system. By using the proposed method of optimal parameters on the SPR signal filtering, the results show that the repeatability error of the position of SPR formant is smaller compared with the fixed parameters, and does not increase with the addition of noise levels.

Hyper-trim shrinkage for denoising of ECG signal

In this paper we present a new method for the pansharpening of multi-spectral satellite imagery. This method is based on a simple explicit image formation model which leads to an ill posed problem that needs to be regularized for best results. We use both Tikhonov (ridge regression) and Total Variation (TV) regularization. We develop the solutions to these two problems and then we address the problem of selecting the optimal regularization parameter λ. We find the value of λ that minimizes Stein's unbiased risk estimate (SURE). For ridge regression this leads to an analytical expression for SURE while for the TV regularized solution we use Monte Carlo SURE where the estimate is obtained by stochastic means. Finally, we present experiment results where we use quality metrics to evaluate the spectral and spatial quality of the resulting pansharpened image.

Asymptotic theory for the regularized system identification has received increasing interests in recent years. In this paper, for the finite impulse response (FIR) model and filtered white noise inputs, we show the convergence in distribution of the Stein's unbiased risk estimator (SURE) based hyper-parameter estimator and find factors that influence its convergence properties. In particular, we consider the ridge regression case to obtain closed-form expressions of the limit of the regression matrix and the variance of the limiting distribution of the SURE based hyper-parameter estimator, and then demonstrate their relation numerically.

We study the effective degrees of freedom of a general class of reduced-rank estimators for multivariate regression in the framework of Stein's unbiased risk estimation. A finite-sample exact unbiased estimator is derived that admits a closed-form expression in terms of the thresholded singular values of the least-squares solution and hence is readily computable. The results continue to hold in the high-dimensional setting where both the predictor and the response dimensions may be larger than the sample size. The derived analytical form facilitates the investigation of theoretical properties and provides new insights into the empirical behaviour of the degrees of freedom. In particular, we examine the differences and connections between the proposed estimator and a commonly-used naive estimator. The use of the proposed estimator leads to efficient and accurate prediction risk estimation and model selection, as demonstrated by simulation studies and a data example.

Multivariate regression is one of the most widely applied multivariate statistical methods with many uses across a range of disciplines. But the number of parameters increases exponentially with dimension and reduced-rank regression (RRR) is a well known approach to dimension reduction. But traditional RRR applies only to an overdetermined system. For increasingly common undetermined systems this issue can be managed by regularization, e.g., with a quadratic penalty. A significant problem is then the choice of the two tuning parameters: one discrete i.e., the rank; the other continuous i.e., the Tikhonov penalty parameter. In this paper we resolve this problem via Stein's unbiased risk estimator (SURE). We compare SURE to cross-validation and apply it on both simulated and real data sets.

This paper addresses the problem of Bayesian wavelet estimating the mean vector of multivariate normal distribution under a multivariate normal prior distribution based on linear and nonlinear exponential balanced loss functions. The covariance matrix of multivariate normal distribution is considered known. Bayes estimators of mean vector parameter of multivariate normal distribution are achieved. Then two soft shrinkage wavelet threshold estimators based on Stein’s unbiased risk estimate (SURE) and Bayes estimators are provided. Finally, the performance of the soft shrinkage wavelet estimators was checked through simulation study and Electrical Grid Stability Simulated data set. Simulation and real data results showed the better performance of SURE thresholds based on linear and nonlinear exponential balanced loss functions compared to other classical wavelet methods. Also, they showed better performance for SURE threshold based on nonlinear exponential balanced loss function in multivariate normal distribution with small dimensions.

In this paper, we propose an advanced color image denoising scheme called multichannel nonlocal means fusion (MNLF), where noise reduction is formulated as the minimization of a penalty function. An inherent feature of color images is the strong interchannel correlation, which is introduced into the penalty function as additional prior constraints to expect a better performance. The optimal solution of the minimization problem is derived, consisting of constructing and fusing multiple nonlocal means (NLM) spanning all three channels. The weights in the fusion are optimized to minimize the overall mean squared denoising error, with the help of the extended and adapted Stein's unbiased risk estimator (SURE). Simulations on representative test images under various noise levels verify the improvement brought by the multichannel NLM, compared to the traditional single-channel NLM. In the meantime, MNLF provides competitive performance both in terms of the color peak signal-to-noise ratio and in perceptual quality when compared with other state-of-the-art benchmarks.