On Convergence in Distribution of Stein's Unbiased Risk Hyper-parameter Estimator for Regularized System Identification

Purpose: Parallel imaging methods in MRI have resulted in faster acquisition times and improved noise performance. ESPIRiT is one such technique that estimates coil sensitivity maps from the auto-calibration region using an eigenvalue-based method. This method requires choosing several parameters for the the map estimation. Even though ESPIRiT is fairly robust to these parameter choices, occasionally, poor selection can result in reduced performance. The purpose of this work is to automatically select parameters in ESPIRiT for more robust and consistent performance across a variety of exams. Theory and Methods: Stein's unbiased risk estimate (SURE) is a method of calculating an unbiased estimate of the mean squared error of an estimator under certain assumptions. We show that this can be used to estimate the performance of ESPIRiT. We derive and demonstrate the use of SURE to optimize ESPIRiT parameter selection. Results: Simulations show SURE to be an accurate estimator of the mean squared error. SURE is then used to optimize ESPIRiT parameters to yield maps that are optimal in a denoising/data-consistency sense. This improves g-factor performance without causing undesirable attenuation. In-vivo experiments verify the reliability of this method. Conclusion: Simulation experiments demonstrate that SURE is an accurate estimate of expected mean squared error. Using SURE to determine ESPIRiT parameters allows for automatic parameter selections.In-vivo results are consistent with simulation and theoretical results.

Ultrasound (US) image quality often suffers from random and speckle noise. Especially in the case of difficult patients, increased presence of random noise leads to compromised signal to noise ratio (SNR). The proposed technique removes random or speckle noise by promoting the low rank component of temporal US image sequences comprising of RF-data or envelope-detected data, respectively. This approach leverages the fact that due to tissue connectedness, tissue motion tends to exhibit limited temporal profiles. An approach to approximate the low rank component of the image sequence involves truncating its singular value decomposition. In the proposed algorithm, this is performed in a block-wise manner. Additionally, an adaptive singular value threshold is proposed based on Stein's unbiased risk estimator. This paper presents applications of this methodology on sequences of RF-data and envelope-detected data.

Recently, total variation (TV)-based regularization has become a standard technique for signal denoising. The reconstruction quality is generally sensitive to the value of regularization parameter. In this work, based on Cham-bolle's algorithm, we develop two data-driven optimization schemes based on minimization of Stein's unbiased risk estimate (SURE)-statistically equivalent to mean squared error (MSE). First, we propose a recursive evaluation of SURE to monitor the estimation error during Chambolle's iteration; the optimal value is then identified by the minimum SURE. Second, for fast optimization, we perform alternating update between regularization parameter and solution within Chambolle's iteration. We exemplify the proposed methods with both 1-D and 2-D signal denoising. Numerical experiments show that the proposed methods lead to highly accurate estimate of regularization parameter and nearly optimal denoising performance.

We consider geostatistical regression models to predict spatial variables of interest, where likelihood‐based methods are used to estimate model parameters. It is known that parameters in the Matérn covariogram cannot be estimated well, even when increasing amounts of data are collected densely in a fixed domain. Although a best linear unbiased predictor has been proposed when model parameters are known, a predictor with estimated parameters is nonlinear and may be not the best in practice. Therefore, we propose an adjusted procedure for the likelihood‐based estimates to improve the predicted ability of the nonlinear spatial predictor. The adjusted parameter estimators based on minimizing a corrected Stein's unbiased risk estimator tend to have less bias than the conventional likelihood‐based estimators, and the resulting spatial predictor is more accurate and more stable. Statistical inference for the proposed method is justified both theoretically and numerically. To verify the practicability of the proposed method, a groundwater data set in Bangladesh is analyzed.

Non-local means (NLM) provides a powerful framework for denoising. However, there are a few parameters of the algorithm-most notably, the width of the smoothing kernel-that are data-dependent and difficult to tune. Here, we propose to use Stein's unbiased risk estimate (SURE) to monitor the mean square error (MSE) of the NLM algorithm for restoration of an image corrupted by additive white Gaussian noise. The SURE principle allows to assess the MSE without knowledge of the noise-free signal. We derive an explicit analytical expression for SURE in the setting of NLM that can be incorporated in the implementation at low computational cost. Finally, we present experimental results that confirm the optimality of the proposed parameter selection.

In this paper we present a new method for the pansharpening of multi-spectral satellite imagery. This method is based on a simple explicit image formation model which leads to an ill posed problem that needs to be regularized for best results. We use both Tikhonov (ridge regression) and Total Variation (TV) regularization. We develop the solutions to these two problems and then we address the problem of selecting the optimal regularization parameter λ. We find the value of λ that minimizes Stein's unbiased risk estimate (SURE). For ridge regression this leads to an analytical expression for SURE while for the TV regularized solution we use Monte Carlo SURE where the estimate is obtained by stochastic means. Finally, we present experiment results where we use quality metrics to evaluate the spectral and spatial quality of the resulting pansharpened image.

Multivariate regression is one of the most widely applied multivariate statistical methods with many uses across a range of disciplines. But the number of parameters increases exponentially with dimension and reduced-rank regression (RRR) is a well known approach to dimension reduction. But traditional RRR applies only to an overdetermined system. For increasingly common undetermined systems this issue can be managed by regularization, e.g., with a quadratic penalty. A significant problem is then the choice of the two tuning parameters: one discrete i.e., the rank; the other continuous i.e., the Tikhonov penalty parameter. In this paper we resolve this problem via Stein's unbiased risk estimator (SURE). We compare SURE to cross-validation and apply it on both simulated and real data sets.

Aiming at solving the problem that the shape feature will be lost while removing the noise in the image, this paper proposes a hybrid shear wave and sure let method. Many research has proven that Shearlets has better representation about distributed discontinuities than traditional wavelets, such as edges, and are ideal tools for edge representation. Therefore, the method proposed in this paper is as follows: firstly, the noise image is decomposed by shear wavelet to obtain from high to low frequency coefficients. Then we use Stein's unbiased risk estimation, which is an accurate and statistically unbiased MSE estimation. At the same time, it's only related to the noise image, not the clean image, to optimize the noise figure. Finally, the denoised image is obtained by inverse shearlet transformation. Experiments show that the proposed algorithm has more advantages than other algorithms.

We propose an unbiased estimate of a filtered version of the mean squared error--the blur-SURE (Stein's unbiased risk estimate)--as a novel criterion for estimating an unknown point spread function (PSF) from the degraded image only. The PSF is obtained by minimizing this new objective functional over a family of Wiener processings. Based on this estimated blur kernel, we then perform nonblind deconvolution using our recently developed algorithm. The SURE-based framework is exemplified with a number of parametric PSF, involving a scaling factor that controls the blur size. A typical example of such parametrization is the Gaussian kernel. The experimental results demonstrate that minimizing the blur-SURE yields highly accurate estimates of the PSF parameters, which also result in a restoration quality that is very similar to the one obtained with the exact PSF, when plugged into our recent multi-Wiener SURE-LET deconvolution algorithm. The highly competitive results obtained outline the great potential of developing more powerful blind deconvolution algorithms based on SURE-like estimates.

We study the effective degrees of freedom of a general class of reduced-rank estimators for multivariate regression in the framework of Stein's unbiased risk estimation. A finite-sample exact unbiased estimator is derived that admits a closed-form expression in terms of the thresholded singular values of the least-squares solution and hence is readily computable. The results continue to hold in the high-dimensional setting where both the predictor and the response dimensions may be larger than the sample size. The derived analytical form facilitates the investigation of theoretical properties and provides new insights into the empirical behaviour of the degrees of freedom. In particular, we examine the differences and connections between the proposed estimator and a commonly-used naive estimator. The use of the proposed estimator leads to efficient and accurate prediction risk estimation and model selection, as demonstrated by simulation studies and a data example.

We propose a new orthonormal wavelet thresholding algorithm for denoising color images that are assumed to be corrupted by additive Gaussian white noise of known intercolor covariance matrix. The proposed wavelet denoiser consists of a linear expansion of thresholding (LET) functions, integrating both the interscale and intercolor dependencies. The linear parameters of the combination are then solved for by minimizing Stein's unbiased risk estimate (SURE), which is nothing but a robust unbiased estimate of the mean squared error (MSE) between the (unknown) noise-free data and the denoised one. Thanks to the quadratic form of this MSE estimate, the parameters optimization simply amounts to solve a linear system of equations. The experimentations we made over a wide range of noise levels and for a representative set of standard color images have shown that our algorithm yields even slightly better peak signal-to-noise ratios than most state-of-the-art wavelet thresholding procedures, even when the latters are executed in an undecimated wavelet representation.

In this paper, we propose an advanced color image denoising scheme called multichannel nonlocal means fusion (MNLF), where noise reduction is formulated as the minimization of a penalty function. An inherent feature of color images is the strong interchannel correlation, which is introduced into the penalty function as additional prior constraints to expect a better performance. The optimal solution of the minimization problem is derived, consisting of constructing and fusing multiple nonlocal means (NLM) spanning all three channels. The weights in the fusion are optimized to minimize the overall mean squared denoising error, with the help of the extended and adapted Stein's unbiased risk estimator (SURE). Simulations on representative test images under various noise levels verify the improvement brought by the multichannel NLM, compared to the traditional single-channel NLM. In the meantime, MNLF provides competitive performance both in terms of the color peak signal-to-noise ratio and in perceptual quality when compared with other state-of-the-art benchmarks.

We propose a bilateral filter with a locally controlled domain kernel for directional edge-preserving smoothing. Traditional bilateral filters use a range kernel, which is responsible for edge preservation, and a fixed domain kernel that performs smoothing. Our intuition is that orientation and anisotropy of image structures should be incorporated into the domain kernel while smoothing. For this purpose, we employ an oriented Gaussian domain kernel locally controlled by a structure tensor. The oriented domain kernel combined with a range kernel forms the directional bilateral filter. The two kernels assist each other in effectively suppressing the influence of the outliers while smoothing. To find the optimal parameters of the directional bilateral filter, we propose the use of Stein's unbiased risk estimate (SURE). We test the capabilities of the kernels separately as well as together, first on synthetic images, and then on real endoscopic images. The directional bilateral filter has better denoising performance than the Gaussian bilateral filter at various noise levels in terms of peak signal-to-noise ratio (PSNR).

This chapter presents a basic overview of wavelet shrinkage and its application to signal denoising. It discusses two wavelet‐based methods used to denoise signals: the VisuShrink method and the SureShrink method. VisuShrink utilizes the wavelet shrinkage algorithm with a universal threshold. The chapter looks at two examples that implement VisuShrink. The first example illustrates the method on the test signal. This example is somewhat artificial since people set the noise level when they form the noisy vector. The second example introduces the idea of image segmentation. In this application, pixels are separated into different regions based on similarity. Denoising the image before segmenetation is applied can sometimes increase the effectiveness of the algorithm. The SureShrink method utilizes Stein's unbiased risk estimator. It calls for a tolerance to be computed for each highpass portion of the wavelet transformation. A test is performed on each highpass portion to determine its sparseness.

Hyper-trim shrinkage for denoising of ECG signal