Variational Bayesian inference based robust multiple measurement sparse signal recovery

Compressive sensing is a sub-Nyquist sampling technique for efficient signal acquisition and reconstruction of sparse or compressible signals. In order to account for the sparsity of the underlying signal of interest, it is common to use sparsifying priors such as Bernoulli–Gaussian-inverse Gamma (BGiG) and Gaussian-inverse Gamma (GiG) priors on the components of the signal. With the introduction of variational Bayesian inference, the sparse Bayesian learning (SBL) methods for solving the inverse problem of compressive sensing have received significant interest as the SBL methods become more efficient in terms of execution time. In this paper, we consider the sparse signal recovery problem using compressive sensing and the variational Bayesian (VB) inference framework. More specifically, we consider two widely used Bayesian models of BGiG and GiG for modeling the underlying sparse signal for this problem. Although these two models have been widely used for sparse recovery problems under various signal structures, the question of which model can outperform the other for sparse signal recovery under no specific structure has yet to be fully addressed under the VB inference setting. Here, we study these two models specifically under VB inference in detail, provide some motivating examples regarding the issues in signal reconstruction that may occur under each model, perform comparisons and provide suggestions on how to improve the performance of each model.

Cauchy greedy algorithm for robust sparse recovery and multiclass classification

Numerous algorithms have been proposed recently for sparse signal recovery in Compressed Sensing (CS). In practice, the number of measurements can be very limited due to the nature of the problem and/or the underlying statistical distribution of the non-zero elements of the sparse signal may not be known a priori. It has been observed that the performance of any sparse signal recovery algorithm depends on these factors, which makes the selection of a suitable sparse recovery algorithm difficult. To take advantage in such situations, we propose to use a fusion framework using which we employ multiple sparse signal recovery algorithms and fuse their estimates to get a better estimate. Theoretical results justifying the performance improvement are shown. The efficacy of the proposed scheme is demonstrated by Monte Carlo simulations using synthetic sparse signals and ECG signals selected from MIT-BIH database.

We study the role of the nonzero entries on the performance limits in support recovery of sparse signals. The key to our results is the recently studied connection between sparse signal recovery and multiple user communication. By leveraging the concept of outage capacity in information theory, we explicitly characterize the impact of the probability distribution imposed on the nonzero entries of the sparse signal on support recovery. When Multiple Measurement Vectors (MMV) are available, we show that the identification of the nonzero rows of the signal is closely connected to decoding the messages from multiple users over a Single-Input Multiple-Output channel. Necessary and sufficient conditions for support (indices of nonzero rows) recovery are provided, and the results allow us to understand the role of correlation of the nonzero entries as well as the role of the rank of the matrix formed from the non-zero entries.

Sparse signal priors help in a variety of modern signal processing tasks. In many cases, a sparse signal needs to be recovered from an underdetermined system of equations. For instance, sparse approximation of a signal with an overcomplete dictionary or reconstruction of a sparse signal from a small number of linear measurements. The reconstruction problem typically requires solving an ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> norm minimization problem. In this paper we present homotopy based algorithms to update the solution of some ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> problems when the system is updated by adding new rows or columns to the underlying system matrix. We also discuss a case where these ideas can be extended to accommodate for more general changes in the system matrix.

Robust Bayesian compressed sensing with outliers

We consider the recovery of sparse signals, f ∈ ℝN, containing at most k ≪ N nonzero entries using linear measurements contaminated with i.i.d. Gaussian background noise. Given this measurement model, we present and analyze a computationally efficient group testing strategy for recovering the support of f and approximating its nonzero entries. In particular, we demonstrate that group testing measurement matrix constructions may be combined with statistical binary detection and estimation methods to produce efficient adaptive sequential algorithms for sparse signal support recovery. Furthermore, when f exhibits sufficient sparsity, we show that these adaptive group testing methods allow the recovery of sparse signals using fewer noisy linear measurements than possible with non-adaptive methods based on Gaussian measurement ensembles. As a result we improve on previous sufficient conditions for sparsity pattern recovery in the noisy sublinear-sparsity regime.

The conditions under which Compressed sensing (CS) succeeds depend on the structure of the measurement matrix. Researches indicated that matrices whose entries are drawn independently from certain probability distributions satisfy the restricted isometry property (RIP) and guarantee exact recovery of a sparse signal with high probability. Motivated by signal processing applications, random filtering with Toeplitz sensing matrices whose elements are drawn from the same distributions were considered and shown to also be sufficient to recover a sparse signal from reduced samples exactly with high probability. In order to achieve reduction of sampling rate in impulse radio Ultra-Wideband (IR-UWB) communication systems, the output streams of UWB channel were periodically down-sampled. Sub-sampling at the receiver leads to quasi-Toeplitz-structured measurement matrices, whose entries are UWB channel coefficients. In this paper, the feasibility of this kind of quasi-Toeplitz matrices as measurement matrices is discussed. It is shown that the quasi-Toeplitz matrices with entries drawn from logarithm normal distribution satisfy RIP with high probability and also ensures the exact reconstruction of the sparse signals. Simulation results validate their performance.

The sparse signal recovery in the standard compressed sensing (CS) problem requires that the sensing matrix be known a priori. Such an ideal assumption may not be met in practical applications where various errors and fluctuations exist in the sensing instruments. This paper considers the problem of compressed sensing subject to a structured perturbation in the sensing matrix. Under mild conditions, it is shown that a sparse signal can be recovered by $\ell_1$ minimization and the recovery error is at most proportional to the measurement noise level, which is similar to the standard CS result. In the special noise free case, the recovery is exact provided that the signal is sufficiently sparse with respect to the perturbation level. The formulated structured sensing matrix perturbation is applicable to the direction of arrival estimation problem, so has practical relevance. Algorithms are proposed to implement the $\ell_1$ minimization problem and numerical simulations are carried out to verify the result obtained.

We introduce a lifted $\ell _1$ (LL1) regularization framework for the recovery of sparse signals. The proposed LL1 regularization is a generalization of several popular regularization methods in the field and is motivated by recent advancements in re-weighted $\ell _1$ approaches for sparse recovery. Through a comprehensive analysis of the relationships between existing methods, we identify two distinct types of lifting functions that guarantee equivalence to the $\ell _0$ minimization problem, which is a key objective in sparse signal recovery. To solve the LL1 regularization problem, we propose an algorithm based on the alternating direction method of multipliers and provide proof of convergence for the unconstrained formulation. Our experiments demonstrate the improved performance of the LL1 regularization compared with state-of-the-art methods, confirming the effectiveness of our proposed framework. In conclusion, the LL1 regularization presents a promising and flexible approach to sparse signal recovery and invites further research in this area.

As an alternative paradigm to the Shannon-Nyquist sampling theorem, compressive sensing enables sparse signals to be acquired by sub-Nyquist analog-to-digital converters thus may launch a revolution in signal collection, transmission and processing. In the practical compressive sensing applications, the sparse signal is always affected by noise and interference, and therefore the recovery performance reduces based on the conventional compressive sensing, especially in the low signal-to-noise scene, the sparse recovery is usually unavailable. In this paper, the influence of noise on recovery performance is analyzed, so as to provide the theoretical basis for the noise folding phenomenon in compressive sensing. From the analysis, we find that the expected noise gain in the random measure process is closely related to the row and column of the measurement matrix. However, only those columns corresponding to the support for the sparse signal contribute to the sparse vector. In the traditional Shannon-Nyquist sampling system, an antialiasing filter is applied before the sampling process, so as to filter the noise beyond the passband of interest. Inspired by the necessity of antialiasing filtering in bandpass signal sampling, we propose a selective measurement scheme, namely adapted compressive sensing, whose measurement matrix can be updated according to the noise information fed back by the processing center. The measurement matrix is specially designed, and the sensing matrix has directivity so that the signal noise can be suppressed. The measurement matrix senses only the spectrum of interest, where the sparse spectrum is most likely to lie. Moreover, we compare the recovery performance of the proposed adaptive scheme with those of the non-adaptive orthogonal matching pursuit algorithm, FOCal underdetermined system solver algorithm, and sarse Bayesian learning algorithm. Extensive numerical experiments show that the proposed scheme has a better improvement in the performance of the sparse signal recovery. From the viewpoint of implementation, the measurement noise should be taken into consideration in the system, and more efficient algorithms will be developed for source pre-estimation at lower signal-to-noise ratio.

Analysis of cascade network consisting of RBF nodes and least square error minimization block for compressed sensing recovery of sparse signals is presented in this paper. The proposed algorithm radial basis function cascade network for sparse signal recovery uses the L 0 norm optimization, L 2 least square method and feedback network model to improve the signal recovery performance and computational time over the existing ANN based and SL0 algorithms. The recovery of spike and pulse current measurement from simulated compressed sensed data for Telemetry application is demonstrated using the new algorithm. The Simulink model for compressed sensing data acquisition process, sparse signal recovery results and algorithm performance evaluation are presented.

The importance of sparse signal structures has been recognized in a plethora of applications ranging from medical imaging to group disease testing to radar technology. It has been shown in practice that various signals of interest may be (approximately) sparsely modeled, and that sparse modeling is often beneficial, or even indispensable to signal recovery. Alongside an increase in applications, a rich theory of sparse and compressible signal recovery has recently been developed under the names compressed sensing (CS) and sparse approximation (SA). This revolutionary research has demonstrated that many signals can be recovered from severely undersampled measurements by taking advantage of their inherent low-dimensional structure. More recently, an offshoot of CS and SA has been a focus of research on other low-dimensional signal structures such as matrices of low rank. Low-rank matrix recovery (LRMR) is demonstrating a rapidly growing array of important applications such as quantum state tomography, triangulation from incomplete distance measurements, recommender systems (e.g., the Netflix problem), and system identification and control. In this dissertation, we examine CS, SA, and LRMR from a theoretical perspective. We consider a variety of different measurement and signal models, both random and deterministic, and mainly ask two questions. How many measurements are necessary? How large is the recovery error? We give theoretical lower bounds for both of these questions, including oracle and minimax lower bounds for the error. However, the main emphasis of the thesis is to demonstrate the efficacy of convex optimization---in particular l1 and nuclear-norm minimization based programs---in CS, SA, and LRMR. We derive upper bounds for the number of measurements required and the error derived by convex optimization, which in many cases match the lower bounds up to constant or logarithmic factors. The majority of these results do not require the restricted isometry property (RIP), a ubiquitous condition in the literature.

Summary form only given. We describe a set of prior distributions whose independent and identically distributed (iid) realizations result in compressible signals. A signal is compressible when the sorted magnitudes of its coefficients exhibit a power-law decay so that the signal can be well-approximated by a sparse signal. By definition, a sparse signal has only a small fraction of non-zero coefficients compared to its ambient dimension. These distributions, dubbed compressible priors, analytically summarize compressible signals and shed new light to the solution of under-determined linear regression problems by building upon the literature on sparse signal recovery. Our main results are summarized as follows: 1) By using order statistics, we show that iid realizations of generalized Pareto, Student's t, lognormal, generalized extreme value, and log-logistics distributions can result in compressible signals. We quantify the compressibility of the signal realizations from these compressible priors as a function of their parameters. 2) We point out a common misconception about the generalized Gaussian distribution (GGD): GGD result in signals that lose their compressibility as their dimensions grow. For instance, special cases of the GGD distribution, e.g., Laplacian distribution, are commonly used as sparsity promoting priors in compressive sensing and sparse Bayesian learning problems where the number of observations is assumed to grow logarithmically with the number of unknowns. We show that signals generated from Laplacian distribution can only be stably embedded into lower dimensions that grow proportional to the signal dimension. Hence, we identify an inconsistency between the decoding algorithms motivated by the GGD distribution and their sparse solutions. 3) We use compressible priors as a scaffold to build new decoding algorithms based on Bayesian inference arguments. The objective of these algorithms is to approximate the signal realization from a compressible prior as opposed to naively producing sparse solutions. Some of these new algorithms are variants of the popular iterative reweighting schemes. We show (i) how tuning of these algorithms explicitly depends on the parameters of the compressible prior of the signal, and (ii) how to learn the parameters of the signal's compressible prior on the fly while recovering the signal. To illustrate the concepts, we study various order statistics of natural images from the Berkeley Segmentation Dataset, including pixel gradients and wavelet coefficients.

Commonly employed reconstruction algorithms in compressed sensing (CS) use the $L_2$ norm as the metric for the residual error. However, it is well-known that least squares (LS) based estimators are highly sensitive to outliers present in the measurement vector leading to a poor performance when the noise no longer follows the Gaussian assumption but, instead, is better characterized by heavier-than-Gaussian tailed distributions. In this paper, we propose a robust iterative hard Thresholding (IHT) algorithm for reconstructing sparse signals in the presence of impulsive noise. To address this problem, we use a Lorentzian cost function instead of the $L_2$ cost function employed by the traditional IHT algorithm. We also modify the algorithm to incorporate prior signal information in the recovery process. Specifically, we study the case of CS with partially known support. The proposed algorithm is a fast method with computational load comparable to the LS based IHT, whilst having the advantage of robustness against heavy-tailed impulsive noise. Sufficient conditions for stability are studied and a reconstruction error bound is derived. We also derive sufficient conditions for stable sparse signal recovery with partially known support. Theoretical analysis shows that including prior support information relaxes the conditions for successful reconstruction. Simulation results demonstrate that the Lorentzian-based IHT algorithm significantly outperform commonly employed sparse reconstruction techniques in impulsive environments, while providing comparable performance in less demanding, light-tailed environments. Numerical results also demonstrate that the partially known support inclusion improves the performance of the proposed algorithm, thereby requiring fewer samples to yield an approximate reconstruction.