Cauchy greedy algorithm for robust sparse recovery and multiclass classification

Sparse Bayesian learning using correlated hyperparameters for recovery of block sparse signals

We consider the recovery of sparse signals, f ∈ ℝN, containing at most k ≪ N nonzero entries using linear measurements contaminated with i.i.d. Gaussian background noise. Given this measurement model, we present and analyze a computationally efficient group testing strategy for recovering the support of f and approximating its nonzero entries. In particular, we demonstrate that group testing measurement matrix constructions may be combined with statistical binary detection and estimation methods to produce efficient adaptive sequential algorithms for sparse signal support recovery. Furthermore, when f exhibits sufficient sparsity, we show that these adaptive group testing methods allow the recovery of sparse signals using fewer noisy linear measurements than possible with non-adaptive methods based on Gaussian measurement ensembles. As a result we improve on previous sufficient conditions for sparsity pattern recovery in the noisy sublinear-sparsity regime.

Numerous algorithms have been proposed recently for sparse signal recovery in Compressed Sensing (CS). In practice, the number of measurements can be very limited due to the nature of the problem and/or the underlying statistical distribution of the non-zero elements of the sparse signal may not be known a priori. It has been observed that the performance of any sparse signal recovery algorithm depends on these factors, which makes the selection of a suitable sparse recovery algorithm difficult. To take advantage in such situations, we propose to use a fusion framework using which we employ multiple sparse signal recovery algorithms and fuse their estimates to get a better estimate. Theoretical results justifying the performance improvement are shown. The efficacy of the proposed scheme is demonstrated by Monte Carlo simulations using synthetic sparse signals and ECG signals selected from MIT-BIH database.

Variational Bayesian inference based robust multiple measurement sparse signal recovery

In this paper, we consider the problem of finding the sparsest solution to underdetermined linear systems. Unlike the literatures which use the ℓ1 regularization to approximate the original problem, we consider the ℓ2/3 regularization which leads to a better approximation but a nonconvex, nonsmooth, and non-Lipschitz optimization problem. Through developing a fixed point representation theory associated with the two thirds thresholding operator for ℓ2/3 regularization solutions, we propose a fixed point iterative thresholding algorithm based on two thirds norm for solving the k-sparsity problems. Relying on the restricted isometry property, we provide subsequentional convergence guarantee for this fixed point iterative thresholding algorithm on recovering a sparse signal. By discussing the preferred regularization parameters and studying the phase diagram, we get an adequate and efficient algorithm for the high-dimensional sparse signal recovery. Finally, comparing with the existing algorithms, such as the standard ℓ1 minimization, the iterative reweighted ℓ2 minimization, the iterative reweighted ℓ1 minimization, and iterative Half thresholding algorithm, we display the results of the experiment which indicate that the two thirds norm fixed point iterative thresholding algorithm applied to sparse signal recovery and large scale imageries from noisy measurements can be accepted as an effective solver for ℓ2/3 regularization.

This study concerns the issue of jointly enhancing noise robustness and promoting signal sparsity in Sparse Bayesian Learning (SBL), which aims at addressing the performance deficiency of sparse signal recovery due to uninformative data with low signal-to-noise ratios. In particular, the authors propose a hierarchical prior noise model with a signal-dependent parametrisation and incorporate it into developing the robust SBL algorithms for sparse signal recovery. The main contribution of the proposed approach is twofold. The first is the new consideration of noise-robustness enhancement in building SBL algorithms, which devotes to noise awareness in counteracting outliers in measurements. Specifically, the idea of signal-sparsity enforcing is extended to build a Least Absolute Deviation like loss criterion with the proposed hierarchical prior model of measurement noise. The second is the novelty of using the signal-dependent parametrisation in the proposed noise model. Indeed, the signal-dependent mechanism plays an indispensable role in producing the reliable noise parameter estimation jointly with updating signal model parameters under the fast SBL framework. In addition to numerical simulation studies, the real-life application of radio tomographic imaging is presented to validate the proposed approach.

In this paper, we introduce a new sparse signal recovery algorithm referred to as the matching pursuit with greedy tree search (GTMP). The tree search in our proposed method is implemented to minimize the cost function to improve the recovery performance of sparse signals. In addition, a pruning strategy is employed to each node of the tree for efficient implementation. In our performance guarantee analysis, we provide the condition that ensures the exact identification of the nonzero locations. Through empirical simulations, we show that GTMP is effective for sparse signal reconstruction and outperforms conventional sparse recovery algorithms.

Recovery of block sparse signals with partially-known block support information is of particular importance in compressed sensing. A uniform sufficient condition guaranteeing stable recovery of non-strictly block $k$-sparse signals is established via the weighted $ℓ_{2,p}−\alpha ℓ_{2,q}$ nonconvex minimization method, and the reconstruction error is precisely bounded in terms of the residual of block-sparsity and the measurement error. Furthermore, a series of contrastive numerical experiments reveal that exploiting the approximate block-sparsity characteristic and the nonuniform prior block support estimate substantially promotes the performance of reconstruction for block-structural signals.

We study the role of the nonzero entries on the performance limits in support recovery of sparse signals. The key to our results is the recently studied connection between sparse signal recovery and multiple user communication. By leveraging the concept of outage capacity in information theory, we explicitly characterize the impact of the probability distribution imposed on the nonzero entries of the sparse signal on support recovery. When Multiple Measurement Vectors (MMV) are available, we show that the identification of the nonzero rows of the signal is closely connected to decoding the messages from multiple users over a Single-Input Multiple-Output channel. Necessary and sufficient conditions for support (indices of nonzero rows) recovery are provided, and the results allow us to understand the role of correlation of the nonzero entries as well as the role of the rank of the matrix formed from the non-zero entries.

Compressive sensing is a sub-Nyquist sampling technique for efficient signal acquisition and reconstruction of sparse or compressible signals. In order to account for the sparsity of the underlying signal of interest, it is common to use sparsifying priors such as Bernoulli–Gaussian-inverse Gamma (BGiG) and Gaussian-inverse Gamma (GiG) priors on the components of the signal. With the introduction of variational Bayesian inference, the sparse Bayesian learning (SBL) methods for solving the inverse problem of compressive sensing have received significant interest as the SBL methods become more efficient in terms of execution time. In this paper, we consider the sparse signal recovery problem using compressive sensing and the variational Bayesian (VB) inference framework. More specifically, we consider two widely used Bayesian models of BGiG and GiG for modeling the underlying sparse signal for this problem. Although these two models have been widely used for sparse recovery problems under various signal structures, the question of which model can outperform the other for sparse signal recovery under no specific structure has yet to be fully addressed under the VB inference setting. Here, we study these two models specifically under VB inference in detail, provide some motivating examples regarding the issues in signal reconstruction that may occur under each model, perform comparisons and provide suggestions on how to improve the performance of each model.

The sparse signal recovery in the standard compressed sensing (CS) problem requires that the sensing matrix be known a priori. Such an ideal assumption may not be met in practical applications where various errors and fluctuations exist in the sensing instruments. This paper considers the problem of compressed sensing subject to a structured perturbation in the sensing matrix. Under mild conditions, it is shown that a sparse signal can be recovered by $\ell_1$ minimization and the recovery error is at most proportional to the measurement noise level, which is similar to the standard CS result. In the special noise free case, the recovery is exact provided that the signal is sufficiently sparse with respect to the perturbation level. The formulated structured sensing matrix perturbation is applicable to the direction of arrival estimation problem, so has practical relevance. Algorithms are proposed to implement the $\ell_1$ minimization problem and numerical simulations are carried out to verify the result obtained.

The importance of sparse signal structures has been recognized in a plethora of applications ranging from medical imaging to group disease testing to radar technology. It has been shown in practice that various signals of interest may be (approximately) sparsely modeled, and that sparse modeling is often beneficial, or even indispensable to signal recovery. Alongside an increase in applications, a rich theory of sparse and compressible signal recovery has recently been developed under the names compressed sensing (CS) and sparse approximation (SA). This revolutionary research has demonstrated that many signals can be recovered from severely undersampled measurements by taking advantage of their inherent low-dimensional structure. More recently, an offshoot of CS and SA has been a focus of research on other low-dimensional signal structures such as matrices of low rank. Low-rank matrix recovery (LRMR) is demonstrating a rapidly growing array of important applications such as quantum state tomography, triangulation from incomplete distance measurements, recommender systems (e.g., the Netflix problem), and system identification and control. In this dissertation, we examine CS, SA, and LRMR from a theoretical perspective. We consider a variety of different measurement and signal models, both random and deterministic, and mainly ask two questions. How many measurements are necessary? How large is the recovery error? We give theoretical lower bounds for both of these questions, including oracle and minimax lower bounds for the error. However, the main emphasis of the thesis is to demonstrate the efficacy of convex optimization---in particular l1 and nuclear-norm minimization based programs---in CS, SA, and LRMR. We derive upper bounds for the number of measurements required and the error derived by convex optimization, which in many cases match the lower bounds up to constant or logarithmic factors. The majority of these results do not require the restricted isometry property (RIP), a ubiquitous condition in the literature.

Analysis of cascade network consisting of RBF nodes and least square error minimization block for compressed sensing recovery of sparse signals is presented in this paper. The proposed algorithm radial basis function cascade network for sparse signal recovery uses the L 0 norm optimization, L 2 least square method and feedback network model to improve the signal recovery performance and computational time over the existing ANN based and SL0 algorithms. The recovery of spike and pulse current measurement from simulated compressed sensed data for Telemetry application is demonstrated using the new algorithm. The Simulink model for compressed sensing data acquisition process, sparse signal recovery results and algorithm performance evaluation are presented.

The problem of sparse signal recovery from lower number of observations is formalized as a constrained minimization problem. Due to the dissimilar requirements of signal processing, the constraints and objective functions differ from each other. This paper presents an algorithm scheme base on alternating iteration to deal with the nonconvex objective function. The proposed method employs the log-penalty and proximity operator to seek the sparse solution. In each iteration, the introduction of nonconvex penalty decreases the penalization of large coefficients which contributes to a faster decrease of the function value. By making use of proximity operator, it reduces the computational complexity and achieves less number of iterations. In addition, the experimental results illustrate the effectiveness of the proposed method as well as the drawbacks of the algorithm.

We introduce a lifted $\ell _1$ (LL1) regularization framework for the recovery of sparse signals. The proposed LL1 regularization is a generalization of several popular regularization methods in the field and is motivated by recent advancements in re-weighted $\ell _1$ approaches for sparse recovery. Through a comprehensive analysis of the relationships between existing methods, we identify two distinct types of lifting functions that guarantee equivalence to the $\ell _0$ minimization problem, which is a key objective in sparse signal recovery. To solve the LL1 regularization problem, we propose an algorithm based on the alternating direction method of multipliers and provide proof of convergence for the unconstrained formulation. Our experiments demonstrate the improved performance of the LL1 regularization compared with state-of-the-art methods, confirming the effectiveness of our proposed framework. In conclusion, the LL1 regularization presents a promising and flexible approach to sparse signal recovery and invites further research in this area.