Proximity operator based alternating iteration algorithm for sparse signal recovery

HKUST Electronic Theses Second-order algorithms for sparse signal recovery from intensity-only measurements by Ruixue Wen thesis 2022 1 online resource (xiv, 113 pages) : illustrations (some color) In this thesis, we study the sparse phase retrieval problem, which is to…Read more ›

The algorithms based on the technique of optimal <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula>-thresholding (OT) were recently proposed for signal recovery, and they are very different from the traditional family of hard thresholding methods. However, the computational cost for OT-based algorithms remains high at the current stage of their development. This stimulates the development of the so-called natural thresholding (NT) algorithm and its variants in this paper. The family of NT algorithms is developed through the first-order approximation of the so-called regularized optimal <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula>-thresholding model, and thus the computational cost for this family of algorithms is significantly lower than that of the OT-based algorithms. The guaranteed performance of NT-type algorithms for signal recovery from noisy measurements is shown under the restricted isometry property and concavity of the objective function of regularized optimal <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula>-thresholding model. Empirical results indicate that the NT-type algorithms are robust and very comparable to several mainstream algorithms for sparse signal recovery.

This study concerns the issue of jointly enhancing noise robustness and promoting signal sparsity in Sparse Bayesian Learning (SBL), which aims at addressing the performance deficiency of sparse signal recovery due to uninformative data with low signal-to-noise ratios. In particular, the authors propose a hierarchical prior noise model with a signal-dependent parametrisation and incorporate it into developing the robust SBL algorithms for sparse signal recovery. The main contribution of the proposed approach is twofold. The first is the new consideration of noise-robustness enhancement in building SBL algorithms, which devotes to noise awareness in counteracting outliers in measurements. Specifically, the idea of signal-sparsity enforcing is extended to build a Least Absolute Deviation like loss criterion with the proposed hierarchical prior model of measurement noise. The second is the novelty of using the signal-dependent parametrisation in the proposed noise model. Indeed, the signal-dependent mechanism plays an indispensable role in producing the reliable noise parameter estimation jointly with updating signal model parameters under the fast SBL framework. In addition to numerical simulation studies, the real-life application of radio tomographic imaging is presented to validate the proposed approach.

Numerous algorithms have been proposed recently for sparse signal recovery in Compressed Sensing (CS). In practice, the number of measurements can be very limited due to the nature of the problem and/or the underlying statistical distribution of the non-zero elements of the sparse signal may not be known a priori. It has been observed that the performance of any sparse signal recovery algorithm depends on these factors, which makes the selection of a suitable sparse recovery algorithm difficult. To take advantage in such situations, we propose to use a fusion framework using which we employ multiple sparse signal recovery algorithms and fuse their estimates to get a better estimate. Theoretical results justifying the performance improvement are shown. The efficacy of the proposed scheme is demonstrated by Monte Carlo simulations using synthetic sparse signals and ECG signals selected from MIT-BIH database.

In this paper, we consider the problem of finding the sparsest solution to underdetermined linear systems. Unlike the literatures which use the ℓ1 regularization to approximate the original problem, we consider the ℓ2/3 regularization which leads to a better approximation but a nonconvex, nonsmooth, and non-Lipschitz optimization problem. Through developing a fixed point representation theory associated with the two thirds thresholding operator for ℓ2/3 regularization solutions, we propose a fixed point iterative thresholding algorithm based on two thirds norm for solving the k-sparsity problems. Relying on the restricted isometry property, we provide subsequentional convergence guarantee for this fixed point iterative thresholding algorithm on recovering a sparse signal. By discussing the preferred regularization parameters and studying the phase diagram, we get an adequate and efficient algorithm for the high-dimensional sparse signal recovery. Finally, comparing with the existing algorithms, such as the standard ℓ1 minimization, the iterative reweighted ℓ2 minimization, the iterative reweighted ℓ1 minimization, and iterative Half thresholding algorithm, we display the results of the experiment which indicate that the two thirds norm fixed point iterative thresholding algorithm applied to sparse signal recovery and large scale imageries from noisy measurements can be accepted as an effective solver for ℓ2/3 regularization.

This paper discusses compressive sensing with sparse measurement matrices. Sparse matrices have several attractive properties, like low computational complexity in both encoding and recovery, easy incremental updates to signals, and low storage requirement, etc. Typical examples of existing algorithms for sparse signal recovery with sparse measurement matrices include convex relaxation, matching pursuit, and Bayesian framework based approaches. In this paper, we propose an alternative technique to this problem. The proposed technique has a linear recovery complexity and relatively good empirical behavior. In this technique, we employ a permutation-based multi-dimensional measurement matrix, which is composed of several sub-matrices, each consisting of a block-diagonal matrix and a random permutation matrix. The measurement symbols generated from the same permutation matrix are referred to as a dimension. Such a measurement matrix brings some useful features to the measurement symbols. Fully exploiting these features enables us to design a simple and effective recovery algorithm. The proposed recovery algorithm employs an iterative process. In each iteration, the algorithm looks for measurement symbols with certain features, and uses such features to recover the source symbols related to these measurements. An iterative process is then applied, along with an interference cancellation operation, to reconstruct all source symbols gradually. The complexity of the proposed algorithm is relatively low, which grows linearly with the source signal length. Numerical results show that the proposed technique empirically offers a much lower sketch length than ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -minimization-based convex relaxation and Bayesian framework based algorithms. It achieves the empirical lower bound of sketch length and linear recovery complexity at the same time.

Numerical algorithms for the l0-norm regularized non-smooth non-convex minimization problems have recently became a topic of great interest within signal processing, compressive sensing, statistics, and machine learning. Nevertheless, the l0-norm makes the problem combinatorial and generally computationally intractable. In this paper, we construct a new surrogate function to approximate l0-norm regularization, and subsequently make the discrete optimization problem continuous and smooth. Then we use the well-known spectral gradient algorithm to solve the resulting smooth optimization problem. Experiments are provided which illustrate this method is very promising.

Sparse optimization problems have been successfully applied to a wide range of research areas, and useful insights and elegant methods for proving the stability and convergence of neurodynamic algorithms have been yielded in previous work. This article develops several neurodynamic algorithms for sparse signal recovery by solving the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula> regularization problem. First, in the framework of the locally competitive algorithm (LCA), modified LCA (MLCA) with finite-time convergence and MLCA with fixed-time convergence are designed. Then, the sliding-mode control (SMC) technique is introduced and modified, i.e., modified SMC (MSMC), which is combined with LCA to design MSMC-LCA with finite-time convergence and MSMC-LCA with fixed-time convergence. It is shown that the solutions of the proposed neurodynamic algorithms exist and are unique under the observation matrix satisfying restricted isometry property (RIP) condition, while finite-time or fixed-time convergence to the optimal points is shown via Lyapunov-based analysis. In addition, combining the notions of finite-time stability (FTS) and fixed-time stability (FxTS), upper bounds on the convergence time of the proposed neurodynamic algorithms are given, and the convergence results obtained for the MLCA and MSMC-LCA with fixed-time convergence are shown to be independent of the initial conditions. Finally, simulation experiments of signal recovery and image recovery are carried out to demonstrate the superior performance of the proposed neurodynamic algorithms.

Ground Penetrating Radar (GPR) is essential for subsurface exploration. Conventional GPR 3D imaging demands dense spatial sampling along regular grids, which is both time-consuming and impractical in complex environments. In this work, we propose a novel method that combines sparse recovery techniques with a placement matrix to merge arbitrarily and sparsely sampled measurements into a regular grid framework. By exploiting the inherent sparsity of subsurface targets and using the Dantzig Selector with cross-validation, our method reconstructs the target reflectivity vector from random spatial sampling. The recovered data is then processed via the Back-Projection Algorithm (BPA) to generate high-resolution 3D images. Simulations demonstrate that our approach not only improves imaging quality under reduced sampling conditions but also efficiently handles arbitrary scanning paths by mapping irregular measurements onto the desired grid.

Compressive sensing is a sub-Nyquist sampling technique for efficient signal acquisition and reconstruction of sparse or compressible signals. In order to account for the sparsity of the underlying signal of interest, it is common to use sparsifying priors such as Bernoulli–Gaussian-inverse Gamma (BGiG) and Gaussian-inverse Gamma (GiG) priors on the components of the signal. With the introduction of variational Bayesian inference, the sparse Bayesian learning (SBL) methods for solving the inverse problem of compressive sensing have received significant interest as the SBL methods become more efficient in terms of execution time. In this paper, we consider the sparse signal recovery problem using compressive sensing and the variational Bayesian (VB) inference framework. More specifically, we consider two widely used Bayesian models of BGiG and GiG for modeling the underlying sparse signal for this problem. Although these two models have been widely used for sparse recovery problems under various signal structures, the question of which model can outperform the other for sparse signal recovery under no specific structure has yet to be fully addressed under the VB inference setting. Here, we study these two models specifically under VB inference in detail, provide some motivating examples regarding the issues in signal reconstruction that may occur under each model, perform comparisons and provide suggestions on how to improve the performance of each model.

In this paper, we propose a novel sparse signal recovery algorithm called Trainable ISTA (TISTA). The proposed algorithm consists of two estimation units such as a linear estimation unit and a minimum mean squared error (MMSE) estimator-based shrinkage unit. The estimated error variance required in the MMSE shrinkage unit is precisely estimated from a tentative estimate of the original signal. The remarkable feature of the proposed scheme is that TISTA includes adjustable variables controlling a step size and the error variance for the MMSE shrinkage. The variables are adjusted by standard deep learning techniques. The number of trainable variables of TISTA is equal to the number of iteration rounds and it is much smaller than that of known learnable sparse signal recovery algorithms. This feature leads to highly stable and fast training processes of TISTA. Computer experiments show that TISTA is applicable to various classes of sensing matrices such as Gaussian matrices, binary matrices and matrices with large condition numbers. Numerical results also demonstrate that TISTA shows significantly faster convergence than those of AMP and LISTA in many cases.

This work discusses the problem of sparse signal recovery when there is correlation among the values of nonzero entries. We examine intra-vector correlation in the context of the block sparse model and inter-vector correlation in the context of the multiple measurement vector model, as well as their combination. Algorithms based on the sparse Bayesian learning are presented and the benefits of incorporating correlation at the algorithm level are discussed. The impact of correlation on the limits of support recovery is also discussed highlighting the different impact intra-vector and inter-vector correlations have on such limits.

In this paper, we deal with the signal recovery problem in compressed sensing, that is, the problem of estimating the original signal from its linear measurements. Recovery algorithms can be mainly classified into two types, optimization based algorithms and statistical modeling based algorithms. Basis pursuit (BP) or basis pursuit denoising (BPDN) is one of the most widely used optimization based recovery algorithms, that minimizes the $\ell_{1}$ norm of the signal or its coefficients in some basis under the constraint that its linear transform is equal to or close to the observation signal. There are various extensions of those algorithms depending on the problem structure. When the original signal is an image, the objective function is often the sum of the $\ell_{1}$ norm of the coefficients of the signal in some basis and a total variation (TV) of the image. It can be considered that it requires the image to be sparse in both the specific transform domain and finite differences at the same time. In this paper, we propose a statistical model that represents those sparsities and the signal recovery algorithm based on the variational method. One of the advantages of the statistical approach is that we can utilize the posterior information of the original signal and it is known that it can be used to construct the compressed sensing measurements adaptively. The proposed recovery algorithm and adaptive construction of the compressed sensing measurements are validated on numerical experiments.

Cauchy greedy algorithm for robust sparse recovery and multiclass classification

In this paper, we propose a novel sparse signal recovery algorithm, DU-PSISTA (Deep Unfolded-Periodic Sketched Iterative Shrinkage-Thresholding Algorithm), which aims to balance computational efficiency and accuracy for recovering high-dimensional sparse signals. The proposed method periodically alternates between standard ISTA and a sketched variant that leverages random projections to reduce the dimensionality of gradient computations. This hybrid structure enables flexible control over the trade-off between accuracy and computational complexity through a pre-configurable period parameter. Furthermore, we incorporate deep unfolding into the algorithm that embeds learnable parameters such as step sizes and thresholding factors into each iteration. These parameters are optimized through data-driven training, enabling the model to adaptively improve convergence and performance. Numerical experiments confirm that our method achieves comparable recovery performance to conventional deep unfolded ISTA while reducing computation, especially when the period parameter and sketch size are properly selected.