Compressive Sensing with Sparse Measurement Matrices

Binary sparse measurement matrices are widely used in compressed sensing (CS) due to their low computational complexity. However, binary sparse measurement matrices perform well in CS-based binary signal recovery only when the source signals are very sparse (e.g., k/n=0.1, where k is the sparsity of the source signal, n is the length of the source signal). In this paper, we propose to construct a non-binary sparse measurement matrix to recover binary source signals which are not so sparse (e.g., k/n=0.2) accurately with few measurements. The novel measurement matrix enables us to design a suboptimal and effective recovery algorithm by fully exploiting the structural features. Moreover, we analyze and estimate the un-recovery probability based on the tree structure to evaluate the recovery performance. The simulation results validate that non-binary sparse measurement matrices can be used to recover binary source signals which are not so sparse, the recovery performance of non-binary sparse measurement matrices is better than that of binary sparse measurement matrices in terms of the un-recovery probability.

The algorithms based on the technique of optimal <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula>-thresholding (OT) were recently proposed for signal recovery, and they are very different from the traditional family of hard thresholding methods. However, the computational cost for OT-based algorithms remains high at the current stage of their development. This stimulates the development of the so-called natural thresholding (NT) algorithm and its variants in this paper. The family of NT algorithms is developed through the first-order approximation of the so-called regularized optimal <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula>-thresholding model, and thus the computational cost for this family of algorithms is significantly lower than that of the OT-based algorithms. The guaranteed performance of NT-type algorithms for signal recovery from noisy measurements is shown under the restricted isometry property and concavity of the objective function of regularized optimal <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula>-thresholding model. Empirical results indicate that the NT-type algorithms are robust and very comparable to several mainstream algorithms for sparse signal recovery.

HKUST Electronic Theses Second-order algorithms for sparse signal recovery from intensity-only measurements by Ruixue Wen thesis 2022 1 online resource (xiv, 113 pages) : illustrations (some color) In this thesis, we study the sparse phase retrieval problem, which is to…Read more ›

This paper discusses compressive sensing with digital sparse signals. The motivation is that most existing sparse signal recovery algorithms, like matching pursuit, convex relaxation and Bayesian framework, do not fully exploit the digital nature of signals when dealing with digital sparse signals, which result in certain performance losses. In this paper, we solve this problem via a permutation-based multi-dimensional sensing matrix and an iterative recovery algorithm with maximum likelihood (ML) local detectors. The sensing matrix considered consists of several sub-matrices, each composed of a random permutation matrix and a block-diagonal matrix. The measurements generated from the same permutation matrix are referred to as a dimension. The block-diagonal matrices allow the use of the low-complexity ML detector in each dimension, which best utilizes the digital nature of signals. The multi-dimensional structure of the sensing matrix enables information exchange between dimensions through an iterative process to achieve a near global-optimal estimation. Numerical results are used to show the rate-distortion performance of the proposed technique. It is shown that it can achieve much better rate-distortion than the existing approaches based on convex relaxation and Bayesian framework with digital source signals.

Deterministic construction of sparse binary matrices via incremental integer optimization

Compressive sensing (CS) is an innovative approach to simultaneously measure and compress signals such as biomedical signals that are sparse or compressible. A major effort in CS is to design a measurement matrix that can be used to encode and compress such signals. The measurement matrix structure has a direct impact on the computational and storage costs as well as the recovered signal quality. Sparse measurement matrices (i.e. with few non-zero elements) may drastically reduce these costs. We propose a permuted Kronecker-based sparse measurement matrix for sensing and data recovery in CS applications. In our study, we use three classes of sub-matrices (normalized Gaussian, Bernoulli, and BCH-based matrices) to create the proposed measurement matrix. Using ECG signals from the MIT-BIH Arrhythmia database, we show that the reconstructed signal quality is comparable to the ones achieved using well known CS methods. Our methodology results in an overall reduction in storage and computations, both during the sensing and recovery process. This approach can be generalized to other classes of eligible measurement matrices in CS.

The problem of sparse signal recovery from lower number of observations is formalized as a constrained minimization problem. Due to the dissimilar requirements of signal processing, the constraints and objective functions differ from each other. This paper presents an algorithm scheme base on alternating iteration to deal with the nonconvex objective function. The proposed method employs the log-penalty and proximity operator to seek the sparse solution. In each iteration, the introduction of nonconvex penalty decreases the penalization of large coefficients which contributes to a faster decrease of the function value. By making use of proximity operator, it reduces the computational complexity and achieves less number of iterations. In addition, the experimental results illustrate the effectiveness of the proposed method as well as the drawbacks of the algorithm.

Numerical algorithms for the l0-norm regularized non-smooth non-convex minimization problems have recently became a topic of great interest within signal processing, compressive sensing, statistics, and machine learning. Nevertheless, the l0-norm makes the problem combinatorial and generally computationally intractable. In this paper, we construct a new surrogate function to approximate l0-norm regularization, and subsequently make the discrete optimization problem continuous and smooth. Then we use the well-known spectral gradient algorithm to solve the resulting smooth optimization problem. Experiments are provided which illustrate this method is very promising.

Sparse optimization problems have been successfully applied to a wide range of research areas, and useful insights and elegant methods for proving the stability and convergence of neurodynamic algorithms have been yielded in previous work. This article develops several neurodynamic algorithms for sparse signal recovery by solving the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula> regularization problem. First, in the framework of the locally competitive algorithm (LCA), modified LCA (MLCA) with finite-time convergence and MLCA with fixed-time convergence are designed. Then, the sliding-mode control (SMC) technique is introduced and modified, i.e., modified SMC (MSMC), which is combined with LCA to design MSMC-LCA with finite-time convergence and MSMC-LCA with fixed-time convergence. It is shown that the solutions of the proposed neurodynamic algorithms exist and are unique under the observation matrix satisfying restricted isometry property (RIP) condition, while finite-time or fixed-time convergence to the optimal points is shown via Lyapunov-based analysis. In addition, combining the notions of finite-time stability (FTS) and fixed-time stability (FxTS), upper bounds on the convergence time of the proposed neurodynamic algorithms are given, and the convergence results obtained for the MLCA and MSMC-LCA with fixed-time convergence are shown to be independent of the initial conditions. Finally, simulation experiments of signal recovery and image recovery are carried out to demonstrate the superior performance of the proposed neurodynamic algorithms.

This study concerns the issue of jointly enhancing noise robustness and promoting signal sparsity in Sparse Bayesian Learning (SBL), which aims at addressing the performance deficiency of sparse signal recovery due to uninformative data with low signal-to-noise ratios. In particular, the authors propose a hierarchical prior noise model with a signal-dependent parametrisation and incorporate it into developing the robust SBL algorithms for sparse signal recovery. The main contribution of the proposed approach is twofold. The first is the new consideration of noise-robustness enhancement in building SBL algorithms, which devotes to noise awareness in counteracting outliers in measurements. Specifically, the idea of signal-sparsity enforcing is extended to build a Least Absolute Deviation like loss criterion with the proposed hierarchical prior model of measurement noise. The second is the novelty of using the signal-dependent parametrisation in the proposed noise model. Indeed, the signal-dependent mechanism plays an indispensable role in producing the reliable noise parameter estimation jointly with updating signal model parameters under the fast SBL framework. In addition to numerical simulation studies, the real-life application of radio tomographic imaging is presented to validate the proposed approach.

A modified multiple OLS (m2OLS) algorithm for signal recovery in compressive sensing

For signals reconstruction based on compressive sensing, to reconstruct signals of higher accuracy with lower compression rates, it is required that there is a smaller mutual coherence between the measurement matrix and the sparsifying matrix. Mutual coherence between the measurement matrix and sparsifying matrix can be expressed indirectly by the property of the Gram matrix. On the basis of the Gram matrix, a new optimization algorithm of acquiring a measurement matrix has been proposed in this paper. Firstly, a new mathematical model is designed and a new method of initializing measurement matrix is adopted to optimize the measurement matrix. Then, the loss function of the new algorithm model is solved by the gradient projection-based method of Gram matrix approximating an identity matrix. Finally, the optimized measurement matrix is generated by minimizing mutual coherence between measurement matrix and sparsifying matrix. Compared with the conventional measurement matrices and the traditional optimization methods, the proposed new algorithm effectively improves the performance of optimized measurement matrices in reconstructing one-dimensional sparse signals and two-dimensional image signals that are not sparse. The superior performance of the proposed method in this paper has been fully tested and verified by a large number of experiments.

Numerous algorithms have been proposed recently for sparse signal recovery in Compressed Sensing (CS). In practice, the number of measurements can be very limited due to the nature of the problem and/or the underlying statistical distribution of the non-zero elements of the sparse signal may not be known a priori. It has been observed that the performance of any sparse signal recovery algorithm depends on these factors, which makes the selection of a suitable sparse recovery algorithm difficult. To take advantage in such situations, we propose to use a fusion framework using which we employ multiple sparse signal recovery algorithms and fuse their estimates to get a better estimate. Theoretical results justifying the performance improvement are shown. The efficacy of the proposed scheme is demonstrated by Monte Carlo simulations using synthetic sparse signals and ECG signals selected from MIT-BIH database.

In this paper, we consider the problem of finding the sparsest solution to underdetermined linear systems. Unlike the literatures which use the ℓ1 regularization to approximate the original problem, we consider the ℓ2/3 regularization which leads to a better approximation but a nonconvex, nonsmooth, and non-Lipschitz optimization problem. Through developing a fixed point representation theory associated with the two thirds thresholding operator for ℓ2/3 regularization solutions, we propose a fixed point iterative thresholding algorithm based on two thirds norm for solving the k-sparsity problems. Relying on the restricted isometry property, we provide subsequentional convergence guarantee for this fixed point iterative thresholding algorithm on recovering a sparse signal. By discussing the preferred regularization parameters and studying the phase diagram, we get an adequate and efficient algorithm for the high-dimensional sparse signal recovery. Finally, comparing with the existing algorithms, such as the standard ℓ1 minimization, the iterative reweighted ℓ2 minimization, the iterative reweighted ℓ1 minimization, and iterative Half thresholding algorithm, we display the results of the experiment which indicate that the two thirds norm fixed point iterative thresholding algorithm applied to sparse signal recovery and large scale imageries from noisy measurements can be accepted as an effective solver for ℓ2/3 regularization.

Wearable ECG monitoring devices, often limited by power constraints, require efficient methods to reduce energy consumption and extend battery life for continuous health tracking. This work proposes a digital compressed sensing approach for ECG (DCS-ECG), featuring a suitable sensing matrix, effective sparse signal recovery algorithms, and an optimized number of bits for quantization (BM). The framework is designed to achieve high compression ratios (CR) with minimal signal distortion and parameter extraction without the reconstruction of the original signal. The following five sensing matrices are examined: deterministic block binary diagonal (DBBD) matrix, random binary (RBin) matrix, random Bernoulli (RBen) matrix, random Gaussian (RG) matrix, and random sparse fixed binary (RSB) matrix. These are paired with a sparsifying matrix (ψ), a hybrid of identity matrix (I), discrete sine transform matrix (S), and discrete cosine transform matrix (C) (i.e. ψ = [I S C]). The following five sparse reconstruction algorithms are evaluated: iterative hard thresholding (IHT) algorithm, orthogonal matching pursuit (OMP) algorithm, smoothed ℓ0 (SL0) algorithm, ℓ1 least square (L1LS) algorithm, and subspace matching pursuit (SMP) algorithm. The proposed DCS-ECG system was tested on ECG signals from the MIT-BIH arrhythmia database. The configuration using the DBBD sensing matrix and OMP recovery algorithm with the sparsifying matrix achieved a low average percentage of root mean square difference (PRD) of 3.75 ± 1.43% at CR = 12.5%, which also offers fast reconstruction. For CR = 12.5% and BM = 8, the CS measurement generation on the Arduino Due consumes 2.02 mJ, while reconstruction on the Raspberry Pi requires 1.13 J. The wireless transmission energy is 1.2973 J. Overall, the proposed CS-based system achieves an 86.24% total energy saving compared to the without CS approach, demonstrating its suitability for low-power IoT-enabled wearable edge ECG health monitoring applications.