Natural Thresholding Algorithms for Signal Recovery With Sparsity

In this paper, we consider the problem of finding the sparsest solution to underdetermined linear systems. Unlike the literatures which use the ℓ1 regularization to approximate the original problem, we consider the ℓ2/3 regularization which leads to a better approximation but a nonconvex, nonsmooth, and non-Lipschitz optimization problem. Through developing a fixed point representation theory associated with the two thirds thresholding operator for ℓ2/3 regularization solutions, we propose a fixed point iterative thresholding algorithm based on two thirds norm for solving the k-sparsity problems. Relying on the restricted isometry property, we provide subsequentional convergence guarantee for this fixed point iterative thresholding algorithm on recovering a sparse signal. By discussing the preferred regularization parameters and studying the phase diagram, we get an adequate and efficient algorithm for the high-dimensional sparse signal recovery. Finally, comparing with the existing algorithms, such as the standard ℓ1 minimization, the iterative reweighted ℓ2 minimization, the iterative reweighted ℓ1 minimization, and iterative Half thresholding algorithm, we display the results of the experiment which indicate that the two thirds norm fixed point iterative thresholding algorithm applied to sparse signal recovery and large scale imageries from noisy measurements can be accepted as an effective solver for ℓ2/3 regularization.

HKUST Electronic Theses Second-order algorithms for sparse signal recovery from intensity-only measurements by Ruixue Wen thesis 2022 1 online resource (xiv, 113 pages) : illustrations (some color) In this thesis, we study the sparse phase retrieval problem, which is to…Read more ›

Sparse optimization problems have been successfully applied to a wide range of research areas, and useful insights and elegant methods for proving the stability and convergence of neurodynamic algorithms have been yielded in previous work. This article develops several neurodynamic algorithms for sparse signal recovery by solving the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula> regularization problem. First, in the framework of the locally competitive algorithm (LCA), modified LCA (MLCA) with finite-time convergence and MLCA with fixed-time convergence are designed. Then, the sliding-mode control (SMC) technique is introduced and modified, i.e., modified SMC (MSMC), which is combined with LCA to design MSMC-LCA with finite-time convergence and MSMC-LCA with fixed-time convergence. It is shown that the solutions of the proposed neurodynamic algorithms exist and are unique under the observation matrix satisfying restricted isometry property (RIP) condition, while finite-time or fixed-time convergence to the optimal points is shown via Lyapunov-based analysis. In addition, combining the notions of finite-time stability (FTS) and fixed-time stability (FxTS), upper bounds on the convergence time of the proposed neurodynamic algorithms are given, and the convergence results obtained for the MLCA and MSMC-LCA with fixed-time convergence are shown to be independent of the initial conditions. Finally, simulation experiments of signal recovery and image recovery are carried out to demonstrate the superior performance of the proposed neurodynamic algorithms.

The problem of sparse signal recovery from lower number of observations is formalized as a constrained minimization problem. Due to the dissimilar requirements of signal processing, the constraints and objective functions differ from each other. This paper presents an algorithm scheme base on alternating iteration to deal with the nonconvex objective function. The proposed method employs the log-penalty and proximity operator to seek the sparse solution. In each iteration, the introduction of nonconvex penalty decreases the penalization of large coefficients which contributes to a faster decrease of the function value. By making use of proximity operator, it reduces the computational complexity and achieves less number of iterations. In addition, the experimental results illustrate the effectiveness of the proposed method as well as the drawbacks of the algorithm.

This study concerns the issue of jointly enhancing noise robustness and promoting signal sparsity in Sparse Bayesian Learning (SBL), which aims at addressing the performance deficiency of sparse signal recovery due to uninformative data with low signal-to-noise ratios. In particular, the authors propose a hierarchical prior noise model with a signal-dependent parametrisation and incorporate it into developing the robust SBL algorithms for sparse signal recovery. The main contribution of the proposed approach is twofold. The first is the new consideration of noise-robustness enhancement in building SBL algorithms, which devotes to noise awareness in counteracting outliers in measurements. Specifically, the idea of signal-sparsity enforcing is extended to build a Least Absolute Deviation like loss criterion with the proposed hierarchical prior model of measurement noise. The second is the novelty of using the signal-dependent parametrisation in the proposed noise model. Indeed, the signal-dependent mechanism plays an indispensable role in producing the reliable noise parameter estimation jointly with updating signal model parameters under the fast SBL framework. In addition to numerical simulation studies, the real-life application of radio tomographic imaging is presented to validate the proposed approach.

This paper discusses compressive sensing with sparse measurement matrices. Sparse matrices have several attractive properties, like low computational complexity in both encoding and recovery, easy incremental updates to signals, and low storage requirement, etc. Typical examples of existing algorithms for sparse signal recovery with sparse measurement matrices include convex relaxation, matching pursuit, and Bayesian framework based approaches. In this paper, we propose an alternative technique to this problem. The proposed technique has a linear recovery complexity and relatively good empirical behavior. In this technique, we employ a permutation-based multi-dimensional measurement matrix, which is composed of several sub-matrices, each consisting of a block-diagonal matrix and a random permutation matrix. The measurement symbols generated from the same permutation matrix are referred to as a dimension. Such a measurement matrix brings some useful features to the measurement symbols. Fully exploiting these features enables us to design a simple and effective recovery algorithm. The proposed recovery algorithm employs an iterative process. In each iteration, the algorithm looks for measurement symbols with certain features, and uses such features to recover the source symbols related to these measurements. An iterative process is then applied, along with an interference cancellation operation, to reconstruct all source symbols gradually. The complexity of the proposed algorithm is relatively low, which grows linearly with the source signal length. Numerical results show that the proposed technique empirically offers a much lower sketch length than ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -minimization-based convex relaxation and Bayesian framework based algorithms. It achieves the empirical lower bound of sketch length and linear recovery complexity at the same time.

Numerical algorithms for the l0-norm regularized non-smooth non-convex minimization problems have recently became a topic of great interest within signal processing, compressive sensing, statistics, and machine learning. Nevertheless, the l0-norm makes the problem combinatorial and generally computationally intractable. In this paper, we construct a new surrogate function to approximate l0-norm regularization, and subsequently make the discrete optimization problem continuous and smooth. Then we use the well-known spectral gradient algorithm to solve the resulting smooth optimization problem. Experiments are provided which illustrate this method is very promising.

Numerous algorithms have been proposed recently for sparse signal recovery in Compressed Sensing (CS). In practice, the number of measurements can be very limited due to the nature of the problem and/or the underlying statistical distribution of the non-zero elements of the sparse signal may not be known a priori. It has been observed that the performance of any sparse signal recovery algorithm depends on these factors, which makes the selection of a suitable sparse recovery algorithm difficult. To take advantage in such situations, we propose to use a fusion framework using which we employ multiple sparse signal recovery algorithms and fuse their estimates to get a better estimate. Theoretical results justifying the performance improvement are shown. The efficacy of the proposed scheme is demonstrated by Monte Carlo simulations using synthetic sparse signals and ECG signals selected from MIT-BIH database.

A modified multiple OLS (m2OLS) algorithm for signal recovery in compressive sensing

Ground Penetrating Radar (GPR) is essential for subsurface exploration. Conventional GPR 3D imaging demands dense spatial sampling along regular grids, which is both time-consuming and impractical in complex environments. In this work, we propose a novel method that combines sparse recovery techniques with a placement matrix to merge arbitrarily and sparsely sampled measurements into a regular grid framework. By exploiting the inherent sparsity of subsurface targets and using the Dantzig Selector with cross-validation, our method reconstructs the target reflectivity vector from random spatial sampling. The recovered data is then processed via the Back-Projection Algorithm (BPA) to generate high-resolution 3D images. Simulations demonstrate that our approach not only improves imaging quality under reduced sampling conditions but also efficiently handles arbitrary scanning paths by mapping irregular measurements onto the desired grid.

The cloud radio access network (C-RAN) is a promising network architecture for future mobile communications, and one practical hurdle for its large scale implementation is the stringent requirement of high capacity and low latency fronthaul connecting the distributed remote radio heads (RRH) to the centralized baseband pools (BBUs) in the C-RAN. To improve the scalability of C-RAN networks, it is very important to take the fronthaul loading into consideration in the signal detection, and it is very desirable to reduce the fronthaul loading in C-RAN systems. In this paper, we consider uplink C-RAN systems and we propose a distributed fronthaul compression scheme at the distributed RRHs and a joint recovery algorithm at the BBUs by deploying the techniques of distributed compressive sensing (CS). Different from conventional distributed CS, the CS problem in C-RAN system needs to incorporate the underlying effect of multi-access fading for the end-to-end recovery of the transmitted signals from the users. We analyze the performance of the proposed end-to-end signal recovery algorithm and we show that the aggregate measurement matrix in C-RAN systems, which contains both the distributed fronthaul compression and multiaccess fading, can still satisfy the restricted isometry property with high probability. Based on these results, we derive tradeoff results between the uplink capacity and the fronthaul loading in C-RAN systems.

In this paper, we develop a low-complexity message passing algorithm for joint support and signal recovery of approximately sparse signals. The problem of recovery of strictly sparse signals from noisy measurements can be viewed as a problem of recovery of approximately sparse signals from noiseless measurements, making the approach applicable to strictly sparse signal recovery from noisy measurements. The support recovery embedded in the approach makes it suitable for recovery of signals with same sparsity profiles, as in the problem of multiple measurement vectors (MMV). Simulation results show that the proposed algorithm, termed as JSSR-MP (joint support and signal recovery via message passing) algorithm, achieves performance comparable to that of sparse Bayesian learning (M-SBL) algorithm in the literature, at one order less complexity compared to the M-SBL algorithm.

Suppose that we observe noisy linear measurements of an unknown signal that can be modeled as the sum of two component signals, each of which arises from a nonlinear sub-manifold of a high-dimensional ambient space. We introduce Successive Projection onto INcoherent manifolds (SPIN), a first-order projected gradient method to recover the signal components. Despite the nonconvex nature of the recovery problem and the possibility of underdetermined measurements, SPIN provably recovers the signal components, provided that the signal manifolds are incoherent and that the measurement operator satisfies a certain restricted isometry property. SPIN significantly extends the scope of current signal recovery models and algorithms for low-dimensional linear inverse problems, and matches (or exceeds) the current state of the art in terms of performance.

Seeking sparse solutions of underdetermined linear systems is required in many areas of engineering and science such as signal and image processing. The efficient sparse representation becomes central in various big or high-dimensional data processing, yielding fruitful theoretical and realistic results in these fields. The mathematical optimization plays a fundamentally important role in the development of these results and acts as the mainstream numerical algorithms for the sparsity-seeking problems arising from big-data processing, compressed sensing, statistical learning, computer vision, and so on. This has attracted the interest of many researchers at the interface of engineering, mathematics and computer science. Sparse Optimization Theory and Methods presents the state of the art in theory and algorithms for signal recovery under the sparsity assumption. The up-to-date uniqueness conditions for the sparsest solution of underdertemined linear systems are described. The results for sparse signal recovery under the matrix property called range space property (RSP) are introduced, which is a deep and mild condition for the sparse signal to be recovered by convex optimization methods. This framework is generalized to 1-bit compressed sensing, leading to a novel sign recovery theory in this area. Two efficient sparsity-seeking algorithms, reweighted l1-minimization in primal space and the algorithm based on complementary slackness property, are presented. The theoretical efficiency of these algorithms is rigorously analysed in this book. Under the RSP assumption, the author also provides a novel and unified stability analysis for several popular optimization methods for sparse signal recovery, including l1-mininization, Dantzig selector and LASSO. This book incorporates recent development and the author's latest research in the field that have not appeared in other books.

Source localization is an important field of research and the received signal strength (RSS)-based method is of particular interest. Further, by exploiting the sparsity of localization problem, the compressive sensing (CS) can be applied to considerably decrease the number of RSS measurements, especially in multi-source scenario. However, most existing CS-based localization methods usually neglect some practical issues. In particular, the sensor positions are assumed be known exactly, while in practice they may not be accurate. Additionally, a uniform Gaussian noise assumption is made that the noise variances of sensors are identical, but in practice the noise should be nonuniform. When these assumptions are violated, the localization performance will deteriorate dramatically. To address such issues, in this paper, we formulate the source localization based on superimposed RSS measurements as a sparse signal recovery problem. Moreover, by regarding the sensor positions as adjustable parameters, the inaccurate sensor positions can be refined through the adjustment of parameters. Following this idea, we develop a novel iterative algorithm for joint signal recovery and parameter optimization based on the variational expectation-maximization algorithm. Consequently, the sensor position uncertainty can be alleviated and thus the signal recovery performance will be improved greatly. Meanwhile, it is also capable of learning the variance of nonuniform noise. Extensive simulation results demonstrate the superiority of the proposed method.