Approximate message passing-based compressed sensing reconstruction with generalized elastic net prior

We consider the linear regression problem, where the goal is to recover the vector $\boldsymbol{x}\in \mathbb {R}^n$ from measurements $\boldsymbol{y}=\boldsymbol{A}\boldsymbol{x}+\boldsymbol{w}\in \mathbb {R}^m$ under known matrix $\boldsymbol{A}$ and unknown noise $\boldsymbol{w}$ . For large i.i.d. sub-Gaussian $\boldsymbol{A}$ , the approximate message passing (AMP) algorithm is precisely analyzable through a state-evolution (SE) formalism, which furthermore shows that AMP is Bayes optimal in certain regimes. The rigorous SE proof, however, is long and complicated. And, although the AMP algorithm can be derived as an approximation of loop belief propagation (LBP), this viewpoint provides little insight into why large i.i.d. $\boldsymbol{A}$ matrices are important for AMP, and why AMP has a state evolution. In this work, we provide a heuristic derivation of AMP and its state evolution, based on the idea of “first-order cancellation,” that provides insights missing from the LBP derivation while being much shorter than the rigorous SE proof.

A common goal in many research areas is to reconstruct an unknown signal x from noisy linear measurements. Approximate message passing (AMP) is a class of low-complexity algorithms for efficiently solving such high-dimensional regression tasks. Often, it is the case that side information (SI) is available during reconstruction. For this reason a novel algorithmic framework that incorporates SI into AMP, referred to as approximate message passing with side information (AMP-SI), has been recently introduced. An attractive feature of AMP is that when the elements of the signal are exchangeable, the entries of the measurement matrix are independent and identically distributed (i.i.d.) Gaussian, and the denoiser applies the same non-linearity at each entry, the performance of AMP can be predicted accurately by a scalar iteration referred to as state evolution (SE). However, the AMP-SI framework uses different entry-wise scalar denoisers, based on the entry-wise level of the SI, and therefore is not supported by the standard AMP theory. In this work, we provide rigorous performance guarantees for AMP-SI when the input signal and SI are drawn i.i.d. according to some joint distribution subject to finite moment constraints. Moreover, we provide numerical examples to support the theory which demonstrate empirically that the SE can predict the AMP-SI mean square error accurately.

We consider the problem of recovering a vector $\beta_o \in \mathbb{R}^p$ from $n$ random and noisy linear observations $y= X\beta_o + w$, where $X$ is the measurement matrix and $w$ is noise. The LASSO estimate is given by the solution to the optimization problem $\hat{\beta}_{\lambda} = \arg \min_{\beta} \frac{1}{2} \|y-X\beta\|_2^2 + \lambda \| \beta \|_1$. Among the iterative algorithms that have been proposed for solving this optimization problem, approximate message passing (AMP) has attracted attention for its fast convergence. Despite significant progress in the theoretical analysis of the estimates of LASSO and AMP, little is known about their behavior as a function of the regularization parameter $\lambda$, or the thereshold parameters $\tau^t$. For instance the following basic questions have not yet been studied in the literature: (i) How does the size of the active set $\|\hat{\beta}^\lambda\|_0/p$ behave as a function of $\lambda$? (ii) How does the mean square error $\|\hat{\beta}_{\lambda} - \beta_o\|_2^2/p$ behave as a function of $\lambda$? (iii) How does $\|\beta^t - \beta_o \|_2^2/p$ behave as a function of $\tau^1, \ldots, \tau^{t-1}$? Answering these questions will help in addressing practical challenges regarding the optimal tuning of $\lambda$ or $\tau^1, \tau^2, \ldots$. This paper answers these questions in the asymptotic setting and shows how these results can be employed in deriving simple and theoretically optimal approaches for tuning the parameters $\tau^1, \ldots, \tau^t$ for AMP or $\lambda$ for LASSO. It also explores the connection between the optimal tuning of the parameters of AMP and the optimal tuning of LASSO.

Approximate message passing (AMP) is an efficient iterative signal recovery algorithm for compressed sensing (CS). For sensing matrices with independent and identically distributed (i.i.d.) Gaussian entries, the behavior of AMP can be asymptotically described by a scaler recursion called state evolution. Orthogonal AMP (OAMP) is a variant of AMP that imposes a divergence-free constraint on the denoiser. In this paper, we extend OAMP to incorporate generic denoisers, hence the name D-OAMP. Our numerical results show that state evolution predicts the performance of D-OAMP well for generic denoisers when i.i.d. Gaussian or partial orthogonal sensing matrices are involved. We compare the performances of denosing-AMP (D-AMP) and D-OAMP for recovering natural images from CS measurements. Simulation results show that D-OAMP outperforms D-AMP in both convergence speed and recovery accuracy for partial orthogonal sensing matrices.

In the grant-free massive machine-type communication (mMTC) scenario, a key challenge is the joint device activity detection and data decoding. The sporadic nature of mMTC makes compressed sensing a promising solution to the activity detection problem. However, the typical two-phase coherent transmission scheme, which divides channel training and data decoding into two separate phases, suffers performance losses, especially when only a few bits of data are transmitted by each active device. This paper follows a newly proposed non-coherent transmission scheme in which the data bits are embedded in the pilot sequences and the BS simultaneously detects active devices and decodes the embedded data bits without explicit channel estimation. To exploit statistical channel information and the specific structure of the sparsity pattern introduced by the non-coherent transmission scheme, i.e., only one row in each section can be non-zero, we propose a receiving method based on the approximate message passing (AMP) algorithm with non-separable minimum mean-squared error denoisers specifically designed for the problem. The corresponding state evolution equations, which can be used to predict the section error rate (SER) performance, is obtained and simplified under certain assumptions. We also derive closed-form expressions of the SER performance based on the state evolution results. Finally, numerical simulations are given to validate the accuracy of the performance analysis and to show the superiority of the proposed receiving method over the conventional method based on AMP with separable denoisers in the literature.

This paper studies a large random matrix system (LRMS) model involving an arbitrary signal distribution and forward error control (FEC) coding. We establish an area property based on the approximate message passing (AMP) algorithm. Under the assumption that the state evolution for AMP is correct for the coded system, the achievable rate of AMP is analyzed. We prove that AMP achieves the constrained capacity of the LRMS with an arbitrary signal distribution provided that a matching condition is satisfied. As a byproduct, we provide an alternative derivation for the constraint capacity of an LRMS using a proved property of AMP. We discuss realization techniques for the matching principle of binary signaling using irregular low-density parity-check (LDPC) codes and provide related numerical results. We show that the optimized codes demonstrate significantly better performance over un-matched ones under AMP. For quadrature phase shift keying (QPSK) modulation, bit error rate (BER) performance within 1 dB from the constrained capacity limit is observed.

Recovering a sparse signal from an undersampled set of random linear measurements is the main problem of interest in compressed sensing. In this paper, we consider the case where both the signal and the measurements are complex-valued. We study the popular recovery method of l1-regularized least squares or LASSO. While several studies have shown that LASSO provides desirable solutions under certain conditions, the precise asymptotic performance of this algorithm in the complex setting is not yet known. In this paper, we extend the approximate message passing (AMP) algorithm to solve the complex-valued LASSO problem and obtain the complex approximate message passing algorithm (CAMP). We then generalize the state evolution framework recently introduced for the analysis of AMP to the complex setting. Using the state evolution, we derive accurate formulas for the phase transition and noise sensitivity of both LASSO and CAMP. Our theoretical results are concerned with the case of i.i.d. Gaussian sensing matrices. Simulations confirm that our results hold for a larger class of random matrices.

We consider the design and analysis of spatially coupled sparse regression codes (SC-SPARCs), which were recently introduced by Barbier et al. for efficient communication over the additive white Gaussian noise channel. SC-SPARCs can be efficiently decoded using an Approximate Message Passing (AMP) decoder, whose performance in each iteration can be predicted via a set of equations called state evolution. In this paper, we give an asymptotic characterization of the state evolution equations for SC-SPARCs. For any given base matrix (that defines the coupling structure of the SC-SPARC) and rate, this characterization can be used to predict whether AMP decoding will succeed in the large system limit. We then consider a simple base matrix defined by two parameters $(\omega, \Lambda)$ , and show that AMP decoding succeeds in the large system limit for all rates $R . The asymptotic result also indicates how the parameters of the base matrix affect the decoding progression. Simulation results are presented to evaluate the performance of SC-SPARCs defined with the proposed base matrix.

Approximate message passing (AMP) algorithms break a (high-dimensional) statistical problem into manageable parts and then repeatedly solve each part in turn, akin to alternating projections. One notable characteristic is that their asymptotic behaviours can be accurately predicted via their associated state evolution equations. Orthogonal AMP (OAMP) was recently developed to avoid the need for computing the so-called Onsager term in traditional AMP algorithms, providing two clear benefits: the derivation of an OAMP algorithm is both straightforward and more broadly applicable. OAMP was originally demonstrated for statistical problems with a single measurement vector and single transform. This paper extends OAMP to statistical problems with multiple measurement vectors (MMVs) and multiple transforms (MTs). We name the resulting algorithms as OAMP-MMV and OAMP-MT, respectively, and their combination as augmented OAMP (A-OAMP). Whereas the extension of traditional AMP algorithms to such problems would be challenging, the orthogonal principle underpinning OAMP makes these extensions straightforward. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">The MMV and MT models are widely applicable to signal processing and communications. We present an example of a MIMO relay system with correlated source data and signal clipping, which can be effectively modelled as a joint MMVMT system. While existing methods encounter challenges when applied to this example, OAMP offers an efficient solution with excellent performance.

We consider compressive imaging problems, where images are reconstructed from a reduced number of linear measurements. Our objective is to improve over current state of the art compressive imaging algorithms in terms of both reconstruction error and runtime. To pursue our objective, we propose a compressive imaging algorithm that employs the approximate message passing (AMP) framework. AMP is an iterative signal reconstruction algorithm that performs scalar denoising of noisy signals. In this work, we apply an adaptive Wiener filter, which is a wavelet-based image denoiser, within AMP. Numerical results show that the proposed algorithm improves over the state of the art in both reconstruction error and runtime.

To improve operation speed of conventional imaging algorithms, an improved AMP(Approximate Message Passing) algorithm using realification, namely RMoAMP algorithm, was proposed. In RMoAMP algorithm, we multiply a realification matrix by the azimuth sparse transform matrix and using the relationship between Kronecker Product and vectorization of matrix in iterative process to reduce the amount of calculation. the simulations show that compared to the traditional reconstruction algorithms such as OMP, SLO and AMP, the proposed algorithm requires less computation time and good imaging results have been obtained. Proposed algorithm can be used in ISAR fast imaging.

This paper focuses on algorithmic approaches to solve the robust face recognition problem where the test face image can be corrupted. The standard approach is to formulate the problem as a sparse recovery problem and solve it using l 1 -minimization. As an alternative, the approximate message passing (AMP) algorithm had been tested but resulted in pessimistic results. Our contribution is to successfully solve this problem using the AMP framework. Recently developed adaptive damping technique has been adopted to address the issue that AMP normally only works well with Gaussian matrices. Statistical models are designed to capture the nature of the signal more authentically. Expectation maximization (EM) method has been used to learn the unknown hyper-parameters of the statistical model in an online fashion. Simulations demonstrate that our method achieves better recognition performance than the impressive benchmark l 1 -minimization, is robust to the initial values of hyper-parameters, and exhibits low computational cost.

In this work, we propose a low-overhead characteristic learning mechanism for the time-varying massive MIMO channels. Specially, we exploit the common sparsity and temporal correlation of the channel. Firstly, using VCR and modeling the temporal correlation as an autoregressive process, we formulate the dynamic massive MIMO channel as a sparse signal model. Then, an expectation maximization (EM) based sparse Bayesian learning (SBL) framework is developed to learn model parameters. To achieve the posteriors of model parameters, approximate message passing (AMP) is utilized in the expectation step. Finally, we demonstrate the performance through numerical simulations.

We consider the problem of decoding a discrete signal of categorical variables from the observation of several histograms of pooled subsets of it. We present an approximate message passing (AMP) algorithm for recovering the signal in the random dense setting where each observed histogram involves a random subset of entries of size proportional to n. We characterize the performance of the algorithm in the asymptotic regime where the number of observations m tends infinity proportionally to n by deriving the corresponding state evolution (SE) equations and studying their dynamics. We initiate the analysis of the multi-dimensional SE dynamics by proving their convergence to a fixed point, along with some further properties of the iterates. The analysis reveals sharp phase transition phenomena where the behavior of AMP changes from exact recovery to weak correlation with the signal, as m/n crosses a threshold. We derive formulae for the threshold in some special cases and show that they accurately match experimental behavior.

In this correspondence paper, we study the application of approximate message passing (AMP) algorithm in the signal detection of index modulation (IM) with multiple active resources (antennas or subcarriers). First, for uplink generalized spatial modulation (GSM), AMP detectors in both spatially uncorrelated and correlated channels are proposed. Specifically, an effective method to utilize the activation pattern constraint of GSM signal vector per user in the posterior probability computation of scalar symbol node is derived. Furthermore, a simple while effective damping strategy is introduced in AMP detector to guarantee convergence in correlated channels. Second, for orthogonal frequency division multiplexing with IM in rapidly time-varying channels, the damped AMP detector is further studied. In addition, the convergence of proposed detectors is analyzed by state evolution. Finally, the effectiveness of proposed AMP detectors is verified by simulations.