Signal Processing With Compressive Measurements

Most of the recent compressive sensing (CS) literature has focused on sparse signal recovery based on compressive measurements. However, exact signal recovery may not be required in certain signal processing applications such as in inference problems. In this paper, we provide performance limits of classification of sparse as well as not necessarily sparse signals based on compressive measurements. When signals are not necessarily sparse, we show that Kullback-Leibler and Chernoff distances between two probability density functions under any two hypotheses are preserved up to a factor of M/N with M(<;N)-length compressive measurements compared to that with N-length original measurements when the pdfs of the original-length observation vectors exhibit certain properties. These results are used to quantify the performance limits in terms of upper and lower bounds on the probability of error in signal classification with M-length compressive measurements. When the signals of interest are sparse in the standard canonical basis, performance limits are derived in terms of lower bounds on the probability of error in classifying sparse signals with any classification rule.

We consider the recovery of sparse signals, f ∈ ℝN, containing at most k ≪ N nonzero entries using linear measurements contaminated with i.i.d. Gaussian background noise. Given this measurement model, we present and analyze a computationally efficient group testing strategy for recovering the support of f and approximating its nonzero entries. In particular, we demonstrate that group testing measurement matrix constructions may be combined with statistical binary detection and estimation methods to produce efficient adaptive sequential algorithms for sparse signal support recovery. Furthermore, when f exhibits sufficient sparsity, we show that these adaptive group testing methods allow the recovery of sparse signals using fewer noisy linear measurements than possible with non-adaptive methods based on Gaussian measurement ensembles. As a result we improve on previous sufficient conditions for sparsity pattern recovery in the noisy sublinear-sparsity regime.

In today's society, we are flooded with massive volumes of data in the order of a billion gigabytes on a daily basis from pervasive sensors. It is becoming increasingly challenging to locally store and transport the acquired data to a central location for signal/data processing (i.e., for inference). To alleviate these problems, it is evident that there is an urgent need to significantly reduce the sensing cost (i.e., the number of expensive sensors) as well as the related memory and bandwidth requirements by developing unconventional sensing mechanisms to extract as much information as possible yet collecting fewer data. The first aim of this thesis is to develop theory and algorithms for data reduction. We develop a data reduction tool called sparse sensing, which consists of a deterministic and structured sensing function (guided by a sparse vector) that is optimally designed to achieve a desired inference performance with the reduced number of data samples. The first part of this thesis is dedicated to the development of sparse sensing mechanisms and convex programs to efficiently design sparse sensing functions. We design sparse sensing functions under the assumption that the data is not yet available and the model information is perfectly known. Sparse sensing offers a number of advantages over compressed sensing (a state-of-the-art data reduction method for sparse signal recovery). One of the major differences is that in sparse sensing the underlying signals need not be sparse. This allows us to consider general signal processing tasks (not just sparse signal recovery) under the proposed sparse sensing framework. Specifically, we focus on fundamental statistical inference tasks, like estimation, filtering, and detection. In essence, we present topics that transform classical (e.g., random or uniform) sensing methods to low-cost data acquisition mechanisms tailored for specific inference tasks. The developed framework can be applied to sensor selection, sensor placement, or sensor scheduling, for example. In the second part of this thesis, we focus on some applications related to distributed sampling using sensor networks. Sensor networks can be used as a spatial sampling device, that is, to faithfully represent distributed signals (e.g., a spatially varying phenomenon such as a temperature field). On top of that, the distributed signals can exist in space and time, where the temporal sampling is achieved using analog-to-digital converters, for example. Each sensor has an independent sample clock, and its stability essentially determines the alignment of the temporal sampling grid across the sensors. Due to imperfections in the oscillator, the sample clocks drift from each other, resulting in the misalignment of the temporal sampling grids. To overcome this issue, we devise a mechanism to distribute the sample clock wirelessly. Specifically, we perform wireless clock synchronization based on the time-of-flight measurements of broadcast messages. In addition, clock synchronization also plays a central role in other time-based sensor network applications such as localization. Localization is increasingly gaining popularity in many applications, especially for monitoring environments beyond human reach, e.g., using robots or drones with several sensor units mounted on it. Consequently we now have to localize more than one sensor or even localize the whole sensing platform. Therefore, we extend the classical localization paradigm to localize a (rigid) sensing platform by exploiting the knowledge of the sensor placement on the platform. In particular, we develop algorithms for rigid body localization, i.e., for estimating the position and orientation of the rigid platform using distance measurements. Given the central role of sensing and sensor networks, the results presented in this thesis impacts a wide range of applications.

Compressive sensing is a sub-Nyquist sampling technique for efficient signal acquisition and reconstruction of sparse or compressible signals. In order to account for the sparsity of the underlying signal of interest, it is common to use sparsifying priors such as Bernoulli–Gaussian-inverse Gamma (BGiG) and Gaussian-inverse Gamma (GiG) priors on the components of the signal. With the introduction of variational Bayesian inference, the sparse Bayesian learning (SBL) methods for solving the inverse problem of compressive sensing have received significant interest as the SBL methods become more efficient in terms of execution time. In this paper, we consider the sparse signal recovery problem using compressive sensing and the variational Bayesian (VB) inference framework. More specifically, we consider two widely used Bayesian models of BGiG and GiG for modeling the underlying sparse signal for this problem. Although these two models have been widely used for sparse recovery problems under various signal structures, the question of which model can outperform the other for sparse signal recovery under no specific structure has yet to be fully addressed under the VB inference setting. Here, we study these two models specifically under VB inference in detail, provide some motivating examples regarding the issues in signal reconstruction that may occur under each model, perform comparisons and provide suggestions on how to improve the performance of each model.

This paper studies the recovery of sparse or compressible signals from a certain number of linear measurements when the signal support is partially known.The reconstruction method is based on a convex minimization which minimize the l 1 norm of the signal over the complement of the known part.We propose a relaxed null space condition and give a new bound via restricted isometry constant to guarantee the signal to be recovered stably.

Summary form only given. We describe a set of prior distributions whose independent and identically distributed (iid) realizations result in compressible signals. A signal is compressible when the sorted magnitudes of its coefficients exhibit a power-law decay so that the signal can be well-approximated by a sparse signal. By definition, a sparse signal has only a small fraction of non-zero coefficients compared to its ambient dimension. These distributions, dubbed compressible priors, analytically summarize compressible signals and shed new light to the solution of under-determined linear regression problems by building upon the literature on sparse signal recovery. Our main results are summarized as follows: 1) By using order statistics, we show that iid realizations of generalized Pareto, Student's t, lognormal, generalized extreme value, and log-logistics distributions can result in compressible signals. We quantify the compressibility of the signal realizations from these compressible priors as a function of their parameters. 2) We point out a common misconception about the generalized Gaussian distribution (GGD): GGD result in signals that lose their compressibility as their dimensions grow. For instance, special cases of the GGD distribution, e.g., Laplacian distribution, are commonly used as sparsity promoting priors in compressive sensing and sparse Bayesian learning problems where the number of observations is assumed to grow logarithmically with the number of unknowns. We show that signals generated from Laplacian distribution can only be stably embedded into lower dimensions that grow proportional to the signal dimension. Hence, we identify an inconsistency between the decoding algorithms motivated by the GGD distribution and their sparse solutions. 3) We use compressible priors as a scaffold to build new decoding algorithms based on Bayesian inference arguments. The objective of these algorithms is to approximate the signal realization from a compressible prior as opposed to naively producing sparse solutions. Some of these new algorithms are variants of the popular iterative reweighting schemes. We show (i) how tuning of these algorithms explicitly depends on the parameters of the compressible prior of the signal, and (ii) how to learn the parameters of the signal's compressible prior on the fly while recovering the signal. To illustrate the concepts, we study various order statistics of natural images from the Berkeley Segmentation Dataset, including pixel gradients and wavelet coefficients.

Cauchy greedy algorithm for robust sparse recovery and multiclass classification

Compressed signal processing (CSP) is a branch of compressive sensing (CS), which gives a direction to solve a class of signal processing problems directly from the compressive measurements of a signal. CSP utilizes the information preserved in the compressive measurements of a signal to solve certain inference problems like: classification, detection, and estimation, without reconstructing the original signal. It further simplifies the signal processing compared to conventional CS by omitting their complex reconstruction stage. This, in turn, reduces the implementation complexity of signal processing systems. This paper investigates the performance of CSP for classification application. After extracting the features from compressive measurements, these features or the data instances are used for classification purpose. Through experimental analysis, it has been found that as the CS undersampling factor is increased, the overlapping among the data instances predominates. This results in a complex decision boundary, which in turn degrades the classification accuracy at higher undersampling factors. To overcome the above issue, this paper proposes the use of a machine learning method known as overlap aware learning along with CSP. This generates a smoother decision boundary and hence improves the classification accuracy at higher undersampling factors. The simulation results show the trend of improved classification accuracy using the proposed method. An analysis of the proposed method has been done on different datasets and based on run-time complexity and complexity vs gain analysis to verify the effectiveness of proposed method.

This paper focuses on the reconstruction of second order statistics of signals under a compressive sensing framework, which can be useful in many detection problems. More specifically, the focus is on general cyclostationary signals that are compressed using random linear projections, and using those compressive measurements, the cyclic power spectrum is retrieved. Subsequently, this can for instance be used to detect the occupation of specific frequency bands, which has applications in cognitive radio. Surprisingly, if the span of the random linear projections is larger than the period of the cyclostationary signals, the cyclic power spectrum can be recovered without putting any sparsity constraints on it, which allows for simple least squares reconstruction methods. This result indicates that significant compression can be realized by directly reconstructing the second-order statistics rather than the random signals themselves.

There has been growing interest in performing signal processing tasks directly on compressive measurements, e.g. low-dimensional linear measurements of signals taken with Gaussian random vectors. In this paper, we present a highly efficient algorithm to recover the covariance matrix of high-dimensional data from compressive measurements. We show that, as the number of data samples increases, the eigenvectors (principal components) of the empirical covariance matrix of a simple matrix-vector multiplication of the compressive measurements converge to the true principal components of the original data. Also, we investigate the perturbation of eigenvalues of the covariance matrix under random projection of the data to find conditions for approximate recovery of them. Furthermore, we introduce an important application of our proposed method for efficient estimation of the parameters of Gaussian Mixture Models from compressive measurements. We present experimental results demonstrating the performance and efficiency of our proposed algorithms.

In this paper, we propose a novel direction-of-arrival (DOA) estimation scheme, which is named the compressive one-bit measurement scheme. In the proposed scheme, the one-bit quantization technique is used to reduce the system cost in terms of the analog-to-digital converter (ADC). However, the one-bit quantization leads to a serious information loss, thus compromising the estimation accuracy. Inspired by the compressive sensing theory, the compressive measurement method is used to expand the receive array aperture. To be specific, the compressive measurement allows more sensors than the number of front-end circuit chains to be used. Thus, the additional information is obtained, and the estimation performance can be improved from the system structure layer. It is noted that by introducing the compression operation, the relationship between the normalized and the original covariance matrix is different to the conventional one-bit quantization structure. Thus, two DOA estimation algorithms, i.e., the iterative compressive measurement-based multiple signal classification (CM-MUSIC) and the compressive sensing (CS)-based algorithm, are proposed for the compressive one-bit measurement scheme. The iterative CM-MUSIC can achieve a high-resolution estimation, whereas the CS-based algorithm can resolve more sources than the sensors. Numerical simulations validate the effectiveness of the proposed scheme.

Sparse Bayesian learning using correlated hyperparameters for recovery of block sparse signals

In this paper, we study the performance limits of recovering the support of a sparse signal based on quantized noisy random projections. Although the problem of support recovery of sparse signals with real valued noisy projections with different types of projection matrices has been addressed by several authors in the recent literature, very few attempts have been made for the same problem with quantized compressive measurements. In this paper, we derive performance limits of support recovery of sparse signals when the quantized noisy corrupted compressive measurements are sent to the decoder over additive white Gaussian noise channels. The sufficient conditions which ensure the perfect recovery of sparsity pattern of a sparse signal from coarsely quantized noisy random projections are derived when the maximum-likelihood decoder is used. More specifically, we find the relationships among the parameters, namely the signal dimension N , the sparsity index K , the number of noisy projections M, the number of quantization levels L, and measurement signal-to-noise ratio which ensure the asymptotic reliable recovery of the support of sparse signals when the entries of the measurement matrix are drawn from a Gaussian ensemble.

We study the role of the nonzero entries on the performance limits in support recovery of sparse signals. The key to our results is the recently studied connection between sparse signal recovery and multiple user communication. By leveraging the concept of outage capacity in information theory, we explicitly characterize the impact of the probability distribution imposed on the nonzero entries of the sparse signal on support recovery. When Multiple Measurement Vectors (MMV) are available, we show that the identification of the nonzero rows of the signal is closely connected to decoding the messages from multiple users over a Single-Input Multiple-Output channel. Necessary and sufficient conditions for support (indices of nonzero rows) recovery are provided, and the results allow us to understand the role of correlation of the nonzero entries as well as the role of the rank of the matrix formed from the non-zero entries.

We develop a general framework for compressive linear-projection measurements with side information. Side information is an additional signal correlated with the signal of interest. We investigate the impact of side information on classification and signal recovery from low-dimensional measurements. Motivated by real applications, two special cases of the general model are studied. In the first, a joint Gaussian mixture model is manifested on the signal and side information. The second example again employs a Gaussian mixture model for the signal, with side information drawn from a mixture in the exponential family. Theoretical results on recovery and classification accuracy are derived. The presence of side information is shown to yield improved performance, both theoretically and experimentally.