Sparse Optimization Theory and Methods

The algorithms based on the technique of optimal <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula>-thresholding (OT) were recently proposed for signal recovery, and they are very different from the traditional family of hard thresholding methods. However, the computational cost for OT-based algorithms remains high at the current stage of their development. This stimulates the development of the so-called natural thresholding (NT) algorithm and its variants in this paper. The family of NT algorithms is developed through the first-order approximation of the so-called regularized optimal <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula>-thresholding model, and thus the computational cost for this family of algorithms is significantly lower than that of the OT-based algorithms. The guaranteed performance of NT-type algorithms for signal recovery from noisy measurements is shown under the restricted isometry property and concavity of the objective function of regularized optimal <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula>-thresholding model. Empirical results indicate that the NT-type algorithms are robust and very comparable to several mainstream algorithms for sparse signal recovery.

Compressed sensing or compressed sampling (CS) has been receiving a lot of interest as a promising method for sparse signal recovery from fewer sensors, while the complexity is not high. Because of the sparsity of the Ultra-Wideband (UWB) channel, CS has been used to the UWB channel estimation to reduce demands and complexity of high-speed analog-to-digital (A/D) converters. In this paper, pre-filtering method for UWB channel estimation based on the theory of CS is developed. An FIR filter is adopted at the transmitter, so that we get a measurement matrix, a quasi-Toeplitz matrix, leading to a better reconstruction accuracy of CS. Furthermore, there is no need to add a measurement matrix element in the receiver, which magnifies the noise in the UWB channel. That also makes the receiver simplified. Extensive simulations show that the proposed approach can successfully suppress the additive white Gaussian noise (AWGN). Finally, when the UWB channel estimation method we proposed is used in the correlation detection of UWB signals, it develops better performance of the bit error rate (BER) at lower sampling rate.

Three‐dimensional (3D) imaging of a moving target can be achieved via a wideband multiple‐input multiple‐output (MIMO) radar with planar array in a single snapshot. The sparse recovery (SR) method has been introduced to deal with the MIMO imaging problem with sparse array recently. For sparse linear array, SR method can successfully achieve the high cross‐range resolution. For sparse planar array, however, it is not effective to use a SR algorithm directly in 2D cross‐range direction, because the final 2D imagery will be blurred by the SR‐induced range cell migration (RCM) in the former cross‐range direction. To remove the RCM, a joint sparsity‐driven method is proposed in this study by exploiting the joint sparsity of echo signal. An improved imagery can be obtained with fewer antennas and higher calculation efficiency. Comparative experiments demonstrate the superiority of the proposed method.

In this paper, a Smart Device (SD) localization method, based on the Path Loss (PL) model of Macro Base Station (MBS) and femtocells, using the convex optimization method is discussed for an Internet of Things (IoT) networks. Localization plays a major role for smart city, smart agriculture, and smart health applications in IoT networks. Global Positioning System (GPS) works well for outdoor positioning but fails to provide accurate locations in an indoor environment and non-line-of-sight (NLOS) paths. We propose the Convex optimization (CO) method that uses the combined effects of the Received Signal Strength (RSS) from macrocells and femtocells. The method requires no additional infrastructure and localizes a Smart Device (SD) in an IoT environment. The Cramèr-Rao Lower Bound (CRLB) is also evaluated to analyze the performance of the estimator. Extensive simulations demonstrate that our proposed method provides an accurate location as compared to Least Square method.

Downward-looking sparse-linear-array 3-D synthetic aperture radar (DLSLA 3-D SAR) cross-track reconstruction usually suffers from incomplete observation and limited resolution. The incomplete observation is caused by the sparse and nonuniform distribution of the equivalent antenna phase centers (APCs) due to the array elements' installation location restriction, loss, or deviation. Sparse recovery methods provide a solution with improved resolution from the incomplete observation for the 3-D imaging scene that behaves with spatial sparsity. However, conventional grid-based sparse recovery (GB-SR) methods are under the assumption that the scatterers are located on the discretized grids; otherwise, the off-grid effect or basis mismatch problem will occur. In this letter, we propose a reweighted scheme-based gridless sparse recovery (GL-SR) method, i.e., reweighted gridless sparse iterative covariance-based estimation (RGLS), for DLSLA 3-D SAR cross-track imaging. The proposed method possesses the merits of gridless SPICE (GLS), i.e., free of off-grid effect and user parameters, and has a statistically more appealing property than GLS by adopting the reweighted scheme. As seen from the experiments that compare the performance of GB-SR and GL-SR methods for DLSLA 3-D SAR cross-track reconstruction, the proposed method performs outstandingly under the circumstance of sparse and nonuniform APCs' distribution.

We analyze boosting algorithms [Ann. Statist. 29 (2001) 1189–1232; Ann. Statist. 28 (2000) 337–407; Ann. Statist. 32 (2004) 407–499] in linear regression from a new perspective: that of modern first-order methods in convex optimization. We show that classic boosting algorithms in linear regression, namely the incremental forward stagewise algorithm ($\text{FS}_{\varepsilon}$) and least squares boosting [LS-BOOST$(\varepsilon)$], can be viewed as subgradient descent to minimize the loss function defined as the maximum absolute correlation between the features and residuals. We also propose a minor modification of $\text{FS}_{\varepsilon}$ that yields an algorithm for the LASSO, and that may be easily extended to an algorithm that computes the LASSO path for different values of the regularization parameter. Furthermore, we show that these new algorithms for the LASSO may also be interpreted as the same master algorithm (subgradient descent), applied to a regularized version of the maximum absolute correlation loss function. We derive novel, comprehensive computational guarantees for several boosting algorithms in linear regression (including LS-BOOST$(\varepsilon)$ and $\text{FS}_{\varepsilon}$) by using techniques of first-order methods in convex optimization. Our computational guarantees inform us about the statistical properties of boosting algorithms. In particular, they provide, for the first time, a precise theoretical description of the amount of data-fidelity and regularization imparted by running a boosting algorithm with a prespecified learning rate for a fixed but arbitrary number of iterations, for any dataset.

The compressed sensing(CS) theory has provided a new solution for the low-rate sampling design of Impulse Radio Ultra-WideBand (IR-UWB) signal. This paper gives full consideration of the influence of quantization noise in IR-UWB system, and designs an effective quantization and reconstruction mechanism with high performance of resisting noise. Based on the analysis of noise distribution characteristics in compressed sampling value, the signal reconstruction model is revised, and then the performances of Dantzig-Selector (DS) method, Subspace Pursuit(SP) method, and traditional reconstruction algorithms are compared. On this basis, a signal reconstruction method (hybrid DS-SP method) with self-adaptability in DS and SP is proposed. Based on hybrid DS-SP method and consideration of overload uniform quantization, overload hybrid DS-SP mechanism is proposed which considered both the digital rear-end and analog front-end of IR-UWB receiver under compressed sensing framework. Simulation results show that the overload hybrid DS-SP mechanism achieves the exceptional reconstruction performance compared to the traditional reconstruction algorithms under different noise conditions with a compromised complexity between DS and SP.

In this article, we investigate jointly sparse signal recovery and jointly sparse support recovery in Multiple Measurement Vector (MMV) models for complex signals, which arise in many applications in communications and signal processing. Recent key applications include channel estimation and device activity detection in MIMO-based grant-free random access which is proposed to support massive machine-type communications (mMTC) for Internet of Things (IoT). Utilizing techniques in compressive sensing, optimization and deep learning, we propose two model-driven approaches, based on the standard auto-encoder structure for real numbers. One is to jointly design the common measurement matrix and jointly sparse signal recovery method, and the other aims to jointly design the common measurement matrix and jointly sparse support recovery method. The proposed model-driven approaches can effectively utilize features of sparsity patterns in designing common measurement matrices and adjusting model-driven decoders, and can greatly benefit from the underlying state-of-the-art recovery methods with theoretical guarantee. Hence, the obtained common measurement matrices and recovery methods can significantly outperform the underlying advanced recovery methods. We conduct extensive numerical results on channel estimation and device activity detection in MIMO-based grant-free random access. The numerical results show that the proposed approaches provide pilot sequences and channel estimation or device activity detection methods which can achieve higher estimation or detection accuracy with shorter computation time than existing ones. Furthermore, the numerical results explain how such gains are achieved via the proposed approaches.

In this paper, we consider the theoretical bound of the probability of error in compressed sensing (CS) with the Bayesian approach. In the detection problem, the signal is sparse and is reconstructed from a compressed measurement vector. Utilizing the oracle estimator in CS, we provide a theoretical bound of the probability of error when the noise in CS is white Gaussian noise (WGN). We show that without any additional information in CS, the probability of error obtained using the signal reconstructed by four recovery algorithms: the basis pursuit denoising (BPDN) algorithm, the Dantzig selector, the orthogonal matching pursuit (OMP) method and the compressive sampling matching pursuit (CoSaMP) algorithm is always larger than the derived theoretical bound. Simulation results demonstrate the effectiveness of our result.

In this paper, combining techniques in compressed sensing, parallel optimization and deep learning, an autoencoder-based approach is proposed to jointly design the common measurement matrix and jointly sparse signal recovery method for complex sparse signals. The encoder achieves noisy linear compression for jointly sparse signals, with a common measurement matrix. The decoder realizes jointly sparse signal recovery based on an iterative parallel-coordinate descent algorithm which is proposed to solve GROUP LASSO in a parallel manner. In particular, the decoder consists of an approximation part which unfolds (several iterations of) the proposed iterative algorithm to obtain an approximate solution of GROUP LASSO and a correction part which reduces the difference between the approximate solution and the actual jointly sparse signals. To our knowledge, this is the first time that an optimization-based jointly sparse signal recovery method is implemented using a neural network. The proposed approach achieves higher recovery accuracy with less computation time than the classic GROUP LASSO method, and the gain significantly increases in the presence of extra structures in sparse patterns. The common measurement matrix obtained by the proposed approach is also suitable for the classic GROUP LASSO method. We consider an application example, i.e., channel estimation in Multiple-Input Multiple-Output (MIMO)-based grant-free massive access for massive machine-type communications (mMTC). By numerical results, we demonstrate the substantial gains of the proposed approach over GROUP LASSO and AMP when the number of jointly sparse signals is not very large.

The cloud radio access network (C-RAN) is a promising network architecture for future mobile communications, and one practical hurdle for its large scale implementation is the stringent requirement of high capacity and low latency fronthaul connecting the distributed remote radio heads (RRH) to the centralized baseband pools (BBUs) in the C-RAN. To improve the scalability of C-RAN networks, it is very important to take the fronthaul loading into consideration in the signal detection, and it is very desirable to reduce the fronthaul loading in C-RAN systems. In this paper, we consider uplink C-RAN systems and we propose a distributed fronthaul compression scheme at the distributed RRHs and a joint recovery algorithm at the BBUs by deploying the techniques of distributed compressive sensing (CS). Different from conventional distributed CS, the CS problem in C-RAN system needs to incorporate the underlying effect of multi-access fading for the end-to-end recovery of the transmitted signals from the users. We analyze the performance of the proposed end-to-end signal recovery algorithm and we show that the aggregate measurement matrix in C-RAN systems, which contains both the distributed fronthaul compression and multiaccess fading, can still satisfy the restricted isometry property with high probability. Based on these results, we derive tradeoff results between the uplink capacity and the fronthaul loading in C-RAN systems.

Compressed sensing (CS) is an emerging signal processing paradigm that enables sub-Nyquist processing of sparse signals such as electrocardiogram (ECG) and electromyogram (EMG) biosignals. Consequently, it can be applied to biosignal acquisition systems to reduce the data rate to realize ultra-low-power performance. CS is compared to conventional and adaptive sampling techniques and several system-level design considerations are presented for CS acquisition systems including sparsity and compression limits, thresholding techniques, encoder bit-precision requirements, and signal recovery algorithms. Simulation studies show that compression factors greater than 16X are achievable for ECG and EMG signals with signal-to-quantization noise ratios greater than 60 dB.

HKUST Electronic Theses Second-order algorithms for sparse signal recovery from intensity-only measurements by Ruixue Wen thesis 2022 1 online resource (xiv, 113 pages) : illustrations (some color) In this thesis, we study the sparse phase retrieval problem, which is to…Read more ›

Most signals can be represented sparsely in a basis. Recently, Compressive Sensing Theorem which offers convex optimization algorithms based on l 1 -minimization for sparse signal recovery is often being used. In this paper, some of the iterative signal recovery algorithms alternative to l 1 -minimization solution which are Orthogonal Matching Pursuit (OMP), Compressive Sampling Matching Pursuit (CoSaMP), Iterative Hard Thresholding (IHT) and Lipschitz Iterative Hard Theresholding (LIHT) are compared in noisy and noiseless conditions with various tests. Iterative algorithms alternative to the l 1 optimization method with similar performance are verified. OMP algorithm that works at higher true reconstruction rates in noisy and noiseless conditions can be preferred instead of convex optimization methods.

Group sparse vectors are the generalization of unstructured sparse vectors, with zero and non-zero elements occurring in groups. In group sparse signal recovery problems, we are interested in finding the signal with the smallest number of active groups that satisfy our observations. Because this problem has exponential complexity, a convex relaxation is typically used, which minimizes the sum of the groups' second norm that satisfies the observations. In this paper, we provide a set of deterministic necessary and sufficient conditions that the sensing matrix should satisfy for equivalence between the l0 solution of a structured group sparse problem and its convex relaxation. These conditions are generalization of the Range Space Property that has been previously proposed for equivalence between the l0- and the l1- norms in non-structured sparse recovery problems. We also provide a sufficient condition for a unique solution of the relaxed convex problem.