On the Optimality of Backward Regression: Sparse Recovery and Subset Selection

The greedy algorithms are efficient ways in reconstructing the sparse signal. Among all the greedy recovery algorithms for practical compressive sampling(CS), Subspace Pursuit(SP) can offer reliable recovery accuracy with low costs. In this paper, the SP algorithm is optimized by reducing the complexity of the least square(LS) caculation in each iteration. The Optimized SP performs well in LTE channel estimation when compared with the other greedy algorithms. The utilization of Partial Fourier Matrix helps reduce the matrix storage in SP hardware implemention. The matrix inverse caculation is also simplified by taking advatage of the Hermite Toeplitz matrix generated from the Fourier Matrix.

In sparse approximation theory, the fundamental problem is to reconstruct a signal AisinRn from linear measurements (A,psii) with respect to a dictionary of psii's. Recently, there is focus on the novel direction of Compressed Sensing where the reconstruction can be done with very few-O(klogn)-linear measurements over a modified dictionary if the signal is compressible, that is, its information is concentrated in k coefficients with the original dictionary. In particular, the results prove that there exists a single O(klogn)timesn measurement matrix such that any such signal can be reconstructed from these measurements, with error at most O(1) times the worst case error for the class of such signals. Compressed sensing has generated tremendous excitement both because of the sophisticated underlying mathematics and because of its potential applications. In this paper, we address outstanding open problems in Compressed Sensing. Our main result is an explicit construction of a non-adaptive measurement matrix and the corresponding reconstruction algorithm so that with a number of measurements polynomial in k, logn, 1/epsiv, we can reconstruct compressible signals. This is the first known polynomial time explicit construction of any such measurement matrix. In addition, our result improves the error guarantee from O(1) to 1+epsiv and improves the reconstruction time from poly(n) to poly (klogn). Our second result is a randomized construction of O(kpolylog(n)) measurements that work for each signal with high probability and gives per-instance approximation guarantees rather than over the class of all signals. Previous work on compressed sensing does not provide such per-instance approximation guarantees; our result improves the best known number of measurements known from prior work in other areas including learning theory, streaming algorithms and complexity theory for this case. Our approach is combinatorial. In particular, we use two parallel sets of group tests, one to filter and the other to certify and estimate; the resulting algorithms are quite simple to implement.

The compressed sensing(CS) theory has provided a new solution for the low-rate sampling design of Impulse Radio Ultra-WideBand (IR-UWB) signal. This paper gives full consideration of the influence of quantization noise in IR-UWB system, and designs an effective quantization and reconstruction mechanism with high performance of resisting noise. Based on the analysis of noise distribution characteristics in compressed sampling value, the signal reconstruction model is revised, and then the performances of Dantzig-Selector (DS) method, Subspace Pursuit(SP) method, and traditional reconstruction algorithms are compared. On this basis, a signal reconstruction method (hybrid DS-SP method) with self-adaptability in DS and SP is proposed. Based on hybrid DS-SP method and consideration of overload uniform quantization, overload hybrid DS-SP mechanism is proposed which considered both the digital rear-end and analog front-end of IR-UWB receiver under compressed sensing framework. Simulation results show that the overload hybrid DS-SP mechanism achieves the exceptional reconstruction performance compared to the traditional reconstruction algorithms under different noise conditions with a compromised complexity between DS and SP.

A number of neural networks can be formulated as the linear-in-the-parameters models. Training such networks can be transformed to a model selection problem where a compact model is selected from all the candidates using subset selection algorithms. Forward selection methods are popular fast subset selection approaches. However, they may only produce suboptimal models and can be trapped into a local minimum. More recently, a two-stage fast recursive algorithm (TSFRA) combining forward selection and backward model refinement has been proposed to improve the compactness and generalization performance of the model. This paper proposes unified two-stage orthogonal least squares methods instead of the fast recursive-based methods. In contrast to the TSFRA, this paper derives a new simplified relationship between the forward and the backward stages to avoid repetitive computations using the inherent orthogonal properties of the least squares methods. Furthermore, a new term exchanging scheme for backward model refinement is introduced to reduce computational demand. Finally, given the error reduction ratio criterion, effective and efficient forward and backward subset selection procedures are proposed. Extensive examples are presented to demonstrate the improved model compactness constructed by the proposed technique in comparison with some popular methods.

Iterative selection and correction based adaptive greedy algorithm for compressive sensing reconstruction

The design of wireless sensor networks for signal processing applications has to consider the limited battery power of the sensor nodes. In this paper, a combined routing path selection and power allocation strategy is presented that is especially designed for distributed detection in sensor networks with a serial topology. The objective is to minimize the global probability of detection error of the serial network under a total network power constraint. The cross-layer approach for the selection of a routing path through the network is based on the local observation SNR of the sensors as well as fading conditions and is efficiently implemented by a backward greedy algorithm. After the routing path is established, a subsequently performed power allocation algorithm aims to optimally distribute a total power budget with respect to the detection performance of the sensor network.

This paper proposes an efficient image reconstruction for compressive sensing (CS) that combines the Lifting Wavelet Transform (LWT) using Symlet 8 (sym8) wavelets with three reconstruction algorithms: Subspace Pursuit (SP), Compressive Sampling Matching Pursuit (CoSaMP), and the Analytic Learned Iterative Shrinkage Thresholding Algorithm (ALISTA). Unlike the conventional Discrete Wavelet Transform (DWT) which relies on computationally intensive convolution operations the LWT provides a faster sparse representation while preserving the sparsity crucial for CS. The proposed approach leverages a key insight: among the four subbands produced by the LWT namely the approximation (CA) and the detail coefficients (LH, HL, HH) only the latter three are inherently sparse. Therefore, compressive sensing is applied exclusively to these detail subbands, while the CA subband is left uncompressed to retain essential low-frequency information. Experiments were conducted on both a natural test image (Lena) and a medical MRI scan, across image resolutions ranging from 200×200 to 512×512 pixels and sampling rates from 10% to 80%. Performance was assessed using the Structural Similarity Index (SSIM) and reconstruction time. Results consistently demonstrate that ALISTA significantly outperforms SP and CoSaMP in both reconstruction fidelity and computational efficiency. At an 80% sampling rate, ALISTA achieves SSIM values of 0.99346 for Lena and 0.98 for the MRI image, compared to approximately 0.97 and 0.94, respectively, for the other two methods. Furthermore, ALISTA maintains remarkably low reconstruction times under 4 seconds even for 512×512-pixel images. These findings confirm that the ALISTA + LWT/sym8 combination offers the best trade-off between image quality and speed, exhibiting robustness across different image types and scales.

This paper proposes an efficient image reconstruction for compressive sensing (CS) that combines the Lifting Wavelet Transform (LWT) using Reverse Biorthogonal 6.8 (rbio6.8) wavelets with three reconstruction algorithms: Subspace Pursuit (SP), Compressive Sampling Matching Pursuit (CoSaMP), and the Analytic Learned Iterative Shrinkage Thresholding Algorithm (ALISTA). Unlike the conventional Discrete Wavelet Transform (DWT) which relies on computationally intensive convolution operations the LWT provides a faster sparse representation while preserving the sparsity crucial for CS. The proposed approach leverages a key insight: among the four subbands produced by the LWT namely the approximation (CA) and the detail coefficients (LH, HL, HH) only the latter three are inherently sparse. Therefore, compressive sensing is applied exclusively to these detail subbands, while the CA subband is left uncompressed to retain essential low-frequency information. Experiments were conducted on both a natural test image (Lena) and a medical MRI scan, across image resolutions ranging from 200×200 to 512×512 pixels and sampling rates from 10% to 80%. Performance was assessed using the Structural Similarity Index (SSIM) and reconstruction time. Results consistently demonstrate that ALISTA significantly outperforms SP and CoSaMP in both reconstruction fidelity and computational efficiency. At an 80% sampling rate, ALISTA achieves SSIM values of 0.99314 for Lena and 0.97688 for the MRI image, compared to approximately 0.97402 and 0.93577, respectively, for the other two methods. Furthermore, ALISTA maintains remarkably low reconstruction times under 4 seconds even for 512×512-pixel images. These findings confirm that the ALISTA + LWT/rbio6.8 combination offers the best trade-off between image quality and speed, exhibiting robustness across different image types and scales.

Orthogonal least squares (OLS) is a classic algorithm for sparse recovery, function approximation, and subset selection. In this paper, we analyze the performance guarantee of the OLS algorithm. Specifically, we show that OLS guarantees the exact reconstruction of any K-sparse vector in K iterations, provided that a sensing matrix has unit ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -norm columns and satisfies the restricted isometry property (RIP) of order K + 1 with δ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K+1</sub> <; C <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</sub> = {1/√K, K = 1, 1/√(K + 1/4), K = 2, 1/√(K + 1/16), K = 3, 1/√K, K ≥ 4}. Furthermore, we show that the proposed guarantee is optimal in the sense that if δ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K+1</sub> ≥ C <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</sub> , then there exists a counterexample for which OLS fails the recovery.

Backward sequential elimination for sparse vector subset selection

Exploiting the sparse user activity induced by sporadic transmission, compressed sensing (CS) has been widely applied in multiple-input multiple-output (MIMO) enabled non-orthogonal multiple access (NOMA) for efficient multiuser detection. However, most of the existing detection schemes in MIMO enabled NOMA systems focus mainly on the spatial structure in user activity induced by multiantenna reception, while the temporal correlation in user activity has rarely been incorporated for further performance improvement. To address this issue, we propose a novel Multiuser Detection framework in MIMO enabled NOMA (MDMN), which explicitly integrates the temporal correlation in user activity into the detection process to improve the user detection accuracy. Specifically, we first formulate multiuser detection with multiantenna reception in MDMN as a block CS problem, by exploiting the spatial correlation in user activity. Furthermore, through incorporating the temporal correlation in active user sets into block CS, an adaptive CS algorithm based on subspace pursuit is developed for MDMN, named spatial-temporal correlation enhanced adaptive subspace pursuit (STASP). In particular, STASP does not require any prior knowledge of the user sparsity, as the cross validation designed in STASP can properly terminate the algorithm. The superior performance of the proposed MDMN framework and the corresponding algorithms is corroborated by extensive simulation results.

Compressed Sensing (CS), a popular technique which seeks to capture a discrete signal with a small number of linear measurements, could be used to compress a signal during the process of sampling. As an iterative greedy reconstruction algorithm for practical CS, sparsity adaptive matching pursuit (SAMP) takes advantage of the capability of signal reconstruction without prior information of the sparsity in the process of resuming the original high-dimension-data from low-dimension measurement. This paper presents a backward and adaptive matching pursuit reconstruction algorithm with fixed step sizes to avoid the overestimation phenomena of SAMP by using a standard regularized approach. Firstly, a fixed and biggish step size is set to make sure the size of support set of the signal to be reconstructed increasing stably. The energy difference between adjacent reconstructed signals is then taken as the halting condition of iteration. A standard regularized approach is employed to post-dispose the final iteration results, which backward eliminates superfluous atoms to acquire exact reconstruction. Experimental results show that such an improvement of SAMP is feasible in technology and effective in acquiring quick and exact reconstruction with sufficient measurement.

Compressed Sensing (CS) is a new framework to measure a signal at sub-Nyquist rate, implying that fewer measurements are needed in reduction of acquisition time, simplification of hardware, reduction of memory space required, etc. In this paper, We demonstrate that ideas from compressed sensing may be exploited in the context of real, no-spare audio signals with short-term stability and focus on two types of greedy reconstruction algorithms: matching pursuit and gradient pursuit, including MP, OMP, StOMP, CoSaMP, GP, CGP, etc. The performance of the greedy reconstruction algorithms is analyzed for original audio signals with the reconstruction time, reconstruction error, SDR, etc. It is demonstrated that the compressed sensing framework may be applied with relatively minor modifications for real audio signals with short-term stability.

In this paper, a fault feature extraction method based on the combination of computational order tracking (COT) and variational mode decomposition (VMD) is proposed to solve the problem of fault feature extraction in a gearbox under variable speed conditions. First, a speed estimation method based on a forward-backward greedy algorithm and Fourier fitting is proposed to solve the speed estimation problem under variable speed conditions. Then, a feature extraction method based on COT, VMD, and fast kurtometry is proposed. COT is used to calculate the order. After the signal is decomposed by VMD, a fast kurtogram is used to filter the modal components with obvious fault features so as to extract fault features. Finally, combined with the analysis of actual equipment examples, the experimental results show that steady-state filtering can effectively filter the background noise and improve the signal-to-noise ratio. The rotation speed estimated by the forward–backward greedy algorithm and Fourier fitting is very close to the actual speed, which verifies that the method proposed in this paper can effectively solve the problem of fault feature extraction of a gearbox under variable speed conditions.

A novel noise reduction algorithm combined with compressive sensing (CS) and lifting wavelet transform (LWT) is proposed in this paper. This algorithm can overcome the limitations of traditional noise reduction methods based on Kalman filtering and wavelet threshold filtering. The characteristics of wavelet time-frequency distribution of the microelectromechanical system (MEMS) gyroscope are discussed to illustrate the demerit of the classical filtering methods. Noise reduction algorithm of MEMS gyroscope signal is studied in detail by combining CS theory with lifting wavelet transform. De-noising effect, time-consumption of computation as well as traditional CS reconstruction algorithms are analyzed. The results show that the signal reconstruction algorithm of conventional matching pursuit (MP) greedy algorithms contains more glitches and computation time-consumption, the basis pursuit de-noising (BPDN) algorithm is better and it has advantages of high computational efficiency and ease of implementation.