TwIST sparse regularization method using cubic B-spline dual scaling functions for impact force identification

Compressed sensing (CS) reconstruction of nonsparse scenes is one of the difficulties in synthetic aperture radar (SAR) imaging technology. Although the conventional CS method with sparse representation has proven applicable for nonsparse SAR reconstruction, its disadvantages are unsatisfactory imaging quality and high computational complexity under downsampling. In this paper, a novel deep learning approach for nonsparse SAR scene reconstruction is proposed based on sparse representation and the iterative shrinkage threshold algorithm (ISTA). Specifically, we first develop a sparse representation-based imaging model associated with the ℓ₁ sparse regularizer in nonlinear transform domains. Then, the advantages of the recurrent neural network (RNN) and convolutional neural network (CNN) are incorporated into an ISTA-inspired deep unfolded network (DUN) called SR-ISTA-Net, in which all the parameters are layer-varied rather than handcrafted. The experiments verify that the proposed SR-ISTA-Net can provide high-quality reconstruction results under nonsparse scenes while substantially reducing imaging time.

Conventional sparse spike deconvolution algorithms that are based on the iterative shrinkage-thresholding algorithm (ISTA) are widely used. The aim of this type of algorithm is to obtain accurate seismic wavelets. When this is not fulfilled, the processing stops being optimum. Using a recurrent neural network (RNN) as deep learning method and applying backpropagation to ISTA, we have developed an RNN-like ISTA as an alternative sparse spike deconvolution algorithm. The algorithm is tested with both synthetic and real seismic data. The algorithm first builds a training dataset from existing well-logs seismic data and then extracts wavelets from those seismic data for further processing. Based on the extracted wavelets, the new method uses ISTA to calculate the reflection coefficients. Next, inspired by the backpropagation through time (BPTT) algorithm, backward error correction is performed on the wavelets while using the errors between the calculated reflection coefficients and the reflection coefficients corresponding to the training dataset. Finally, after performing backward correction over multiple iterations, a set of acceptable seismic wavelets is obtained, which is then used to deduce the sequence of reflection coefficients of the real data. The new algorithm improves the accuracy of the deconvolution results by reducing the effect of wrong seismic wavelets that are given by conventional ISTA. In this study, we account for the mechanism and the derivation of the proposed algorithm, and verify its effectiveness through experimentation using theoretical and real data.

Iterative shrinkage-thresholding algorithms provide simple methods to recover sparse signals from compressed measurements. In this paper, we propose a new class of iterative shrinkage-thresholding algorithms which preserve the computational simplicity and improve iterative estimation by incorporating a soft support detection. Indeed, at each iteration, by learning the components that are likely to be nonzero from the current signal estimation using Bayesian techniques, the shrinkage-thresholding step is adaptively tuned and optimized. Unlike other adaptive methods, we are able to prove, under suitable conditions, the convergence of the proposed methods. Moreover, we show through numerical experiments that the proposed methods outperform classical shrinkage-thresholding in terms of rate of convergence and of sparsity-undersampling tradeoff.

Iterative shrinkage/thresholding algorithm (ISTA) is a well-studied method for finding sparse solutions to ill-posed inverse problems. In this letter, we present a data-driven scheme for learning optimal thresholding functions for ISTA. The proposed scheme is obtained by relating iterations of ISTA to layers of a simple deep neural network (DNN) and developing a corresponding error backpropagation algorithm that allows to fine-tune the thresholding functions. Simulations on sparse statistical signals illustrate potential gains in estimation quality due to the proposed data adaptive ISTA.

Electrical Resistance Tomography (ERT) aims to estimate the electrical properties at the interior of an object from current-voltage measurements on its boundary. Image reconstruction of ERT is a nonlinear and ill-posed inverse problem. Regularization technique is used to solve the inverse problems and two-step iterative shrinkage/thresholding algorithm (TwIST) is a relatively effective method. In order to further improve the noise immunity of iterative threshold algorithm, an improved two-step iterative shrinkage/thresholding algorithm based on 3D modelling is presented in this paper. We transform the sensitivity matrix by singular value decomposition (SVD), and then remove the small singular values which are easy to cause the instability of the solution. Both simulation and experimental results demonstrate that the proposed method can improve the imaging quality obviously.

Compressive sensing (CS) is a technique that enables the recovery of sparse signals using fewer measurements than traditional sampling methods. To address the computational challenges of CS reconstruction, our objective is to develop an interpretable and concise neural network model for reconstructing natural images using CS. We achieve this by mapping one step of the iterative shrinkage thresholding algorithm (ISTA) to a deep network block, representing one iteration of ISTA. To enhance learning ability and incorporate structural diversity, we integrate aggregated residual transformations (ResNeXt) and squeeze‐and‐excitation mechanisms into the ISTA block. This block serves as a deep equilibrium layer connected to a semi‐tensor product network for convenient sampling and providing an initial reconstruction. The resulting model, called MsDC‐DEQ‐Net, exhibits competitive performance compared to state‐of‐the‐art network‐based methods. It significantly reduces storage requirements compared to deep unrolling methods, using only one iteration block instead of multiple iterations. Unlike deep unrolling models, MsDC‐DEQ‐Net can be iteratively used, gradually improving reconstruction accuracy while considering computation tradeoffs. Additionally, the model benefits from multiscale dilated convolutions, further enhancing performance.

Structural damage identification is essentially an inverse problem. Ill-posedness is a common obstacle encountered in solving such an inverse problem, especially in the context of a sensitivity-based model updating for damage identification. Tikhonov regularization, also termed as ℓ2-norm regularization, is a common approach to handle the ill-posedness problem and yields an acceptable and smooth solution. Tikhonov regularization enjoys a more popular application as its explicit solution, computational efficiency, and convenience for implementation. However, as the ℓ2-norm term promotes smoothness, the solution is sometimes over smoothed, especially in the case that the sensor number is limited. On the other side, the solution of the inverse problem bears sparse properties because typically, only a small number of components of the structure are damaged in comparison with the whole structure. In this regard, this paper proposes an alternative way, sparse regularization, or specifically ℓ1-norm regularization, to handle the ill-posedness problem in response sensitivity-based damage identification. The motivation and implementation of sparse regularization are firstly introduced, and the differences with Tikhonov regularization are highlighted. Reweighting sparse regularization is adopted to enhance the sparsity in the solution. Simulation studies on a planar frame and a simply supported overhanging beam show that the sparse regularization exhibits certain superiority over Tikhonov regularization as less false-positive errors exist in damage identification results. The experimental result of the overhanging beam further demonstrates the effectiveness and superiorities of the sparse regularization in response sensitivity-based damage identification. Copyright © 2015 John Wiley & Sons, Ltd.

The fast compressed sensing (CS) reconstruction for natural image processing aims to infer the original pixel vector from randomized measurements as correctly as possible. Current first-order proximal mainstream schemes, including alternating direction methods of multipliers (ADMM) and the iterative shrinkage thresholding algorithm (ISTA), have advantages in convergence and stability at the expense of computing speed and complex tuning strategy. This paper proposes the deep iterative shrinkage network for real-time applications, where network layers replace the iterative optimization steps. The hyperparameters associated with shrinkage thresholds and step sizes are adjusted by standard deep learning updates. Numerical results suggest that the proposed method has flexibility and effectiveness in image reconstruction with different compression ratios while preserving the computational simplicity of end-to-end methods.

Discretized tensor-based model of total focusing method: A sparse regularization approach for enhanced ultrasonic phased array imaging

First-order methods are used widely for large scale optimization problems in signal/image processing and machine learning, because their computation depends mildly on the problem dimension. Nesterov's fast gradient method (FGM) has the optimal convergence rate among first-order methods for smooth convex minimization; its extension to non-smooth case, the fast iterative shrinkage-thresholding algorithm (FISTA), also satisfies the optimal rate; thus both algorithms have gained great interest. We recently introduced a new optimized gradient method (OGM) (for smooth convex functions) having a theoretical convergence speed that is 2× faster than Nesterov's FGM. This paper further discusses the convergence analysis of OGM and explores its fast convergence on an image restoration problem using a smoothed total variation (TV) regularizer. In addition, we empirically investigate the extension of OGM to nonsmooth convex minimization for image restoration with l 1 -sparsity regularization.

Elimination of accelerometer mass loading effects in sparse identification of impact forces

Hyperspectral images have high spectral resolution but low spatial resolution, which results in a large number of mixed pixels. As an economical and effective means to improve image quality, the fusion of hyperspectral and multispectral data from different sensors can achieve the reconstruction of super-resolution images. As a representative of fusion method, coupled non-negative matrix factorization is an ill-posed problem, in which the number of endmembers was set to be no less than the groundtruth without requiring an accurate value. However, this often results in spectral shadows and spatial information redundancy, especially when the observed images are contaminated by noise. To address these problems above, this article incorporates sparse and low-rank regularization to reformulate a bi-convex fusion problem for the removal of shadows and noise, in which the logarithmic sum function is employed to suppress the small singular components of endmember and abundance matrices. Then, an efficient solver is designed to obtain the closed-form solutions via matrix-vector operators, in which the alternating direction method of multipliers is utilized to split the variables using equality constraints. The experimental results of real datasets demonstrate that the proposed fusion method can effectively enhance the quality of reconstructed super-resolution images especially in high-noise environments, which also verifies the validation of incorporated regularization.

Uncovering brain activity from magnetoencephalography (MEG) data requires solving an ill-posed inverse problem, greatly confounded by noise, interference, and correlated sources. Sparse reconstruction algorithms, such as Champagne, show great promise in that they provide focal brain activations robust to these confounds. In this paper, we address the technical considerations of statistically thresholding brain images obtained from sparse reconstruction algorithms. The source power distribution of sparse algorithms makes this class of algorithms ill-suited to “conventional” techniques. We propose two non-parametric resampling methods hypothesized to be compatible with sparse algorithms. The first adapts the maximal statistic procedure to sparse reconstruction results and the second departs from the maximal statistic, putting forth a less stringent procedure that protects against spurious peaks. Simulated MEG data and three real data sets are utilized to demonstrate the efficacy of the proposed methods. Two sparse algorithms, Champagne and generalized minimum-current estimation (G-MCE), are compared to two non-sparse algorithms, a variant of minimum-norm estimation, sLORETA, and an adaptive beamformer. The results, in general, demonstrate that the already sparse images obtained from Champagne and G-MCE are further thresholded by both proposed statistical thresholding procedures. While non-sparse algorithms are thresholded by the maximal statistic procedure, they are not made sparse. The work presented here is one of the first attempts to address the problem of statistically thresholding sparse reconstructions, and aims to improve upon this already advantageous and powerful class of algorithm.

Synthetic aperture radar tomography (TomoSAR) has been widely employed in 3-D urban mapping. However, state-of-the-art super-resolving TomoSAR algorithms are computationally expensive, because conventional numerical solvers need to solve the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$l_{2^{-}}l_{1}$</tex> mix norm minimization. This paper proposes a computationally efficient super-resolving To-moSAR inversion algorithm based on deep learning. We studied the potential of deep learning to mimic a conventional <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$l_{2}-l_{1}$</tex> mix norm solver, i.e. iterative shrinkage thresholding algorithm (ISTA), and proposed several improvements of the complex-valued learned ISTA for TomoSAR inversion. Investigation on the super-resolution ability and estimator efficiency of the proposed algorithm shows that the proposed algorithm approaches the Cramer Rao lower bound (CRLB) with a computational efficiency more than 100 times better than the conventional solver.

We present two model-based neural network architectures purposed for sporadic user detection and channel estimation in massive machine-type communications. In the scenario under consideration, a base station assigns the users a set of pilot sequences that is linearly dependent, but because user activity is sporadic the detection/estimation problem is amenable to sparse recovery algorithms. Further, we consider a millimeter-wave wireless channel, so that the channel vectors are sparse in a known dictionary. We apply the deep unfolding framework to design custom neural network layers by unrolling two iterative optimization algorithms: (1) linearized alternating direction method of multipliers, which we apply to a constrained convex problem, and (2) vector approximate message passing featuring a novel denoiser based on the iterative shrinkage thresholding algorithm. The networks thus inherit domain knowledge as encapsulated by the signal model, and suitable operations as informed by the algorithms—in the same spirit as convolutional networks that exploit structure inherent in images and audio, except grounded in optimization and statistics. The networks, trained on synthetic data generated from the block-fading millimeter-wave multiple access channel model, offer improved complexity and accuracy relative to their iterative counterparts, and are potentially a boon to cell-free MIMO systems.