A novel dictionary learning method for sparse representation with nonconvex regularizations

MRI reconstruction via enhanced group sparsity and nonconvex regularization

In order to analyse synthetic aperture radar (SAR) images of the sea surface, ship wake detection is essential for extracting information on the wake generating vessels. One possibility is to assume a linear model for wakes, in which case detection approaches are based on transforms such as Radon and Hough. These express the bright (dark) lines as peak (trough) points in the transform domain. In this paper, ship wake detection is posed as an inverse problem, which the associated cost function including a sparsity enforcing penalty, i.e. the generalized minimax concave (GMC) function. Despite being a non-convex regularizer, the GMC penalty enforces the overall cost function to be convex. The proposed solution is based on a Bayesian formulation, whereby the point estimates are recovered using maximum a posteriori (MAP) estimation. To quantify the performance of the proposed method, various types of SAR images are used, corresponding to TerraSAR-X, COSMO-SkyMed, Sentinel-1, and ALOS2. The performance of various priors in solving the proposed inverse problem is first studied by investigating the GMC along with the L1, Lp, nuclear and total variation (TV) norms. We show that the GMC achieves the best results and we subsequently study the merits of the corresponding method in comparison to two state-of-the-art approaches for ship wake detection. The results show that our proposed technique offers the best performance by achieving 80% success rate.

Sparse reconstruction for blade tip timing signal using generalized minimax-concave penalty

The goal of dictionary learning algorithms is to learn a set of atoms called dictionary from a set of training data such that each training data can be represented sparsely by the dictionary. Most dictionary learning algorithms use two alternating steps, sparse coding and dictionary update, to solve this problem. In this paper, we propose a new online dictionary learning algorithm with a novel dictionary update step. In the new update method, only the atoms involved in the sparse representation of new training data are adaptively updated. The adaptive update also includes using the previous training data that have common atoms with the new training data in their sparse representation. The proposed algorithm reduces the computational complexity by reducing the number of atoms updating at each iteration and the number of training data contributing in the dictionary update. In the proposed algorithm, different weights are given to training data, so each has the proper level of influence in the dictionary update. The simulation results show this improves the performance of the algorithm, in convergence speed and representation error. We also show that our algorithm has the ability of full recovery of dictionary with only one trial over training data and does not need any dictionary pruning to remove the unused atoms of the dictionary. We compare our proposed algorithm with well-known online and batch based methods using synthetic and autoregressive data.

Feature selection aims to select a subset of features from high-dimensional data according to a predefined selecting criterion. Sparse learning has been proven to be a powerful technique in feature selection. Sparse regularizer, as a key component of sparse learning, has been studied for several years. Although convex regularizers have been used in many works, there are some cases where nonconvex regularizers outperform convex regularizers. To make the process of selecting relevant features more effective, we propose a novel nonconvex sparse metric on matrices as the sparsity regularization in this paper. The new nonconvex regularizer could be written as the difference of the $\ell _{2,1}$ norm and the Frobenius ( $\ell _{2,2}$ ) norm, which is named the $\ell _{2,1-2}$ . To find the solution of the resulting nonconvex formula, we design an iterative algorithm in the framework of ConCave-Convex Procedure (CCCP) and prove its strong global convergence. An adopted alternating direction method of multipliers is embedded to solve the sequence of convex subproblems in CCCP efficiently. Using the scaled cluster indictors of data points as pseudolabels, we also apply $\ell _{2,1-2}$ to the unsupervised case. To the best of our knowledge, it is the first work considering nonconvex regularization for matrices in the unsupervised learning scenario. Numerical experiments are performed on real-world data sets to demonstrate the effectiveness of the proposed method.

Dictionary learning is still a challenging problem in signal and image processing. In this paper, we propose an efficient block-structured incoherent dictionary learning algorithm for sparse representations of image signals. The constrained minimization of dictionary learning is achieved by iteratively alternating between sparse coding and dictionary update. Without relying on any prior knowledge of the group structure for the input data, we develop a two-stage clustering method that identifies the underlying block structure of the dictionary under certain restricted constraints. The two-stage clustering method mainly consists of affinity propagation and agglomerative hierarchical clustering. To meet the conditions of both the upper bound and the lower bound of the mutual coherence of dictionary atoms, we introduce a regularization term for the objective function to adjust the block coherence of the overcomplete dictionary. The experiments on synthetic data and real images demonstrate that the proposed dictionary learning algorithm has lower representation error, higher visual quality and better reconstructed results than most of the state-of-the-art methods.

Dictionary learning aims to find a dictionary that can sparsely represent the training data. Methods in the literature typically solve the problem by alternating between two stages: sparse coding and dictionary update. In this paper, we propose a novel dictionary learning algorithm using rank-one projection (ROP). The key contribution is that we cast dictionary learning as an optimization with respect to a single variable which is a set of rank one matrices. The resulting algorithm is hence single-stage. An alternating direction method of multipliers (ADMM) is derived to solve the optimization problem and a lower bound of penalty parameter is computed to guarantee a global convergence despite non-convexity of the optimization formulation. From practical point of view, ROP reduces the number of tuning parameters required in benchmark algorithms. Numerical tests demonstrate that ROP outperforms benchmark algorithms for both synthetic and real data.

Both interclass variances and intraclass similarities are crucial for improving the classification performance of discriminative dictionary learning (DDL) algorithms. However, existing DDL methods often ignore the combination between the interclass and intraclass properties of dictionary atoms and coding coefficients. To address this problem, in this paper, we propose a discriminative Fisher embedding dictionary learning (DFEDL) algorithm that simultaneously establishes Fisher embedding models on learned atoms and coefficients. Specifically, we first construct a discriminative Fisher atom embedding model by exploring the Fisher criterion of the atoms, which encourages the atoms of the same class to reconstruct the corresponding training samples as much as possible. At the same time, a discriminative Fisher coefficient embedding model is formulated by imposing the Fisher criterion on the profiles (row vectors of the coding coefficient matrix) and coding coefficients, which forces the coding coefficient matrix to become a block-diagonal matrix. Since the profiles can indicate which training samples are represented by the corresponding atoms, the proposed two discriminative Fisher embedding models can alternatively and interactively promote the discriminative capabilities of the learned dictionary and coding coefficients. The extensive experimental results demonstrate that the proposed DFEDL algorithm achieves superior performance in comparison with some state-of-the-art dictionary learning algorithms on both hand-crafted and deep learning-based features.

Image deblurring based on sparse regularization has garnered significant attention, but there are still certain limitations that need to be addressed. For instance, convex sparse regularization tends to exhibit biased estimation, which can adversely impact the deblurring performance, while non-convex sparse regularization poses challenges in terms of solving techniques. Furthermore, the performance of the traditional iterative algorithm also needs to be improved. In this paper, we propose an image deblurring method based on convex non-convex (CNC) sparse regularization and a plug-and-play (PnP) algorithm. The utilization of CNC sparse regularization not only mitigates estimation bias but also guarantees the overall convexity of the image deblurring model. The PnP algorithm is an advanced learning-based optimization algorithm that surpasses traditional optimization algorithms in terms of efficiency and performance by utilizing the state-of-the-art denoiser to replace the proximal operator. Numerical experiments verify the performance of our proposed algorithm in image deblurring.

This study aimed to explore the effect of ultrasound imaging in the diagnosis and evaluation of acupoint application in the treatment of idiopathic edema. In this study, an ultrasound imaging diagnosis based on the segmentation dictionary learning (S-DL) algorithm was proposed. In addition, the autoencoding algorithm (ASE) was compared with the traditional dictionary learning (DL) algorithm. The treatment effect, associated quantitative integral, and quality of life score of patients in two groups were compared. The results showed that the peak signal-to-noise ratio (PSNR), root mean square error (RMSE), and structural similarity (SSIM) of the S-DL algorithm were 32.45 dB, 0.654, and 0.0012, respectively, which were quite different compared to the ASE and DL algorithms, showing statistical significance (P < 0.05). As the noise level increased, the image reconstruction quality gradually decreased, but the S-DL algorithm obtained better image quality than the DL and ASE algorithms, and the difference was statistically great (P < 0.05). There was no significant difference in the average age and average course of the disease between the experimental group and the control group (P > 0.05). The overall treatment effect of patients in the experimental group was 96.77%, while that in the control group was 45.16%, and the difference between the two was statistically significant (P < 0.05). After treatment, the semiquantitative scores of fatigue, dizziness, palpitation, frequent urination, urgent urination, and dyspepsia of the experimental group were 1.18, 0.39, 0.72, 1.21, and 0.87, respectively, which were much lower than those of the control group statistically (P < 0.05). The score of quality of life of the experimental group of patients after treatment was 91.27 points, and that of the control group was 82.35 points, showing statistically great difference (P < 0.05). It showed that the algorithm performance of S-DL was relatively good, and the acupoint application therapy was better than traditional western medicine in the treatment of idiopathic edema, which reduces the discomfort of patients to a certain extent and improves the quality of life of patients.

Group-based sparse representation for image compressive sensing reconstruction with non-convex regularization

Most of the research on dictionary learning has focused on developing algorithms under the assumption that data is available at a centralized location. But often the data is not available at a centralized location due to practical constraints like data aggregation costs, privacy concerns, etc. Using centralized dictionary learning algorithms may not be the optimal choice in such settings. This motivates the design of dictionary learning algorithms that consider distributed nature of data as one of the problem variables. Just like centralized settings, distributed dictionary learning problem can be posed in more than one way depending on the problem setup. Most notable distinguishing features are the online versus batch nature of data and the representative versus discriminative nature of the dictionaries. In this paper, several distributed dictionary learning algorithms that are designed to tackle different problem setups are reviewed. One of these algorithms is cloud K-SVD, which solves the dictionary learning problem for batch data in distributed settings. One distinguishing feature of cloud K-SVD is that it has been shown to converge to its centralized counterpart, namely, the K-SVD solution. On the other hand, no such guarantees are provided for other distributed dictionary learning algorithms. Convergence of cloud K-SVD to the centralized K-SVD solution means problems that are solvable by K-SVD in centralized settings can now be solved in distributed settings with similar performance. Finally, cloud K-SVD is used as an example to show the advantages that are attainable by deploying distributed dictionary algorithms for real world distributed datasets.

Retinal blood vessels are considered as being interference on the retinal images for the task of detecting significant features of the most frequent eye diseases. If these blood vessel structures could be suppressed, it might lead to a more accurate segmentation of retinal lesions as well as a better extraction of textural features to be used for pathology detection. This work proposes, as a novelty, the use of sparse representations and dictionary learning techniques for retinal vessel inpainting. The dictionary learning algorithms used in this paper were the Recursive Least Square Dictionary Learning (RLS-DL), and Online Dictionary Learning (ODL). We tested the performance of the algorithm for grayscale and RGB images from the DRIVE public database, employing different neighbourhoods and sparseness factors. An average recovery error smaller than 0.022 was achieved. The results suggest that the use of sparse representations and dictionaries learned by RLS-DLA performs very well for inpainting of retinal blood vessels.

In this paper we show that the computational complexity of the iterative thresholding and K-residual-means (ITKrM) algorithm for dictionary learning can be significantly reduced by using dimensionality-reduction techniques based on the Johnson–Lindenstrauss lemma. The dimensionality reduction is efficiently carried out with the fast Fourier transform. We introduce the iterative compressed-thresholding and K-means (IcTKM) algorithm for fast dictionary learning and study its convergence properties. We show that IcTKM can locally recover an incoherent, overcomplete generating dictionary of $$K$$ atoms from training signals of sparsity level $$S$$ with high probability. Fast dictionary learning is achieved by embedding the training data and the dictionary into $$m< d$$ dimensions, and recovery is shown to be locally stable with an embedding dimension which scales as low as $$m = O(S\log ^4S\log ^3 K)$$. The compression effectively shatters the data dimension bottleneck in the computational cost of ITKrM, reducing it by a factor O(m/d). Our theoretical results are complemented with numerical simulations which demonstrate that IcTKM is a powerful, low-cost algorithm for learning dictionaries from high-dimensional data sets.

Dictionary learning algorithms have received widespread acceptance when it comes to data analysis and signal representation problems. However, most existing algorithms assume isotropic noise. This is a restrictive assumption as the noise across samples may be nonuniform in a number of real world application. The aim of this article is to propose a sequential dictionary learning algorithm for measurement data vectors affected by nonuniform noise. As most of the existing algorithms, the proposed method is a two stages iterative algorithm where the effect of the noise nature appears in the sparse coding and dictionary update stages. The cost function used to derive the dictionary update stage is developed based on the link between singular value decomposition and rank one matrix approximation. Results on the simulated dictionary recovery problem and an application on face recognition dataset shows that the proposed algorithm outperforms current state of the art dictionary learning methods.