Discriminative low-rank representation with Schatten-p norm for image recognition

Subspace clustering based on latent low rank representation with Frobenius norm minimization

Recently, low-rank representation (LRR) methods have been widely applied for hyperspectral anomaly detection, due to their potentials in separating the backgrounds and anomalies. However, existing LRR models generally convert 3-D hyperspectral images (HSIs) into 2-D matrices, inevitably leading to the destruction of intrinsic 3-D structure properties in HSIs. To this end, we propose a novel tensor low-rank and sparse representation (TLRSR) method for hyperspectral anomaly detection. A 3-D TLR model is expanded to separate the LR background part represented by a tensorial background dictionary and corresponding coefficients. This representation characterizes the multiple subspace property of the complex LR background. Based on the weighted tensor nuclear norm and the LF,1 sparse norm, a dictionary is designed to make its atoms more relevant to the background. Moreover, a principal component analysis (PCA) method can be assigned as one preprocessing step to exact a subset of HSI bands, retaining enough the HSI object information and reducing computational time of the postprocessing tensorial operations. The proposed model is efficiently solved by the well-designed alternating direction method of multipliers (ADMMs). A comparison with the existing algorithms via experiments establishes the competitiveness of the proposed method with the state-of-the-art competitors in the hyperspectral anomaly detection task.

Improving generalization of double low-rank representation using Schatten-p norm

Low-rank representation (LRR) efficiently performs the subspace segmentation and feature extraction from corrupted data. However, there are three disadvantages in existing LRR techniques. First, the inference algorithm of LRR (as a generative model) is computationally expensive. Second, LRR ignores the discriminative information for image classification. Third, although the robust representation is implemented by recovering the low-rank components and the sparse noises, it has been limited due to the constrained assumption that noises is sparse. To solve these problems, and inspired by Denoising Autoencoders (DAE) and Contractive Autoencoders (CAE), this paper proposes a discriminative low-rank representations framework (DLRR) for image classification. We directly learn a discriminative projection dictionary that results in fast inference. Simultaneously, DLRR can obtain a robust representation from any corrupted input. Our implementation of DLRR achieves state-of-the-art results on artificial dataset and dataset of Olivetti Face Patches.

Rapid advancement and active research in computer vision applications and 3D imaging have made a high demand for efficient depth image estimation techniques. The depth image acquisition, however, is typically challenged due to poor hardware performance and high computation cost. To tackle such limitations, this paper proposes an efficient approach for depth image reconstruction using low rank (LR) and total variation (TV) regularizations. The key idea is LR incorporates non-local depth information and TV takes into account the local spatial consistency. The proposed model reformulates the task of depth image estimate as a joint LR-TV regularized minimization problem, in which LR is used to approximate the low-dimensional structure of the depth image, and TV is employed to promote the depth sparsity in the gradient domain. Furthermore, this paper introduces an algorithm based on alternating direction method of multipliers (ADMM) for solving the minimization problem, whose solution provides an estimate of the depth map from incomplete pixels. Experimental results are conducted and the results show that the proposed approach is very effective at estimating high-quality depth images and is robust to different types of data missing models.

Accurate depth reconstruction is vital for numerous applications including autonomous vehicles, virtual reality, and robot perception. However, the depth imaging is challenging because of limited hardware operations, resource-constrained limitations, and incomplete data measurements. To address such shortcomings, this paper introduces an imaging model for efficient depth image estimation from incomplete depth pixels using non-local low-rank (NLLR) and total variation (TV) representations. The motivation is that NLLR is used to model global similar structure among depth patches, and the TV is incorporated to capture the correlations among local depth pixels. We reformulate the problem of depth reconstruction as a regularized least squares minimization problem with the non-local LR and TV regularizers. Furthermore, this paper proposes an iterative algorithm using the alternating direction method of multipliers (ADMM) to solve the optimization model, yielding an estimate of the depth map from far reduced data points. Experimental results on benchmark datasets validate the efficiency of the proposed approach.

Recently, compressed sensing (CS) has been an effective data compression technique for telemonitoring of multichannel electroencephalogram (EEG) signals through wireless body-area networks. Most of the existing multichannel EEG CS methods ignore the noise or only consider the Gaussian noise. However, there are also some other types of noise, such as heavy-tailed impulsive noise. In this paper, to overcome the above mentioned problems, we propose a novel multichannel EEG CS method based on sparse and low rank representation in the presence of mixed noise (SLRMN), which can take both Gaussian noise and impulsive noise into consideration. Moreover, we develop the alternative direction method of multipliers (ADMM) to solve the proposed SLRMN. The experimental results demonstrate the advantage of the proposed SLRMN over state-of-the-art multichannel EEG CS methods in the presence of mixed noise.

Attention reweighted sparse subspace clustering

We analyze the convergence rate of the alternating direction method of multipliers (ADMM) for minimizing the sum of two or more nonsmooth convex separable functions subject to linear constraints. Previous analysis of the ADMM typically assumes that the objective function is the sum of only two convex functions defined on two separable blocks of variables even though the algorithm works well in numerical experiments for three or more blocks. Moreover, there has been no rate of convergence analysis for the ADMM without strong convexity in the objective function. In this paper we establish the global R-linear convergence of the ADMM for minimizing the sum of any number of convex separable functions, assuming that a certain error bound condition holds true and the dual stepsize is sufficiently small. Such an error bound condition is satisfied for example when the feasible set is a compact polyhedron and the objective function consists of a smooth strictly convex function composed with a linear mapping, and a nonsmooth $$\ell _1$$l1 regularizer. This result implies the linear convergence of the ADMM for contemporary applications such as LASSO without assuming strong convexity of the objective function.

Recently, we have proposed combining the alternating direction method of multipliers (ADMM) with a Gaussian back substitution procedure for solving the convex minimization model with linear constraints and a general separable objective function, i.e., the objective function is the sum of many functions without coupled variables. In this paper, we further study this topic and show that the decomposed subproblems in the ADMM procedure can be substantially alleviated by linearizing the involved quadratic terms arising from the augmented Lagrangian penalty. When the resolvent operators of the separable functions in the objective have closed-form representations, embedding the linearization into the ADMM subproblems becomes necessary to yield easy subproblems with closed-form solutions. We thus show theoretically that the blend of ADMM, Gaussian back substitution and linearization works effectively for the separable convex minimization model under consideration.

Recently, the alternating direction method of multipliers (ADMM) has been used effectively to solve the multi-agent unconstrained optimization problems, where the objective function is the sum of privately known local objective functions of agents. In this paper, first, with the help of the edge-node incidence matrix, an unconstrained optimization problem is transformed into an equivalent optimization problem with only equality constraint and, thus, can be dealt with the ADMM conveniently. Second, a novel distributed inexact consensus ADMM is proposed to enable the agents to reach consensus on the optimal solution of the optimization problem. At the same time, the analysis of the linear convergence of the proposed algorithm is also provided under some mild conditions. Finally, some simulation results are presented to demonstrate the better effectiveness of the proposed algorithm than the standard consensus-based ADMM algorithm.

<p>Total variation (TV) regularization is a powerful tool in image denoising, but it often exhibits limited performance in preserving edges. In contrast, non-convex TV regularization can more effectively preserve edges and contours, albeit posing challenges when solving. Recently, the convex non-convex (CNC) strategy has emerged as a potent approach that allows incorporating non-convex TV regularization terms while maintaining the overall convexity of the objective function. Its superior performance has been validated through various numerical experiments; however, theoretical analysis remains lacking. In this paper, we provided theoretical analysis of the performance of the CNC-TV denoising model. By utilizing the oracle inequality, we derived an improved upper bound on its performance compared to TV regularization. In addition, we devised an alternating direction method of multipliers (ADMM) algorithm to address the proposed model and verified its convergence properties. Our proposed model has been validated through numerical experiments in 1D and 2D denoising, demonstrating its exceptional performance.</p>

We consider the problem of restoring images corrupted by Poisson noise. Under the framework of maximum a posteriori estimator, the problem can be converted into a minimization problem where the objective function is composed of a Kullback-Leibler (KL)-divergence term for the Poisson noise and a total variation (TV) regularization term. Due to the logarithm function in the KL-divergence term, the non-differentiability of TV term and the positivity constraint on the images, it is not easy to design stable and efficiency algorithm for the problem. Recently, many researchers proposed to solve the problem by alternating direction method of multipliers (ADMM). Since the approach introduces some auxiliary variables and requires the solution of some linear systems, the iterative procedure can be complicated. Here we formulate the problem as two new constrained minimax problems and solve them by Chambolle-Pocks first order primal-dual approach. The convergence of our approach is guaranteed by their theory. Comparing with ADMM approaches, our approach requires about half of the auxiliary variables and is matrix-inversion free. Numerical results show that our proposed algorithms are efficient and outperform the ADMM approach.

Multiview subspace clustering is one of the most widely used methods for exploiting the internal structures of multiview data. Most previous studies have performed the task of learning multiview representations by individually constructing an affinity matrix for each view without simultaneously exploiting the intrinsic characteristics of multiview data. In this article, we propose a multiview low-rank representation (MLRR) method to comprehensively discover the correlation of multiview data for multiview subspace clustering. MLRR considers symmetric low-rank representations (LRRs) to be an approximately linear spatial transformation under the new base, that is, the multiview data themselves, to fully exploit the angular information of the principal directions of LRRs, which is adopted to construct an affinity matrix for multiview subspace clustering, under a symmetric condition. MLRR takes full advantage of LRR techniques and a diversity regularization term to exploit the diversity and consistency of multiple views, respectively, and this method simultaneously imposes a symmetry constraint on LRRs. Hence, the angular information of the principal directions of rows is consistent with that of columns in symmetric LRRs. The MLRR model can be efficiently calculated by solving a convex optimization problem. Moreover, we present an intuitive fusion strategy for symmetric LRRs from the perspective of spectral clustering to obtain a compact representation, which can be shared by multiple views and comprehensively represents the intrinsic features of multiview data. Finally, the experimental results based on benchmark datasets demonstrate the effectiveness and robustness of MLRR compared with several state-of-the-art multiview subspace clustering algorithms.

Low-rank Parameterization of Volumetric Domains for Isogeometric Analysis