Logarithmic Schatten- p Norm Minimization for Tensorial Multi-View Subspace Clustering.

Multi-view subspace clustering, which aims to partition a dataset into its relevant subspaces based on their multi-view features, has been widely applied to identify various characteristics of datasets. The typical model of multi-view subspace clustering in literature often makes use of the nuclear norm to seek the underlying low-rank representation. However, due to the sum property of the singular values defined by tensor nuclear norm, the existing multi-view subspace clustering does not well handle the noise and the illumination variations embedded in multi-view data. To address and improve the robustness and clustering performance, we propose a new nonconvex multi-view subspace clustering model via tensor minimax concave penalty (MCP) approximation associated with rank minimization (NMSC-MCP), which can simultaneously construct the low-rank representation tensor and affinity matrix in a unified framework. Specifically, the nonconvex MCP approximation rank function is adopted to as a tighter tensor rank approximation to discriminate the dimension of features so that better accuracy can be achieved. In addition, we also address the local structure by including both hyper-Laplacian regularization and auto-weighting scheme into the objective function to promote the clustering performance. A corresponding iterative algorithm is then developed to solve the proposed model and the constructed iterative sequence generated by the proposed algorithm is shown to converge to the desirable KKT critical point. Extensive experiments on benchmark datasets have demonstrate the highly desirable effectiveness of our proposed method.

Nonconvex low-rank tensor approximation with graph and consistent regularizations for multi-view subspace learning

In this paper, we focus on the multi-view subspace clustering problem under the framework of third order tensor. Recently, tensor nuclear norm (TNN) has been widely used in the multi-view subspace clustering problem. It is known that TNN is a convex surrogate of the tensor rank. However, TNN is linearly proportional to the sum of the singular values. Thus solving the TNN minimization problem will over-penalize the large singular values and lead to a sub-optimal solution. To address this issue, in this paper, a novel tighter tensor log-determinant (TLD) regularizer is proposed to better approximate the tensor rank. Although the proposed method is non-convex, a closed-form solution has been deduced via solving the Euler-Lagrange equation. A corresponding algorithm associated with augmented Lagrangian multipliers is established and the constructed convergent sequence to the Karush-Kuhn-Tucker (KKT) critical point solution is mathematically validated in detail. Extensive experiments indicate that the proposed model achieves significant improvements compared to the state-of-the-art convex subspace clustering models.

Low-rank tensor completion (LRTC) has gained significant attention due to its powerful capability of recovering missing entries. However, it has to repeatedly calculate the time-consuming singular value decomposition (SVD). To address this drawback, we, based on the tensor-tensor product (t-product), propose a new LRTC method-the unified tensor factorization (UTF)-for 3-way tensor completion. We first integrate the tensor factorization (TF) and the tensor nuclear norm (TNN) regularization into a framework that inherits the benefits of both TF and TNN: fast calculation and convex optimization. The conditions under which TF and TNN are equivalent are analyzed. Then, UTF for tensor completion is presented and an efficient iterative updated algorithm based on the alternate direction method of multipliers (ADMM) is used for our UTF optimization, and the solution of the proposed alternate minimization algorithm is also proven to be able to converge to a Karush–Kuhn–Tucker (KKT) point. Finally, numerical experiments on synthetic data completion and image/video inpainting tasks demonstrate the effectiveness of our method over other state-of-the-art tensor completion methods.

Two new low rank tensor completion methods based on sum nuclear norm

Recently, a growing number of researchers have focused on multi-view subspace clustering (MSC) due to its potential for integrating heterogeneous data. However, current MSC methods remain challenged by limited robustness and insufficient exploitation of cross-view high-order latent information for clustering advancement. To address these challenges, we develop a novel MSC framework termed TMSC-TNNBDR, a tensorized MSC framework that leverages t-SVD based tensor nuclear norm (TNN) regularization and block diagonal representation (BDR) learning to unify view consistency and structural sparsity. Specifically, each subspace representation matrix is constrained by a block diagonal regularizer to enforce cluster structure, while all matrices are aggregated into a tensor to capture high-order interactions. To efficiently optimize the model, we developed an optimization algorithm based on the inexact augmented Lagrange multiplier (ALM). The TMSC-TNNBDR exhibits both optimized block-diagonal structure and low-rank properties, thereby enabling enhanced mining of latent higher-order inter-view correlations while demonstrating greater resilience to noise. To investigate the capability of TMSC-TNNBDR, we conducted several experiments on certain datasets. Benchmarking on circumscribed datasets demonstrates our method’s superior clustering performance over comparative algorithms while maintaining competitive computational overhead.

Low-Rank Tensor Recovery (LRTR), the higher order generalization of Low-Rank Matrix Recovery (LRMR), is especially suitable for analyzing multi-linear data with gross corruptions, outliers and missing values, and it attracts broad attention in the fields of computer vision, machine learning and data mining. This paper considers a generalized model of LRTR and attempts to recover simultaneously the low-rank, the sparse, and the small disturbance components from partial entries of a given data tensor. Specifically, we first describe generalized LRTR as a tensor nuclear norm optimization problem that minimizes a weighted combination of the tensor nuclear norm, the l1-norm and the Frobenius norm under linear constraints. Then, the technique of Alternating Direction Method of Multipliers (ADMM) is employed to solve the proposed minimization problem. Next, we discuss the weak convergence of the proposed iterative algorithm. Finally, experimental results on synthetic and real-world datasets validate the efficiency and effectiveness of the proposed method.

An increasing amount of attention have been attracted in multi-view subspace clustering, whose impressive performance can be achieved by means of the self-expressive property, under the assumption of linear relations between multi-view data samples. However, most of them fail to recover the nonlinear relations between multi-view data for deeper study, and additionally they are incapable of discovering the comprehensiveness and higher-order correlations among multiple views. To deal with these challenges, we propose a novel model termed Coupled Learning for Kernel Representation and Graph Tensor (CLKT) in Multi-view Subspace Clustering, where both nonlinear relations and higher-order correlations among multiple affinity graphs are jointly learned in a unified framework. Optimal solutions of the proposed method can be obtained by an alternative minimizing optimization strategy. Extensive experiments on six real-world datasets indicate the superiority of CLKT compared with the state-of-the-art methods.KeywordsMulti-view clusteringKernel learningLow-rank tensor representation

Tensorial Multi-view Clustering (TMC), a prominent approach in multi-view clustering, leverages low-rank tensor learning to capture high-order correlation among views for consistent clustering structure identification. Despite its promising performance, the TMC algorithms face three key challenges: 1). The severe computational burden makes it difficult for TMC methods to handle large-scale datasets. 2). Estimation bias problem caused by the convex surrogate of the tensor rank. 3). Lack of explicit balance of consistency and complementarity. Being aware of these, we propose a basic framework Efficient and Scalable Tensorial Multi-View Subspace Clustering (ESTMC) for large-scale multi-view clustering. ESTMC integrates anchor representation learning and non-convex function-based low-rank tensor learning with a Generalized Non-convex Tensor Rank (GNTR) into a unified objective function, which enhances the efficiency of the existing subspace-based TMC framework. Furthermore, a novel model ESTMC-C with the proposed Enhanced Tensor Rank (ETR), Consistent Geometric Regularization (CGR), and Tensorial Exclusive Regularization (TER) is extended to balance the learning of consistency and complementarity among views, delivering divisible representations for the clustering task. Efficient iterative optimization algorithms are designed to solve the proposed ESTMC and ESTMC-C, which enjoy time-economical complexity and exhibit theoretical convergence. Extensive experimental results on various datasets demonstrate the superiority of the proposed algorithms as compared to state-of-the-art methods.

This paper addresses the multi-view subspace clustering problem and proposes the self-paced enhanced low-rank tensor kernelized multi-view subspace clustering (SETKMC) method, which is based on two motivations: (1) singular values of the representations and multiple instances should be treated differently. The reasons are that larger singular values of the representations usually quantify the major information and should be less penalized; samples with different degrees of noise may have various reliability for clustering. (2) many existing methods may cause the degraded performance when multi-view features reside in different nonlinear subspaces. This is because they usually assumed that multiple features lie within the union of several linear subspaces. SETKMC integrates the nonconvex tensor norm, self-paced learning, and kernel trick into a unified model for multi-view subspace clustering. The nonconvex tensor norm imposes different weights on different singular values. The self-paced learning gradually involves instances from more reliable to less reliable ones while the kernel trick aims to handle the multi-view data in nonlinear subspaces. One iterative algorithm is proposed based on the alternating direction method of multipliers. Extensive results on seven real-world datasets show the effectiveness of the proposed SETKMC compared to fifteen state-of-the-art multi-view clustering methods.

<p style='text-indent:20px;'>In 2011, Kilmer and Martin proposed tensor singular value decomposition (T-SVD) for third order tensors. Since then, T-SVD has applications in low rank tensor approximation, tensor recovery, multi-view clustering, multi-view feature extraction, tensor sketching, etc. By going through the Discrete Fourier Transform (DFT), matrix SVD and inverse DFT, a third order tensor is mapped to an f-diagonal third order tensor. We call this a Kilmer-Martin mapping. We show that the Kilmer-Martin mapping of a third order tensor is invariant if that third order tensor is taking T-product with some orthogonal tensors. We define singular values and T-rank of that third order tensor based upon its Kilmer-Martin mapping. Thus, tensor tubal rank, T-rank, singular values and T-singular values of a third order tensor are invariant when it is taking T-product with some orthogonal tensors. Some properties of singular values, T-rank and best T-rank one approximation are discussed.</p>

Incomplete multiview clustering (IMC) methods have achieved remarkable progress by exploring the complementary information and consensus representation of incomplete multiview data. However, to our best knowledge, none of the existing methods attempts to handle the uncoupled and incomplete data simultaneously, which affects their generalization ability in real-world scenarios. For uncoupled incomplete data, the unclear and partial cross-view correlation introduces the difficulty to explore the complementary information between views, which results in the unpromising clustering performance for the existing multiview clustering methods. Besides, the presence of hyperparameters limits their applications. To fill these gaps, a novel uncoupled IMC (UIMC) method is proposed in this article. Specifically, UIMC develops a joint framework for feature inferring and recoupling. The high-order correlations of all views are explored by performing a tensor singular value decomposition (t-SVD)-based tensor nuclear norm (TNN) on recoupled and inferred self-representation matrices. Moreover, all hyperparameters of the UIMC method are updated in an exploratory manner. Extensive experiments on six widely used real-world datasets have confirmed the superiority of the proposed method in handling the uncoupled incomplete multiview data compared with the state-of-the-art methods.

Multi-view clustering can mine the underlying structure of multi-view data and has attracted increasing attention. Most existing multi-view clustering methods either construct the similarity matrix from the global level through self-representation learning or construct the similarity matrix from the local level through graph learning. Spectral clustering method can be used to yield the clustering results based on the similarity matrix. However, the similarity matrix that only considers global information or local information is not robust. Moreover, separating the similarity matrix learning and clustering as two steps may lead to sub-optimal clustering results. To address these issues, we propose in this paper, a multi-view subspace clustering with local and global information (MVSCLG) method. Our method combines the self-representation learning and graph learning to learn a similarity matrix with global and local information, and simultaneously utilizes the spectral decomposition and the spectral rotation techniques to yield the clustering results. We also develop an effective optimization algorithm to solve the resulting optimization problem. The effectiveness and superiority of this method are verified on four multi-view benchmark data sets.

An efficient non-convex total variation approach for image deblurring and denoising

Multiview subspace clustering (MSC) has attracted growing attention due to the extensive value in various applications, such as natural language processing, face recognition, and time-series analysis. In this article, we are devoted to address two crucial issues in MSC: 1) high computational cost and 2) cumbersome multistage clustering. Existing MSC approaches, including tensor singular value decomposition (t-SVD)-MSC that has achieved promising performance, generally utilize the dataset itself as the dictionary and regard representation learning and clustering process as two separate parts, thus leading to the high computational overhead and unsatisfactory clustering performance. To remedy these two issues, we propose a novel MSC model called joint skinny tensor learning and latent clustering (JSTC), which can learn high-order skinny tensor representations and corresponding latent clustering assignments simultaneously. Through such a joint optimization strategy, the multiview complementary information and latent clustering structure can be exploited thoroughly to improve the clustering performance. An alternating direction minimization algorithm, which owns low computational complexity and can be run in parallel when solving several key subproblems, is carefully designed to optimize the JSTC model. Such a nice property makes our JSTC an appealing solution for large-scale MSC problems. We conduct extensive experiments on ten popular datasets and compare our JSTC with 12 competitors. Five commonly used metrics, including four external measures (NMI, ACC, F-score, and RI) and one internal metric (SI), are adopted to evaluate the clustering quality. The experimental results with the Wilcoxon statistical test demonstrate the superiority of the proposed method in both clustering performance and operational efficiency.