Multi-view data clustering via non-negative matrix factorization with manifold regularization

Clustering of multi-view relational data based on particle swarm optimization

In recent years, datasets which exist in present world are comprising of various representations of the data or in multiview environment, which frequently give the important data to each other. Multi-view clustering based on Non-negative matrix factorization (NMF) has turned to be a very hot direction of research in the field of Pattern Reognition, Machine Learning (ML), and data mining. and data mining due to unsupervised confuse information of Numerous Views. The main problem of employing NMF to multi-view clustering is how to define the factorizations to give significant and commensurate clustering solutions. Specially, multi-view clustering based NMF has achieved extensive attention due to its dimensionality reduction property. Existing methods based on NMF barely produced meaningful clustering solution from heterogeneous numerous views due to their complementary behaviors. To address this issue, we design a innovative NMF technique based Multiview clustering approach, which gives the more meaningful and compatible clustering solution over Numerous Views. The main outcome of the work, is to a design combined NMF method with view weight and constraint co-efficient which will bring the clustering solution to a common point for each view. The effectiveness of propose method is validated by conducting the experiments on real-world datasets.

Multi-view data are becoming common in real-world modeling tasks and many multi-view data clustering algorithms have thus been proposed. The existing algorithms usually focus on the cooperation of different views in the original space but neglect the influence of the hidden information among these different visible views, or they only consider the hidden information between the views. The algorithms are therefore not efficient since the available information is not fully excavated, particularly the otherness information in different views and the consistency information between them. In practice, the otherness and consistency information in multi-view data are both very useful for effective clustering analyses. In this study, a Multi-View clustering algorithm developed with the Cooperation of Visible and Hidden views, i.e., MV-Co-VH, is proposed. The MV-Co-VH algorithm first projects the multiple views from different visible spaces to the common hidden space by using the non-negative matrix factorization (NMF) strategy to obtain the common hidden view data. Collaborative learning is then implemented in the clustering procedure based on the visible views and the shared hidden view. The results of extensive experiments on UCI multi-view datasets and real-world image multi-view datasets show that the clustering performance of the proposed algorithm is competitive with or even better than that of the existing algorithms.

Nonnegative Matrix Factorization (NMF), as a popular feature extraction technique, has recently attracted increasing attentions in high-dimensional data analysis, due to the strong ability of dimension reduction and semantic representation. In order to utilize the partial label information in practice, several semi-supervised NMF methods have been proposed. However, existing semi-supervised NMF variants only consider the global label constraint of data, but ignore the local label information embedded in geometric structure, which is very crucial for data representation. To address the above issue, we propose an improved NMF method, namely local and global regularized semi-supervised NMF (LGNMF), by considering the local and global label constraints simultaneously. Specifically, the local label constraint is depicted in a graph, which is pre-computed by metric learning. In addition, to explore the global label information, we construct an indicator matrix to restrict the coefficient matrix in LGNMF. For the formulated LGNMF, a multiplicative update rule (MUR) is developed. Extensive experiments on several real datasets demonstrate the superiority of the proposed method over the state-of-the-art methods in terms of clustering accuracy and normalized mutual information.

Nonnegative matrix factorization (NMF) has been successfully used in different applications including computer vision, pattern recognition and text mining. NMF aims to decompose a data matrix into the product of two matrices (respectively denoted as the basis vectors and the encoding vectors), whose entries are constrained to be nonnegative. Unlike the ordinary NMF, we propose a novel NMF, denoted as MMNMF, which considers both geometrical information and discriminative information hidden in the data. The geometrical information is discovered by minimizing the distance among the encoding vectors, while the discriminative information is uncovered by maximizing the distance among base vectors. Clustering experiments are performed on the real-world data sets of faces, images, and documents to demonstrate the effectiveness of the proposed algorithm.

Depth-image-based rendering (DIBR) oriented view synthesis has been widely employed in the current depth-based 3-D video systems by synthesizing a virtual view from an arbitrary viewpoint. However, holes may appear in the synthesized view due to disocclusion, thus significantly degrading the quality. Consequently, efforts have been made on developing effective and efficient hole-filling algorithms. Current hole-filling techniques generally extrapolate/interpolate the hole regions with the neighboring information based on an assumption that the texture pattern in the holes is similar to that of the neighboring background information. However, in many scenarios, especially of complex texture, the assumption may not hold. In other words, hole-filling techniques can only provide an estimation for a hole which may not be good enough or may even be erroneous considering a wide variety of complex scene of images. In this paper, we first examine the view interpolation with multiple reference views, demonstrating that the problem of emerging holes in a target virtual view can be greatly alleviated by making good use of other neighboring complementary views in addition to its two (commonly used) most neighboring primary views. The effects of using multiple views for view extrapolation in reducing holes are also investigated in this paper. In view of the 3D Video and ongoing free-viewpoint TV standardization, we propose a new view synthesis framework, which employs multiple views to synthesize output virtual views. Furthermore, a scheme of selective warping of complementary views is developed by efficiently locating a small number of useful pixels in the complementary views for hole reduction, to avoid full warping of additional complementary views thus lowering greatly the warping complexity. Experimental results show that the hole size based on two primary reference views may be reduced by up to about 70% with the help of two complementary reference views in the case of view interpolation, while the hole size based on one primary reference view may be reduced by about 27% with the help of one more complementary reference view in view extrapolation. Moreover, it is shown that by using one more pair of views in view interpolation and one more view in view extrapolation, 10% hole pixels may be reduced additionally.

With the incredible growth of high-dimensional data such as microarray gene expression data and web blogs from internet, the researchers are desirable to develop new clustering techniques to address the critical problem created by irrelevant dimensions. Properties of Nonnegative Matrix Factorization (NMF) as a clustering method were studied by relating its formulation to other methods such as K-means clustering. In this paper, by introducing clustering indicator constraints on NMF and incorporating manifold regularization to preserve geometric structures, we propose a novel manifold regularized NMF method that can simultaneously learn subspace and do clustering. As a result, our clustering results can directly assign cluster label to data points. Extensive experimental results show that our method outperforms related other methods.

Non-negative matrix factorization (NMF) is widely used in feature extraction and dimension reduction fields. Essentially, it is an optimization problem to determine two non-negative low rank matrices \(W_{m \times k}\) and \(H_{k \times n}\) for a given matrix \(A_{m \times n}\), satisfying \(A_{m \times n} \approx W_{m \times k}H_{k \times n}\). In this paper, a novel approach to improve the image decomposing and reconstruction effects by introducing the Singular Value Decomposing (SVD)-based initialization scheme of factor matrices W and H, and another measure called choosing rule to determine the optimum value of factor rank k, are proposed. The input image is first decomposed using SVD to get its singular values and corresponding eigenvectors. Then, the number of main components as the rank value k is extracted. Then, the singular values and corresponding eigenvectors are used to initialize W and H based on selected rank k. Finally, convergent results are obtained using multiplicative and additive update rules. However, iterative NMF algorithms’ convergence is very slow on most platforms limiting its practicality. To this end, a parallel implementation frame of described improved NMF algorithm using CUDA, a tool for algorithms parallelization on massively parallel processors, i.e., many-core graphics processors, is presented. Experimental results show that our approach can get better decomposing effect than traditional NMF implementations and dramatic accelerate rate comparing to serial schemes as well as existing distributed-system implementations.

Sparse nonnegative matrix factorization with ℓ0-constraints

In this letter, we consider the nonnegative matrix factorization (NMF) problem and several NMF variants. Two approaches based on DC (difference of convex functions) programming and DCA (DC algorithm) are developed. The first approach follows the alternating framework that requires solving, at each iteration, two nonnegativity-constrained least squares subproblems for which DCA-based schemes are investigated. The convergence property of the proposed algorithm is carefully studied. We show that with suitable DC decompositions, our algorithm generates most of the standard methods for the NMF problem. The second approach directly applies DCA on the whole NMF problem. Two algorithms-one computing all variables and one deploying a variable selection strategy-are proposed. The proposed methods are then adapted to solve various NMF variants, including the nonnegative factorization, the smooth regularization NMF, the sparse regularization NMF, the multilayer NMF, the convex/convex-hull NMF, and the symmetric NMF. We also show that our algorithms include several existing methods for these NMF variants as special versions. The efficiency of the proposed approaches is empirically demonstrated on both real-world and synthetic data sets. It turns out that our algorithms compete favorably with five state-of-the-art alternating nonnegative least squares algorithms.

Multi-view subspace clustering has been an important and powerful tool for partitioning multi-view data, especially multi-view high-dimensional data. Despite great success, most of the existing multi-view subspace clustering methods still suffer from three limitations. First, they often recover the subspace structure in the original space, which can not guarantee the robustness when handling multi-view data with nonlinear structure. Second, these methods mostly regard subspace clustering and affinity matrix learning as two independent steps, which may not well discover the latent relationships among data samples. Third, many of them ignore the different importance of multiple views, whose performance may be badly affected by the low-quality views in multi-view data. To overcome these three limitations, this paper develops a novel subspace clustering method for multi-view data, termed Kernelized Multi-view Subspace Clustering via Auto-weighted Graph Learning (KMSC-AGL). Specifically, the proposed method implicitly maps the multi-view data from linear space into nonlinear space via kernel-induced functions, so as to exploit the nonlinear structure hidden in data. Furthermore, our method aims to enhance the clustering performance by learning a set of view-specific representations and their affinity matrix in a general framework. By integrating the view weighting strategy into this framework, our method can automatically assign the weights to different views, while learning an optimal affinity matrix that is well-adapted to the subsequent spectral clustering. Extensive experiments are conducted on a variety of multi-view data sets, which have demonstrated the superiority of the proposed method.

In the era of big data, data may come from multiple sources, known as multi-view data. Multi-view clustering aims at generating better clusters by exploiting complementary and consistent information from multiple views rather than relying on the individual view. Due to inevitable system errors caused by data-captured sensors or others, the data in each view may be erroneous. Various types of errors behave differently and inconsistently in each view. More precisely, error could exhibit as noise and corruptions in reality. Unfortunately, none of the existing multi-view clustering approaches handle all of these error types. Consequently, their clustering performance is dramatically degraded. In this paper, we propose a novel Markov chain method for Error-Robust Multi-View Clustering (EMVC). By decomposing each view into a shared transition probability matrix and error matrix and imposing structured sparsity-inducing norms on error matrices, we characterize and handle typical types of errors explicitly. To solve the challenging optimization problem, we propose a new efficient algorithm based on Augmented Lagrangian Multipliers and prove its convergence rigorously. Experimental results on various synthetic and real-world datasets show the superiority of the proposed EMVC method over the baseline methods and its robustness against different types of errors.

Existing stereoscopic image quality assessment (SIQA) methods are mostly based on the luminance information, in which color information is not sufficiently considered. Actually, color is part of the important factors that affect human visual perception, and nonnegative matrix factorization (NMF) and manifold learning are in line with human visual perception. We propose an SIQA method based on learning binocular manifold color visual properties. To be more specific, in the training phase, a feature detector is created based on NMF with manifold regularization by considering color information, which not only allows parts-based manifold representation of an image, but also manifests localized color visual properties. In the quality estimation phase, visually important regions are selected by considering different human visual attention, and feature vectors are extracted by using the feature detector. Then the feature similarity index is calculated and the parts-based manifold color feature energy (PMCFE) for each view is defined based on the color feature vectors. The final quality score is obtained by considering a binocular combination based on PMCFE. The experimental results on LIVE I and LIVE Π 3-D IQA databases demonstrate that the proposed method can achieve much higher consistency with subjective evaluations than the state-of-the-art SIQA methods.

Nonnegative matrix factorization (NMF) aims to decompose a given data matrix X into the product of two lower-rank nonnegative factor matrices UV T . Graph regularized NMF (GNMF) is a recently proposed NMF method that preserves the geometric structure of X during such decomposition. Although GNMF has been widely used in computer vision and data mining, its multiplicative update rule (MUR) based solver suffers from both slow convergence and non-stationarity problems. In this paper, we propose a new efficient GNMF solver called rank-one residue approximation (RRA). Different from MUR, which updates both factor matrices ( U and V) as a whole in each iteration round, RRA updates each of their columns by approximating the residue matrix by their outer product. Since each column of both factor matrices is updated optimally in an analytic formulation, RRA is theoretical and empirically proven to converge rapidly to a stationary point. Moreover, since RRA needs neither extra computational cost nor parametric tuning, it enjoys a similar simplicity to MUR but performs much faster. Experimental results on real-world datasets show that RRA is much more efficient than MUR for GNMF. To confirm the stationarity of the solution obtained by RRA, we conduct clustering experiments on real-world image datasets by comparing with the representative solvers such as MUR and NeNMF for GNMF. The experimental results confirm the effectiveness of RRA.KeywordsNonnegative matrix factorizationManifold regularizationRank-one residue iterationBlock coordinate descent

Multiview clustering (MVC), which aims to explore the underlying cluster structure shared by multiview data, has drawn more research efforts in recent years. To exploit the complementary information among multiple views, existing methods mainly learn a common latent subspace or develop a certain loss across different views, while ignoring the higher level information such as basic partitions (BPs) generated by the single-view clustering algorithm. In light of this, we propose a novel marginalized multiview ensemble clustering (M2VEC) method in this paper. Specifically, we solve MVC in an EC way, which generates BPs for each view individually and seeks for a consensus one. By this means, we naturally leverage the complementary information of multiview data upon the same partition space. In order to boost the robustness of our approach, the marginalized denoising process is adopted to mimic the data corruptions and noises, which provides robust partition-level representations for each view by training a single-layer autoencoder. A low-rank and sparse decomposition is seamlessly incorporated into the denoising process to explicitly capture the consistency information and meanwhile compensate the distinctness between heterogeneous features. Spectral consensus graph partitioning is also involved by our model to make M2VEC as a unified optimization framework. Moreover, a multilayer M2VEC is eventually delivered in a stacked fashion to encapsulate nonlinearity into partition-level representations for handling complex data. Experimental results on eight real-world data sets show the efficacy of our approach compared with several state-of-the-art multiview and EC methods. We also showcase our method performs well with partial multiview data.