Learning Latent Features With Pairwise Penalties in Low-Rank Matrix Completion

Wigner distribution deconvolution (WDD) is a decades-old method for recovering phase from intensity measurements. Although the technique offers an elegant linear solution to the quadratic phase retrieval problem, it has seen limited adoption due to its high computational/memory requirements and the fact that the technique often exhibits high noise sensitivity. Here, we propose a method for noise suppression in WDD via low-rank noisy matrix completion. Our technique exploits the redundancy of an object's phase space to denoise its WDD reconstruction. We show in model calculations that our technique outperforms other WDD algorithms as well as modern iterative methods for phase retrieval such as ptychography. Our results suggest that a class of phase retrieval techniques relying on regularized direct inversion of ptychographic datasets (instead of iterative reconstruction techniques) can provide accurate quantitative phase information in the presence of high levels of noise.

Localizing a cloud of points from noisy measurements of a subset of pairwise distances has applications in various areas, such as sensor network localization and reconstruction of protein conformations from NMR measurements. In [1], Drineas et al. proposed a natural two-stage approach, named SVD-MDS, for this purpose. This approach consists of a low-rank matrix completion algorithm, named SVD-Reconstruct, to estimate random missing distances, and the classic multidimensional scaling (MDS) method to estimate the positions of nodes. In this paper, we present a detailed analysis for this method. More specifically, we first establish error bounds for Euclidean distance matrix (EDM) completion in both expectation and tail forms. Utilizing these results, we then derive the error bound for the recovered positions of nodes. In order to assess the performance of SVD-Reconstruct, we present the minimax lower bound of the zero-diagonal, symmetric, low-rank matrix completion problem by Fano's method. This result reveals that when the noise level is low, the SVD-Reconstruct approach for Euclidean distance matrix completion is suboptimal in the minimax sense; when the noise level is high, SVD-Reconstruct can achieve the optimal rate up to a constant factor.

In paper, we focus on the computation of the principal components for a general tensor, known as the tensor principal component analysis (PCA) problem. It has been proven that the general tensor PCA problem is reducible to its matricization form when the order is even. Usually it is considered as low-rank matrix completion problems theoretically. It is common to consider nuclear norm as a surrogate of the rank operator since it is the tightest convex lower bound of the rank operator under certain condition. However, most nuclear norm minimization based approaches involve numbers of singular value decomposition (SVD) operations. Given a matrix X E Rm× n, the time complexity of SVD operation is O(mn2), which brings prohibitive computational burden to apply these methods in real applications. However, the problem is non-convex, and the proximal mapping associated with non-convex regularization is not easy to compute. It is always solved by the Linearized Alternating Direction Method of Multipliers (LADMM). Despite the success of LADMM in practice, it remains unknown if LADMM is convergent in solving such non-convex compositely regularized optimization. In this paper, we firstly present a detailed convergence analysis of the LADMM algorithm for solving non-convex compositely regularized optimization with a large class of non-convex penalties. Furthermore, we propose a new efficient and scalable algorithm for matrix principal component analysis called Proximal Linearized Alternating Direction Method of Multipliers for Principal Component Analysis(PLADMPCA). Different from traditional matrix factorization methods, PLADMPCA utilizes the linearization technique to formulate the matrix as an outer product of vectors, which greatly improves the computational efficacy compared to matrix factorization method. We empirically evaluate the proposed algorithm PLADMPCA on synthetic tensor data with different order. Results have shown that PLADMPCA have much better computational cost to matrix factorization based method. At the same time, it outperforms the state-of-art SVD based matrix completion algorithms by similar or better reconstruction accuracy with enormous advantages on efficiency.

Low-rank matrix completion, which aims to recover a matrix with many missing values, has attracted much attention in many fields of computer science. A low-rank matrix fitting (LMaFit) method has been proposed for fast matrix completion recently. However, this method cannot converge accurately on matrices of real-world images. For improving the accuracy of LMaFit method, an improved low-rank matrix fitting (ILMF) method based on the weighted [Formula: see text] norm minimization is proposed in this paper, where the [Formula: see text] norm is the summation of the [Formula: see text]-power [Formula: see text] of [Formula: see text] norms of rows in a matrix. In the proposed method, i.e. the ILMF method, the incomplete matrix that may be corrupted by noises is decomposed into the summation of a low-rank matrix and a noise matrix at first. Then, a weighted [Formula: see text] norm minimization problem is solved by using an alternating direction method for improving the accuracy of matrix completion. Experimental results on real-world images show that the ILMF method has much better performances in terms of both the convergence accuracy and convergence speed than the compared methods.

Low-rank matrix completion is a hot topic in the field of machine learning. It is widely used in image processing, recommendation systems and subspace clustering. However, the traditional method uses the nuclear norm to approximate the rank function, which leads to only the suboptimal solution. Inspired by the closed-form formulation of L_{2/3} regularization, we propose a new truncated schatten 2/3-norm to approximate the rank function. Our proposed regularizer takes full account of the prior rank information and achieves a more accurate approximation of the rank function. Based on this regularizer, we propose a new low-rank matrix completion model. Meanwhile, a fast and efficient algorithm are designed to solve the proposed model. In addition, a rigorous mathematical analysis of the convergence of the proposed algorithm is provided. Finally, the superiority of our proposed model and method is investigated on synthetic data and recommender system datasets. All results show that our proposed algorithm is able to achieve comparable recovery performance while being faster and more efficient than state-of-the-art methods.

Several complex tasks that arise in organizations can be simplified by mapping them into a matrix completion problem. In this paper, we address a key challenge faced by our company: predicting the efficiency of artists in rendering visual effects (VFX) in film shots. We tackle this challenge by using a two-fold approach: first, we transform this task into a constrained matrix completion problem with entries bounded in the unit interval [0,1]; second, we propose two novel matrix factorization models that leverage our knowledge of the VFX environment. Our first approach, expertise matrix factorization (EMF), is an interpretable method that structures the latent factors as weighted user-item interplay. The second one, survival matrix factorization (SMF), is instead a probabilistic model for the underlying process defining employees' efficiencies. We show the effectiveness of our proposed models by extensive numerical tests on our VFX dataset and two additional datasets with values that are also bounded in the [0,1] interval.

Matrix completion by deep matrix factorization

The matrix completion problem consists of finding or approximating a low-rank matrix based on a few samples of this matrix. We propose a new algorithm for matrix completion that minimizes the least-square distance on the sampling set over the Riemannian manifold of fixed-rank matrices. The algorithm is an adaptation of classical nonlinear conjugate gradients, developed within the framework of retraction-based optimization on manifolds. We describe all the necessary objects from differential geometry necessary to perform optimization over this low-rank matrix manifold, seen as a submanifold embedded in the space of matrices. In particular, we describe how metric projection can be used as retraction and how vector transport lets us obtain the conjugate search directions. Finally, we prove convergence of a regularized version of our algorithm under the assumption that the restricted isometry property holds for incoherent matrices throughout the iterations. The numerical experiments indicate that our approach scales very well for large-scale problems and compares favorably with the state-of-the-art, while outperforming most existing solvers.

DOA estimation via shift-invariant matrix completion

In this paper, a novel video super-resolution image reconstruction algorithm is proposed. We design a patch-based low rank matrix completion algorithm. The proposed algorithm addresses the problem of generating a high-resolution (HR) image from several low-resolution (LR) images, based on sparse representation and low-rank matrix completion. The approach represents observed LR frames in the form of sparse matrices and rearranges those frames into low dimensional constructions. Experimental results demonstrate that, high-frequency details in the super resolved images are recovered from the LR frames. The gains in terms of PSNR and SSIM are significant.

Theoretical convergence guarantees versus numerical convergence behavior of the holomorphically embedded power flow method

Accurate node localization is essential for many Internet of Things (IoT) applications. However, incomplete and noisy distance measurements often degrade the reliability of the Euclidean Distance Matrix (EDM), which is critical for range-based localization. To address this issue, a Low-Rank Matrix Completion approach based on nonconvex rank approximation (LRMCN) is proposed to recover the true EDM. First, the observed EDM is decomposed into a low-rank matrix representing the true distances and a sparse matrix capturing noise. Second, a nonconvex surrogate function is used to approximate the matrix rank, while the l1-norm is utilized to model the sparsity of the noise component. Third, the resulting optimization problem is solved using the Alternating Direction Method of Multipliers (ADMMs). This enables accurate recovery of a complete and denoised EDM from incomplete and corrupted measurements. Finally, relative node locations are estimated using classical multi-dimensional scaling, and absolute coordinates are determined based on a small set of anchor nodes with known locations. The experimental results show that the proposed method achieves superior performance in both matrix completion and localization accuracy, even in the presence of missing and corrupted data.

Recommender systems have become essential in the digital era to help users navigate overwhelming content. This study develops a book recommendation system using three collaborative filtering methods: user-based, item-based, and matrix factorization using singular value decomposition. We evaluate the system on a real-world dataset of 1,149,780 book ratings from 278,858 users across 271,360 books. A subset of 500 active users is used for experimental evaluation. The models are assessed using root mean square error and mean absolute error to measure rating prediction accuracy. The results show that the item-based collaborative filtering method achieves the best accuracy (root mean square error 7.362; mean absolute error 6.761), slightly outperforming the user-based approach (7.365; 6.809) and the matrix factorization method (7.643; 7.413). We analyze the results to understand the performance differences, noting the stability of item similarity as a key factor and the need for optimal tuning in the matrix factorization model. In conclusion, item-based collaborative filtering proved most effective for this context. This work provides insights into the comparative performance of foundational recommendation techniques and highlights practical considerations for improving book recommender systems.

Low Rank Matrix Completion (LRMC) is widely used in the analysis of incomplete datasets. In this paper, we propose a novel FPGA-based accelerator to speedup a matrix-factorization-based LRMC algorithm that uses stochastic gradient descent. The accelerator is a multi-pipelined architecture with parallel pipelines processing distinct data from a shared on-chip buffer. We propose two distinct on-chip buffer architectures based on a design-space exploration of the performance tradeoffs offered by two competing design methodologies: memory-efficiency versus concurrent conflict-free accesses. Our first design (i.e., memory-efficient design) organizes the buffer into banks and maximally utilizes available on-chip memory for matrix chunk processing without requiring complex address translation tables for on-chip addressing; however, it could incur bank conflicts when concurrent accesses to the same bank occur. The second design (i.e., bank-conflict-free design) exploits parallel multiport memory access and completely eliminates bank conflicts by duplicating the stored data; however, it has much higher on-chip RAM consumption. Intuitively, design one enables (slower) acceleration of (larger) chunks of the input matrix whereas design two enables (faster) processing of (smaller) matrix chunks but requires more iterations for processing the complete matrix. We propose a simple but efficient partitioning approach for supporting large input matrices that do not fit in the on-chip memory of FPGA. We also develop algorithmic optimizations based on matching to reduce data dependencies for parallel pipeline execution. We implement our designs on a state-of-the-art UltraScale+ FPGA device. We use real-life datasets for the evaluation and compare these two designs by varying the number of pipelines. The data dependency optimization results in at least 21.6 x data dependency reduction and improves the execution time by up to 66.3 x compared with non-optimized baseline designs. The memory-efficient design is also shown to be more scalable than the bank-conflict-free design. Compared with the state-of-the-art multi-core implementation and GPU implementation, the bank-conflict-free design achieves 5.4 x and 5.2 x speedup, respectively; the memory-efficient design achieves 16.7 x and 16.2 x speedup, respectively.

The matrix factorization recommender system based on manifold regularization, taking into account the similarity of local neighbors and manifold structure, can improve the quality of a recommendation system. However, the similarity between samples may not be accurate due to the sparsity of the data or the incompleteness of the tag information. Therefore, we propose a new model called SI-GMF (Similarity-learning-based Improved Graph Regularized matrix Factorization) by embedding the new similarity measure strategy in GMF (Graph Regularized matrix Factorization) framework, and induce three new matrix factorization algorithms (SI-GMF_1, SI-GMF_2, SI-GMF_3) based on three initial similarities by employing three different similarity measures. The solutions to the newly developed algorithms can be effectively obtained by SGD method. The experimental results show that the newly designed algorithms significantly improve the accuracy of a recommender system.