Global Optimality in Inductive Matrix Completion

Matrix completion (MC) with additional information has found wide applicability in several machine learning applications. Among algorithms for solving such problems, Inductive Matrix Completion(IMC) has drawn a considerable amount of attention, not only for its well established theoretical guarantees but also for its superior performance in various real-world applications. However, IMC based methods usually place very strong constraints on the quality of the features(side information) to ensure accurate recovery, which might not be met in practice. In this paper, we propose Goal-directed Inductive Matrix Completion(GIMC) to learn a nonlinear mapping of the features so that they satisfy the required properties. A key distinction between GIMC and IMC is that the feature mapping is learnt in a supervised manner, deviating from the traditional approach of unsupervised feature learning followed by model training. We establish the superiority of our method on several popular machine learning applications including multi-label learning, multi-class classification, and semi-supervised clustering.

The task of estimating a matrix given a sample of observed entries is known as the \emph{matrix completion problem}. Most works on matrix completion have focused on recovering an unknown real-valued low-rank matrix from a random sample of its entries. Here, we investigate the case of highly quantized observations when the measurements can take only a small number of values. These quantized outputs are generated according to a probability distribution parametrized by the unknown matrix of interest. This model corresponds, for example, to ratings in recommender systems or labels in multi-class classification. We consider a general, non-uniform, sampling scheme and give theoretical guarantees on the performance of a constrained, nuclear norm penalized maximum likelihood estimator. One important advantage of this estimator is that it does not require knowledge of the rank or an upper bound on the nuclear norm of the unknown matrix and, thus, it is adaptive. We provide lower bounds showing that our estimator is minimax optimal. An efficient algorithm based on lifted coordinate gradient descent is proposed to compute the estimator. A limited Monte-Carlo experiment, using both simulated and real data is provided to support our claims.

With the development of Internet of Things technology, the will to make cities smarter is growing. In the context of smart cities, where people are surrounded by a tremendous number of points of interests (POIs), POI recommendation is of great significance. The massive amount of user check-in data collected by location-based social networks (LBSNs) can help users explore new places via POI recommendations. With rich auxiliary data becomeing available in LBSNs, purely exploiting users' check-in information for POI recommendation is not sufficient. Although several attempts have been done to employ geographical influence to enhance POI recommendation, they simply use a certain distribution function to measure the geographical influence between POIs, which may lead to biased results. To this end, this article proposes a geography-aware inductive matrix completion (GAIMC) approach for personalized POI recommendation. Specifically, the GAIMC model consists of two parts, including geographic feature extraction via a Gaussian mixture model (GMM) and inductive matrix completion for recommendation. The GAIMC model first captures the geographical influence among users and POIs by the GMM, which can mine clustering information with hierarchical structures based on the users' check-in data and POIs' location information. Then, a matrix-completion method called inductive matrix completion, which can incorporate geographical features with the user-POI association metric, is utilized to recommend POIs. The experimental results on two real-world LBSN data sets demonstrate that our proposed model can achieve the best recommendation performance in comparison with the state-of-the-art counterparts.

In this work, we address the problem of low-rank matrix recovery from its under-sampled projections. The recovery is formulated as a Schatten-p norm minimization problem. We proposed a novel algorithm to solve the Schatten-p norm minimization problem based on the FOCUSS (FOCally Under-determined System Solver) approach. We compared our proposed method with state-of-the-art solvers. Experimental evaluation was carried out on two problems - matrix completion and image inpainting. For matrix completion, our proposed method showed better recovery rate than other methods. In the image inpainting problem, our method yields 1.5 dB improvement over the nearest competing algorithm.

Inductive Matrix Completion (IMC) is an important class of matrix completion problems that allows direct inclusion of available features to enhance estimation capabilities. These models have found applications in personalized recommendation systems, multilabel learning, dictionary learning, etc. This paper examines a general class of noisy matrix completion tasks where the underlying matrix is following an IMC model i.e., it is formed by a mixing matrix (a priori unknown) sandwiched between two known feature matrices. The mixing matrix here is assumed to be well approximated by the product of two sparse matrices - referred here to as “sparse factor models.” We leverage the main theorem of [1] and extend it to provide theoretical error bounds for the sparsity-regularized maximum likelihood estimators for the class of problems discussed in this paper. The main result is general in the sense that it can be used to derive error bounds for various noise models. In this paper, we instantiate our main result for the case of Gaussian noise and provide corresponding error bounds in terms of squared loss.

Many studies have clarified that microRNAs (miRNAs) are associated with many human diseases. Therefore, it is essential to predict potential miRNA-disease associations for disease pathogenesis and treatment. Numerous machine learning and deep learning approaches have been adopted to this problem. In this paper, we propose a Neural Inductive Matrix completion-based method with Graph Autoencoders (GAE) and Self-Attention mechanism for miRNA-disease associations prediction (NIMGSA). Some of the previous works based on matrix completion ignore the importance of label propagation procedure for inferring miRNA-disease associations, while others cannot integrate matrix completion and label propagation effectively. Varying from previous studies, NIMGSA unifies inductive matrix completion and label propagation via neural network architecture, through the collaborative training of two graph autoencoders. This neural inductive matrix completion-based method is also an implementation of self-attention mechanism for miRNA-disease associations prediction. This end-to-end framework can strengthen the robustness and preciseness of both matrix completion and label propagation. Cross validations indicate that NIMGSA outperforms current miRNA-disease prediction methods. Case studies demonstrate that NIMGSA is competent in detecting potential miRNA-disease associations.

Most existing network representation learning algorithms focus on network structures for learning. However, network structure is only one kind of view and feature for various networks, and it cannot fully reflect all characteristics of networks. In fact, network vertices usually contain rich text information, which can be well utilized to learn text-enhanced network representations. Meanwhile, Matrix-Forest Index (MFI) has shown its high effectiveness and stability in link prediction tasks compared with other algorithms of link prediction. Both MFI and Inductive Matrix Completion (IMC) are not well applied with algorithmic frameworks of typical representation learning methods. Therefore, we proposed a novel semi-supervised algorithm, tri-party deep network representation learning using inductive matrix completion (TDNR). Based on inductive matrix completion algorithm, TDNR incorporates text features, the link certainty degrees of existing edges and the future link probabilities of non-existing edges into network representations. The experimental results demonstrated that TFNR outperforms other baselines on three real-world datasets. The visualizations of TDNR show that proposed algorithm is more discriminative than other unsupervised approaches.

formula omitted] completions on partial matrices

Popular microblogging sites such as Tumblr have attracted hundreds of millions of users as a content sharing platform, where users can create rich content in the form of posts that are shared with other users who follow them. Due to the sheer amount of posts created on such services, an important task is to make quality recommendations of blogs for users to follow. Apart from traditional recommender system settings where the follower graph is the main data source, additional side-information of users and blogs such as user activity (e.g., like and reblog) and rich content (e.g., text and images) are also available to be exploited for enhanced recommendation performance. In this paper, we propose a novel boosted inductive matrix completion method (BIMC) for blog recommendation. BIMC is an additive low-rank model for user-blog preferences consisting of two components; one component captures the low-rank structure of follow relationships and the other captures the latent structure using side-information. Our model formulation combines the power of the recently proposed inductive matrix completion (IMC) model (for side-information) together with a standard matrix completion (MC) model (for low-rank structure). Furthermore, we utilize recently developed deep learning techniques to obtain semantically rich feature representations of text and images that are incorporated in BIMC. Experiments on a large-scale real-world dataset from Tumblr illustrate the effectiveness of the proposed BIMC method.

This paper considers recovery of matrices that are low rank or approximately low rank from linear measurements corrupted with additive noise. We study minimization of the difference of Nuclear and Frobenius norms (abbreviated as [Formula: see text] norm) as a nonconvex and Lipschitz continuous metric for solving this noisy low rank matrix recovery problem. We mainly study two types of bounded observation noisy low rank matrix recovery problems, including the [Formula: see text]-norm bounded noise and the Dantizg Selector noise. Based on the matrix restricted isometry property (abbreviated as M-RIP), we prove that this [Formula: see text] norm-based minimization method can stably recover a (approximately) low rank matrix in the two types bounded noisy low rank matrix recovery problems. In addition, we use the truncated difference of Nuclear and Frobenius norms (denoted as the truncated [Formula: see text] norm) to recover a low rank matrix when the observation noise is the Dantizg Selector noise. We give the stable recovery result for this truncated [Formula: see text] norm minimization in Dantizg Selector noise case when the linear measurement map satisfies the M-RIP condition.

We consider a matrix completion problem that exploits social graph as side information. We develop a computationally efficient algorithm that achieves the optimal sample complexity for the entire regime of graph information under the multiple cluster setting (to be detailed). The key idea is to incorporate a switching mechanism which selects the information employed in the first clustering step, between the following two types: graph & matrix ratings. Our experimental results on both synthetic and real data corroborate our theoretical result as well as demonstrate that our algorithm outperforms prior algorithms that leverage graph side information.

The essence of distantly supervised relation extraction is that it is an incomplete multi-label classification problem with sparse and noisy features. To tackle the sparsity and noise challenges, we propose solving the classification problem using matrix completion on factorized matrix of minimized rank. We formulate relation classification as completing the unknown labels of testing items (entity pairs) in a sparse matrix that concatenates training and testing textual features with training labels. Our algorithmic framework is based on the assumption that the rank of item-byfeature and item-by-label joint matrix is low. We apply two optimization models to recover the underlying low-rank matrix leveraging the sparsity of feature-label matrix. The matrix completion problem is then solved by the fixed point continuation (FPC) algorithm, which can find the global optimum. Experiments on two widely used datasets with different dimensions of textual features demonstrate that our low-rank matrix completion approach significantly outperforms the baseline and the state-of-the-art methods.

This paper studies the problem of the restoration of images corrupted by mixed Gaussian-impulse noise. In recent years, low-rank matrix reconstruction has become a research hotspot in many scientific and engineering domains such as machine learning, image processing, computer vision and bioinformatics, which mainly involves the problem of matrix completion and robust principal component analysis, namely recovering a low-rank matrix from an incomplete but accurate sampling subset of its entries and from an observed data matrix with an unknown fraction of its entries being arbitrarily corrupted, respectively. Inspired by these ideas, we consider the problem of recovering a low-rank matrix from an incomplete sampling subset of its entries with an unknown fraction of the samplings contaminated by arbitrary errors, which is defined as the problem of matrix completion from corrupted samplings and modeled as a convex optimization problem that minimizes a combination of the nuclear norm and the -norm in this paper. Meanwhile, we put forward a novel and effective algorithm called augmented Lagrange multipliers to exactly solve the problem. For mixed Gaussian-impulse noise removal, we regard it as the problem of matrix completion from corrupted samplings, and restore the noisy image following an impulse-detecting procedure. Compared with some existing methods for mixed noise removal, the recovery quality performance of our method is dominant if images possess low-rank features such as geometrically regular textures and similar structured contents; especially when the density of impulse noise is relatively high and the variance of Gaussian noise is small, our method can outperform the traditional methods significantly not only in the simultaneous removal of Gaussian noise and impulse noise, and the restoration ability for a low-rank image matrix, but also in the preservation of textures and details in the image.

Since lots of miRNA-disease associations have been verified, it is meaningful to discover more miRNA-disease associations for serving disease diagnosis and prevention of human complex diseases. However, it is not practical to identify potential associations using traditional biological experimental methods since the process is expensive and time consuming. Therefore, it is necessary to develop efficient computational methods to accomplish this task. In this work, we introduced a matrix completion model with dual Laplacian regularization (DLRMC) to infer unknown miRNA-disease associations in heterogeneous omics data. Specifically, DLRMC transformed the task of miRNA-disease association prediction into a matrix completion problem, in which the potential missing entries of the miRNA-disease association matrix were calculated, the missing association can be obtained based on the prediction scores after the completion procedure. Meanwhile, the miRNA functional similarity and the disease semantic similarity were fully exploited to serve the miRNA-disease association matrix completion by using a dual Laplacian regularization term. In the experiments, we conducted global and local Leave-One-Out Cross Validation (LOOCV) and case studies to evaluate the efficacy of DLRMC on the Human miRNA-disease associations dataset obtained from the HMDDv2.0 database. As a result, the AUCs of DLRMC is 0.9174 and 0.8289 in global LOOCV and local LOOCV, respectively, which significantly outperform a variety of previous methods. In addition, in the case studies on four significant diseases related to human health including Colon Neoplasms, Kidney neoplasms, Lymphoma and Prostate neoplasms, 90%, 92%, 92% and 94% out of the top 50 predicted miRNAs has been confirmed, respectively.

The matrix completion problem can be stated as follows: Suppose $X\in \mathbb{R}^{n_{1}\times n_{2}}$ is unknown except for an upper bound $r$ on its rank. By measuring a small number $m\ll n_{1}n_{2}$ of elements of $X$ , is it possible to recover $X$ exactly, or at least, to construct a reasonable approximation of $X$ ? There are two existing approaches to choosing the elements to be measured, namely: choosing them at random or in some deterministic fashion. Practically all of the existing literature focuses on the case where the sampled locations are chosen at random. In contrast, the focus in the present paper is on deterministic methods for choosing the elements to be sampled. Specifically, we choose the measurement matrix to be the adjacency or biadjacency matrix of a so-called Ramanujan graph. Existing explicit techniques for constructing Ramanujan graphs lead to graphs whose degree $d$ is bounded by $n^{1/3}$ where $n$ is the number of vertices. However, for the matrix completion problem, this degree is too small. We point out that the well-known Lubotzky-Phillips-Sarnak construction of Ramanujan graphs can be used to generate graphs of very high degree, and use these for the matrix construction problem. Then we carry out numerical studies of “phase transition” in the matrix completion problem, by comparing the behavior of sampling at random with choosing the sampled locations using a Ramanujan graph. Specifically, we show that for a fixed sampling pattern, there is a maximum rank $\bar{r}$ for which randomly generated matrices of rank $r\leq\overline{r}$ can be recovered with probability one, and the recovery probability drops very sharply to zero if $r$ is increased by just two or three beyond $\bar{r}$ . The technique proposed here is applicable only to the recovery of square matrices. The recovery of rectangular matrices requires so-called asymmetric Ramanujan graphs, and is studied in a separate paper.