Robust online matrix completion on graphs

As a convex relaxation of the rank minimization model, the nuclear norm minimization (NNM) problem has been attracting significant research interest in recent years. The standard NNM regularizes each singular value equally, composing an easily calculated convex norm. However, this restricts its capability and flexibility in dealing with many practical problems, where the singular values have clear physical meanings and should be treated differently. In this paper we study the weighted nuclear norm minimization (WNNM) problem, which adaptively assigns weights on different singular values. As the key step of solving general WNNM models, the theoretical properties of the weighted nuclear norm proximal (WNNP) operator are investigated. Albeit nonconvex, we prove that WNNP is equivalent to a standard quadratic programming problem with linear constrains, which facilitates solving the original problem with off-the-shelf convex optimization solvers. In particular, when the weights are sorted in a non-descending order, its optimal solution can be easily obtained in closed-form. With WNNP, the solving strategies for multiple extensions of WNNM, including robust PCA and matrix completion, can be readily constructed under the alternating direction method of multipliers paradigm. Furthermore, inspired by the reweighted sparse coding scheme, we present an automatic weight setting method, which greatly facilitates the practical implementation of WNNM. The proposed WNNM methods achieve state-of-the-art performance in typical low level vision tasks, including image denoising, background subtraction and image inpainting.

Robust low-rank matrix completion (RMC), or robust principal component analysis with partially observed data, has been studied extensively for computer vision, signal processing and machine learning applications. This problem aims at decomposing a partially observed matrix into the superposition of a low-rank matrix and a sparse matrix, where the sparse matrix captures the grossly corrupted entries of the matrix. A widely used approach to tackle RMC is to consider a convex formulation, which minimizes the nuclear norm of the low-rank matrix (to promote low-rankness) plus the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _1$</tex-math></inline-formula> norm of the sparse matrix (to promote sparsity). In this paper, motivated by some recent works on low-rank matrix completion and Riemannian optimization, we formulate this problem as a nonsmooth Riemannian optimization problem over Grassmann manifold. This new formulation is scalable as the low-rank matrix is factorized to the multiplication of two much smaller matrices. We then propose an alternating manifold proximal gradient continuation method to solve the proposed new formulation. Convergence rate of the proposed algorithm is rigorously analyzed. Numerical results on both synthetic data and real data on background extraction from surveillance videos are reported to demonstrate the advantages of the proposed new formulation and algorithm over several popular existing approaches.

On robust matrix completion with prescribed eigenvalues

Matrix completion is a ubiquitous tool in machine learning and data analysis. Most work in this area has focused on the number of observations necessary to obtain an accurate low-rank approximation. In practice, however, the cost of observations is an important limiting factor, and experimentalists may have on hand multiple modes of observation with differing noise-vs-cost trade-offs. This paper considers matrix completion subject to such constraints: a budget is imposed and the experimentalist's goal is to allocate this budget between two sampling modalities in order to recover an accurate low-rank approximation. Specifically, we consider that it is possible to obtain low noise, high cost observations of individual entries or high noise, low cost observations of entire columns. We introduce a regression-based completion algorithm for this setting and experimentally verify the performance of our approach on both synthetic and real data sets. When the budget is low, our algorithm outperforms standard completion algorithms. When the budget is high, our algorithm has comparable error to standard nuclear norm completion algorithms and requires much less computational effort.

Large graphs are prevalent in many applications and enable a variety of information dissemination processes, e.g., meme, virus, and influence propagation. How can we optimize the underlying graph structure to affect the outcome of such dissemination processes in a desired way (e.g., stop a virus propagation, facilitate the propagation of a piece of good idea, etc)? Existing research suggests that the leading eigenvalue of the underlying graph is the key metric in determining the so-called epidemic threshold for a variety of dissemination models. In this paper, we study the problem of how to optimally place a set of edges (e.g., edge deletion and edge addition) to optimize the leading eigenvalue of the underlying graph, so that we can guide the dissemination process in a desired way. We propose effective, scalable algorithms for edge deletion and edge addition, respectively. In addition, we reveal the intrinsic relationship between edge deletion and node deletion problems. Experimental results validate the effectiveness and efficiency of the proposed algorithms.

This paper presents a new technique for visualizing large, complex collections of data. The size and dimensionality of these datasets make them challenging to display in an effective manner. The images must show the global structure of spatial relationships within the dataset, yet at the same time accurately represent the local detail of each data element being visualized. We propose combining ideas from information and scientific visualization together with a navigation assistant, a software system designed to help users identify and explore areas of interest within their data. The assistant locates data elements of potential importance to the user, clusters them into spatial regions, and builds underlying graph structures to connect the regions and the elements they contain. Graph traversal algorithms, constraint-based viewpoint construction, and intelligent camera planning techniques can then be used to design animated tours of these regions. In this way, the navigation assistant can help users to explore any of the areas of interest within their data. We conclude by demonstrating how our assistant is being used to visualize a multidimensional weather dataset.

Many of the existing data base models have an underlying graph structure. Even certain models based on set theory rather than graph theory are translatable into some graph representation, an important example being the relational model.

BackgroundVisualization concerns the representation of data visually and is an important task in scientific research. Protein-protein interactions (PPI) are discovered using either wet lab techniques, such mass spectrometry, or in silico predictions tools, resulting in large collections of interactions stored in specialized databases. The set of all interactions of an organism forms a protein-protein interaction network (PIN) and is an important tool for studying the behaviour of the cell machinery. Since graphic representation of PINs may highlight important substructures, e.g. protein complexes, visualization is more and more used to study the underlying graph structure of PINs. Although graphs are well known data structures, there are different open problems regarding PINs visualization: the high number of nodes and connections, the heterogeneity of nodes (proteins) and edges (interactions), the possibility to annotate proteins and interactions with biological information extracted by ontologies (e.g. Gene Ontology) that enriches the PINs with semantic information, but complicates their visualization.MethodsIn these last years many software tools for the visualization of PINs have been developed. Initially thought for visualization only, some of them have been successively enriched with new functions for PPI data management and PIN analysis. The paper analyzes the main software tools for PINs visualization considering four main criteria: (i) technology, i.e. availability/license of the software and supported OS (Operating System) platforms; (ii) interoperability, i.e. ability to import/export networks in various formats, ability to export data in a graphic format, extensibility of the system, e.g. through plug-ins; (iii) visualization, i.e. supported layout and rendering algorithms and availability of parallel implementation; (iv) analysis, i.e. availability of network analysis functions, such as clustering or mining of the graph, and the possibility to interact with external databases.ResultsCurrently, many tools are available and it is not easy for the users choosing one of them. Some tools offer sophisticated 2D and 3D network visualization making available many layout algorithms, others tools are more data-oriented and support integration of interaction data coming from different sources and data annotation. Finally, some specialistic tools are dedicated to the analysis of pathways and cellular processes and are oriented toward systems biology studies, where the dynamic aspects of the processes being studied are central.ConclusionA current trend is the deployment of open, extensible visualization tools (e.g. Cytoscape), that may be incrementally enriched by the interactomics community with novel and more powerful functions for PIN analysis, through the development of plug-ins. On the other hand, another emerging trend regards the efficient and parallel implementation of the visualization engine that may provide high interactivity and near real-time response time, as in NAViGaTOR. From a technological point of view, open-source, free and extensible tools, like Cytoscape, guarantee a long term sustainability due to the largeness of the developers and users communities, and provide a great flexibility since new functions are continuously added by the developer community through new plug-ins, but the emerging parallel, often closed-source tools like NAViGaTOR, can offer near real-time response time also in the analysis of very huge PINs.

BackgroundMixed graphical models (MGMs) are graphical models learned over a combination of continuous and discrete variables. Mixed variable types are common in biomedical datasets. MGMs consist of a parameterized joint probability density, which implies a network structure over these heterogeneous variables. The network structure reveals direct associations between the variables and the joint probability density allows one to ask arbitrary probabilistic questions on the data. This information can be used for feature selection, classification and other important tasks.ResultsWe studied the properties of MGM learning and applications of MGMs to high-dimensional data (biological and simulated). Our results show that MGMs reliably uncover the underlying graph structure, and when used for classification, their performance is comparable to popular discriminative methods (lasso regression and support vector machines). We also show that imposing separate sparsity penalties for edges connecting different types of variables significantly improves edge recovery performance. To choose these sparsity parameters, we propose a new efficient model selection method, named Stable Edge-specific Penalty Selection (StEPS). StEPS is an expansion of an earlier method, StARS, to mixed variable types. In terms of edge recovery, StEPS selected MGMs outperform those models selected using standard techniques, including AIC, BIC and cross-validation. In addition, we use a heuristic search that is linear in size of the sparsity value search space as opposed to the cubic grid search required by other model selection methods. We applied our method to clinical and mRNA expression data from the Lung Genomics Research Consortium (LGRC) and the learned MGM correctly recovered connections between the diagnosis of obstructive or interstitial lung disease, two diagnostic breathing tests, and cigarette smoking history. Our model also suggested biologically relevant mRNA markers that are linked to these three clinical variables.ConclusionsMGMs are able to accurately recover dependencies between sets of continuous and discrete variables in both simulated and biomedical datasets. Separation of sparsity penalties by edge type is essential for accurate network edge recovery. Furthermore, our stability based method for model selection determines sparsity parameters faster and more accurately (in terms of edge recovery) than other model selection methods. With the ongoing availability of comprehensive clinical and biomedical datasets, MGMs are expected to become a valuable tool for investigating disease mechanisms and answering an array of critical healthcare questions.

This chapter focuses on the structure-based classification of websites according to their hypertext type or genre. A website usually consists of several web pages. Its structure is given by their hyperlinks resulting in a directed graph. In order to represent the logical structure of a website, the underlying graph structure is represented as a so-called directed Generalized Tree (GT), in which a rooted spanning tree represents the logical core structure of the site. The remaining arcs are classified as reflexive, lateral, and vertical up- and downward arcs with respect to this kernel tree.

To represent complex structure, graph data is widely used in diverse applications. Among various techniques to extract meaningful information in graph data, graph clustering is an important task for the discovery of an underlying graph structure. However, the volume of graph data becomes large and increases fast recently as well as traditional clustering algorithms become computationally expensive as the size of data to be clustered increases. In this paper, we propose an efficient graph clustering algorithm running on Pregel which is one of prominent parallel processing models for large-scale graph data. Graph clustering technique merges the nodes into clusters such that the nodes in a cluster are strongly connected each other. To seek strongly connected nodes efficiently, we utilize Min-Hash which calculates similarity between vertices and/or clusters. In our experimental study, we demonstrate the efficiency and scalability of our parallel algorithm compared to existing algorithms.

The optimal transport problem has recently developed into a powerful framework for various applications in estimation and control. Many of the recent advances in the theory and application of optimal transport are based on regularizing the problem with an entropy term, which connects it to the Schrödinger bridge problem and thus to stochastic optimal control. Moreover, the entropy regularization makes the otherwise computationally demanding optimal transport problem feasible even for large scale settings. This has led to an accelerated development of optimal transport based methods in a broad range of fields. Many of these applications have an underlying graph structure, for instance, information fusion and tracking problems can be described by trees. In this work we consider multimarginal optimal transport problems with a cost function that decouples according to a tree structure. The entropy regularized multimarginal optimal transport problem can be viewed as a generalization of the Schrödinger bridge problem with the same tree-structure, and by utilizing these connections we extend the computational methods for the classical optimal transport problem in order to solve structured multimarginal optimal transport problems in an efficient manner. In particular, the algorithm requires only matrix-vector multiplications of relatively small dimensions. We show that the multimarginal regularization introduces less diffusion, compared to the commonly used pairwise regularization, and is therefore more suitable for many applications. Numerical examples illustrate this, and we finally apply the proposed framework for the tracking of an ensemble of indistinguishable agents.

Network Coding encourages information coding across a communication network. While the necessity, benefit and complexity of network coding are sensitive to the underlying graph structure of a network, existing theory on network coding often treats the network topology as a black box, focusing on algebraic or information theoretic aspects of the problem. This work aims at an in-depth examination of the relation between algebraic coding and network topologies. We mathematically establish a series of results along the direction of: if network coding is necessary/beneficial, or if a particular finite field is required for coding, then the network must have a corresponding hidden structure embedded in its underlying topology, and such embedding is computationally efficient to verify. Specifically, we first formulate a meta-conjecture, the NC-Minor Conjecture, that articulates such a connection between graph theory and network coding, in the language of graph minors. We next prove that the NC-Minor Conjecture is almost equivalent to the Hadwiger Conjecture, which connects graph minors with graph coloring. Such equivalence implies the existence of K4, K5, K6, and KO(q/ log q) minors, for networks requiring F3, F4, F5 and Fq, respectively. We finally prove that network coding can make a difference from routing only if the network contains a K4 minor, and this minor containment result is tight. Practical implications of the above results are discussed.

1. MODULAR ORIGAMI. The traditional Japanese art of paper folding known as origami has become a very popular recreational activity throughout the world, appealing to both children and adults. Over the last century, a number of variations and off-shoots of the traditional art form have appeared, and one of the most interesting of these is known as ‘modular’ origami (or sometimes ‘unit’ origami). This usually involves building large geometric structures from several smaller, individually folded ‘modules’ which are structurally identical and fitted together without adhesive (see [13] for some detailed examples and instructions). Such structures can be built with as few as 6 modules, or as many as 900 (or even 1720 in a model built by the author and a friend during their idle undergraduate years). Models created in this way are often spectacular and decorative, especially when different colors are used for the modules. In trying to find the most decorative ways of using different colors, one is confronted with some interesting questions not only in aesthetics, but also in mathematics. For instance, there are a couple of coloring conditions which usually produce spectacular models: (i) no two modules of the same color are joined; (ii) the distribution of colors over the model is ‘symmetric’ or ‘regular’ in some sense. These conditions give rise to questions in graph theory, since it turns out that most modular origami models have an underlying graph structure: a module can be thought of as an edge, and a point at which two or more modules meet can be thought of as a vertex. These underlying graphs often correspond to geometric solids, with the graph of the dodecahedron arising especially frequently. This graph-theoretic model gives an alternative way of understanding the above conditions: a coloring satisfying condition (i) corresponds to an edge-coloring of the underlying graph, or in other words an edge-decomposition in which each subgraph has valency 1 (this means that in each subgraph, each vertex is incident with exactly one edge, and so the subgraph consists of disjoint edges). The problem of satisfying condition (ii) can be solved by finding a transitive decomposition of the underlying graph (we introduce transitive decompositions in the next section). A coloring satisfying either one of these conditions will usually yield an attractive model; however, it is sometimes possible to find colorings which satisfy both. This involves finding a transitive decomposition in which each subgraph has valency 1; we call this a transitive 1-decomposition. In Section 3.1 we construct a coloring based on such a transitive 1-decomposition. 1.1. Transitive Decompositions of Graphs. The essential idea of a transitive decomposition of a graph has been around for a long time (since the 1970s), and has appeared in a number of different guises (for instance, in [3, 6, 9, 10, 11, 12]); but only recently has it been studied extensively in its own right. Any graph can be broken down into

Motoo's combinatorial central limit theorem for serial rank statistics