A dynamical systems approach to weighted graph matching

Graph and hypergraph matching are fundamental problems in computer vision, which have been receiving a steadily growing attention in the last two decades due to the recent modern algorithms and models that are both fast and accurate. Graph and hypergraph matching are successful in tackling many real-world tasks that require 2D and 3D feature matching and alignment. Matching is a general problem in vision and graphs have always been quintessential for modeling information that has both local and longer range connections. Therefore, graph matching will continue to influence computer vision and robotics research, crossing over many different mathematical and computational models, including Bayesian networks, conditional and Markov random fields, spectral graph models, and, more recently, deep neural networks. While graph matching is limited to using pairwise relationships, hypergraph matching permits the use of relationships between sets of features of any order. Consequently, it carries the promise to make matching more robust to changes in scale, deformations, and outliers. In this chapter, we present several key ideas, computational models, and optimization methods that introduced in the literature the possibility to learn graph and hypergraph matching in a semi-supervised or even completely unsupervised fashion. We start by describing two methods for solving graph and hypergraph matching based on spectral matching and integer projected fixed point methods, introduced in Chap. 1. Then, based on the same intuition (also discussed in Chap. 1) that correct assignments form a strong cluster of agreements, we follow up by introducing the main approach to learning as it applies to graph and hypergraph matching, as well as the more general Maximum A Posteriori (MAP) inference problem in probabilistic graphical models (MRFs and CRFs). In the last part of the chapter, we demonstrate experimentally the effectiveness of the algorithms presented and, whenever possible, provide extensive comparisons to other top published methods. Moreover, we show that our learning approach is general and can significantly boost the performance of different state-of-the-art matching algorithms in the literature.

A graph can be exactly specified by the spectrum and corresponding eigenvectors of its adjacency matrix. This provides a solid foundation for spectrum based graph matching. However, most previous methods ignore the spectral multiplicity, which may significantly affect the matching accuracy. In this paper, we address the problem of minimizing the matching error when graph spectral multiplicity is involved. We first model spectral multiplicity by the sub-eigenspace rotation matrix R, and integrate R into the spectrum based graph matching model. We then focus on the exact graph matching problem, and show how to establish the vertex-to-vertex correspondence by iteratively optimizing the sub-eigenspace rotation matrix R and the permutation matrix P. A reliable matching initialization method is proposed to make this process converge rapidly. Finally, we extend the approach to the inexact graph matching problem by optimally warping two graphs to the same size. The proposed approach is robust and efficient. We support our approach with numerical experiments and demonstrate its effectiveness in the practical application of uncalibrated stereo matching.

Image registration is an important problem for image processing and computer vision with many proposed applications in medical image analysis.^{1, 2} Image registration techniques attempt to map corresponding features between two images. The problem is particularly difficult as anatomy is subject to elastic deformations. This paper considers this problem in the context of graph matching. Firstly, weighted Region Adjacency Graphs (RAGs) are constructed from each image using an approach based on watershed saliency. ³ The vertices of the RAG represent salient regions in the image and the (weighted) edges represent the relationship (bonding) between each region. Correspondences between images are then determined using a weighted graph matching method. Graph matching is considered to be one of the most complex problems in computer vision, due to its combinatorial nature. Our approach uses a multi-spectral technique to graph matching first proposed by Umeyama⁴ to find an approximate solution to the weighted graph matching problem (WGMP) based on the singular value decomposition of the adjacency matrix. Results show the technique is successful in co-registering 2-D MRI images and the method could be useful in co-registering 3-D volumetric data (e.g. CT, MRI, SPECT, PET etc.).

We propose a systematic approach for registering cross-source point clouds that come from different kinds of sensors. This task is especially challenging due to the presence of significant missing data, large variations in point density, scale difference, large proportion of noise, and outliers. The robustness of the method is attributed to the extraction of macro and micro structures. Macro structure is the overall structure that maintains similar geometric layout in cross-source point clouds. Micro structure is the element (e.g., local segment) being used to build the macro structure. We use graph to organize these structures and convert the registration into graph matching. With a novel proposed descriptor, we conduct the graph matching in a discriminative feature space. The graph matching problem is solved by an improved graph matching solution, which considers global geometrical constraints. Robust cross source registration results are obtained by incorporating graph matching outcome with RANSAC and ICP refinements. Compared with eight state-of-the-art registration algorithms, the proposed method invariably outperforms on Pisa Cathedral and other challenging cases. In order to compare quantitatively, we propose two challenging cross-source data sets and conduct comparative experiments on more than 27 cases, and the results show we obtain much better performance than other methods. The proposed method also shows high accuracy in same-source data sets.

Graph matching, set covering and partitioning problems are both theoretically and practically important in decision support systems. Numerous situations have been modeled as graph matching, set covering and partitioning problems. These situations include applications in the areas of task assignment, marriage problem, airline crew scheduling, truck deliveries or vehicle routing, political redistricting, etc. Relational database systems currently support these applications merely by providing some programming languages which are used to write the programs associated with the data. These programming languages are complete as opposed to existing query languages like SQL. These languages, since they are complete, have the usual complexity of a powerful language and lack the simplicity of query languages. This increases application development effort. Relational operators and the query language SQL are extended in this present work in order to enable these problems to be stated and solved directly by the relational database systems leading to benefits of improved data independence, increased productivity, and better performance. The relational operators are extended with six operators, i.e., match, maxmatch, cover, mincover, partition, and minpartition. Properties of these operators are described. Some of these operators may possibly take a very long time in finding a solution. Genetic algorithms are proposed. These algorithms are bounded by a polynomial time. These algorithms therefore enable DBMS to be capable of responding to queries involving proposed operators in a given time constraint. The database operations and query languages extended in this research provide a powerful tool for users as a database‐query tool which supports some decision support systems for information retrieval from a database.

Quadratic assignment problems arise in a wide variety of domains, spanning operations research, graph theory, computer vision, and neuroscience, to name a few. The graph matching problem is a special case of the quadratic assignment problem, and graph matching is increasingly important as graph-valued data is becoming more prominent. With the aim of efficiently and accurately matching the large graphs common in big data, we present our graph matching algorithm, the Fast Approximate Quadratic assignment algorithm. We empirically demonstrate that our algorithm is faster and achieves a lower objective value on over 80% of the QAPLIB benchmark library, compared with the previous state-of-the-art. Applying our algorithm to our motivating example, matching C. elegans connectomes (brain-graphs), we find that it efficiently achieves performance.

As a fundamental problem in pattern recognition, graph matching has applications in a variety of fields, from computer vision to computational biology. In graph matching, patterns are modeled as graphs and pattern recognition amounts to finding a correspondence between the nodes of different graphs. Many formulations of this problem can be cast in general as a quadratic assignment problem, where a linear term in the objective function encodes node compatibility and a quadratic term encodes edge compatibility. The main research focus in this theme is about designing efficient algorithms for approximately solving the quadratic assignment problem, since it is NP-hard. In this paper we turn our attention to a different question: how to estimate compatibility functions such that the solution of the resulting graph matching problem best matches the expected solution that a human would manually provide. We present a method for learning graph matching: the training examples are pairs of graphs and the 'labels' are matches between them. Our experimental results reveal that learning can substantially improve the performance of standard graph matching algorithms. In particular, we find that simple linear assignment with such a learning scheme outperforms Graduated Assignment with bistochastic normalisation, a state-of-the-art quadratic assignment relaxation algorithm.

In this paper, matching pairs of stocahstically generated graphs in the presence of generalized seed side-information is considered. The graph matching problem emerges naturally in various applications such as social network de-anonymization, image processing, DNA sequencing, and natural language processing. A pair of randomly generated labeled Erdös-Rényi graphs with pairwise correlated edges are considered. It is assumed that the matching strategy has access to the labeling of the vertices in the first graph, as well as a collection of shortlists — called ambiguity sets — of possible labels for the vertices of the second graph. The objective is to leverage the correlation among the edges of the graphs along with the side-information provided in the form of ambiguity sets to recover the labels of the vertices in the second graph. This scenario can be viewed as a generalization of the seeded graph matching problem, where the ambiguity sets take a specific form such that the exact labels for a subset of vertices in the second graph are known prior to matching. A matching strategy is proposed which operates by evaluating the joint typicality of the adjacency matrices of the graphs. Sufficient conditions on the edge statistics as well as ambiguity set statistics are derived under which the proposed matching strategy successfully recovers the labels of the vertices in the second graph. Additionally, Fano-type arguments are used to derive necessary conditions for successful seeded graph matching.

A graph matching algorithm based on concavely regularized convex relaxation

This paper addresses the multi-attributed graph matching problem considering multiple attributes jointly while preserving the characteristics of each attribute. Since most of conventional graph matching algorithms integrate multiple attributes to construct a single attribute in an oversimplified way, the information from multiple attributes are not often fully exploited. In order to solve this problem, we propose a novel multi-layer graph structure that can preserve the particularities of each attribute in separated layers. Then, we also propose a multi-attributed graph matching algorithm based on the random walk centrality for the proposed multi-layer graph structure. We compare the proposed algorithm with other state-of-the-art graph matching algorithms based on the single-layer structure using synthetic and real datasets, and prove the superior performance of the proposed multi-layer graph structure and matching algorithm.

We focus on the problem of graph matching that is fundamental in computer vision and machine learning. Many state-of-the-arts frequently formulate it as integer quadratic programming, which incorporates both unary and second-order terms. This formulation is in general NP-hard thus obtaining an exact solution is computationally intractable. Therefore most algorithms seek the approximate optimum by relaxing techniques. This paper commences with the finding of the “circular” character of solution chain obtained by the iterative Gradient Assignment (via Hungarian method) in the discrete domain, and proposes a method for guiding the solver converging to a fixed point, resulting a convergent algorithm for graph matching in discrete domain. Furthermore, we extend the algorithms to their counterparts in continuous domain, proving the classical graduated assignment algorithm will converge to a double-circular solution chain, and the proposed Soft Constrained Graduated Assignment (SCGA) method will converge to a fixed (discrete) point, both under wild conditions. Competitive performances are reported in both synthetic and real experiments.

Optimization problems over permutation matrices appear widely in facility layout, chip design, scheduling, pattern recognition, computer vision, graph matching, etc. Since this problem is NP-hard due to the combinatorial nature of permutation matrices, we relax the variable to be the more tractable doubly stochastic matrices and add an $L_p$-norm ($ 0 < p < 1$) regularization term to the objective function. The optimal solutions of the $L_p$-regularized problem are the same as the original problem if the regularization parameter is sufficiently large. A lower bound estimation of the nonzero entries of the stationary points and some connections between the local minimizers and the permutation matrices are further established. Then we propose an $L_p$ regularization algorithm with local refinements. The algorithm approximately solves a sequence of $L_p$ regularization subproblems by the projected gradient method using a nonmonotone line search with the Barzilai--Borwein step sizes. Its performance can be fu...

Graph matching, as one of the most fundamental problems in graph database, has a wide range of applications. Due to the large scale of graph database and the hardness of graph matching, it is an attractive alternative to make use of the cloud to store massive data graphs and conduct the complex graph matching. To protect the privacy, data graphs are usually encrypted before being outsourced to the cloud. A few schemes have been proposed to support graph matching query over encrypted graphs. However, none of them can realize efficient subgraph extraction when the matched subgraph needs to be exactly located at the data graph. The graph user has to perform the complex subgraph isomorphism operation to extract the isomorphic subgraph from the matched data graph in state-of-the-art schemes. The time complexity of judging subgraph isomorphism is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$O(n!n^{2})$</tex-math></inline-formula> at most, where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$n$</tex-math></inline-formula> is the number of vertices in the data graph. In order to solve this problem, we propose a privacy-preserving graph matching query scheme supporting quick subgraph extraction in this paper. In our design, two non-colluding cloud servers are adopted to accomplish the matching operation jointly. Neither of them can infer the plaintexts of graphs. The first cloud server can prune some impossible matched vertices from the data graph, and get a possible matched matrix to represent which vertices in the data graph might match with the vertices in the query graph. The second cloud server performs the related matrix operations based on its stored information and the information sent by the first cloud server. The two cloud servers jointly verify the correctness of the matched matrix. The graph user can directly and quickly extract the matrix of the subgraph isomorphic to the query graph from the data graph matrix based on the matched matrix, and further recover this subgraph. No subgraph isomorphism operation is involved in the procedure of subgraph extraction for the graph user. The time complexity of subgraph extraction is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$O(m^{2})$</tex-math></inline-formula> in our scheme, where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$m$</tex-math></inline-formula> is the number of vertices in the query graph. The extensive experiments with real-world database demonstrate the efficiency of the proposed privacy-preserving graph matching scheme.

The most reliable method for solving linear equations of the least squares principle, which can be used to solve incorrect geodetic problems, is based on matrix factorization, which is called a singular expansion. Some other methods require less machine time and memory. However, they are less effective in taking into account the errors of the source information, rounding errors, and linear dependence. The methodology of such research is that for any matrix A and any two orthogonal matrices U and V, there is a matrix , which is determined as T U AV  . The idea of a singular decomposition is that by choosing the right matrices U and V, you can convert most elements of the matrix  to zero and make it diagonal with non-negative elements. The novelty and relevance of scientific solutions lie in the feasibility of using a singular decomposition of the matrix to obtain linear equations of the least squares method, which can be used to solve incorrect geodetic problems. The purpose of scientific research is to obtain a stable solution of parametric equations of corrections to the results of mea- surements in incorrect geodetic problems. Based on the performed research on the application of the singular decomposition method for solving incorrect geodetic problems, we can summarize the following results. A singular expansion of a real matrix A is any factorization T A U W  of a matrix with orthogonal columns U, an orthogonal matrix W , and a diagonal matrix , the elements of which are called singular numbers of the matrix A, and the columns of matrices U and W — left and right singular vectors. If the matrix A has a full rank, then its solution will be unique and stable, which can be obtained by different methods. However, the method of singular decomposition, in contrast to other methods, makes it possible to solve problems with incomplete rank. Research shows that the method of solving normal equations by sequential exclusion of unknowns (Gaussian method), which is quite common in geod- esy, does not provide stable solutions for poorly conditioned or incorrect geodetic problems. Therefore, in the case of unstable systems of equations, it is proposed to use the method of singular matrix decomposition, which in computational mathematics is called SVD. The SVD singular decomposition method makes it possible to obtain stable solutions to both stable and by nature unstable problems. This possibility to solve incorrect geodetic problems is associated with the application of some limit , the choice of which can be made by the relative errors of the matrix of coefficients of parametric equations of corrections A and the vector of results of geodetic measurements L . Moreover, the solution of the system of normal equations obtained by the SVD method will have the shortest length. Thus, applying the apparatus of the singular decomposition of the matrix of coefficients of parametric equations of correc- tions to the results of geodetic measurements, we obtained new formulas for estimating the accuracy of the least squares method in solving incorrect geodetic problems. The derived formulas have a compact form and allow the easy calculation of elements  and XQ estimates of accuracy, almost ignoring the complex procedure of rotation of the matrix of coefficients of normal equations.

Graph matching problem that incorporates pair-wise constraints can be formulated as Quadratic Assignment Problem(QAP). The optimal solution of QAP is discrete and combinational, which makes QAP problem NP-hard. Thus, many algorithms have been proposed to find approximate solutions. In this paper, we propose a new algorithm, called Nonnegative Orthogonal Graph Matching (NOGM), for QAP matching problem. NOGM is motivated by our new observation that the discrete mapping constraint of QAP can be equivalently encoded by a nonnegative orthogonal constraint which is much easier to implement computationally. Based on this observation, we develop an effective multiplicative update algorithm to solve NOGM and thus can find an effective approximate solution for QAP problem. Comparing with many traditional continuous methods which usually obtain continuous solutions and should be further discretized, NOGM can obtain a sparse solution and thus incorporates the desirable discrete constraint naturally in its optimization. Promising experimental results demonstrate benefits of NOGM algorithm.