The complexity of data exchange

Schema mappings are high-level specifications that describe the relationship between two database schemas. They are considered to be the essential building blocks in data exchange and data integration, and have been the focus of extensive research investigations. Since in real-life applications schema mappings can be quite complex, it is important to develop methods and tools for illustrating, explaining, and deriving schema mappings. A promising approach to this effect is to use "good" data examples that illustrate the schema mapping at hand.

Data exchange is the problem of transforming data structured according to a source schema into data structured according to a target schema, via a mapping specified by rules in the form of source-to-target tuple generating dependencies. In this context, given a source instance and a mapping, there might be more than one valid target instance that satisfies the mapping. This issue contradicts the main goal of exchanging data, namely to have a materialised target instance that can be used to answer queries over the target schema without reference to the original source instance. In this paper we introduce and solve the novel problem of definability abduction, which aims at finding extensions to the initial schema mappings to guarantee the uniqueness of the materialised target instance. We consider several semantic criteria to select reasonable extensions and provide provably sound and complete algorithms to generate these additions. We also do a complexity analysis in different data exchange settings, also with source and target dependencies.KeywordsData ExchangeSchema MappingConjunctive QueryQuery AnsweringTarget DatabaseThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This paper presents a schema mapping technique for XML-based geodata translation/transformation. XML is becoming the universal language for data exchange over the Internet. For geodata exchange, OGC defined the XML-based GML(Geographical Markup Language). Defining XML-based data format alone is not enough for data exchange. Associated techniques are equally important. The proposed schema mapping technique enables to a large extent automatic generation of geodata translator/convertor. Given a pair of source and target data schemas, a user specifies elements correspondence between them in a declarative way. The proposed schema mapping processor takes the schema mapping specification as input and generate geodata translator as output. Due to the automatic generation, semantic reconciliation between source and target schemas is very much facilitated.

The task of combining data residing at different sources to provide the user a unified view is known as data integration. Schema mappings act as glue between the global schema and the source schemas of a data integration system. Global-and-local-as-view (GLAV) is one the approaches for specifying the schema mappings. Tableaux are used for expressing queries and functional dependencies on a single database. We investigate a general technique for expressing a GLAV mapping by a tabular structure called mapping assertion tableaux (MAT). In a similar way, we also express the tuple generating dependency (tgd) and equality generating dependency (egd) constraints by tabular forms, called tabular tgd (TTGD) and tabular egd (TEGD), respectively. A set consisting of the MATs, TTGDs and TEGDs are called schema mapping tableaux (SMT). We present algorithms that use SMT as operator on an instance of the source schema to produce an instance of the target schema. We show that the target instances computed by the SMT are `minimal' and `most general' in nature. We also define the notion of equivalence between the schema mappings of two data integration systems and present algorithms that optimize schema mappings through the manipulation of the SMT.

A schema mapping is a high-level declarative specification of how data structured under one schema, called the source schema, is to be transformed into data structured under a possibly different schema, called the target schema. We demonstrate SPIDER, a prototype tool for debugging schema mappings, where the language for specifying schema mappings is based on a widely adopted formalism. We have built SPIDER on top of a data exchange system, Clio, from IBM Almaden Research Center. At the heart of SPIDER is a data-driven facility for understanding a schema mapping through the display of routes. A route essentially describes the relationship between source and target data with the schema mapping. In this demonstration, we showcase our route engine, where we can display one or all routes starting from either source or target data, as well as the intermediary data and schema elements involved. In addition, we demonstrate standard debugging features for schema mappings that we have also built, such as computing and exploring routes step-by-step, stopping or pausing the computation with breakpoints, performing guided computation of routes by taking human input into account, as well as tracking the state of the target instance during the process of computing routes.

Approximate top-K answering under uncertain schema mappings

Relational schema mappings have been extensively studied in connection with data integration and exchange problems, but mappings between XML schemas have not received the same amount of attention. Our goal is to develop a theory of expressive XML schema mappings. Such mappings should be able to use various forms of navigation in a document, and specify conditions on data values. We develop a language for XML schema mappings, and concentrate on three types of problems: static analysis of mappings, their complexity, and their composition. We look at static analysis problems related to various flavors of consistency: for example, whether it is possible to map some document of a source schema into a document of the target schema, or whether all documents of a source schema can be mapped. We classify the complexity of these problems. We then move to the complexity of mappings themselves, i.e., recognizing pairs of documents such that one can be mapped into the other, and provide a classification based on sets of features used in mappings. Finally we look at composition of XML schema mappings. We study its complexity and show that it is harder to achieve closure under composition for XML than for relational mappings. Nevertheless, we find a robust class of XML schema mappings that have good complexity properties and are closed under composition.

Recently, two relaxed notions of equivalence of schema mappings have been introduced, which provide more potential of optimizing schema mappings than logical equivalence: data exchange (DE) equivalence and conjunctive query (CQ) equivalence. In this work, we systematically investigate these notions of equivalence for mappings consisting of s-t tgds and target egds and/or target tgds. We prove that both CQ- and DE-equivalence are undecidable and so are some important optimization tasks (like detecting if some dependency is redundant). However, we also identify an important difference between the two notions of equivalence: CQ-equivalence remains undecidable even if the schema mappings consist of s-t tgds and target dependencies in the form of key dependencies only. In contrast, DE-equivalence is decidable for schema mappings with s-t tgds and target dependencies in the form of functional and inclusion dependencies with terminating chase property.

Recently, two relaxed notions of equivalence of schema mappings have been introduced, which provide more potential of optimizing schema mappings than logical equivalence: data exchange (DE) equivalence and conjunctive query (CQ) equivalence. In this work, we systematically investigate these notions of equivalence for mappings consisting of s-t tgds and target egds and/or target tgds. We prove that both CQ- and DE-equivalence are undecidable and so are some important optimization tasks (like detecting if some dependency is redundant). However, we also identify an important difference between the two notions of equivalence: CQ-equivalence remains undecidable even if the schema mappings consist of s-t tgds and target dependencies in the form of key dependencies only. In contrast, DE-equivalence is decidable for schema mappings with s-t tgds and target dependencies in the form of functional and inclusion dependencies with terminating chase property.

Data exchange deals with inserting data from one database into another database having a different schema. Fagin et al. [2005] have shown that among the universal solutions of a solvable data exchange problem, there exists—up to isomorphism—a unique most compact one, “the core”, and have convincingly argued that this core should be the database to be materialized. They stated as an important open problem whether the core can be computed in polynomial time in the general setting where the mapping between the source and target schemas is given by source-to-target constraints that are arbitrary tuple generating dependencies (tgds) and target constraints consisting of equality generating dependencies (egds) and a weakly acyclic set of tgds. In this article, we solve this problem by developing new methods for efficiently computing the core of a universal solution. This positive result shows that data exchange based on cores is feasible and applicable in a very general setting. In addition to our main result, we use the method of hypertree decompositions to derive new algorithms and upper bounds for query containment checking and computing cores of arbitrary database instances. We also show that computing the core of a data exchange problem is fixed-parameter intractable with respect to a number of relevant parameters, and that computing cores is NP-complete if the rule bodies of target tgds are augmented by a special predicate that distinguishes a null value from a constant data value.

Scene classification is usually based on supervised learning, but collecting instances is expensive and time-consuming. TrAdaBoost has achieved great success in transferring source instances to target images, but it has problems, such as the excessive focus on instances harder to classify, the rapid convergence speed of the source instances, and the weight mismatch caused by the big gap between the number of source and target instances, leading to decreased classification accuracy. In this paper, in order to address these problems, classical particle swarm optimization (PSO) is modified to select the optimal feature subspace for classifying the “harder” and “easier” instances by reducing unimportant dimensions. A modified correction factor is proposed by considering the classification accuracy of the instances from both domains, to decrease the convergence speed. Iterative selective TrAdaBoost is also proposed to reduce the weight mismatch by removing the indiscriminate source instances. The experimental results obtained with three benchmark data sets confirm that the proposed method outperforms most of the previous methods of scene classification with limited target samples. It is also proved that modified PSO for optimal feature subspace selection, the modified correction factor, and iterative selective TrAdaBoost are all effective in improving the classification accuracy, giving improvements of 3.6%, 4.3%, and 2.7%, and these three contributions together increase the classification accuracy by about 8% in total.

Schema mapping is becoming pervasive in all data transformation, exchange, and integration tasks. It brings to the surface the problem of differences and mismatches between heterogeneous formats and models, respectively, used in source and target databases to be mapped one to another. In this chapter, we start by describing the problem of schema mapping, its background, and technical implications. Then, we outline the early schema mapping systems, along with the new generation of schema mapping tools. Moving from the former to the latter entailed a dramatic change in the performance of mapping generation algorithms. Finally, we conclude the chapter by revisiting the query answering techniques allowed by the mappings, and by discussing useful applications and future and current developments of schema mapping tools.

The Clio project provides tools that vastly simplify information integration. Information integration requires data conversions to bring data in different representations into a common form. Key contributions of Clio are the definition of non-procedural schema mappings to describe the relationship between data in heterogeneous schemas, a new paradigm in which we view the mapping creation process as one of query discovery, and algorithms for automatically generating queries for data transformation from the mappings. Clio provides algorithms to address the needs of two major information integration problems, namely, data integration and data exchange. In this chapter, we present our algorithms for both schema mapping creation via query discovery, and for query generation for data exchange. These algorithms can be used in pure relational, pure XML, nested relational, or mixed relational and nested contexts.

With the popularity of the internet, more and more data are generated on internet. Because of the usability of Extensible Markup Language(XML for short), more data is organized by XML document format. Because of the flexibility of XML, data organized by XML have a variety of organizational formats which brings a lot of inconvenience to data management. In particular, when the large-scale data operations are performed on XML data, for example data integration, model change, and so on, there are many problems. One of the current implementations is to use Data Exchange to carry out the above operations. The works of predecessors mainly are to analyze the characteristics of Schema Mapping on XML, and institute Data Exchange rules. These rules only consider the data integrity, reliability, but don't consider the quality of the data after conversion. This paper proposes the concept of quality assurance mechanisms. Firstly we discuss that a new model with quality assurance, and provide a suitable method for this model. Then we propose the strategy of weak branch's convergence on the basis of Schema. In the end theoretical analysis and experimental results show that the method is correct and feasible.

Recent results in schema-mapping and data-exchange research may be considered the starting point for a new generation of systems, capable of dealing with a significantly larger class of applications. In this paper we demonstrate the first of these second-generation systems, called ++Spicy. We introduce a number of scenarios from a variety of data management tasks, such as data fusion, data cleaning, and ETL, and show how, based on the system, schema mappings and data exchange techniques can be very effectively applied to these contexts. We compare ++Spicy to the previous generations of tools, to show that this is much-needed advancement in the field.