Using Conditional Functional Dependency to Discover Abnormal Data in RDF Graphs

The representation and the exchange of information originating from different data sources is an increasingly common need for companies and industries to integrate their operations and also to publish and trade information with government and other enterprises. For this purpose, there are many standards based on XML language that were created to allow effective data communication and exchange in a particular domain. In order to ensure data quality for XML data, this paper presents an approach based on conditional functional dependencies verification. Conditional dependencies are an extension of traditional database dependencies with the ability to enforce bindings of semantically related data values. The basis of our verification method is a generic grammarware for validating XML integrity constraints in one tree traversal. We use an attribute grammar to describe XML documents and constraints. Keywords: conditional functional dependencies, data quality, XML integrity constraints.

A RDF graph is typically stored in XML file or relational database. However, when it becomes a large RDF graph, an alternative way to handle the storing and query RDF graph or linked data is to use MapReduce algorithm and Hadoop framework. In this paper, we propose a supporting tool to perform data transfer and query on big RDF graph. We intend to reduce the access time and query response time by using Hadoop Framework. The RDF/XML or linked data is converted into a huge set of N-triples and they are uploaded onto Hadoop storing in data nodes of Hadoop Distributed File System (HDFS). The query of RDF graph in terms of SPARQL is analyzed and converted into a specific N-triple format as to search the answer using Jena Algebra. The MapReduce algorithm is developed to relevantly manipulate the RDF graph.

To help users get familiar with large RDF graphs, RDF summarization techniques can be used. In this work, we study quotient summaries of RDF graphs, that is: graph summaries derived from a notion of equivalence among RDF graph nodes. We make the following contributions: (i) four novel summaries which are often small and easy-to-comprehend, in the style of E–R diagrams; (ii) efficient (amortized linear-time) algorithms for computing these summaries either from scratch, or incrementally, reflecting additions to the graph; (iii) the first formal study of the interplay between RDF graph saturation in the presence of an RDFS ontology, and summarization; we provide a sufficient condition for a highly efficient shortcut method to build the quotient summary of a graph without saturating it; (iv) formal results establishing the shortcut conditions for some of our summaries and others from the literature; (v) experimental validations of our claim within a tool available online.

Many techniques for cleaning dirty data are based on enforcing some set of integrity constraints. Conditional functional dependencies (CFDs) are a combination of traditional Functional dependencies (FDs) and association rules, and are widely used as a constraint formalism for data cleaning. However, the discovery of such CFDs has received limited attention. In this paper, we regard CFDs as an extension of association rules, and present three general methodologies for (approximate) CFD discovery, each using a different way of combining pattern mining for discovering the conditions (the “C” in CFD) with FD discovery. We discuss how existing algorithms fit into these three methodologies, and introduce new techniques to improve the discovery process. We show that the right choice of methodology improves performance over the traditional CFD discovery method CTane. Code related to this paper is available at: https://github.com/j-r77/cfddiscovery, https://codeocean.com/2018/06/20/discovering-conditional-functional-dependencies/code.

The structure of the ever increasing large RDF repositories is too complex to allow non-expert users extract useful information from them. Keyword search is an interesting alternative but in the context of RDF graph data, where query answers are RDF graph fragments, itfaces two major problems: the query quality answer problem and the result computation algorithm scalability problem. In this paper we focus on empowering keyword search on RDF data by exploiting personalized information. We proposean original approach which exploits the structural summary of the RDF graph to generate pattern graphs for the input keyword query. Pattern graphs are structured conjunctive queries and are seen as possible interpretations of the unstructured keyword query. Personalized information is represented as collections of profile graphs, a concept similar to pattern graphs. The ran king of the results is achieved by measuring graph similarity between the user profile graph and the generated pattern graphs. Novel similarity metrics have been introduced which consider intrinsic and extrinsic similarity and take into account both structural and semantic characteristics of the pattern and profile graphs. Effectiveness and efficiency experimental results show that our approach can tackle the two major problems that hinder the widespread use of keyword search on RDF data.

RDF and RDFS have recently become very popular as frameworks for representing data and meta-data in form of a domain description, respectively. RDF data can also be thought of as graph data. In this paper, we focus on keyword-based querying of RDF data. In the existing approaches for answering such keyword queries, keywords are mapped to nodes in the graph and their neighborhoods are explored to extract subgraph(s) of the data graph that contain(s) information relevant to the query. In order to restrict the computational effort, a fixed distance bound is used to define the neighborhoods of nodes. In this paper we present an elegant algorithm for keyword query processing on RDF data that does not assume such a fixed bound. The approach adopts a pruned exploration mechanism where closely related nodes are identified, subgraphs are pruned and joined using suitable hook nodes. The system dynamically manages the distance depending on the closeness between the keywords. The working of the algorithm is illustrated using a fragment of AIFB institute data represented as an RDF graph.

Several anonymization techniques, such as generalization and bucketization, have been designed for privacy preserving microdata publishing. Recently it has become major importance in research areas and numerous approaches have been proposed to preserve privacy in literature. In order to solve this issue, recent work study the problem of privacy preservation hazard for mined Conditional Functional Dependency (CFD) against (d,l) inference model using Compact frequent pattern growth branch sort algorithm (CFPGBS) methods. Boyce–Codd normal Form (BCNF) method is presented for publishing multiple subtables and it is anonymized through (d, l)- inference model. But the major problem of the work is that initial partitioning is done without considering the importance of the feature, so it contains all quasi identifier features. It becomes more complex to perform initial partitioning if the data samples becomes large,thus may also reduce the privacy results. To overcome this issue in this paper presents a hybrid feature selection method which removes unimportant quasi identifier in the final Boyce–Codd normal Form (BCNF) table. The proposed hybrid feature selection method follows the procedure of hybrid firefly algorithm with harmony search (HFAHS) algorithm to select optimal feature in the table for initial partition of top down approach. For selected features expectation Maximization (EM) based Gaussian mixture distribution (EM-GMM) based top down initial partition is done to group the partition data from the result d,l -inference model along with changed publishing data from BCNF. Experimental results show that the proposed BCNF- d,l -inference model can adeptly anonymize the microdata with less information loss when compared to existing methods against CFD. The effectiveness and privacy results of proposed methods are also compared with existing conventional methods. Our workload experiments confirm that proposed preserves better utility, privacy than d,l -inference model against CFD attack and is more effective in workloads with selection of optimal quasi identifiers.

Smart online university party building system based on association rules data mining is designed and shown in the paper. According to the need of data patching, the association rules related to the target attribute will be searched by reasonably reducing the value of support, that is, find the corresponding conditional functional dependencies. For enhancing the data robustness, the neural model is applied into the training stage of the model. Furthermore, the system is implemented through the platform design. The user interface and the experience are jointly improved through the test.

Conditional functional dependencies (CFDs) have recently been proposed as a useful integrity constraint to summarize data semantics and identify data inconsistencies. A CFD augments a functional dependency (FD) with a pattern tableau that defines the context (i.e., the subset of tuples) in which the underlying FD holds. While many aspects of CFDs have been studied, including static analysis and detecting and repairing violations, there has not been prior work on generating pattern tableaux, which is critical to realize the full potential of CFDs. This paper is the first to formally characterize a "good" pattern tableau, based on naturally desirable properties of support, confidence and parsimony. We show that the problem of generating an optimal tableau for a given FD is NP-complete but can be approximated in polynomial time via a greedy algorithm. For large data sets, we propose an "on-demand" algorithm providing the same approximation bound, that outperforms the basic greedy algorithm in running time by an order of magnitude. For ordered attributes, we propose the range tableau as a generalization of a pattern tableau, which can achieve even more parsimony. The effectiveness and efficiency of our techniques are experimentally demonstrated on real data.

A variety of data dependencies have been proposed for data cleaning, including conditional functional dependencies, editing rules and so on. Fixing rule is a newly proposed class of data dependencies for data repairing with high accuracy. However, to our knowledge, algorithms for automatically designing fixing rules have not been developed. In this paper, a workflow of generating fixing rules has been proposed and an efficient algorithm for automatically producing fixing rules from constant CFDs has been developed. The algorithm has been proved to have the ability to avoid generating conflict FRs, and experiments show that the algorithm scales well with the size of input and can output a consistent set of FRs with fairly good usability.

Data cleaning is an important problem and data quality rules are the most promising way to face it with a declarative approach. Previous work has focused on specific formalisms, such as functional dependencies (FDs), conditional functional dependencies (CFDs), and matching dependencies (MDs), and those have always been studied in isolation. Moreover, such techniques are usually applied in a pipeline or interleaved. In this work we tackle the problem in a novel, unified framework. First, we let users specify quality rules using denial constraints with ad-hoc predicates. This language subsumes existing formalisms and can express rules involving numerical values, with predicates such as “greater than” and “less than”. More importantly, we exploit the interaction of the heterogeneous constraints by encoding them in a conflict hypergraph. Such holistic view of the conflicts is the starting point for a novel definition of repair context which allows us to compute automatically repairs of better quality w.r.t. previous approaches in the literature. Experimental results on real datasets show that the holistic approach outperforms previous algorithms in terms of quality and efficiency of the repair.

This paper proposes a method on solving the problem of dealing with dirty data in the database. Considering the complexity of the structure of the data, based on the previous methods that work on this problem, our method combines the methods that use regular expression and methods that use conditional functional dependencies, to complete the data quality improvement. This method uses dependencies to improve the repairing speed and the searching time on the data. The repairing based on the regular expression is regular while there exist questions that the repairing efficient is influenced by the amount of data. When dealing with the database from company Standard Solution Group (SSG) which is from the reality world data, we have tried other related methods and inspired by these methods, we propose this method. The experiments on the data from SSG shows that this method is much efficient.

RDF data is complex; exploring it is hard, and can be done through many different metaphors. We have developed and propose to demonstrate Spade, a tool helping users discover meaningful content of an RDF graph by showing them the results of aggregation (OLAP-style) queries automatically identified from the data. Spade chooses aggregates that are visually interesting , a property formally based on statistic properties of the aggregation query results. While well understood for relational data, such exploration raises multiple challenges for RDF: facts, dimensions and measures have to be identified (as opposed to known beforehand); as there are more candidate aggregates, assessing their interestingness can be very costly; finally, ontologies bring novel specific challenges but also novel opportunities, enabling ontology-driven exploration from an aggregate initially proposed by the system. Spade is a generic, extensible framework , which we instantiated with: ( i ) novel methods for enumerating candidate measures and dimensions in the vast space of possibilities provided by an RDF graph; ( ii ) a set of aggregate interestingness functions; ( iii ) ontology-based interactive exploration, and ( iv ) efficient early-stop techniques for estimating the interestingness of an aggregate query. The demonstration will comprise interactive scenarios on a variety of large, interesting RDF graphs.

Data quality is a critical problem for the Semantic Web. We propose that the degree to which a triple deviates from similar triples can be an important heuristic for identifying errors. Inspired by data dependency, which has shown promise in database data quality research, we introduce Semantic Dependency to assess quality of Semantic Web data. The system first builds a summary graph for finding candidate semantic dependencies. Each semantic dependency has a probability according to its instantiations and is subsequently adjusted based on the inconsistencies among them. Then triples can get a posterior probability of normality based on what semantic dependencies can support each of them. Repeating the iteration above, the proposed approach detects abnormal Semantic Web data. Experiments have shown that the system is efficient on data set with 10M triples and has more than a ten percent F-score improvement over our previous system.

The typical RDF querying method is a structured query language such as SPARQL. However, a structured query language requires issuers to understand the data model and query language syntax, limiting the accessibility of RDF data. The keyword query is regarded as an intuitive way for searching the RDF data. This paper proposes a keyword query approach based on the community structure of the RDF entity graph, which utilizes the characteristics of the community structure on the RDF graph. We propose a novel RDF entity graph model by designing an efficient entity index mechanism by treating the RDF entity as the virtual document, which reduces the graph scale while improving the efficiency of retrieving entities. We design an effective hierarchical and overlapping community-based index mechanism on the RDF entity graph, improving the efficiency and accuracy of the subgraph (sub-community) matching. Our approach takes full advantage of the schema of the RDF data and the community structural characteristic of the RDF entity graph. We evaluate our approach using two real-world datasets, and the experiment demonstrates the effectiveness and efficiency of our approach.