USING SEMANTIC WEB TECHNOLOGIES IN HETEROGENEOUS DISTRIBUTED DATABASE SYSTEM: A CASE STUDY FOR MANAGING ENERGY DATA ON MOBILE DEVICES

Graph model are increasingly popular among modern applications for its ability to model complex relationships between entities. Users tend to query the data as a graph with graph operations (e.g., graph navigation and exploration). However, a large fraction of the data resides in relational databases or other storage systems. Challenges arise in uniformly querying multiple heterogeneous data sources as a graph. Traditional solutions are limited by time-consuming data integration, expensive development effort, and incomplete query requirements. Thus, we developed Lynx, a general graph query framework, to simplify querying graph data by converting complex statements into basic graph operations. Instead of connecting directly to the data sources, Lynx retrieves data through user-implemented interfaces for those graph operations. We demonstrate Lynx's capabilities through real-world scenarios, showcasing Lynx's ability to process graph queries on multiple heterogeneous data sources and also to be used as a generic graph query engine development framework.

Selection of genes that are differentially expressed and critical to a particular biological process has been a major challenge in post-array analysis. Recent development in bioinformatics has made various data sources available such as mRNA and miRNA expression profiles, biological pathway and gene annotation, etc. Efficient and effective integration of multiple data sources helps enrich our knowledge about the involved samples and genes for selecting genes bearing significant biological relevance. In this work, we studied a novel problem of multi-source gene selection: given multiple heterogeneous data sources (or data sets), select genes from expression profiles by integrating information from various data sources. We investigated how to effectively employ information contained in multiple data sources to extract an intrinsic global geometric pattern and use it in covariance analysis for gene selection. We designed and conducted experiments to systematically compare the proposed approach with representative methods in terms of statistical and biological significance, and showed the efficacy and potential of the proposed approach with promising findings.

Semantic Web for Health Care and Life Sciences: a review of the state of the art

NoSQL and relational databases have been integrated in response to the increasing demand for scalable and efficient data management in hybrid cloud environments. The differences in data structures and query processing methods between these databases present both challenges and opportunities when designing an optimized hybrid system. This study explores the integration of NoSQL and relational databases to maximize data storage, retrieval, and processing efficiency across multiple cloud systems. NoSQL databases excel in handling unstructured and semi-structured data, offering flexibility and scalability, whereas relational databases provide robust consistency and structured query capabilities. Despite the wide availability of relational databases, certain applications require the dynamic adaptability of NoSQL systems, making integration a viable solution. The research evaluates key performance parameters, including query execution speed, scalability, data consistency, and resource utilization. Cloudsim is used for simulation, allowing for an in-depth comparison between standalone and hybrid database models. Experimental results indicate that the proposed hybrid model improves query performance by 30%, reduces latency by 20%, and enhances scalability by 40% compared to using relational or NoSQL databases alone. The novelty of this approach lies in its ability to overcome the limitations of each database type by their strengths in a dynamic cloud environment. The results highlight the effectiveness of hybrid database integration in optimizing cloud-based data management while ensuring seamless operation, adaptability, and resource efficiency.

ObjectiveBiomedical research increasingly relies on the integration of information from multiple heterogeneous data sources. Despite the fact that structural and terminological aspects of interoperability are interdependent and rely on a common set of requirements, current efforts typically address them in isolation. We propose a unified ontology-based knowledge framework to facilitate interoperability between heterogeneous sources, and investigate if using the LexEVS terminology server is a viable implementation method.Materials and methodsWe developed a framework based on an ontology, the general information model (GIM), to unify structural models and terminologies, together with relevant mapping sets. This allowed a uniform access to these resources within LexEVS to facilitate interoperability by various components and data sources from implementing architectures.ResultsOur unified framework has been tested in the context of the EU Framework Program 7 TRANSFoRm project, where it was used to achieve data integration in a retrospective diabetes cohort study. The GIM was successfully instantiated in TRANSFoRm as the clinical data integration model, and necessary mappings were created to support effective information retrieval for software tools in the project.ConclusionsWe present a novel, unifying approach to address interoperability challenges in heterogeneous data sources, by representing structural and semantic models in one framework. Systems using this architecture can rely solely on the GIM that abstracts over both the structure and coding. Information models, terminologies and mappings are all stored in LexEVS and can be accessed in a uniform manner (implementing the HL7 CTS2 service functional model). The system is flexible and should reduce the effort needed from data sources personnel for implementing and managing the integration.

Modern large-scale information systems often use multiple database management systems, not all of which are necessarily relational. In recent years, NoSQL databases have gained acceptance in certain domains while relational databases remain de facto standard in many others. Many "legacy" information systems also use relational databases. Unlike relational database systems, NoSQL databases do not have a common data model or query language, making it difficult for users to access data in a uniform manner when using a combination of relational and NoSQL databases or simply several different NoSQL database systems. Therefore, the need for uniform data access from such a variety of data sources becomes one of the central problems for data integration. In this paper we provide an overview of the main problems, methods, and solutions for data integration between relational and NoSQL databases, as well as between different NoSQL databases. We focus mainly on the problems of structural, syntactic, and semantic heterogeneity and on proposed solutions for uniform data access, emphasizing some of the more recent proposals.

• Offers a new vision on automated data harmonization in Dataspace (DS) systems. • Introduces LLM-based methods for scalable DS ingestion of heterogeneous datasources. • Presents a system with Harmonizer, Transformer, Evaluator components for ingestion. • Demonstrated an automated data ingestion prototype using LLM agents. • Validates the system with healthcare use case harmonizing heterogeneous data sources. Dataspaces (DS) enable stakeholders to collaborate on innovative, data-driven services by integrating data across domains. However, the realization and adoption of DS remain challenging due to domain-specific heterogeneity at the system, service, and data levels. While system and service-level heterogeneity can often be addressed through standards, data-level heterogeneity, namely data structures and semantics variations, remains challenging. To effectively ingest data into the DS, two communication endpoints must correctly interpret each other’s data models, therefore, DS ecosystems rely on “harmonization”, the process of generating a unified target data model from heterogeneous sources and transforming incoming data accordingly. Currently, harmonization and transformation are performed manually whenever new data sources are integrated. This is time-consuming, costly, and difficult to scale, posing a critical barrier to the realization and adoption of DS in practice. This study proposes a novel methodology for automated data harmonization during ingestion into DS ecosystems. The approach integrates harmonization, transformation, and human-in-the-loop evaluation within an automated system powered by modern LLM-based AI agents. These agents address data-level heterogeneity and generate harmonized target data models, representing a substantial departure from current manually-handled data harmonization. The system is validated through a healthcare use case, demonstrating its practical feasibility for harmonization during data ingestion into the DS. Overall, this work provides a foundational step toward seamless, efficient, and scalable data integration in DS. By automating data harmonization, it delivers substantial value to industry digital solutions as well as domains where data heterogeneity persists, including IoT or Big Data platforms.

Entity alignment is the problem of identifying which entities in a data source refer to the same real-world entity in the others. Identifying entities across heterogeneous data sources is paramount to many research fields, such as data cleaning, data integration, information retrieval and machine learning. The aligning process is not only overwhelmingly expensive for large data sources since it involves all tuples from two or more data sources, but also need to handle heterogeneous entity attributes. In this paper, we propose an unsupervised approach, called EnAli, to match entities across two or more heterogeneous data sources. EnAli employs a generative probabilistic model to incorporate the heterogeneous entity attributes via employing exponential family, handle missing values, and also utilize the locality sensitive hashing schema to reduce the candidate tuples and speed up the aligning process. EnAli is highly accurate and efficient even without any ground-truth tuples. We illustrate the performance of EnAli on re-identifying entities from the same data source, as well as aligning entities across three real data sources. Our experimental results manifest that our proposed approach outperforms the comparable baseline.

In this paper, we propose new approaches for action and event recognition by leveraging a large number of freely available Web videos (e.g., from Flickr video search engine) and Web images (e.g., from Bing and Google image search engines). We address this problem by formulating it as a new multi-domain adaptation problem, in which heterogeneous Web sources are provided. Specifically, we are given different types of visual features (e.g., the DeCAF features from Bing/Google images and the trajectory-based features from Flickr videos) from heterogeneous source domains and all types of visual features from the target domain. Considering the target domain is more relevant to some source domains, we propose a new approach named multi-domain adaptation with heterogeneous sources (MDA-HS) to effectively make use of the heterogeneous sources. In MDA-HS, we simultaneously seek for the optimal weights of multiple source domains, infer the labels of target domain samples, and learn an optimal target classifier. Moreover, as textual descriptions are often available for both Web videos and images, we propose a novel approach called MDA-HS using privileged information (MDA-HS+) to effectively incorporate the valuable textual information into our MDA-HS method, based on the recent learning using privileged information paradigm. MDA-HS+ can be further extended by using a new elastic-net-like regularization. We solve our MDA-HS and MDA-HS+ methods by using the cutting-plane algorithm, in which a multiple kernel learning problem is derived and solved. Extensive experiments on three benchmark data sets demonstrate that our proposed approaches are effective for action and event recognition without requiring any labeled samples from the target domain.

Personalized treatments and targeted therapies are the most promising approaches to treat complex diseases, especially for cancer. However, drug resistance is often acquired after treatments. To overcome or reduce drug resistance, treatments using drug combinations have been actively investigated in the literature. Existing methods mainly focus on chemical properties of drugs for potential combination therapies without considering relationships among different diseases. Also, they often do not consider the rich knowledge of drugs and diseases, which can enhance the prediction of drug combinations. This motivates us to develop a new computational method that can predict the beneficial drug combinations. We propose DrugCom, a tensor-based framework for computing drug combinations across different diseases by integrating multiple heterogeneous data sources of drugs and diseases. DrugCom first constructs a primary third-order tensor (i.e., drug×drug ×disease) and several similarity matrices from multiple data sources regarding drugs (e.g., chemical structure) and diseases (e.g., disease phenotype). DrugCom then formulates an objective function, which simultaneously factorizes coupled tensor and matrices to reveal the molecular mechanisms of drug synergy. We adopt the alternating direction method of multipliers algorithm to effectively solve the optimization problem. Extensive experimental studies using real-world datasets demonstrate superior performance of DrugCom.

Stakeholders in Open Source Software (OSS) projects need to determine whether a project is likely to sustain for a sufficient period of time in order to justify their investments into this project. In an OSS project context, there are typically several data sources and OSS processes relevant for determining project health indicators. However, even within one project these data sources often are technically and/or semantically heterogeneous, which makes data collection and analysis tedious and error prone. In this paper, the authors propose and evaluate a framework for OSS data analysis (FOSSDA), which enables the efficient collection, integration, and analysis of data from heterogeneous sources. Major results of the empirical studies are: (a) the framework is useful for integrating data from heterogeneous data sources effectively and (b) project health indicators based on integrated data analyses were found to be more accurate than analyses based on individual non-integrated data sources.

Relatively mature integration technologies are available that make application integration tractable from an engineering perspective. However, integrating multiple heterogeneous data sources into applications is still a time-consuming, costly and error-prone engineering task, because existing integration technologies are based on static integration architectures. Such architectures tightly couple a data source to the data integration infrastructure. This paper describes the architecture and evolving prototype implementation of the Data Concierge, which provides a dynamic solution to integrate heterogeneous data sources. The core architecture and mechanisms in the Data Concierge can be used for dynamically attaching to a previously unidentified source of information, without the need for a pre-existing adapter component. In this paper, an ontology based API description mechanism supported by the Data Concierge is described, along with the architecture and prototype tools for describing the metadata necessary to facilitate dynamic integration. The use of an ontology and associated state machine to generate API calls to access data sources is illustrated using an FTP data source.

This study intends to help data mining developers to get better performance when obtaining the most frequent N-grams in Text Mining projects. The process of building new variables is one of the oldest and still challenging problems in Data Mining projects. The most frequent N-grams are commonly used as input variables in Text Mining projects. The N-grams represent the occurrence of N items in sequence in a given text. The items can be letters or words. This paper presents a performance comparison between the two main approaches of data storage, relational and NoSQL databases in the task of obtaining the most frequent N-grams. Validation of the study was executed using a database from a known benchmark from an international competition organized by PAN@CLEF 2013. The one-tailed paired t-test showed that NoSQL approach is statistically superior to the relational approach with a confidence level of 95%.

A data lake refers to an extremely large data resource or repository. Data lakes store large amounts of data and use advanced analytics to pair data from multiple sources with different types of structured, semi-structured, and unstructured information. NoSQL databases such as Mongodb, Redis, Neo4j, and Cassandra are nontabular and they store data differently rather than use relational tables. NoSQL databases come in many forms, mostly documents, key values, wide columns, and graphs based on their data model. NoSQL gives less complicated scalability and higher overall performance as compared with traditional relational databases. NoSQL databases can store different types of data, but they cannot fully support Automation, Consistency, Isolation, and endurance (ACID) features, i.e., trigger functions in multi-transaction management, because a NoSQL database uses a non-relational database system. Thus, an interpreter is necessary for SQL-to-NoSQL queries. We used ANTLR (ANother Tool for Language Recognition), which has five main stages: The input SQL query, the tokenizer, the parser, the parser tree, and lastly the generation of the query results in NoSQL. The tool gave users to write a flexible multi-pass language parser that is expected to solve problems in querying complex ACID functions and other problems in complex queries in NoSQL databases. In the measurement, analysis, and evaluation of the translation results through the comparison of each NoSQL criterion against a Relational Database Management System (MySQL), the scores obtained were as follows. The performance criterion achieved the highest score (98.40%) by the MongoDB database, followed by scalability (97.40%) and accuracy (97.00%). The criterion with the lowest score was complexity (91.65%).

This paper proposes the system realizing integrated use of Relational Databases and a NoSQL Database. A heterogeneous information source integration system has been proposed. It enables equi-join and projection of tables without converting data. With this system, it became possible to handle various databases at the same time by using JDBC, but the databases which can be handled are only relational databases. Therefore, we add functions to the system to handle the NoSQL database which has been increasingly used in recent years. The table join execution time and the memory consumption of the system are experimentally evaluated. It showed that the performance is tolerable in practical use as compared with integrated use of only relational database. We also showed that it is practical to change the join order or to register column families which are unique elements of HBase.