A unified structural/terminological interoperability framework based on LexEVS: application to TRANSFoRm.

Entity alignment is the problem of identifying which entities in a data source refer to the same real-world entity in the others. Identifying entities across heterogeneous data sources is paramount to many research fields, such as data cleaning, data integration, information retrieval and machine learning. The aligning process is not only overwhelmingly expensive for large data sources since it involves all tuples from two or more data sources, but also need to handle heterogeneous entity attributes. In this paper, we propose an unsupervised approach, called EnAli, to match entities across two or more heterogeneous data sources. EnAli employs a generative probabilistic model to incorporate the heterogeneous entity attributes via employing exponential family, handle missing values, and also utilize the locality sensitive hashing schema to reduce the candidate tuples and speed up the aligning process. EnAli is highly accurate and efficient even without any ground-truth tuples. We illustrate the performance of EnAli on re-identifying entities from the same data source, as well as aligning entities across three real data sources. Our experimental results manifest that our proposed approach outperforms the comparable baseline.

Selection of genes that are differentially expressed and critical to a particular biological process has been a major challenge in post-array analysis. Recent development in bioinformatics has made various data sources available such as mRNA and miRNA expression profiles, biological pathway and gene annotation, etc. Efficient and effective integration of multiple data sources helps enrich our knowledge about the involved samples and genes for selecting genes bearing significant biological relevance. In this work, we studied a novel problem of multi-source gene selection: given multiple heterogeneous data sources (or data sets), select genes from expression profiles by integrating information from various data sources. We investigated how to effectively employ information contained in multiple data sources to extract an intrinsic global geometric pattern and use it in covariance analysis for gene selection. We designed and conducted experiments to systematically compare the proposed approach with representative methods in terms of statistical and biological significance, and showed the efficacy and potential of the proposed approach with promising findings.

Graph model are increasingly popular among modern applications for its ability to model complex relationships between entities. Users tend to query the data as a graph with graph operations (e.g., graph navigation and exploration). However, a large fraction of the data resides in relational databases or other storage systems. Challenges arise in uniformly querying multiple heterogeneous data sources as a graph. Traditional solutions are limited by time-consuming data integration, expensive development effort, and incomplete query requirements. Thus, we developed Lynx, a general graph query framework, to simplify querying graph data by converting complex statements into basic graph operations. Instead of connecting directly to the data sources, Lynx retrieves data through user-implemented interfaces for those graph operations. We demonstrate Lynx's capabilities through real-world scenarios, showcasing Lynx's ability to process graph queries on multiple heterogeneous data sources and also to be used as a generic graph query engine development framework.

Stakeholders in Open Source Software (OSS) projects need to determine whether a project is likely to sustain for a sufficient period of time in order to justify their investments into this project. In an OSS project context, there are typically several data sources and OSS processes relevant for determining project health indicators. However, even within one project these data sources often are technically and/or semantically heterogeneous, which makes data collection and analysis tedious and error prone. In this paper, the authors propose and evaluate a framework for OSS data analysis (FOSSDA), which enables the efficient collection, integration, and analysis of data from heterogeneous sources. Major results of the empirical studies are: (a) the framework is useful for integrating data from heterogeneous data sources effectively and (b) project health indicators based on integrated data analyses were found to be more accurate than analyses based on individual non-integrated data sources.

Technological advances in high-throughput techniques and efficient data gathering methods, coupled computational biology efforts, have resulted in a vast amount of life science data often available in distributed and heterogeneous repositories. These repositories contain information such as sequence and structure data, annotations for biological data, results of complex computations, genetic sequences and multiple bio-datasets. However, the heterogeneity of these data, have created a need for research in resource integration and platform independent processing of investigative queries, involving heterogeneous data sources. When processing huge amounts of data, information integration is one of the most critical issues, because it’s crucial to preserve the intrinsic semantics of all the merged data sources. This integration would allow the proper organization of data, fostering the analysis and access the information to accomplish critical tasks, such as the processing of micro-array data to study protein function and medical researches in making detailed studies of protein structures to facilitate drug design (Ignacimuthu, 2005). Furthermore, DNA micro-array research community urgently requires technology to allow up-to-date micro-array data information to be found, accessed and delivered in a secure framework (Sinnot, 2007). Several research disciplines, such as Bioinformatics, where information integration is critical, could benefit from harnessing the potential of a new approach: the Semantic Web (SW). The SW term was coined by Berners-Lee, Hendler and Lassila (2001) to describe the evolution of a Web that consisted of largely documents for humans to read towards a new paradigm that included data and information for computers to manipulate. The SW is about adding machine-understandable and machine-processable metadata to Web resource through its key-enabling technology: ontologies (Fensel, 2002). Ontologies are a formal explicit and shared specification of a conceptualization. The SW was conceived as a way to solve the need for data integration on the Web. This article expounds SAMIDI, a Semantics-based Architecture for Micro-array Information and Data Integration. The most remarkable innovation offered by SAMIDI is the use of semantics as a tool for leveraging different vocabularies and terminologies and foster integration. SAMIDI is composed of a methodology for the unification of heterogeneous data sources from the analysis of the requirements of the unified data set and a software architecture.

• Offers a new vision on automated data harmonization in Dataspace (DS) systems. • Introduces LLM-based methods for scalable DS ingestion of heterogeneous datasources. • Presents a system with Harmonizer, Transformer, Evaluator components for ingestion. • Demonstrated an automated data ingestion prototype using LLM agents. • Validates the system with healthcare use case harmonizing heterogeneous data sources. Dataspaces (DS) enable stakeholders to collaborate on innovative, data-driven services by integrating data across domains. However, the realization and adoption of DS remain challenging due to domain-specific heterogeneity at the system, service, and data levels. While system and service-level heterogeneity can often be addressed through standards, data-level heterogeneity, namely data structures and semantics variations, remains challenging. To effectively ingest data into the DS, two communication endpoints must correctly interpret each other’s data models, therefore, DS ecosystems rely on “harmonization”, the process of generating a unified target data model from heterogeneous sources and transforming incoming data accordingly. Currently, harmonization and transformation are performed manually whenever new data sources are integrated. This is time-consuming, costly, and difficult to scale, posing a critical barrier to the realization and adoption of DS in practice. This study proposes a novel methodology for automated data harmonization during ingestion into DS ecosystems. The approach integrates harmonization, transformation, and human-in-the-loop evaluation within an automated system powered by modern LLM-based AI agents. These agents address data-level heterogeneity and generate harmonized target data models, representing a substantial departure from current manually-handled data harmonization. The system is validated through a healthcare use case, demonstrating its practical feasibility for harmonization during data ingestion into the DS. Overall, this work provides a foundational step toward seamless, efficient, and scalable data integration in DS. By automating data harmonization, it delivers substantial value to industry digital solutions as well as domains where data heterogeneity persists, including IoT or Big Data platforms.

Relatively mature integration technologies are available that make application integration tractable from an engineering perspective. However, integrating multiple heterogeneous data sources into applications is still a time-consuming, costly and error-prone engineering task, because existing integration technologies are based on static integration architectures. Such architectures tightly couple a data source to the data integration infrastructure. This paper describes the architecture and evolving prototype implementation of the Data Concierge, which provides a dynamic solution to integrate heterogeneous data sources. The core architecture and mechanisms in the Data Concierge can be used for dynamically attaching to a previously unidentified source of information, without the need for a pre-existing adapter component. In this paper, an ontology based API description mechanism supported by the Data Concierge is described, along with the architecture and prototype tools for describing the metadata necessary to facilitate dynamic integration. The use of an ontology and associated state machine to generate API calls to access data sources is illustrated using an FTP data source.

BackgroundProtein-protein interaction (PPI) prediction remains a central task in systems biology to achieve a better and holistic understanding of cellular and intracellular processes. Recently, an increasing number of computational methods have shifted from pair-wise prediction to network level prediction. Many of the existing network level methods predict PPIs under the assumption that the training network should be connected. However, this assumption greatly affects the prediction power and limits the application area because the current golden standard PPI networks are usually very sparse and disconnected. Therefore, how to effectively predict PPIs based on a training network that is sparse and disconnected remains a challenge.ResultsIn this work, we developed a novel PPI prediction method based on deep learning neural network and regularized Laplacian kernel. We use a neural network with an autoencoder-like architecture to implicitly simulate the evolutionary processes of a PPI network. Neurons of the output layer correspond to proteins and are labeled with values (1 for interaction and 0 for otherwise) from the adjacency matrix of a sparse disconnected training PPI network. Unlike autoencoder, neurons at the input layer are given all zero input, reflecting an assumption of no a priori knowledge about PPIs, and hidden layers of smaller sizes mimic ancient interactome at different times during evolution. After the training step, an evolved PPI network whose rows are outputs of the neural network can be obtained. We then predict PPIs by applying the regularized Laplacian kernel to the transition matrix that is built upon the evolved PPI network. The results from cross-validation experiments show that the PPI prediction accuracies for yeast data and human data measured as AUC are increased by up to 8.4 and 14.9% respectively, as compared to the baseline. Moreover, the evolved PPI network can also help us leverage complementary information from the disconnected training network and multiple heterogeneous data sources. Tested by the yeast data with six heterogeneous feature kernels, the results show our method can further improve the prediction performance by up to 2%, which is very close to an upper bound that is obtained by an Approximate Bayesian Computation based sampling method.ConclusionsThe proposed evolution deep neural network, coupled with regularized Laplacian kernel, is an effective tool in completing sparse and disconnected PPI networks and in facilitating integration of heterogeneous data sources.

Purpose of ReviewThe complexity of the data collected for sleep research is increasing, and the focal point of sleep research is dependent on a higher number of data sources. Data collected for sleep studies often includes both subjective and objective measurements of sleep quality and is gathered over a more extended period, e.g., for weeks, months, or even years. However, this variety and volume of data make it challenging and time-consuming for researchers to utilize. Therefore, sophisticated data structures are necessary to utilize data in sleep research.Recent FindingsThis paper explores how heterogeneous data sources can be represented in a homogeneous database design. The following research questions drove our work: (i) How can we represent sleep data from heterogeneous sources in a homogenous digital platform database? and (ii) How can a data source pipeline transform various data sources into a homogeneous data format?SummaryThis paper’s main contributions are conceptualizing the design and development of a homogeneous database and digital platform architecture and a data source pipeline that fits well for sleep research in particular and healthcare research in general.

In the last years, enterprises increase their effort to collect large amounts of data from many heterogeneous data sources and store it in modern architectures like data lakes. However, this approach faces different drawbacks for finding and understanding data sources. Ontology-Based Data Access (OBDA) originating from the Semantic Web enables a homogeneous access to the data sources by using a mapping, called semantic model, between a data source and a target ontology. However, OBDA requires a detailed ontology, which is usually created by ontology engineers and domain experts resulting along with high effort for designing and maintaining. To overcome these limitations, we develop an approach consisting of a knowledge graph, which features an internal growing ontology and linked data-source specific semantic models. The ontology continuously evolves on-demand based on newly added data sources along with their corresponding semantic models, which are created by domain experts. To ensure the knowledge graph's stability, we develop an intuitive user-oriented assistant and combine it with a semi-supervised evolving strategy that assists the user with the help of external knowledge bases. We evaluate accuracy and usability of our approach by conducting a user study with a heterogeneous group of participants that define semantic models upon pre-defined data sets. The results show that semantic models become more objective and consistent with our provided user assistant and thus lead to a knowledge graph with higher interconnectivity and stability.

Managing energy data from multiple distributed and heterogeneous sources is an important issue worldwide. This study focused on using semantic web technologies in an energy data management system. Specifically, we provided a federated approach a mediator server that allows users to access to multiple heterogeneous data sources, including four typical types of databases in energy data resources: relational database, Triplestore, NoSQL database, and XML. A proposed architecture based on this approach is then presented and our solution can realize the data acquisition and integration without the need to rewrite or transform the local data into a unified data. We further examined our architecture by a case study in a Swiss energy company and tested the system based on

Stimulated by the arrival of the big data era, various and heterogeneous data sources such as data in social networks, mobile devices and sensor data for users have emerged, mirroring characteristics and preferences of data owners. These data sources are often used to construct user profiles so as to facilitate personalized services like recommendations or personalized data access. In the context of second language learning, learner data involve learning logs, standard test results, and individual learning preferences and styles. Given its attribute of reflecting the characteristics of learners, such data can be exploited to build the learner profiles. However, these data sources possibly include noises or bias, and hence influence the reliability of the correspondingly constructed learner profiles. Consequently, the inaccurate profiles may result in ineffective learning tasks that are generated by e-Learning systems. To tackle this issue, it is significant and critical to evaluate the accuracy of learner profiles. In a response to this call, we propose a novel metric named “profile mean square error” to examine the accuracy of learner profiles founded upon diverse sources. We also demonstrate how to construct various learner profiles though applying different data sources such as learning logs, standard test results, and personal learning preferences in e-Learning systems and pedagogical activities. Moreover, we conduct an experimental study among some second language learners, the results of which illustrate that the most accurate profiles are generated from multiple data sources if they are integrated in a rational way.

This paper presents the developed algorithm of complex data processing, aimed at forming a reliable information flow from the initial data coming from heterogeneous and heterogeneous sources in terms of reliability and its software implementation. The algorithm is based on the concept of adaptive assessment of reliability of data and their sources, considering the level of consistency, the degree of conflict and confirmation of information. In modern conditions of intensive development of digital technologies and increase in the volume of processed information the task of forming a reliable information flow from heterogeneous data sources, which may differ in the level of reliability, structural characteristics and the degree of consistency of the information provided, is of relevance. The proposed algorithm of complexized data processing is based on the concept of adaptive assessment of reliability of both the data and their sources, considering such criteria as the level of consistency, the degree of conflict and confirmation of information. The developed algorithm is aimed at ensuring objective filtering, integration and optimal use of incoming data.

Combining in-car multiple sensors measuring parameters that can be used to improve both safety and efficiency with a plethora of external data sources (e.g., traffic conditions, weather) which, if properly used, can significantly improve the overall trip experience. One source that can help the navigation and provide "context awareness", especially for autonomous driving, are the High Definition (HD) maps, which have recently witnessed a tremendous growth of popularity in vehicular technology and use. As they are limited to a particular geographic area with respect to a given point along a trip, different portions need to be downloaded (and processed) on multiple occasions throughout a given trip, along with the other data from internal and external sources. We take a first step towards formalizing the problem of Predicting Map Data Consumption (PMDC) in the future time instants for a given trip, based on a (time) window from its history, and investigate the use of Long Short-Term Memory (LSTM) networks - a special type of Recurrent Neural Networks (RNN). Significant efforts were focused on generating an appropriate dataset for this study, towards which we fused the information available in multiple heterogeneous data sources. We conducted experimental observations demonstrating the benefits of the proposed approach.

The analysis of modern production environments requires a clear and understandable uniform visualization of the information from all levels of the production. This visualization has to be on the one hand light weight to avoid cognitive overload of the users and on the other hand rich enough to provide information about all production information sources. To solve this issue we propose to use a unified component-based visualization where single items can be exchanged to fulfill the informational needs of the users from heterogeneous data sources.