English

Data Warehouses (DWs) are currently considered to be the cornerstone of Business Intelligence (BI) systems. Security is a key issue in DWs since the business information that they manage is crucial and highly sensitive, and should be carefully protected. However, the increasing amount of data available on the Web signifies that more and more DW systems are considering the Web as the primary data source through which to populate their DWs. XML is therefore widely accepted as being the principal means through which to provide easier data and metadata interchange among heterogeneous data sources from the Web and the DW systems.Although security issues have been considered during the whole development process of traditional DWs, current research lacks approaches with which to consider security when the target platform is based on the Web and XML technologies. The idiosyncrasy of the unstructured and semi-structured data available on the Web definitely requires particular security rules that are specifically tailored to these systems in order to permit their particularities to be captured correctly.In order to tackle this situation, in this paper, we propose a methodological approach based on the Model Driven Architecture (MDA) for the development of Secure XML DWs. We therefore specify a set of transformation rules that are able to automatically generate not only the corresponding XML structure of the DW from secure conceptual DW models, but also the security rules specified within the DW XML structure, thus allowing us to implement both aspects simultaneously. A case study is provided at the end of the paper to show the benefits of our approach.

Data warehousing is a collection of concepts and tools which aim at providing and maintaining a set of integrated data (the data warehouse – DW ) for business decision support within an organization. They extract data from different operational data sources, and after some cleansing and transformation procedures data are integrated and loaded into a central repository to enable analysis and mining. Data and metadata lineage are important processes for data analysis. The first allows users to trace warehouse data items back to the original source item from which they were derived and the latter shows which operations have been performed to achieve that target data. This work proposes integrating metadata captured during transformation processes using the CWM metadata standard in order to enable data and metadata lineage. Additionally it presents a tool specially developed for performing this task.

In data warehouse (DW) development, a series of mappings must be specified between user concepts and data source elements, in order to identify which sources must undergo an integration process. Until now, these mappings are either assumed to be implied by name matching or identified according to the designer's experience. Then, the result is implemented as Extraction/Transformation/Loading (ETL) processes. Since ETL processes relate elements at the logical level, designers cannot adequately analyze how a change in requirements or in the data sources affects the analysis capabilities. Furthermore, this approach makes it difficult to perform incremental changes in DW design, requiring in some cases to perform the whole analysis again. In this paper we present a set of semantic mappings that relate user concepts specified by requirements to those obtained from data sources. In turn, this allows us to accurately identify how any potential change affects the different structures and ETL processes. As a DW evolves over time, our approach easily allows us to incorporate new concepts, as well as any change introduced at requirements or data sources into the DW repository with no need to redesign the whole DW. In order to show the application of our proposal, we show a real case study focusing on the Digital library of the University of Alicante.

In the past few years, several conceptual approaches have been proposed for the specification of the main multidimensional (MD) properties of the data warehouse (DW) repository. However, most of them deal with isolated aspects of the DW and do not provide designers with an integrated and standard method for designing the whole DW life cycle (ETL processes, data sources, DW repository and so on).Some approaches are depending on specific platform or neglecting important issues in DW design life cycle. Extraction-transformation-loading (ETL) processes play an important role in data warehouse architecture because they are responsible of integrating data from heterogeneous data sources into the DW repository. This paper proposes a conceptual model to refresh data warehouse by (insert, update, delete) data using ETL processes and considering DW security requirements. Firstly, the proposed ETL model is based on Unified Modeling Language (UML), which allows us to accomplish the conceptual modeling of ETL processes .secondly; this part focuses on how to integrate the proposed ETL model with the DW model. The proposed DW model depends on the Model Driven Architecture (MDA). MDA is a standard framework for software development that addresses the complete life cycle of designing, deploying, integrating, and managing applications by using models in software development. This paper proposes an integrated conceptual model for addressing temporal data warehouse security requirements (CMTDWS).

Chapter 64 - Exploiting Versions for On-line Data Warehouse Maintenance in MOLAP Servers

Extraction-transformation-loading (ETL) process in data warehouse development perform data extraction from various resources, transform the data into suitable format and loadit into data warehouse storage. In the ETL process, there is data cleansing process function that handles redundancy, inconsistency and integrity data. ETL process will move data from the source to the integration layer (data store in data warehouse). In the integration layer, the data can be grouped into smaller scope and more specific for the requirement in other repositories called data marts. Reporting program of data warehouse will be associated with a data mart as its data source. In this research, the data warehouse is built to handle the ETL process. The data warehouse build metadata to support the process. The metadata construction for ETL processes will lead to ETL programs with high degree of reusability. The conclusion from this research is the use of dynamic ETL process (using metadata ETL) is required when ETL process is dealing with the operational system that still unstable and likely to change the database schema. Dynamic ETL process is also needed to address the increase requirement for report from the users.

An MDA approach for the development of data warehouses

Materialized views (MV) at the data warehouse (DW) can be kept up to date in response to changes in data sources without accessing data sources for additional information. This process is usually refered to as “self maintenance of views”. A number of algorithms have been proposed for self maintenance of views, which use auxiliary views (AV) to keep some additional information in DW. In this paper we propose an algorithm for self maintainability of multiple MVs using the above approach. Our algorithm generates a simple maintenance query to incrementally maintain an MV along with its AV at DW. The algorithm maintains these views by minimizing the number and the size of the AVs. Our approach provides better insight into view maintenance issues by exploiting the dependencies and constraints that might exist in the data sources and multiple MVs at DW.

Efficient maintenance of basic statistical functions in data warehouses

Data warehouses tend to evolve, because of changes in data sources and business requirements of users. All these kinds of changes must be properly handled, therefore, data warehouse development is never-ending process. In this paper we propose the evolution-oriented user-centric data warehouse design, which on the one hand allows to manage data warehouse evolution automatically or semi-automatically, and on the other hand it provides users with the understandable, easy and transparent data analysis possibilities. The proposed approach supports versions of data warehouse schemata and data semantics.

Data that needs to be documented for clinical studies has often been acquired and documented in clinical routine. Usually this data is manually transferred to Case Report Forms (CRF) and/or directly into an electronic data capture (EDC) system. To enhance the documentation process of a large clinical follow-up study targeting patients admitted for acutely decompensated heart failure by accessing the data created during routine and study visits from a hospital information system (HIS) and by transferring it via a data warehouse (DWH) into the study's EDC system. This project is based on the clinical DWH developed at the University of Würzburg. The DWH was extended by several new data domains including data created by the study team itself. An R user interface was developed for the DWH that allows to access its source data in all its detail, to transform data as comprehensively as possible by R into study-specific variables and to support the creation of data and catalog tables. A data flow was established that starts with labeling patients as study patients within the HIS and proceeds with updating the DWH with this label and further data domains at a daily rate. Several study-specific variables were defined using the implemented R user interface of the DWH. This system was then used to export these variables as data tables ready for import into our EDC system. The data tables were then used to initialize the first 296 patients within the EDC system by pseudonym, visit and data values. Afterwards, these records were filled with clinical data on heart failure, vital parameters and time spent on selected wards. This solution focuses on the comprehensive access and transformation of data for a DWH-EDC system linkage. Using this system in a large clinical study has demonstrated the feasibility of this approach for a study with a complex visit schedule.

Big data represents a new technology for managing data with high velocity, volume, variety and contributes to creating value for companies. As quoted in1, capturing all queries made on the company website or from customer support calls, emails or chat lines, regardless of their outcome, may have significant value in identifying emerging trends. The Big Data Era has largely contributed in accelerating the development strategic plans issued from governments and research organisms, coving the management, exploitation and analysis of these data by taking into account the different V's of Big Data. Among these plans, we can cite for instance the development of: (a) large-scale platforms (ex. data-clusters, distributed data clusters), (b) Software Defined Environments (SDE) (ex. IBM SDE), (c) advanced programming paradigms (ex. map-reduce, Spark, etc.), Data Analytics Tools (Rapid Miner, Google Fusion Tables, Solver), (d) Visualization tools (Google Chart, Tableau, Oracle Visual Analyzer), and (h) high quality and valuable Knowledge Bases (KB), constructed either by academicians (e.g., Cvc, DBpedia, Freebase, and YAGO) and industrials (e.g., Google Knowledge Graph, Facebook Knowledge Graph, Amazon Knowledge Graph, Credit Rating Agencies, Enterprise Knowledge Base, etc.). In this talk, we would like to foster the creation of a think tank dedicated to getting the best from Big Data V's and the efforts related to it to revisit our research activities without compromising them. In this talk, we would like to share the experience conducted with our Model and Data Engineering Team of the LIAS Laboratory at ISAE-ENSMA, which aims at the design of data warehousing applications. Based on the literature, this design is based on two main approaches: (i) a supply-driven approach (also called data-driven) that starts with an analysis of operational data sources in order to identify all the available data and (ii) a user-driven approach (also known as requirement-driven or goal-orientated) which stems from the determination of the information requirements of different business users. Several studies and experiments show that resorting to these two approaches entails a high risk for companies, since some functional requirements cannot be satisfied. This is due to the lack of relevant data in sources. In parallel, reference studies have identified the crucial role of knowledge bases (KB) for analytical tasks, by offering analysts more entities (people, places, products, etc.). The availability of a huge, high quality valuable KB is an asset for data warehousing designers and decision-makers to construct/exploit a valuable data warehouse. So, faced with this situation, we here present a value-driven approach that revisits the traditional life cycle of the design of data warehouses, by considering KB as an external resource. These different phases are illustrated via the YAGO KB.

Traditionally data warehouses were queried by high level users (executive management, business analysts) only. As the range of potential users with data warehouse access is steadily growing, this assumption is no longer appropriate and the necessity of proper access control mechanisms arises. The security capabilities of available commercial OLAP systems are highly proprietary and the syntax of their security constraints is not suitable for design and documentation purposes. Also, approaches trying to derive the access control policies from the operational data sources have not been very successful, as the relational model is predominate in operational systems while OLAP systems make use of the non-traditional multidimensional model. Access control schemes do not map easily. We approach the issue from the application side by introducing a methodology and a language for conceptual OLAP security design.

Research in suicide risk prediction often suffers from the lack of comprehensive data on patient suicide death, which differs from suicide attempt or suicidal behaviors. This study aimed to develop a population-wide multi-source harmonized data warehouse suitable for suicide death risk prediction. The Maryland Suicide Data Warehouse (MSDW) was conceived as a statewide database that addresses limitations in prior suicide research. To develop MSDW, multiple patient-level statewide data sources were linked using the statewide health information exchange infrastructure. Manner of death, the standard outcome in suicide death research, was determined by the state medical examiner. Health services data were linked from multiple data sources such as electronic health records, hospital discharge data, and administrative insurance claims. Data were structured as a common format that preserves observations at their lowest level of analysis. Data features were included based on known or hypothesized psychiatric or suicide risk factors. The warehouse contains a mix of records across data sources for patient diagnoses, clinical encounters, procedures, area of residence, pharmacy fills and laboratory findings. MSDW represents 104,517 decedents reported by the OCME between 2012 and 2020, 5,059 classified as suicides. The MSDW is a statewide data warehouse that allows users to conduct population health research, predictive modeling and observational studies for multiple outcomes. It has multiple overlapping clinical records that improve the completeness and timeliness of data. It is a high-quality statewide data warehouse for conducting suicide prediction research and assessing risk for surveillance and intervention.

Data warehouses collect and maintain large amounts of data from several distributed and heterogeneous data sources. Because of security reasons, operational requirements, and technical feasibility it is often impossible for data warehouses to access the data sources directly. Instead data warehouses have to replicate legacy information as detail data in order to be able to maintain their summary data.In this paper we investigate how to minimize the amount of detail data stored in a data warehouse. More specifically, we identify the minimal amount of data that has to be replicated in order to maintain, either incrementally or by recomputation, summary data defined in terms of generalized project-select-join (GPSJ) views. We show how to minimize the number of tuples and attributes in the current detail tables and even aggregate them where possible. The amount of data to be stored in current detail tables is minimized by exploiting smart duplicate compression in addition to local and join reductions. We identify situations where it becomes possible to omit the typically huge fact table and prove that these techniques in concert ensure that the current detail data is minimal in the sense that no subset of it permits to accurately maintain the same summary data. Finally, we sketch how existing maintenance methods can be adapted to use the minimal detail tables we propose.KeywordsFact TableLocal ReductionBase TableHeterogeneous Data SourceStar SchemaThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.