Web Data Warehousing Convergence

This article proposes a data warehouse integration technique that combines data and documents from different underlying documents and database design approaches. The well-defined and structured data such as relational, object-oriented and object relational data, semi-structured data such as XML, and unstructured data such as HTML documents are integrated into a Web data warehouse system. The user specified requirements and data sources are combined to assist with the definitions of the hierarchical structures, which serve specific requirements and represent a certain type of data semantics using object-oriented features including inheritance, aggregation, association, and collection. A conceptual integrated data warehouse model is then specified based on a combination of user requirements and data source structure, which creates the need for a logical integrated data warehouse model. A case study is then developed into a prototype in a Web-based environment that enables the evaluation. The evaluation of the proposed integration Web data warehouse methodology includes the verification of correctness of the integrated data, and the overall benefits of utilizing this proposed integration technique.

Data Warehouses (DWs) are currently considered to be the cornerstone of Business Intelligence (BI) systems. Security is a key issue in DWs since the business information that they manage is crucial and highly sensitive, and should be carefully protected. However, the increasing amount of data available on the Web signifies that more and more DW systems are considering the Web as the primary data source through which to populate their DWs. XML is therefore widely accepted as being the principal means through which to provide easier data and metadata interchange among heterogeneous data sources from the Web and the DW systems.Although security issues have been considered during the whole development process of traditional DWs, current research lacks approaches with which to consider security when the target platform is based on the Web and XML technologies. The idiosyncrasy of the unstructured and semi-structured data available on the Web definitely requires particular security rules that are specifically tailored to these systems in order to permit their particularities to be captured correctly.In order to tackle this situation, in this paper, we propose a methodological approach based on the Model Driven Architecture (MDA) for the development of Secure XML DWs. We therefore specify a set of transformation rules that are able to automatically generate not only the corresponding XML structure of the DW from secure conceptual DW models, but also the security rules specified within the DW XML structure, thus allowing us to implement both aspects simultaneously. A case study is provided at the end of the paper to show the benefits of our approach.


 
 
 The data warehouse is a very famous solution for analyzing business data from heterogeneous sources. Unfortunately, a data warehouse only can analyze structured data. Whereas, nowadays, thanks to the popularity of social media and the ease of creating data on the web, we are experiencing a flood of unstructured data. Therefore, we need an approach that can "structure" the unstructured data into structured data that can be processed by the data warehouse. To do this, we propose a schema extraction approach using Google Cloud Platform that will create a schema from unstructured data. Based on our experiment, our approach successfully produces a schema from unstructured data. To the best of our knowledge, we are the first in using Google Cloud Platform for extracting a schema. We also prove that our approach helps the database developer to understand the unstructured data better.
 
 

The existing big data platforms take data through distributed processing platforms and store them in a data lake. The architectures such as Lambda and Kappa address the real-time and batch processing of data. Such systems provide real time analytics on the raw data and delayed analytics on the curated data. The data denormalization, creation and maintenance of a columnar dimensional data warehouse is usually time consuming with no or limited support for unstructured data. The system introduced in this paper automatically creates and dynamically maintains its data warehouse as a part of its big data pipeline in addition to its data lake. It creates its data warehouse on structured, semi-structured and unstructured data. It uses Machine Learning to identify and create dimensions. It also establishes relations among data from different data sources and creates the corresponding dimensions. It dynamically optimizes the dimensions based on the crowd sourced data provided by end users and also based on query analysis.

The contemporary landscape of data warehousing is undergoing a revolutionary transformation propelled by the integration of Artificial Intelligence (AI). This paper explores the intersection of AI and data warehousing, unraveling the dynamics that fuel this revolution and examining the profound implications for businesses and data management practices. The traditional paradigm of data warehousing has relied on structured data and predefined schemas, limiting its adaptability to the dynamic nature of modern datasets. The advent of AI injects a new dimension, enabling data warehouses to evolve into intelligent, adaptive entities capable of handling diverse data types, volumes, and velocities. This abstract encapsulates the essence of the research, delving into key themes that define the AI-driven revolution in data warehousing. The paper begins by surveying the historical trajectory of data warehousing, highlighting the challenges posed by the increasing complexity and heterogeneity of contemporary data sources. As businesses grapple with unstructured data, streaming data, and the need for real-time insights, the limitations of traditional data warehousing architectures become apparent. The introduction of AI-driven solutions revolutionizes data warehousing in several dimensions. Machine learning algorithms are harnessed for automating data integration, cleansing, and transformation processes, mitigating the manual labor associated with traditional ETL (Extract, Transform, Load) methods. Deep learning techniques, such as neural networks, unlock the potential to uncover complex patterns within massive datasets, enhancing predictive analytics and decision support capabilities. Moreover, the abstract explores the role of AI in enabling self-optimizing data warehouses. Adaptive query optimization, automated indexing, and real-time performance tuning emerge as pivotal components, ensuring that data warehouses evolve in response to changing workloads and user patterns. Ethical considerations and responsible AI practices within the context of data warehousing are also addressed. The abstract concludes by underlining the transformative impact of AI-driven solutions on the efficiency, agility, and strategic value of data warehousing, offering a glimpse into the future where intelligent data warehouses play a central role in shaping data-driven enterprises.

Very little effort has been put into developing a data warehouse for research literature mining due to research literature being semi-structured in nature where there is not any definitive design framework for such case. This study proposes a feasible data warehouse framework that can support the semi-structured research literature mining. Furthermore, this study also determines the core functional components that are unexplored yet required to realize the implementation of the design. In order to identify the challenges of implementing a semi-structured data warehouse, a prototype data warehouse was developed using the real world data sources of research literature published in “Computer in Human Behaviour” and “Information Systems” journals from the year 2008 to 2017 as the sample semi-structured data for proof of concept. The study results show that the designed data warehouse framework was able to support semi-structured data mining for revealing hidden pattern or trend of certain aspects of the research literature data.

Extraction-transformation-loading (ETL) process in data warehouse development perform data extraction from various resources, transform the data into suitable format and loadit into data warehouse storage. In the ETL process, there is data cleansing process function that handles redundancy, inconsistency and integrity data. ETL process will move data from the source to the integration layer (data store in data warehouse). In the integration layer, the data can be grouped into smaller scope and more specific for the requirement in other repositories called data marts. Reporting program of data warehouse will be associated with a data mart as its data source. In this research, the data warehouse is built to handle the ETL process. The data warehouse build metadata to support the process. The metadata construction for ETL processes will lead to ETL programs with high degree of reusability. The conclusion from this research is the use of dynamic ETL process (using metadata ETL) is required when ETL process is dealing with the operational system that still unstable and likely to change the database schema. Dynamic ETL process is also needed to address the increase requirement for report from the users.

In the digital era, the rapid growth of social media and online platforms has led to an explosion of unstructured textual data that holds significant business value. Traditional marketing strategies, once reliant on structured data such as demographics and purchase history, now benefit from insights derived from text analytics and sentiment analysis. This paper explores the integration of structured and unstructured data to strengthen marketing intelligence and customer segmentation. By utilizing text mining techniques and Natural Language Processing (NLP), unstructured data such as customer reviews and comments can be analyzed to extract sentiments, identify emerging trends, and refine customer relationship strategies. The study proposes an integrated framework that combines data extraction, transformation, and loading (ETL) processes with a data warehouse system for unified analysis. Using clustering algorithms such as K-Means and visualization tools, insights into customer behavior, preferences, and market segmentation are revealed. The paper also discusses the challenges of handling multilingual and context-dependent text, ethical and privacy considerations, and the technical architecture necessary for business intelligence implementation. Findings suggest that effective integration of textual analytics with structured data can lead to more informed decision-making, improved marketing strategies, and stronger customer engagement.

The proliferation of projects that are part of the semantic Web is truly impressive. In fact, ontologies become increasingly present in information systems, they constitute great data sources that arouse the interest of being analyzed. Ontologies are used for standardizing, structuring and formalizing the Web, Web Service, E-learning systems, and other fields. Regarding multidimensional approaches, researches in this field have focused on direct Data Warehouse conception from an ontology, which do not integrate the intervention of the expert in this process. In this case, the transformations are global and not very customizable; it can reproduce the inherent defects from the data sources into the resulting data warehouse. In this paper, we propose a new multi-approach model based on the coupling of relational database design approaches from an ontology with Data Warehouse design approaches from a relational database. Our model is semi-automatic allowing Data Warehouse design from an ontology by giving the designer more ability to intervene in this process and closely control the transformations. To assess the usefulness of our approach, we evaluated it by applying it on an example case study. The results of the example show that our approach is more accurate in terms of useful data filtering and adaptation of the multidimensional model to the end-users business-needs.

Warehousing for a decision support is a technique for retrieval and integration of data from distributed, autonomous, possibly heterogeneous, information sources. A data warehouse collects integrated information, available for queries and analysis. A materialized view of the system is kept at a site called the data warehouse, and user queries are processed using this view. The view has to be maintained to reflect the updates done against the base relations stored at the various data sources. A more efficient solution to be reflected updates done at each source is to materialize the relevant subsets of the data at the base relations pertaining to the actual view and process user queries against this local data. The updates done against the base relations are propagated to the materialized view and the view is maintained incrementally. However, earlier algorithms for incremental view maintenance either required a quiescent state at the data warehouse or required exponential message amounts in terms of the data sources. Therefore, in this paper, we proposed an enhanced algorithm and the algorithm instead does not require that the data warehouse be in a quiescent state for incorporating the new views and also the message amount between warehouse and sources can be reduced.

Data is an important asset and a fundamental requirement for building valuable information for organizations. Association of Information Systems Students of Unsika (Himsika) as a university organization provides many events to develop student’s academic and professional skills. A post-event evaluation through a feedback survey was conducted and stored in Google Sheets spreadsheet format. However, the current analysis process using spreadsheets lacks standardization, making it difficult to compare satisfaction rates over time and between events. Additionally, the lack of standardization leads to semi-structured data on spreadsheets, with varying question formats and meanings. To address these limitations, implementing a centralized data warehouse is proposed as a solution. The data warehouse would provide a structured and standardized approach to analyzing event feedback, enabling better comparisons and evaluation of management quality within Himsika. The research aims to design a data warehouse that supports multidimensional analysis. As a way to simplify and optimize analytical queries, the data structure is standardized in the data warehouse. The Four-step Dimensional Design method is applied in designing dimensional modeling on the data warehouse, consisting of four stages including selecting the business process, declaring the grain, identifying the dimensions, and identifying the facts. The design process resulted in 4 dimensions of events, dim_instances, dim_degree_programs, and dim_professions, and a fact table called fact_rates_by_responses. Overall, the proposed data warehouse and dimensional modeling approach aim to enhance the analysis and evaluation of Himsika’s events.

A Data warehouse (DW) is characterized by a complex architecture, designed in order to integrate data derived from operational data sources (DS), hence providing advanced analytical tools of these data. The DW is highly dependent on its DS. Hence, evolutions of the DS schema need to be propagated to the DW schema and content. This paper presents a model-driven approach for the evolution of a multidimensional DW. It is based on two evolution models: a first evolution model for the DS and another for the DW. These two models concern the data structure aspects as well as the evolution operations. The transition between these two models is performed through specific transformation rules defined in QVT (Query\View\Transformation).

In data warehouse (DW) development, a series of mappings must be specified between user concepts and data source elements, in order to identify which sources must undergo an integration process. Until now, these mappings are either assumed to be implied by name matching or identified according to the designer's experience. Then, the result is implemented as Extraction/Transformation/Loading (ETL) processes. Since ETL processes relate elements at the logical level, designers cannot adequately analyze how a change in requirements or in the data sources affects the analysis capabilities. Furthermore, this approach makes it difficult to perform incremental changes in DW design, requiring in some cases to perform the whole analysis again. In this paper we present a set of semantic mappings that relate user concepts specified by requirements to those obtained from data sources. In turn, this allows us to accurately identify how any potential change affects the different structures and ETL processes. As a DW evolves over time, our approach easily allows us to incorporate new concepts, as well as any change introduced at requirements or data sources into the DW repository with no need to redesign the whole DW. In order to show the application of our proposal, we show a real case study focusing on the Digital library of the University of Alicante.

In a data warehouse (DW) environment, when the operational environment does not posses or does not want to inform the data about the changes that occurred, controls have to be implemented to enable detection of these changes and to reflect them in the DW environment. The main scenarios are: i) the impossibility to instrument the DBMS (triggers, transaction log, stored procedures, replication, materialized views, old and new versions of data, etc) due to security policies, data property or performance issues; ii) the lack of instrumentation resources on the DBMS; iii) the use of legacy technologies such file systems or semi-structured data; iv) application proprietary databases and ERP systems. In another article [1], we presented the development and implementation of a technique that was derived for the comparison of database snapshots, where we use signatures to mark and detect changes. The technique is simple and can be applied to all four scenarios above. To prove the efficiency of our technique, in this article we do comparative performance tests between these approaches. We performed two benchmarks: the first one using synthetic data and the second one using the real data from a case study in the data warehouse project developed for Rio Sul Airlines, a regional aviation company belonging to the Brazil-based Varig group. We also describe the main approaches to solve the detection of changes in data origin.KeywordsPerformance TestData WarehouseOrigin AreaCleaning ManagementView MaintenanceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Chapter 64 - Exploiting Versions for On-line Data Warehouse Maintenance in MOLAP Servers