Materialized view selection and maintenance using multi-query optimization

Materialized views (MV) at the data warehouse (DW) can be kept up to date in response to changes in data sources without accessing data sources for additional information. This process is usually refered to as “self maintenance of views”. A number of algorithms have been proposed for self maintenance of views, which use auxiliary views (AV) to keep some additional information in DW. In this paper we propose an algorithm for self maintainability of multiple MVs using the above approach. Our algorithm generates a simple maintenance query to incrementally maintain an MV along with its AV at DW. The algorithm maintains these views by minimizing the number and the size of the AVs. Our approach provides better insight into view maintenance issues by exploiting the dependencies and constraints that might exist in the data sources and multiple MVs at DW.

Data Warehouse (DW) systems maintain sensitive and crucial information, which is integrated from various heterogenous sources of organization. With the ever increasing deployment and usage of networks, these systems are becoming more vulnerable to malicious attacks. With the increased number of attacks, intrusion detection has become vital part of Information Security. In this paper, we have proposed a model for analyzing and detecting anomalous events based on user behavior analysis through usage patterns, user profiles and session management. After monitoring the events in the system, if any intrusion activity occurs, then alerts are issued to system administrators. Since a user profile is not necessarily fixed but rather it evolves with changing time, so a dynamic user behavior modeling is represented as a sequence of events and combination of fact and dimension tables accessed by the users. In this way, DW systems may be protected by the malicious attacks.

A data warehouse mainly stores integrated information over data from many different remote data sources for query and analysis. The integrated information at the data warehouse is stored in the form of materialized views. Using these materialized views, user queries may be answered quickly and efficiently as the information may be directly available. These materialized views must be maintained in answer to actual relation updates in the different remote sources. One of the issues related to materialized views is that whether they should be recomputed or they should be adapted incrementally after every change in the base relations. View maintenance is the process of updating a materialized view in response to changes to the underlying data is called view maintenance. There are several algorithms developed by different authors to ease the problem of view maintenance for data warehouse systems. In this paper, we have provided a comprehensive study on research works of different authors related to DW view maintenance considering various parameters and presented the same in tabular way.

Virtualization is very important factor to reduce cost of data warehouse deployment. It can reduce cost as well as tremendous pressure of managing devices, storage servers, application models and man power. In current era, data warehouse is very effective concept that can make huge impact in organizational decision support system. Unlike database systems, data warehouse systems take large amount of time and cost to deploy and develop in-house system for an organization. This is the reason that people now think about cloud computing as a solution of the problem instead of implementing their own data warehouse system. This paper is about, how cloud computing environment can be established as an alternative of data warehouse system. It will also throw some light about better environment choice for the organizational need. Organizational data warehouse and EC2 (Elastic Cloud Computing) are discussed with different parameters like ROI, security, scalability, robustness of data, maintenance of system etc. Paper will help in comparative analysis of both the systems.

A Data warehouse is a structure that stores big amount of data. This data is exploited in the best possible ways in order to improve the efficiency of decision-making. The huge volume of data makes answering queries complex and time-consuming. Therefore, materialized views are used in order to reduce the query processing time. Since materializing all views is not possible, due to space and maintenance constraints, materialized view selection became one of the crucial decisions in designing a data warehouse for optimal efficiency. In this paper, we propose a discrete - quantum based - version of Differential Evolution DE algorithm named QDE algorithm to solve the materialized view selection (MVS) problem with space constraint. This algorithm is a merging of the original DE with Quantum Evolutionary (QEA) algorithm. The experimental results show the efficiency of the proposed algorithm compared to well known algorithms used to solve MVS problem with space constraint such as HRUA and GEA.

Materialized view selection under the maintenance time constraint

A data warehouse is designed for answering analytical queries. These queries are usually long, complex and ad hoc in nature and their response time is high when processed against an exponentially growing data warehouse. Materialising views has been found to be an effective tool for reducing this response time. All views cannot be materialised on account of resource constraints. Further, optimal view selection is shown to be an NP-complete problem. This necessitates selection of an appropriate set of views, from amongst all possible views that reduce the query response time. Most of the view selection algorithms are greedy or evolutionary. In this paper, a differential evolution view selection algorithm (DEVSA) that selects the Top-K views from a multi-dimensional lattice is proposed. Further, it is shown that DEVSA when compared to the greedy, genetic and memetic-based view selection algorithms selects comparatively better quality views for higher dimensional datasets.

Maeka Health Promotion Hospital conducts a survey on health conditions and collects the data involving activities or organization operations each year, particularly, the data on the elderly health care having too much. This might lead to difficulties in searching and analyzing the information. To solve this problem, it is essential to rely on tools and information technology in making a decision of the administrator rapidly and correctly as well as being the information for caring for and rehabilitating the elderly's health. The objectives of this study are to develop the data warehouse system and to create data visualization to depict the online report in the form of a dashboard. The research results indicate that the data warehouse system in supporting the elderly’s health care consists of four reports such as the report on the elderly’s score Activity of Daily Living (ADL), the statistical report on the elderly’s living, the reports on the elderly’s behavior, ailment and physical disabled and the statistical report on the elderly's health problem and the depression condition evaluation. Meanwhile, the user's satisfaction with the data warehouse system is at a good level ( = 4.21, S.D. = 0.08).

We introduce an object-oriented approach to transform a star-schema of a legacy data warehouse system into a Dimensional Object Model (DOM) (Firestone, 1998), in order to take advantage of the flexibility of the object paradigm. We have applied the Object-Oriented Software Engineering (OOSE) process, proposed by (Jacobson et al., 1992), to describe the life cycle of a data warehouse and to clarify the context of our intervention. By means of OOSE we also define the target architecture and the components that will ensure the development of reuse-supporting data warehouse systems. In a sense, a change tolerant architecture is proposed, as referred to by (Jacobson, 1998). The solution proposed applies the object paradigm for the star-schema decomposition in a three-layer architecture, separating the components in logical, interface and data management layers. This decomposition minimizes the dependence among the components and increases their reusability. The main contribution of this work is that, by transforming the star-schema into a DOM, we are preparing the system to evolve through any object-oriented data warehouse development methodology. In a context of many already existing data warehouse systems based on relational models, this transformation sounds especially interesting.

A data warehouse is a central repository of historical data designed primarily to support analytical processing. These analytical queries are exploratory, long and complex in nature. Further, the rapid and continuous growth in the size of data warehouse increases the response times of such queries. Query response times need to be reduced in order to speedup decision making. This problem, being an NP-Complete problem, can be appropriately dealt with by using swarm intelligence techniques. One such technique, i.e. the set-based particle swarm optimization (SPSO), has been proposed to address this problem. Accordingly, a SPSO based view selection algorithm (SPSOVSA), which selects the Top-K views from a multidimensional lattice, is proposed. Experimental based comparison of SPSOVSA with the most fundamental view selection algorithm shows that SPSOVSA is able to select comparatively better quality Top-K views for materialization. The materialization of these selected views would improve the performance of analytical queries and lead to efficient decision making.

In order to efficiently serve a high volume of queries, LinkedIn's distributed graph service caches computationally-expensive materialized views. Despite this, even basic view maintenance has been a scalability bottleneck. In this paper, we introduce Partial Update, which efficiently applies incremental view maintenance in the absence of deletions in a materialized view. Without expensive transaction handling, Partial Update guarantees eventual consistency across source of truth databases, materialized views and base relations stores. By design, its delta change computation logic only produces lightweight queries, reducing any need to augment the underlying data storage systems. This allows Partial Update to seamlessly integrate on top of traditional eventually consistent systems. In LinkedIn's production systems, Partial Update achieves up to a 50% reduction in both maintenance time and network bandwidth utilization. The improved view maintenance scalability allows us to improve the quality of cached data by a factor of 20. We consider that Partial Update is general enough to be applicable to the majority of select-project-join view maintenance strategies in eventually consistent distributed data stores.

A general procedure for highway maintenance planning at the project level is developed in this study. Three key models used in the planning process are explored as well; the performance prediction model, the thickness equivalent model and the cost-benefit analysis model. The process of maintenance planning can be divided into six steps and the maintenance plan optimization is based on Life Cycle Cost Analysis (LCCA). This study examines a real asphalt highway in Northern China to explain in detail how to develop project-level planning. The planning results include both maintenance strategies and maintenance time. According to the feedback from the field research, the maintenance plan made by this way is in agreement with the local working experience and quantifies the experience.

A Data Warehouse (DW) can be abstractly seen as a set of materialized views defined over a set of remote data sources. A DW is intended to satisfy a set of queries. The views materialized in a DWrelate to each other in a complex manner, through common subexpressions, in order to guarantee high query performance and low view maintenance cost. DWs are time varying. As time passes new materialized views are added in order to satisfy new queries or for performance reasons while old queries are dropped. The evolution of a DWcan result in a redundant set of materialized views.In this paper we address the problem of detecting redundant views in a given DW view selection, that is, views that can be removed from the DW without negatively affecting the query evaluation or the view maintenance process. Using an AND/OR dag representation for multiple queries and views, we first provide a method for detecting materialized views that are not needed in the process of propagating source relation changes to the DW. Then, we use this method to detect materialized views that are redundant. As a side effect, our approach shows how source relation changes can be propagated to the DW materialized views by exploiting common subexpressions between views and by using other materialized views that are not affected by these changes.

The study investigates the practice of data warehouse system selection decisions for Small and Medium-sized Enterprises (SMEs) in Taiwan. As data warehouse system evaluation and selection are often costly and time consuming, the need for a decision-aid approach to software selection is obvious, especially for SMEs. The aim of this study is to determine significant factors that influence data warehouse system selection of SMEs. Both technical and managerial factors are considered in structuring an evaluation hierarchy based on the Analytic Hierarchy Process (AHP). The results indicate that SMEs select data warehouse systems that focus mainly on cost and vendor criteria.

Decision makers in the tourism sector face real challenges. There is no chance to access a unified data source that can supply information to meet their inquiries and expectations. In this paper, we propose a data warehouse system for supporting the decision makers in the Egyptian tourism sector. The methodology followed to build this system is descried in details to guide the other developing countries who are trying to support their decision makers in the tourism sector by building a similar data warehouse system. In addition, we evaluated the tourism data warehouse system. The results of evaluation indicated that it can meet the needs and requirements of the decision makers in the tourism sector.