A case for dynamic view management

Pre-computation and materialization of views with aggregate functions is a common technique in Data Warehouses. Due to the complex structure of the warehouse and the different profiles of the users who submit queries, there is need for tools that will automate the selection and management of the materialized data. In this paper we present DynaMat, a system that dynamically materializes information at multiple levels of granularity in order to match the demand (workload) but also takes into account the maintenance restrictions for the warehouse, such as down time to update the views and space availability. DynaMat unifies the view selection and the view maintenance problems under a single framework using a novel “goodness” measure for the materialized views. DynaMat constantly monitors incoming queries and materializes the best set of views subject to the space constraints. During updates, DynaMat reconciles the current materialized view selection and refreshes the most beneficial subset of it within a given maintenance window. We compare DynaMat against a system that is given all queries in advance and the pre-computed optimal static view selection. The comparison is made based on a new metric, the Detailed Cost Savings Ratio introduced for quantifying the benefits of view materialization against incoming queries. These experiments show that DynaMat's dynamic view selection outperforms the optimal static view selection and thus, any sub-optimal static algorithm that has appeared in the literature.

A company, majorly company that active in commercial (profit orientation) need to analyze their sales performance. By analyzing sales performance, company can increase their sales performance. One of method to analyze sales performance is by collecting historical data that relates to sales and then process that data so that produce information that show company sales performance. A data warehouse is a set of data that has characteristic subject oriented, time variant, integrated, and nonvolatile that help company management in processing of decision making. Design of data warehouse is started from collecting data that relate to sales such as product, customer, sales area, sales transaction, etc. After collecting the data, next is data extraction and transformation. Data extraction is a process f or selecting data that will be loaded into data warehouse. Data transformation is making some change to the data afte r extracted to be more consistent. After transformation processing, data are loaded into data warehouse. Data in data warehouse is processed by OLAP (On Line Analytical Processing) to produce information. Information that are produced from data processing by OLAP are chart and query reporting. Chart reporting are sales chart based on cement type, sales chart based on sales area, sales chart based on plant, monthly and year ly sales chart, and chart based on customer feedback. Query reporting are sales based on cement type, sales area, plant and customer. Keywords: Data warehouse; OLAP; Sales performance analysis; Ready mix market

On-Line Analytical Processing (OLAP) tools are frequently used in business, science and health to extract useful knowledgefrom massive databases. An important and hard optimization problem in OLAP data warehouses is the view selection problem, consisting of selecting a set of aggregate views of the data for speeding up future query processing. A common variant of the view selection problem addressed in the literature minimizes the sum of maintenance cost and query time on the view set. Converting what is inherently an optimization problem with multiple conflicting objectives into one with a single objective ignores the need and value of a variety of solutions offering various levels of trade-off between the objectives. We apply two non-elitist multiobjective evolutionary algorithms (MOEAs) to view selection under a size constraint. Our emphasis is to determine the suitability of the combination of MOEAs with constraint handling to the view selection problem, compared to a widely used greedy algorithm. We observe that the evolutionary process mimics that of the greedy in terms of the convergence process in the population. The MOEAs are competitive with the greedy on a variety of problem instances, often finding solutions dominating it in a reasonable amount of time.

SummaryThe Model‐Driven Architecture (MDA) is an approach that aligns modeling and automation for software development. By applying such an approach to data warehouse (DW) projects, we can minimize a great deal of time and cost. Furthermore, most of OnLine Analytical Processing (OLAP) platforms seem to be like black boxes that provide wizards only to business intelligence developers to create and manipulate OLAP objects without allowing their sustainability and migration from a platform to another. That is why many works in the literature have proposed using the MDA approach in DW projects. However, most of them have mainly focused on the generation of the DW relational model from the conceptual one, and they overlooked the OLAP model and the cube implementation.To deal with this problem, we propose in this paper an MDA solution to automate the process of getting OLAP cube and its implementation through a set of metamodels and automatic transformations among them. In fact, the proposal generates the OLAP and DW relational models (PSMs) from the conceptual one, using also a PDM model that describes the target business intelligence platform. After that, the source code to create the cube is got from both PSM models. For this aim, we define a set of transformation rules implemented using the Atlas transformation language. Finally, a case study will be provided to validate our approach.

Data cube has been playing an essential role in fast OLAP (online analytical processing) in many data warehouses. The pre-computation of data cubes is critical for improving the OLAP response time of in large high-dimensional data warehouses. However, as the sizes of data warehouses grow, the time it takes to perform this pre-computation becomes a significant performance bottleneck. In a high dimensional data warehouse, it might not be practical to build all these cuboids and their indices. In this paper, we propose a hierarchical cubing algorithm to partition the high dimensional data cube into low dimensional cube segments. It permits a significant reduction of CPU and I/O overhead for many queries by restricting the number of cube segments to be processed for both the fact table and bitmap indices. Experimental results show that the proposed method is significantly more efficient than other existing cubing methods.

Decision makers query enterprise information stored in Data Warehouses (DW) by using tools (such as On-Line Analytical Processing (OLAP) tools) which employ specific views or cubes from the corporate DW or Data Marts, based on multidimensional modelling. Since the information managed is critical, security constraints have to be correctly established in order to avoid unauthorized access. In previous work we defined a Model-Driven based approach for developing a secure DW repository by following a relational approach. Nevertheless, it is also important to define security constraints in the metadata layer that connects the DW repository with the OLAP tools; that is, over the same multidimensional structures that end users manage. This paper incorporates a proposal for developing secure OLAP applications within our previous approach: it improves a UML profile for conceptual modelling; it defines a logical metamodel for OLAP applications; and it defines and implements transformations from conceptual to logical models, as well as from logical models to secure implementation in a specific OLAP tool (SQL Server Analysis Services).

Business data collection is growing exponentially in recent years. A variety of industries and businesses have adopted new technologies of data storages such as data warehouses. On Line Analytical Processing (OLAP) has become an important tool for executives, managers, and analysts to explore, analyze, and extract interesting patterns from enormous amount of data stored in data warehouses and multidimensional databases. However, it is difficult for human analysts to interpret and extract meaningful information from large amount of data if the data is presented in textual form as relational tables. Visualization and interactive tools employ graphical display formats that help analysts to understand and extract useful information fast from huge data sets. This paper presents a new visual interactive exploration technique for an analysis of multidimensional databases. Users can gain both overviews and refine views on any particular region of interest of data cubes through the combination of interactive tools and navigational functions such as drilling down, rolling up, and slicing. Our technique allows users who are not experts in OLAP technology to explore and analyze OLAP data cubes and data warehouses without generating sophisticated queries. Furthermore, the visualization in our technique displays the exploration path enhancing the user's understanding of the exploration.

Models of multi-dimensional analysis for qualitative data and its application

Chapter 64 - Exploiting Versions for On-line Data Warehouse Maintenance in MOLAP Servers

Data warehouses are a collection of several databases, whose goal is to help different companies and corporations make important decisions about their activities. These decisions are taken from the analyses that are made to the data within the data warehouse. These data are taken from data that companies and corporations collect on daily basis from their branches that may be located in different cities, regions, states and continents. Data that are entered to data warehouses are historical data and they represent that part of data that is important for making decisions. These data go under a transformation process in order to accommodate with the structure of the objects within the databases in the data warehouse. This is done because the structure of the relational databases is not similar with the structure of the databases (multidimensional databases) within the data warehouse. The first ones are optimized for transactions on daily basis like: entering, changing, deleting and retrieving data through simple queries, the second ones are optimized for retrieving data through multidimensional queries, which enable us to extract important information. This information helps to make important decisions by learning which are the weak points and the strong points of the company, in order to invest more on the weak points and to strengthen the strong points, increasing the profits of the company. The goal of this paper is to treat data analyses for decision making from a data warehouse by using OLAP (online analytical processing) analysis. For this treatment we used the Analysis Services of Microsoft SQL Server 2016 platform. We analyzed the data of an IT Store with branches in different cities in Kosovo and came to a conclusion for some sales trends. This paper emphasizes the role of data warehouses in decision making.

Radio Frequency Identification (RFID) technology is used in many applications for monitoring object movement. The use of RFID in supply chain management systems enables to track the movement of products from suppliers to warehouses, store backrooms, and eventually points of sale. The vast amount of data resulting from the proliferation of RFID readers and tags poses challenges for data management and analytics. RFID data warehousing can enhance data quality and consistency, and give great potential benefits for Online Analytical Processing (OLAP) applications. Traditional data warehouses are built primarily on relational database management systems. However, the size of RFID data being collected and analyzed in the industry for business intelligence is growing rapidly, making traditional warehousing solutions prohibitively expensive. Hive is an open-source data warehousing solution built on top of Hadoop which is a popular Big Data computing framework. This paper presents alternative RFID data warehouse designs which can handle a large amount of RFID data and support a variety of OLAP queries. The proposed approaches are implemented on Hive and evaluated for query performance in cloud computing environment.

Traditional data warehousing technologies and On-Line Analytical Processing (OLAP) are unable to analyze textual data. Moreover, as OLAP queries of a decision-maker are generally related to a context, contextual information must be taken into account during the exploitation of data warehouses. Thus, we propose a contextual text cube model denoted CXT-Cube which considers several contextual factors during the OLAP analysis in order to better consider the contextual information associated with textual data. CXT-Cube is characterized by several contextual dimensions, each one related to a contextual factor. In addition, we extend our aggregation OLAP operator for textual data ORank (OLAP-Rank) to consider all the contextual factors defined in our CXT-Cube model. To validate our model, we perform an experimental study and the preliminary results show the importance of our approach for integrating textual data into a data warehouse and improving the decision-making.

In today's knowledge-, service-, and cloud-based economy, businesses accumulate massive amounts of data from a variety of sources. In order to understand businesses one may need to perform considerable analytics over large hybrid collections of heterogeneous and partially unstructured data that is captured related to the process execution. This data, usually modeled as graphs, increasingly come to show all the typical properties of big data: wide physical distribution, diversity of formats, non-standard data models, independently-managed and heterogeneous semantics. We use the term big process graph to refer to such large hybrid collections of heterogeneous and partially unstructured process related execution data. Online analytical processing (OLAP) of big process graph is challenging as the extension of existing OLAP techniques to analysis of graphs is not straightforward. Moreover, process data analysis methods should be capable of processing and querying large amount of data effectively and efficiently, and therefore have to be able to scale well with the infrastructure's scale. While traditional analytics solutions (relational DBs, data warehouses and OLAP), do a great job in collecting data and providing answers on known questions, key business insights remain hidden in the interactions among objects: it will be hard to discover concept hierarchies for entities based on both data objects and their interactions in process graphs. In this paper, we introduce a framework and a set of methods to support scalable graph-based OLAP analytics over process execution data. The goal is to facilitate the analytics over big process graph through summarizing the process graph and providing multiple views at different granularity. To achieve this goal, we present a model for process OLAP (P-OLAP) and define OLAP specific abstractions in process context such as process cubes, dimensions, and cells. We present a MapReduce-based graph processing engine, to support big data analytics over process graphs. We have implemented the P-OLAP framework and integrated it into our existing process data analytics platform, ProcessAtlas, which introduces a scalable architecture for querying, exploration and analysis of large process data. We report on experiments performed on both synthetic and real-world datasets that show the viability and efficiency of the approach.

Building Data Warehouse for Diseases Registry: First Step for Clinical Data Warehouse

As an analysis tool either of on-line analytical processing (OLAP) or data mining has its own strong points and weaknesses. It is an effective way to enhance the power and flexibility of data mining in data warehouses and large databases by integrating data mining with OLAP to offset their weaknesses. In the paper, a proposal of integrating data mining and OLAP is put forward. The mechanism of the on-line analytical data mining in data warehouses is described. Aspects which are necessary for developing a successful on-line analytical data mining system is also discussed.