CAM2S: An Integrated Indexing Structure for Spatial Objects Generating Data Streams

One of the important research and technological issues in data warehouse performance is the optimization of analytical queries. Most of the research have been focusing on optimizing such queries by means of materialized views, data and index partitioning, as well as various index structures including: join indexes, bitmap join indexes, multidimensional indexes or index-based multidimensional clusters. These structures neither well support navigation along dimension hierarchies nor optimize joins with the Time dimension, which in practice is used in the majority of analytical queries. In this chapter we overview the basic index structures, namely: a bitmap index, a join index, and a bitmap join index. Based on these indexes, we show how to build another index, called Time-HOBI, for optimizing queries that address the Time dimension and compute aggregates along dimension hierarchies. We further discuss the extension of the index with additional data structure for storing aggregate values along the hierarchical structure of the index. The aggregates are used for speeding up aggregate queries along dimension hierarchies. Furthermore, we show how the index is used for answering queries in an example data warehouse. Finally, we discuss its performance-related characteristics, based on experiments.

Virtualization is very important factor to reduce cost of data warehouse deployment. It can reduce cost as well as tremendous pressure of managing devices, storage servers, application models and man power. In current era, data warehouse is very effective concept that can make huge impact in organizational decision support system. Unlike database systems, data warehouse systems take large amount of time and cost to deploy and develop in-house system for an organization. This is the reason that people now think about cloud computing as a solution of the problem instead of implementing their own data warehouse system. This paper is about, how cloud computing environment can be established as an alternative of data warehouse system. It will also throw some light about better environment choice for the organizational need. Organizational data warehouse and EC2 (Elastic Cloud Computing) are discussed with different parameters like ROI, security, scalability, robustness of data, maintenance of system etc. Paper will help in comparative analysis of both the systems.

Analysis of two-step index structure for complex spatial objects

Data Warehouse (DW) systems maintain sensitive and crucial information, which is integrated from various heterogenous sources of organization. With the ever increasing deployment and usage of networks, these systems are becoming more vulnerable to malicious attacks. With the increased number of attacks, intrusion detection has become vital part of Information Security. In this paper, we have proposed a model for analyzing and detecting anomalous events based on user behavior analysis through usage patterns, user profiles and session management. After monitoring the events in the system, if any intrusion activity occurs, then alerts are issued to system administrators. Since a user profile is not necessarily fixed but rather it evolves with changing time, so a dynamic user behavior modeling is represented as a sequence of events and combination of fact and dimension tables accessed by the users. In this way, DW systems may be protected by the malicious attacks.

In the era of Big Data, more and more stream data is available. In the same way, Decision Support Systems (DSS) tools, such as data warehouses and alert systems, become more and more sophisticated, and conceptual modeling tools are consequently mandatory for successfully DSS projects. Formalisms such as UML and ER have been widely used in the context of classical information and data warehouse systems, but they have not been investigated yet for stream data warehouses to deal with alert systems. Therefore, in this article, the authors introduce the notion of Active Stream Data Warehouse (ASDW) and this article proposes a UML profile for designing Active Stream Data Warehouses. Indeed, this article extends the ICSOLAP profile to take into account continuous and window OLAP queries. Moreover, this article studies the duality of the stream and OLAP decision-making process and the authors propose a set of ECA rules to automatically trigger OLAP operators. The UML profile is implemented in a new OLAP architecture, and it is validated using an environmental case study concerning the wind monitoring.

Data Warehouses (DWs) are currently considered to be the cornerstone of Business Intelligence (BI) systems. Security is a key issue in DWs since the business information that they manage is crucial and highly sensitive, and should be carefully protected. However, the increasing amount of data available on the Web signifies that more and more DW systems are considering the Web as the primary data source through which to populate their DWs. XML is therefore widely accepted as being the principal means through which to provide easier data and metadata interchange among heterogeneous data sources from the Web and the DW systems.Although security issues have been considered during the whole development process of traditional DWs, current research lacks approaches with which to consider security when the target platform is based on the Web and XML technologies. The idiosyncrasy of the unstructured and semi-structured data available on the Web definitely requires particular security rules that are specifically tailored to these systems in order to permit their particularities to be captured correctly.In order to tackle this situation, in this paper, we propose a methodological approach based on the Model Driven Architecture (MDA) for the development of Secure XML DWs. We therefore specify a set of transformation rules that are able to automatically generate not only the corresponding XML structure of the DW from secure conceptual DW models, but also the security rules specified within the DW XML structure, thus allowing us to implement both aspects simultaneously. A case study is provided at the end of the paper to show the benefits of our approach.

An efficient index structure for complex multi-dimensional objects is one of the most challenging requirements in non-traditional applications such as geographic information systems, computer-aided design, and multimedia databases. In this paper we first propose a main memory data structure for complex multi-dimensional objects. Then, we present an extension of the existing multi-dimensional index structure. Among existing multi-dimensional index structures, the popular Ra-tree is selected. The Ra-tree is coupled with the main memory data structure to improve the performance of spatial query processing. An analytical model is developed for our index structure. Experimental results show that the analytical model is accurate, the relative error being below 15%. The performance of our index structure is compared with that of a state-of-the-art index structure by experimental measurements. Our index structure outperforms the state-of-the-art index structure due to its ability to reduce a large amount of storage.

With the rapid development of network data communication technology, the complexity of the network environment makes the data in the data stream have uncertain characteristics, and a large number of data streams are generated in many fields. Therefore, it is necessary to find an efficient and accurate processing method to process a large amount of data in a computer network. This paper takes the skyline probability calculation and the actual efficiency of the unknown object index structure in the skyline query processing technology on the uncertain data stream as the research object. Based on the skyline query technology, we study an efficient skyline query processing method on the unknown network traffic based on the model SUMG. The method includes two algorithms: dynamic modeling algorithm DMG and skyline query algorithm GST. The DMG algorithm samples the data in the sliding window of the uncertain data stream and establishes a model to convert the data stream into the parameter stream in the uncertain object probability density function, the GST algorithm establishes the R-tree index structure, which is in order to reduce the amount of calculation, it uses the parameter flow of the model. In both methods, the set of local skyline results is first obtained at the distributed node, and the skyline query is performed again on the union of the local skyline results to obtain the global skyline result set. The experimental results show that compared with the skyline query method BNL for unknown network traffic without an index structure, the SUMG method can not only effectively model the link-type unknown object to assist the skyline query, but also effectively prune the uncertain data object and improve the skyline. The distributed skyline query method can cope with the skyline query task on the distributed data stream.

It is frequently the case that spatial queries require a result set of objects whose - however this may be more precisely defined - is the same as that of the query window. In this paper we present an approach which considerably improves query performance in such cases. By adding a dimension to the schema we make the index structure explicitly aware of the of a spatial object. The additional dimension causes the index structure to cluster objects not only by geographic location but also by scale. By matching scales of the query window and the objects, the query then automatically considers only relevant objects. Thus, for example, a query window encompassing an entire world map of political boundaries might return only national borders. Note that scale is not necessarily synonymous with size. This approach improves performance by both narrowing the initial selection criteria and by eliminating the need for subsequent filtering of the query result. In our performance measurements on databases with up to 40 million spatial objects, the introduction of a dimension decreased I/O by up to 4 orders of magnitude. The performance gain largely depends on the object distribution.We investigate a broad set of parameters that affect performance and show that many typical applications could benefit considerably from this technique. Its scalability is demonstrated by showing that the benefit increases with the size of the query and/or of the database. The technique is simple to apply and can be used with any multidimensional index structure that can index spatial extents and can be efficiently generalized to three or more dimensions. In our tests we have used the BANG index structure.

We introduce an object-oriented approach to transform a star-schema of a legacy data warehouse system into a Dimensional Object Model (DOM) (Firestone, 1998), in order to take advantage of the flexibility of the object paradigm. We have applied the Object-Oriented Software Engineering (OOSE) process, proposed by (Jacobson et al., 1992), to describe the life cycle of a data warehouse and to clarify the context of our intervention. By means of OOSE we also define the target architecture and the components that will ensure the development of reuse-supporting data warehouse systems. In a sense, a change tolerant architecture is proposed, as referred to by (Jacobson, 1998). The solution proposed applies the object paradigm for the star-schema decomposition in a three-layer architecture, separating the components in logical, interface and data management layers. This decomposition minimizes the dependence among the components and increases their reusability. The main contribution of this work is that, by transforming the star-schema into a DOM, we are preparing the system to evolve through any object-oriented data warehouse development methodology. In a context of many already existing data warehouse systems based on relational models, this transformation sounds especially interesting.

PurposeDespite the increasing role of the data warehouse as a supportive decision-making tool in today's business world, academic research for measuring its effectiveness has been lacking. This paucity of academic interest stimulated us to evaluate data warehousing effectiveness in the organizational context of Jordanian banks.Design/methodology/approachThis paper develops a theoretical model specific to the data warehouse system domain that builds on the DeLone and McLean model. The model is empirically tested by means of structural equation modelling applying the partial least squares approach and using data collected in a survey questionnaire from 127 respondents at Jordanian banks.FindingsEmpirical data analysis supported that data quality, system quality, user satisfaction, individual benefits and organizational benefits have made strong contributions to data warehousing effectiveness in our organizational data context.Practical implicationsThe results provide a better understanding of the data warehouse effectiveness and its importance in enabling the Jordanian banks to be competitive.Originality/valueThis study is indeed one of the first empirical attempts to measure data warehouse system effectiveness and the first of its kind in an emerging country such as Jordan.

The Extract-Transform-Load (ETL) process in data warehousing involves extracting data from source databases, transforming it into a form suitable for research and analysis, and loading it into a data warehouse. ETL processes can use complex transformations involving sources and targets that use different schemas, databases, and technologies, which make ETL implementations fault-prone. In this paper, we present an approach for validating ETL processes using automated balancing tests that check for various types of discrepancies between the source and target data. We formalize three categories of properties, namely, completeness, consistency, and syntactic validity that must be checked during testing. Our approach uses the rules provided in the ETL specifications to generate source-to-target mappings, from which balancing test assertions are generated for each property. We evaluated the approach on a real-world health data warehouse project and revealed 11 previously undetected faults. Using mutation analysis, we demonstrated that our auto-generated assertions can detect faults in the data inside the target data warehouse.

This research aims to investigate how sales type, product type, and price influence net profit in an F&B outlet using a Data Warehouse system. A quantitative method was applied by integrating daily transaction data into a Data Warehouse architecture built through an Extract, Transform, and Load (ETL) process, allowing the data to be more organized and easier to analyze. To address heteroskedasticity and obtain more reliable coefficient estimates, multiple linear regression with HC3 robust standard errors was used. The results show that price and sales type have a significant and positive effect on net profit, while product type shows mixed effects depending on the category. The regression model, with an R² value of 0.994, indicates that these three variables explain most of the variation in net profit. Overall, the findings highlight how structured data processing through a Data Warehouse can support profitability analysis and improve decision-making in F&B operations.

For a large company the speed of processing and distribution of data is very important. However, at Company XYZ, this has not yet been implemented in its entirety. This hinders the progress of the company. To be able to solve these problems, a storage system is needed that can access data more quickly and effectively by using a data warehouse system. This study discusses the design and construction of a data warehouse using the Kimball method. The data warehouse schema is created using SQL Server Management Studio and Microsoft SQL Server Integration Service. The analysis results from the data warehouse are displayed in the form of a dashboard using Microsoft PowerBI and a web-based application. Judging from the dashboard display that has been made, the production environment at PT Bio Farma is stable so it does not interfere with production quality. The data is entered into a web-based application, with the percentage of application feasibility of 84% after testing.

To solve the problem of data quality in data warehouse, the process of data stream in data warehouse and the factors influencing data quality were analyzed; a method for controlling data quality based on quality grading was proposed and the frame of data quality grading system was established. In each process of ETL, the data quality grading system would execute script to calculate values, such as total, component and standard measures in accordance with the definition of Unit Measure, so as to get representation indexes; and then the quality report would be generated according to the indexes and the quality would be graded. The method is applicable to various data warehouse systems.