A Study On Query Optimization for Federated Database Systems

We study the problem of query optimization in federated relational database systems. The nature of federated databases explicitly decouples many aspects of the optimization process, often making it imperative for the optimizer to consult underlying data sources while doing cost-based optimization. This not only increases the cost of optimization, but also changes the trade-offs involved in the optimization process significantly. The dominant cost in the decoupled optimization process is the "cost of costing" that traditionally has been considered insignificant. The optimizer can only afford a few rounds of messages to the underlying data sources and hence the optimization techniques in this environment must be geared toward gathering all the required cost information with minimal communication. In this paper, we explore the design space for a query optimizer in this environment and demonstrate the need for decoupling various aspects of the optimization process. We present minimum-communication decoupled variants of various query optimization techniques, and discuss tradeoffs in their performance in this scenario. We have implemented these techniques in the Cohera federated database system and our experimental results, somewhat surprisingly, indicate that a simple two-phase optimization scheme performs fairly well as long as the physical database design is known to the optimizer, though more aggressive algorithms are required otherwise.

The literature on physical database design in general, and on file segmentation in particular, typically ignores any consideration of data security and the cost to enforce it. If records can be physically designed so that all data elements in a given record type have identical security restrictions for a given user, then data element level security enforcement can be transformed into the less costly file level security enforcement for that user and rie. Similarly, if all record types have identical security restrictions, file based security might be sufficient. This paper extends an earlier model for file segmentation to include security considerations. The extended model embeds the security measures into the logical file structure and exploits a four category taxonomy of security restriction types. The model is used to generalize the interaction between element level selective security and physical database design.

S.70-79

2 - Manipulating Query Expressions

Extending classical functional dependencies for physical database design

The architecture of ADZE, an adaptive query optimizing system for federated databases, is presented. ADZE applies an explanation-based learning technique to learn from failures. It creates selection rules that guide the optimizer in situations where it has previously failed in selecting an optimal strategy. Other learning techniques, such as empirical learning and caching of values, are also used by the system in the process of creating and refining selection rules. The relevance of explanation-based learning to this type of application is discussed.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

The two stages of database design are logical and physical design. Logical database design, also referred to as , is done first. It is the process of studying the workings of a business organization, constructing a set of tables to store the business data, and understanding the constraints on the data, the dependencies between the tables, and the business rules concerning the data. The logical database design process is conducted without reference to any specific database technology such as Oracle or Microsoft SQL Server. Tools such as SQL Developer Data Modeler and Toad Data Modeler can be used for the purpose. Physical database design follows logical database design. First, the logical model is mapped to the proprietary features of the chosen database technology. Security requirements, integrity requirements, and business rules are also implemented. Finally, you consider performance: the ability of the database engine to handle work requests efficiently. This typically involves the creation of indexes on the data, and Oracle Database provides a wealth of indexing mechanisms to choose from.

Article Free Access Share on Query optimization and processing in federated database systems Authors: Ee-Peng Lim Department of Computer Science, University of Minnesota, Minneapolis, MN Department of Computer Science, University of Minnesota, Minneapolis, MNView Profile , Jaideep Srivastava Department of Computer Science, University of Minnesota, Minneapolis, MN Department of Computer Science, University of Minnesota, Minneapolis, MNView Profile Authors Info & Claims CIKM '93: Proceedings of the second international conference on Information and knowledge managementDecember 1993 Pages 720–722https://doi.org/10.1145/170088.170469Online:01 December 1993Publication History 5citation658DownloadsMetricsTotal Citations5Total Downloads658Last 12 Months18Last 6 weeks8 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Data mining algorithms are often embedded in more complex systems, serving as the provider of data for internal decision making within these systems. In this paper we address an interesting problem of using data mining techniques for database query optimization. We introduce the concept of conditional cardinality patterns and design an algorithm to compute the required values for a given database schema. However applicable to any database system, our solution is best suited for data warehouse environments due to the special characteristics of both database schemata being used and queries being asked. We verify our proposal experimentally by running our algorithm against the state-of-the-art database query optimizer. The results of conducted experiments show that our algorithm outperforms traditional cost-based query optimizer with respect to the accuracy of cardinality estimation for a wide range of queries.KeywordsData WarehouseDatabase SchemaQuery OptimizationData Mining AlgorithmFact TableThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Query processing is that method or technique in area of mobile atmosphere which deal using mobile environment. This research in that way user can think advancement using query processing and it takes more use for speedily accessing global query processing and it is joint venture of query processing between dissimilar sites includes fixed server movable computer. The necessity saving of energy and also the existence of lopsided features in a moveable computing atmosphere, the predictable query processing for a scattered database cannot be straightly genuine to a moveable computing organization. The mobile environment is a collection of mobile diverse hosts, which are enabled to communicate using wireless links. These wireless links are change giving to the natures of mobile networks. Moreover, nodes in the ad-hoc network have to link without any centralized or help. The usability for the user also changes using query processing tool. Thus, this mechanism that allows the sharing of functionality among different devices in same environment change the way of user communication for searching fast query time in mobile computing environment. The aim of this research is to make innovation in query optimization technique in mobile computing environment. This research focuses on improvement of various query optimization techniques for highly effective and trustworthy query optimization methods.

Due to the ubiquity of spatial data applications and the large amounts of spatial data that these applications generate and process, there is a pressing need for scalable spatial query processing. In this paper, we present new techniques for spatial query processing and optimization in an in-memory and distributed setup to address scalability. More specifically, we introduce new techniques for handling query skew that commonly happens in practice, and minimizes communication costs accordingly. We propose a distributed query scheduler that uses a new cost model to minimize the cost of spatial query processing. The scheduler generates query execution plans that minimize the effect of query skew. The query scheduler utilizes new spatial indexing techniques based on bitmap filters to forward queries to the appropriate local nodes. Each local computation node is responsible for optimizing and selecting its best local query execution plan based on the indexes and the nature of the spatial queries in that node. All the proposed spatial query processing and optimization techniques are prototyped inside Spark, a distributed memory-based computation system. Our prototype system is termed LocationSpark. The experimental study is based on real datasets and demonstrates that LocationSpark can enhance distributed spatial query processing by up to an order of magnitude over existing in-memory and distributed spatial systems.

Due to the increasing complexity in application workloads and query engines, database administrators are turning to automated tuning tools that systematically explore the space of physical design alternatives. A critical element of such tuning is physical database design since the choice of physical structures has a significant impact on the perfor

On the selection of secondary indices in relational databases

A statistical profile summarizes the instances of a database. It describes aspects such as the number of tuples, the number of values, the distribution of values, the correlation between value sets, and the distribution of tuples among secondary storage units. Estimation of database profiles is critical in the problems of query optimization, physical database design, and database performance prediction. This paper describes a model of a database of profile, relates this model to estimating the cost of database operations, and surveys methods of estimating profiles. The operators and objects in the model include build profile, estimate profile, and update profile. The estimate operator is classified by the relational algebra operator (select, project, join), the property to be estimated (cardinality, distribution of values, and other parameters), and the underlying method (parametric, nonparametric, and ad-hoc). The accuracy, overhead, and assumptions of methods are discussed in detail. Relevant research in both the database and the statistics disciplines is incorporated in the detailed discussion.

Rapidly increasing storage device capacities at ever decreasing costs have resulted in mushrooming of publicly available large data sets on the Web. In this paper, we describe a novel approach to teaching relational database course by using such data repositories. We demonstrate our approach using the Amazon.com product database, though the approach is generic and is applicable to other data repositories. The Amazon database is supposedly the largest product database ever in existence. We have used the Amazon Web Services API and .NET/C# application to extract a subset of the product database to enhance student learning in a relational database course. This realistic data served various activities of the course and provided a rich backdrop to demonstrate more interesting features of SQL and Oracle cost-based query optimization. Central to the course is a semester-long team project. We discuss the details of data extraction from Amazon.com, conceptual and logical data modeling, logical and physical database design, database creation and data loading, database querying, and database application development.