Incorporating load factor into the scheduling of soft real-time transactions for main memory databases

Index is indispensable in database system to speeding up data access. Commonly used indexes in main memory databases are B+-Tree, T-Tree and their variants. Cost model of main memory database are more complex than disk-oriental database, and relatively little work has been done on this area. This paper establishes a cost model for B+-Tree and CSB+-Tree, analyzes their main performance factors. We perform several experiment evaluations on the cost model.

The declining cost of main memory and need for high performance database systems have recently inspired research into systems with massive amounts of memory and the ability to store complete databases in main memory It is recognized that in this framework the issues concerned with efficient database recovery are more complex than in traditional DBMS systems. In this paper we define a classification method for main memory database (MMDB) recovery techniques and discuss the results of a performance analysis conducted examining the different classes.

There are some shortcomings for the cache sensitivity of the index in main memory database, so a new index structure is proposed. T -tree index is studied individually ever before, so as Hash index. Combined with the analysis of the two index structure, a new index structure called the T-Hash tree is introduced. Through analyzing the times of the T -Hash tree cache sensitive and testing the performance of the query, insert, delete operation, the results show that the T -Hash tree can effectively reduce the times of cache sensitive, and as the amount of the data is large, the query, insert, delete efficiency of the T -Hash tree is higher than the T tree.

The objective of this paper is to develop and analyze high performance hash based search methods for main memory databases. We define optimal search in main memory databases as the search that requires at most one key comparison to locate a record. Existing hashing techniques become impractical when they are adapted to yield optimal search in main memory databases because of their large directory size. Multi-directory hashing techniques can provide significantly improved directory utilization over single-directory hashing techniques. A multi-directory hashing scheme, called fast search multi-directory hashing, and its generalization, called controlled search multi-directory hashing, are presented. Both methods achieve linearly increasing expected directory size with the number of records. Their performance is compared to existing alternatives.

Database systems normally have been designed with the assumption that main memory is too small to hold the entire database. With the decreasing cost and increasing performance of semiconductor memories, future database systems may be constructed that keep most or all of the data for the database in main memory. The challenge in the design of these systems is to provide fast transaction processing, to effectively use multiple processors, and to perform a fast restart after a crash. This correspondence presents a method of performing crash recovery for these systems.

In the embedded environment, the rapid and efficient recovery is needed in the event of the real-time main memory database system failures. Checkpointing strategy is very important for system recovery as one of the most important techniques of real-time main memory database. A fuzzy checkpointing strategy based on the material embedded real-time main memory database system ART-II (an active real-time main memory database) is suggested, and a redo point strategy for this checkpointing strategy is also developed. The performance test shows that this checkpointing strategy and the redo point strategy can decrease the ratio of the transactions missing the deadline and improve the performance of the system.

The data intensive analytical workload becomes heavy burden for OLAP engine with increasing data volume, user population and query complexity. Large capacity random access memory, multi-level cache and multi-core hardware are main streams of computer. We propose a hardware-aware OLAP model named MOSS-DB which optimizes storage model according to data access features of dimensional tables and fact tables. A hard disk & main memory two-level storage model is employed to support directly dimensional tuple accessing join operator(DDTA-JOIN), DDTA-JOIN simplifies OLAP query processing by replacing traditional join operation with directly accessing dimensional tuple with memory address. So the star schema can be seen as virtual de-normalized table, OLAP query is also simplified to table scan, select and project operations. Query processing on sequence data structure is more suitable for multi-core parallel processing. Our proposal allows massive data DRDB(Disk Resident Database) storage technique to cooperate with MMDB(Main-Memory Database) processing technique, which breaks the main memory capacity limitation. The DDTA-JOIN operation can save cost for index, hash table, etc. For multi-core era, MOSS-DB can flexibly use parallel processing capability of CPU by dynamically dividing fact table into multiple scan partitions and gain maximum cache profit for shared dimensional data. In experiments, we measure that MOSS-DB outperforms conventional DRDB system, and it also outperforms MMDB in SSB testing.

As the request of database services about the intelligent community, in this paper, an intense research for the Distributed Real-Time Database (DRTDB) about the intelligent community has been conducted. And then a database service architecture about the intelligent community has been obtained. Besides, the data storage scheme of Main Memory Database (MMDB) and distributed real-time transaction processing technologies have been studied and applied to this intelligent community. The experiments for inserting and querying functions on DRTDB and common database have been done. Finally, the conclusion shows that the technologies of DRTDB and MMDB in this paper can really meet the requirements of intelligent community in the aspect of real-time data access.

Many database replication protocols are based on the deferred update replication technique. In deferred update replication, transactions are executed by a single server, and certified and possibly committed by every server. Thus, servers must store a full copy of the database. This assumption is detrimental to performance since servers may not be able to cache the entire database in main memory. This paper introduces RAM-DUR, a variation of deferred update replication whereby transaction execution is in-memory only. RAM-DUR's key insight is a sophisticated distributed cache mechanism that provides high performance and strong consistency without the limitations of existing solutions (e.g., no single server must have enough memory to cache the entire database). In addition to presenting RAM-DUR, we detail its implementation, and provide an extensive analysis of its performance.

Main memories are becoming sufficiently large that most OLTP databases can be stored entirely in main memory, but this may not be the best solution. OLTP workloads typically exhibit skewed access patterns where some records are hot (frequently accessed) but many records are cold (infrequently or never accessed). It is more economical to store the coldest records on secondary storage such as flash. As a first step towards managing cold data in databases optimized for main memory we investigate how to efficiently identify hot and cold data. We propose to log record accesses - possibly only a sample to reduce overhead - and perform offline analysis to estimate record access frequencies. We present four estimation algorithms based on exponential smoothing and experimentally evaluate their efficiency and accuracy. We find that exponential smoothing produces very accurate estimates, leading to higher hit rates than the best caching techniques. Our most efficient algorithm is able to analyze a log of 1B accesses in sub-second time on a workstation-class machine.

The authors examine the effect of different partitioning techniques on the MMDB (main memory database) reload problem in terms of the number of I/Os for reload and number of MM references during transaction processing. The best technique is the one that yields the minimum overall cost with regard to both properties. It is shown that horizontal and single vertical partitioning are actually the only possible candidates. Physical vertical never yields the best result. In some very rare cases, group vertical outperforms the other techniques. If the database system encountered performs more selections than projections and joins, and more tuple modifications or tuple deletions than tuple insertions, then horizontal is the best technique. Otherwise, single vertical is the chosen technique. It is also shown that, if reload is the only concern, that is, if the transaction performance is not taken into account, then single vertical is always the best choice. >

Recovery in main memory database systems for engineering design applications

The performance of two main-memory database system (MDBS) algorithms is studied. M.H. Eich's (Proc. of the Fifth Int. Workshop on Database Machines., Oct. 1987) and T.J. Lehman's (Ph. D. Thesis, Univ. of Wisconsin-Madison, Aug 1986) algorithms are selected for this study. The investigation indicates that the shadow approach, as used by Eich, has some advantages over the update in-place strategy of Lehman. The shadow approach has faster response for normal transactions even though transaction commit is slower compared to update in-place approach. It is concluded that irrespective of the recovery algorithm an efficient load balancing plays an important part in the performance of the system. It is suggested that processors should be allocated to any kind of activity on a demand basis. It is discovered that individual recovery of lockable units as used in Lehman's algorithm is a better choice since it increases the system availability after a system failure. It also allows background recovery to go in parallel to normal transaction processing. >

Considering the frequent requirements for cooperative simulators in experiments of naval electronics information system and taking the expansibility of simulators and the portability of models into account, a simulator software framework for naval electronic information system is designed in this paper. The framework has a hierarchical structure, introduces the main memory database (MMDB), and a set of tools is designed in this framework to realize the major function of it. Based on the framework, the developing process of simulators is discussed; and taking the radar simulator as an example, the simulator is constructed to show the implementability and features of the framework.

Traditional real-time systems have largely avoided the use of disks due to their relative slow speeds and their unpredictability. However, many real-time applications including multimedia systems and real-time database applications benefit significantly from the use of disks to store and access real-time data. We investigate the problem of obtaining guaranteed timely access to files on a disk in a real-time system. Our study focuses on several aspects of this problem of providing a real-time filesystem. First, we consider the use of two real-time disk scheduling algorithms: earliest deadline scheduling and just-in-time scheduling, a variation of aperiodic servers for the disk. The latter algorithm is designed to improve disk throughput that can be hurt when a real-time scheduling algorithm such as EDF is applied directly. Admission control policies with practically acceptable properties of performance and usability are provided. Next, we design and implement a real-time filesystem on the RT-Mach microkernel-based system running a real-time shell. The new interface we develop is based on RT-Mach's resource reservation paradigm and provides guaranteed and timely access for multiple concurrent applications requiring disk bandwidth with different timing and volume requirements. Finally, we perform a detailed performance evaluation of the real-time filesystem including its raw performance. We show the following positive but rather surprising result: our real-time scheduling filesystem not only provides guaranteed and timely access but also does so at relatively high levels of throughput. Traditional disk scheduling algorithms offer completely unacceptable file access latencies for real-time applications and do so only at slightly higher throughput.