Distributed Information Storage Architecture

Our research into multimedia information management and storage focuses on enabling database systems to deal with multimedia content at a high level of abstraction. We examine the database system's role in managing multimedia data. Database systems need to do more than providing a document model or binary large objects to implement multimedia database systems. We examine what the database system's role is in managing multimedia data. We then show how the role of the database system merges with the role of media servers that is an extension of traditional file servers with isochronous multimedia data delivery capabilities. The database system provides richer data management opportunities than media servers because of its increased knowledge of the stored multimedia data. This allows the database system to optimize and better manage system resources and multimedia data delivery.

In 1987, Gray and Putzolo presented the five-minute rule, which was reviewed and renewed ten years later in 1997. With the advent of flash memory in the gap between traditional RAM main memory and traditional systems, the five-minute rule now applies to large pages appropriate for today's disks and their fast transfer bandwidths, and it also applies to flash disks holding small pages appropriate for their fast access latency.Flash memory fills the gap between RAM and disks in terms of many metrics: acquisition cost, access latency, transfer bandwidth, spatial density, and power consumption. Thus, within a few years, flash memory will likely be used heavily in operating systems, file systems, and database systems. Research into appropriate system architectures is urgently needed.The basic software architectures for exploiting flash in these systems are called extended buffer pool and extended disk here. Based on the characteristics of these software architectures, an argument is presented why operating systems and file systems on one hand and database systems on the other hand will best benefit from flash memory by employing different software architectures.

In order to store and manage the basic information and inspection task for overhead high voltage power transmission line inspection robots, a database system is built up. This database system can store and manage such information as sensor information, historical data, report, inspection task, and line fault detected. Moreover, there is a simple line fault expert database in the database system developed for automatic line fault detection of the robots. SQLite is adopted as the backend database in this system. According to the specific requirements of the actual inspection application, the E-R model of the database system is designed, which is composed of two parts: sensor data model and line inspection model. Application experiments reveal that this database system can accomplish the information storage and management work for inspection robots.

Indexes are commonly employed to retrieve a portion of a file or to retrieve its records in a particular order. An accurate performance model of indexes is essential to the design, analysis, and tuning of file management and database systems, and particularly to database query optimization. Many previous studies have addressed the problem of estimating the number of disk page fetches when randomly accessing k records out of N given records stored on T disk pages. This paper generalizes these results, relaxing two assumptions that usually do not hold in practice: unlimited buffer and unique records for each key value. Experiments show that the performance of an index scan is very sensitive to buffer size limitations and multiple records per key value. A model for these more practical situations is presented and a formula derived for estimating the performance of an index scan. We also give a closed-form approximation that is easy to compute. The theoretical results are validated using the R * distributed relational database system. Although we use database terminology throughout the paper, the model is more generally applicable whenever random accesses are made using keys.

A new system architecture shares certain characteristics with database systems, expert systems, functional programming languages, and spreadsheet systems, but is very different from any of these.

The workstation file system for the Cedar programming environment was modified to improve its robustness and performance. Previously, the file system used hardware-provided labels on disk blocks to increase robustness against hardware and software errors. The new system does not require hardware disk labels, yet is more robust than the old system. Recovery is rapid after a crash. The performance of operations on file system metadata, e.g., file creation or open, is greatly improved.The new file system has two features that make it atypical. The system uses a log, as do most database systems, to recover metadata about the file system. To gain performance, it uses group commit, a concept derived from high performance database systems. The design of the system used a simple, yet detailed and accurate, analytical model to choose between several design alternatives in order to provide good disk performance.

Conventional storage and retrieval of information from telemedicine environments is usually based on ordinary database systems. Aspects such as scalability, information distribution, high performance system techniques and operational costs are well known challenges to be circumvented in the research for novel proposals in the field of large-scale telemedicine systems. In this paper we present an architecture that targets high performance levels in storing and retrieving DICOM medical images, adopting a distributed approach in a cluster configuration. Our proposal has two main components: the first element is a data model that is based on image hierarchy, considering the hierarchical data format 5 (HDF5). The second component is a distributed file system, characterised by the parallel virtual file system (PVFS) that was employed in this proposal as a distributed storage data system. As a result, this paper presents a differentiated approach for storage and retrieval of information for a telemedicine environment. Experimental results, utilising the architecture, indicate an enhanced level of performance around 16% in terms of storage process. This number represents an improved performance in comparison to a conventional database system.

The future of computer aided design is in object oriented programming. If the database community hopes to participate in this future, it must reexamine some basic assumptions about the architecture of database systems. Database system functionality can be added to object systems but if the performance cost is too high, it will never survive. Below are some suggestions for what can be done at a reasonable performance cost. The object oriented paradigm provides a more practical approach to the partitioning of the global database than horizontal and vertical partitioning of relational tables. Each partition should itself be an independent database containing related data such as the geometry of a part or the spacial relationship of parts in an assembly. A meta-database would be used to control access to collections of these partitions. A collection of partitions comprise the database for a user's design session. The overhead of traditional database transaction management is not acceptable for high performance CAD systems. With the partitioning scheme described above, transaction management can be performed at a partition/session granularity. Once the user has composed the collection of partitions, he has a single user database. There is no need for concurrency control or transaction logging except at the meta-database level. This type of transaction management can in fact be more functional than traditional transaction management, allowing for versioning, long transactions, integrity checking and archival. Object oriented databases need a message model, not a data model. Any object which responds to the same messages as an object of “Duck” class (walk and quack) is, for all intents and purposes, a duck. An attempt to design a data model based on instance variables of an object or based on collections of objects of like class violates the data abstraction facilities of object oriented languages and diminishes their power. An attempt to implement a relational database system with an object oriented language yields a relational database system where you get abstract data types for free. It does not yield an object oriented database system. For object oriented queries, the message is the media. A query can be transformed into an execution plan consisting of messages sent to database objects. Optimization decisions can be made by sending messages to referenced objects. Collection classes can be implemented for new access methods with cost and selectivity methods to provide optimization information. In this way, the query language can grow with the application. Data representation is an important aspect of object oriented systems. Most object systems are typeless in that all instance variables of an object are object references. For performance sake, object systems should provide enough of a type mechanism to allow simple data items (integers, floats, characters … ) to be represented in the form intrinsic to the machine. Methods can then be compiled for access to typed data. In conclusion, object systems provide enormous potential for the development of CAD systems. Performance influences the approach taken to an application. WYSIWYG publishing applications were not attempted until performance was adequate. Functionality is what sells CAD systems. Database system functionality can be added to object systems at a reasonable cost.

Over the last several years, a number of hardware and software systems have been developed which map entire files directly into the virtual memory address spaces used by programs. Since all file contents are directly addressable, there is no need for a programmer to issue explicit file system actions, such as Read or Write. In addition, all of the buffer management problems are eliminated, since programmers do not have to squeeze pieces of large files into small virtual spaces. Although these advantages are tempting, we find that database systems have gone their own way. In this paper, we will look at two particular approaches to database system design, and see how (and why) they interface to file systems as they do. We will then look at the potential advantages and implications of working more closely with virtual memory management, and describe some of the functions and constraints that would have to be supported by a generalized page manager.

SQLite, a widely used database engine, has been previously transformed to run on a bare PC without the support of any OS or kernel. However, the transformed SQLite database was stored in main memory i.e., it had no file system. This paper extends the transformation process to enable bare PC SQLite to work with standard file system interfaces based on the FAT32 file specification. It further presents mechanisms and programming interfaces for a bare machine file system integrated with SQLite that uses a removable USB flash drive. The bare SQLite database and file system can interoperate with conventional OS-based database systems. It can be adapted in the future to work with bare Web browsers, large bare databases, other bare applications, and bare mobile devices.

The architecture of the system is divided into some modules according to the operating level: interface layer, the database engine layer, and data access layer, database maintenance module and data storage module. This paper discusses in detail the data definition design and the important function of EDB and realizes the EDB management implementation details. The paper introduce in details the EDB system architecture, the design of physical storage structure and basic table structure. Through the actual use, the database system is suitable for embedded systems with limited resources, has good portability.

Managing many computers is difficult. Recent virtualization trends exacerbate this problem by making it easy to create and deploy multiple virtual appliances per physical machine, each of which can be configured with different applications and utilities. This results in a huge scaling problem for large organizations as management overhead grows linearly with the number of appliances. To address this problem, we introduce Strata, a system that combines unioning file system and package management semantics to enable more efficient creation, provisioning and management of virtual appliances. Unlike traditional systems that depend on monolithic file systems, Strata uses a collection of individual sotware layers that are composed together into the Virtual Layered File System (VLFS) to provide the traditional file system view. Individual layers are maintained in a central repository and shared across all file systems that use them. Layer changes and upgrades only need to be done once in the repository and are then automatically propagated to all virtual appliances, resulting in management overhead independent of the number of appliances. Our Strata Linux prototype requires only a single loadable kernel module providing the VLFS support and doesn't require any application or source code level kernel modifications. Using this prototype, we demonstrate how Strata enables fast system provisioning, simplifies system maintenance and upgrades, speeds system recovery from security exploits, and incurs only modest performance overhead.

We have performed a study of the usage of the Windows NT File System through long-term kernel tracing. Our goal was to provide a new data point with respect to the 1985 and 1991 trace-based File System studies, to investigate the usage details of the Windows NT file system architecture, and to study the overall statistical behavior of the usage data.In this paper we report on these issues through a detailed comparison with the older traces, through details on the operational characteristics and through a usage analysis of the file system and cache manager. Next to architectural insights we provide evidence for the pervasive presence of heavy-tail distribution characteristics in all aspect of file system usage. Extreme variances are found in session inter-arrival time, session holding times, read/write frequencies, read/write buffer sizes, etc., which is of importance to system engineering, tuning and benchmarking.

The United States Air Force (USAF) is implementing an integrated net-centric system of systems for airborne operations in support of the global war on terror (GWOT). The GWOT demands that a successful architecture framework transforms and delivers net-centric assets to the war-fighter in a timely manner. A critical component of this implementation is the transformation of legacy strategic air platforms into net-centric air power assets operating within a system of systems. The System Architectural (SA) framework, and the Department of Defense Architectural Framework (DoDAF) are ways of managing complexity and organizing information within a system of systems network. This paper will explore and compare these architectural frameworks; show examples used in a system of systems network; and illustrate how the DoDAF can successfully define the transformation of a legacy weapon system into a net-centric asset.

With current systems, some important complex queries may take days to complete because of: (1) the volume of data to be processed, (2) limited aggregate resources. Introducing parallelism addresses the first problem. Cheaper, but powerful computing resources solve the second problem. According to a survey by Brodie,1 only 10% of computerized data is in data bases. This is an argument for both more variety and volume of data to be moved into data base systems. We conjecture that the primary reasons for this low percentage are that data base management systems (DBMSs) still need to provide far greater functionality and improved performance compared to a combination of application programs and file systems. This paper addresses the issues and solutions relating to intraquery parallelism in a relational DBMS supporting SQL. Instead of focussing only on a few algorithms for a subset of the problems, we provide a broad framework for the study of the numerous issues that need to be addressed in supporting parallelism efficiently and flexibly. We also discuss the impact that parallelization of complex queries has on short transactions which have stringent response time constraints. The pros and cons of the shared nothing, shared disks and shared everything architectures for parallelism are enumerated. The impact of parallelism on a number of components of an industrial-strength DBMS are pointed out. The different stages of query processing during which parallelism may be gainfully employed are identified. The interactions between parallelism and the traditional systems' pipelining technique are analyzed. Finally, the performance implications of parallelizing a specific complex query are studied. This gives us a range of sample points for different parameters of a parallel system architecture, namely, I/O and communication bandwidth as a function of aggregate MIPS.