DFS

Despite continual improvements in the performance and reliability of large scale file systems, the management of user-defined file system metadata has changed little in the past decade. The mismatch between the size and complexity of large scale data stores and their ability to organize and query their metadata has led to a de facto standard in which raw data is stored in traditional file systems, while related, application-specific metadata is stored in relational databases. This separation of data and semantic metadata requires considerable effort to maintain consistency and can result in complex, slow, and inflexible system operation. To address these problems, we have developed the Quasar File System (QFS), a metadata-rich file system in which files, user-defined attributes, and file relationships are all first class objects. In contrast to hierarchical file systems and relational databases, QFS defines a graph data model composed of files and their relationships. QFS incorporates Quasar, an XPATH-extended query language for searching the file system. Results from our QFS prototype show the effectiveness of this approach. Compared to the de facto standard, the QFS prototype shows superior ingest performance and comparable query performance on user metadata-intensive operations and superior performance on normal file metadata operations.

Modern file systems associate the deletion of a file with the immediate release of storage, and file writes with the irrevocable change of file contents. We argue that this behavior is a relic of the past, when disk storage was a scarce resource. Today, large cheap disks make it possible for the file system to protect valuable data from accidental delete or overwrite.This paper describes the design, implementation, and performance of the Elephant file system, which automatically retains all important versions of user files. Users name previous file versions by combining a traditional pathname with a time when the desired version of a file or directory existed. Storage in Elephant is managed by the system using file-grain user-specified retention policies. This approach contrasts with checkpointing file systems such as Plan-9, AFS, and WAFL that periodically generate efficient checkpoints of entire file systems and thus restrict retention to be guided by a single policy for all files within that file system.Elephant is implemented as a new Virtual File System in the FreeBSD kernel.

Memory-centric computing demands careful organization of the virtual address space, but traditional methods for doing so are inflexible and inefficient. If an application wishes to address larger physical memory than virtual address bits allow, if it wishes to maintain pointer-based data structures beyond process lifetimes, or if it wishes to share large amounts of memory across simultaneously executing processes, legacy interfaces for managing the address space are cumbersome and often incur excessive overheads. We propose a new operating system design that promotes virtual address spaces to first-class citizens, enabling process threads to attach to, detach from, and switch between multiple virtual address spaces. Our work enables data-centric applications to utilize vast physical memory beyond the virtual range, represent persistent pointer-rich data structures without special pointer representations, and share large amounts of memory between processes efficiently.

Over the last several years, a number of hardware and software systems have been developed which map entire files directly into the virtual memory address spaces used by programs. Since all file contents are directly addressable, there is no need for a programmer to issue explicit file system actions, such as Read or Write. In addition, all of the buffer management problems are eliminated, since programmers do not have to squeeze pieces of large files into small virtual spaces. Although these advantages are tempting, we find that database systems have gone their own way. In this paper, we will look at two particular approaches to database system design, and see how (and why) they interface to file systems as they do. We will then look at the potential advantages and implications of working more closely with virtual memory management, and describe some of the functions and constraints that would have to be supported by a generalized page manager.

No abstract available.

Pangaea is a wide-area file system that supports data sharing among a community of widely distributed users. It is built on a symmetrically decentralized infrastructure that consists of commodity computers provided by the end users. Computers act autonomously to serve data to their local users. When possible, they exchange data with nearby peers to improve the system's overall performance, availability, and network economy. This approach is realized by aggressively creating a replica of a file whenever and wherever it is accessed.This paper presents the design, implementation, and evaluation of the Pangaea file system. Pangaea offers efficient, randomized algorithms to manage highly dynamic and potentially large groups of file replicas. It applies optimistic consistency semantics to replica contents, but it also offers stronger guarantees when required by the users. The evaluation demonstrates that Pangaea outperforms existing distributed file systems in large heterogeneous environments, typical of the Internet and of large corporate intranets.

Careful simulation‐based evaluation plays an important role in the design of file and disk systems. We describe here a particular approach to such evaluations that combines techniques in workload synthesis, file system modeling, and detailed disk behavior modeling. Together, these make feasible the detailed simulation of I/O hardware and file system software. In particular, using the techniques described here is likely to make comparative file system studies more accurate.In addition to these specific contributions, the paper makes two broader points. First, it argues that detailed models are appropriate and necessary in many cases. Second, it demonstrates that detailed models need not be difficult or time consuming to construct or execute.

As one of the workload characteristics, the anomaly behaviorsin real workload have been recognized as a critical requirement for file system design. In this paper, a set of traces collected from typically realistic file system have been analyzed. Studying the correlations of I/O request inter-arrival times shows that it is necessary to examine the self-similarity in file system workload. And then the phenomenon of self-similarity which had been initially observed in network and disk I/O workload has also been visually and statistically observed in file system workload used in this paper. Furthermore, this property is further explored and we find that the self-similarity of I/O accesses in those sets of traces studied in this paper is mainly due to reads. This study is extremely important to provide useful insight into the design and analysis of file system for performance gains, as well as to model file system I/O workloads.

A global and parallel file system for grids

As bandwidth constraints on LAN/WAN environments decrease, the demand for distributed services will continue to increase. In particular, the proliferation of user-level applications requiring high-capacity distributed file storage systems will demand that such services be universally available. At the same time, the advent of high-speed networks have made the deployment of application and communication solutions based upon an Intelligent Mobile Agent (IMA) framework practical. Agents have proven to present an ideal development paradigm for the creation of autonomous large-scale distributed systems, and an agent-based communication scheme would facilitate the creation of independently administered distributed file services. This thesis thus outlines an architecture for such a distributed file system based upon an IMA communication framework.

The ARINC-653 standard, dictating the use of time and space partitioning in real-time avionics systems, has been widely embraced by the industry. We present the design of real-time file system (RTFS), a file system that complies with the ARINC-653 standard and the emerging standard for a file system for ARINC-653. RTFS provides realtime accesses to data stored on a variety of mass storage device. In addition to an interface complying with emerging standard, RTFS provides an application interface that complies with a subset of the POSIX standard. RTFS is implemented as a normal ARINC-653 partition. Task partitions communicate their file operation requests to RTFS via queuing ports; such ports are also used to deliver the responses from RTFS to the task partitions. The temporal behavior of RTFS is predictable and the response times for file operations are bounded. The design of RTFS handles a mix of hard and soft real-time file access requests. RTFS implements metadata journaling using on-board non-volatile memory devices to provide fast file updates and fast file system recovery on faults. Finally, RTFS includes facilities to support network-centric operations and a set of design and maintenance tools. This paper overviews the design of RTFS and describes the realization of the many unique features of RTFS.

NFS is considered to be the bottleneck in cluster computing environment because of its limited resources and centralized data management. With the development of hardware, NFS server has more than one I/O channel, more storage space and more powerful CPU. In this paper, we describe the design and the implementation of a new logical file system called United-FS. It can make storage devices connected to multiple I/O channels work concurrently and cooperatively. It can be exported by NFS server to provide a single file system image to clients by hiding a variety of native file systems built on different type of storage devices. This paper also compares the United-FS with the software RAID system both from theoretical analysis and experiments. The results show that United-FS is much more flexible and its performance is better than software RAID in most cases.

Mobile ad hoc networks (MANETs) are networks with self-organizing capabilities and without a fixed infrastructure. Wireless nodes communicate among themselves using multi-hop radio relaying, without requiring the packets to pass through a central access point or a base station. Routing is a function of speed and stability with which the nodes are able to acquire addresses. Efficient routing forms the basis of a fast and reliable communication network. In a highly mobile and infrastructure-less scenario, pre-configuration of addresses is not possible. Therefore node addresses need to be configured dynamically with minimum delay and packet loss. The main task of an address auto-configuration protocol is to manage the resource address space. It must be able to select, allocate, and assign a unique network address to an un-configured node. This paper proposes a new address auto-configuration protocol for mobile ad hoc networks. The scheme uses virtual address space for addressing new nodes joining a network. The aim is to map one point from virtual address sheet to exactly one new node. The reason for using the term "virtual" is that the whole corresponding address space is a 2D flat sheet and each point of this sheet is virtually mapped to a node in MANET. The protocol uses coordinate values for generating addresses.

This paper proposes a backup system based on mirroring filesystem GMFS has been developed to mirror data in realtime on the filesystem layer. The GMFS is a stackable filesystem which flexibly mirrors without changing the existing environment by operating as a wrapper of other filesystems.Because the conventional mirroring technology utilizes the mirroring function on the device layer or needs a special filesystem, the allocation of the disk and the specific format of the filesystem are needed, and so the disk design is fixed. Therefore, the conventional mirroring technology cannot adjust when the mirroring function not assumed will be needed later. In this situation, the mechanism that adds the mirroring function without changing the existing disk design is necessary. The GMFS conducts the operation of other filesystems transparently, thereby users need not be aware of the GMFS. The conventional filesystem looks as if it performs mirroring data by itself. GMFS can therefore add the function that mirrors in realtime without destroying the existing environment.GMFS uses NFS which is a typical network file system to communicate with an existing environment. The throughput of reading and writing has been improved by adopting the method to call system function of NFS from the inside of the filesystem.We developed this filesystem, and evaluated the performance from the viewpoint of throughput and system call speed and CPU loads. As a result, it was shown that there was no problem in the viewpoint of the performance compared with the conventional filesystem, and the throughput of the read and write of GMFS was 2.0 times faster than conventional mirroring filesystem.

Vendors cannot provide all the operating-system services that users demand. So, there has been a persistent desire to make operating systems more flexible and customizable. Naturally, object-oriented technology has come to bear on this area. However the ease of use of many such solutions has been disappointing. Ease of use is a key feature of Frigate, an object-oriented file system. It differs from most file system designs in that it's for ordinary users, not sophisticated operating system gurus. Frigate's modular, extensible framework allows new extensions to be "plugged in" on the fly. It is fully implemented and supports a set of example file system extensions.