A Time and Space Partitioned Avionics Real-Time File System

Careful simulation‐based evaluation plays an important role in the design of file and disk systems. We describe here a particular approach to such evaluations that combines techniques in workload synthesis, file system modeling, and detailed disk behavior modeling. Together, these make feasible the detailed simulation of I/O hardware and file system software. In particular, using the techniques described here is likely to make comparative file system studies more accurate.In addition to these specific contributions, the paper makes two broader points. First, it argues that detailed models are appropriate and necessary in many cases. Second, it demonstrates that detailed models need not be difficult or time consuming to construct or execute.

The problem of scheduling disk requests in a personal hard real-time read/write file system is examined. We propose a fixed-period scan (FSCAN) approach for disk scheduling in our file system. The idea is to use the CSCAN policy to pick up the data blocks requested by a periodic preemptive schedule. The approach trades disk block size and memory buffer size for higher performance. We derive the worst-case seek and rotational overhead for the FSCAN algorithm, and we show that the worst case seek overhead can be measured empirically for a large class of seek functions. Using this approach and utilizing measured seek functions from real disk drives, we show that these policies can transfer data at 40-70% of the maximum transfer rate of modern disk drives, depending an the file system parameters. A configuration program is developed to automatically test and configure the FSCAN algorithm for modern hard disks. The design and testing of this program are described.

Despite continual improvements in the performance and reliability of large scale file systems, the management of user-defined file system metadata has changed little in the past decade. The mismatch between the size and complexity of large scale data stores and their ability to organize and query their metadata has led to a de facto standard in which raw data is stored in traditional file systems, while related, application-specific metadata is stored in relational databases. This separation of data and semantic metadata requires considerable effort to maintain consistency and can result in complex, slow, and inflexible system operation. To address these problems, we have developed the Quasar File System (QFS), a metadata-rich file system in which files, user-defined attributes, and file relationships are all first class objects. In contrast to hierarchical file systems and relational databases, QFS defines a graph data model composed of files and their relationships. QFS incorporates Quasar, an XPATH-extended query language for searching the file system. Results from our QFS prototype show the effectiveness of this approach. Compared to the de facto standard, the QFS prototype shows superior ingest performance and comparable query performance on user metadata-intensive operations and superior performance on normal file metadata operations.

Modern file systems associate the deletion of a file with the immediate release of storage, and file writes with the irrevocable change of file contents. We argue that this behavior is a relic of the past, when disk storage was a scarce resource. Today, large cheap disks make it possible for the file system to protect valuable data from accidental delete or overwrite.This paper describes the design, implementation, and performance of the Elephant file system, which automatically retains all important versions of user files. Users name previous file versions by combining a traditional pathname with a time when the desired version of a file or directory existed. Storage in Elephant is managed by the system using file-grain user-specified retention policies. This approach contrasts with checkpointing file systems such as Plan-9, AFS, and WAFL that periodically generate efficient checkpoints of entire file systems and thus restrict retention to be guided by a single policy for all files within that file system.Elephant is implemented as a new Virtual File System in the FreeBSD kernel.

As one of the workload characteristics, the anomaly behaviorsin real workload have been recognized as a critical requirement for file system design. In this paper, a set of traces collected from typically realistic file system have been analyzed. Studying the correlations of I/O request inter-arrival times shows that it is necessary to examine the self-similarity in file system workload. And then the phenomenon of self-similarity which had been initially observed in network and disk I/O workload has also been visually and statistically observed in file system workload used in this paper. Furthermore, this property is further explored and we find that the self-similarity of I/O accesses in those sets of traces studied in this paper is mainly due to reads. This study is extremely important to provide useful insight into the design and analysis of file system for performance gains, as well as to model file system I/O workloads.

Vendors cannot provide all the operating-system services that users demand. So, there has been a persistent desire to make operating systems more flexible and customizable. Naturally, object-oriented technology has come to bear on this area. However the ease of use of many such solutions has been disappointing. Ease of use is a key feature of Frigate, an object-oriented file system. It differs from most file system designs in that it's for ordinary users, not sophisticated operating system gurus. Frigate's modular, extensible framework allows new extensions to be "plugged in" on the fly. It is fully implemented and supports a set of example file system extensions.

Embedded systems such as PVR, set-top box, HDTV put unique demand on I/O subsystem design. Underlying software, particularly file system, needs to be elaborately designed so that it can meet tight constraints of consumer electronics platform: performance, price, reliability, and etc. In this work, we develop state-of-art file system elaborately tailored for A/V workload. There are two design objectives in our file system: performance and support for logical level abstraction. For performance, we develop a number of novel features: extent based allocation, single level file structure with block index augmentation scheme, aggressive free block allocation to minimize disk fragmentation, elaborate file system meta data layout, separation of name space data and file data and etc. HERMES enables the user to view file as a collection of semantic units (frame or audio samples). HERMES file system encompasses most of state-of-the-art file system technologies published in preceding works. Via extensive physical experiment, we verify that HERMES file system successfully addresses the original issues: good scalability, predictable I/O latency (minimizing variability in I/O latency), efficient disk head movement and etc. This is the result of harmonious effort of large I/O size, aggressive free block allocation algorithm, data block placement strategy, file organization, layout of HERMES file system and etc. The result of performance experiments indicate that HERMES file system prototype successfully meets the file system constraints for high volume and high bandwidth multimedia application. HERMES file system exhibits superior performance to EXT2 file system (Linux) and XFS file system (SGI).

Input/Output (I/O) operations can represent a significant proportion of the run-time of parallel scientific computing applications. Although there have been several advances in file format libraries, file system design and I/O hardware, a growing divergence exists between the performance of parallel file systems and the compute clusters that they support. In this paper, we document the design and application of the RIOT I/O toolkit (RIOT) being developed at the University of Warwick with our industrial partners at the Atomic Weapons Establishment and Sandia National Laboratories. We use the toolkit to assess the performance of three industry-standard I/O benchmarks on three contrasting supercomputers, ranging from a mid-sized commodity cluster to a large-scale proprietary IBM BlueGene/P system. RIOT provides a powerful framework in which to analyse I/O and parallel file system behaviour—we demonstrate, for example, the large file locking overhead of IBM's General Parallel File System, which can consume nearly 30% of the total write time in the FLASH-IO benchmark. Through I/O trace analysis, we also assess the performance of HDF-5 in its default configuration, identifying a bottleneck created by the use of suboptimal Message Passing Interface hints. Furthermore, we investigate the performance gains attributed to the Parallel Log-structured File System (PLFS) being developed by EMC Corporation and the Los Alamos National Laboratory. Our evaluation of PLFS involves two high-performance computing systems with contrasting I/O backplanes and illustrates the varied improvements to I/O that result from the deployment of PLFS (ranging from up to 25× speed-up in I/O performance on a large I/O installation to 2× speed-up on the much smaller installation at the University of Warwick).

We introduce Pastis, a completely decentralized multi-user read-write peer-to-peer file system. In Pastis every file is described by a modifiable inode-like structure which contains the addresses of the immutable blocks in which the file contents are stored. All data are stored using the Past distributed hash table (DHT), which we have modified in order to reduce the number of network messages it generates, thus optimizing replica retrieval. Pastis' design is simple compared to other existing systems, as it does not require complex algorithms like Byzantine-fault tolerant (BFT) replication or a central administrative authority. It is also highly scalable in terms of the number of network nodes and users sharing a given file or portion of the file system. Furthermore, Pastis takes advantage of the fault tolerance and good locality properties of its underlying storage layer, the Past DHT. We have developed a prototype based on the FreePastry open-source implementation of the Past DHT. We have used this prototype to evaluate several characteristics of our file system design. Supporting the close-to-open consistency model, plus a variant of the read-your-writes model, our prototype shows that Pastis is between 1.4 to 1.8 times slower than NFS. In comparison, Ivy and Oceanstore are between two to three times slower than NFS.

Fault-tolerance is very important in cluster computing. Many famous cluster-computing systems have implemented fault-tolerance by using checkpoint/restart mechanism. But existent checkpointing algorithms can not restore the states of a file system when roll-backing the running of a program, so there are many restrictions on file accesses in existent fault-tolerance systems. SCR algorithm, an algorithm based on atomic operation and consistent schedule, which can restore the states of file systems, is present in this paper. In SCR algorithm, system calls on file sytems are classified into idempotent operations and non-idempotent operations. A non-idempotent operation modifies a file system's states, and an idempotent operation does not. SCR algorithm dynamically follows the tracks of a program's running, logs each non-idempotent operation used by the program and the information that can restore the operation in disks. When checkpointing roll-backing the program, SCR algorithm will revert the file system states to the last checkpoint time. By using SCR algorithm, users are allowed to use any file operation in their programs.

ARINC 653 introduces the concept of partition that allows time and space isolation in real-time avionic systems. Tasks are assigned to partitions according to various objective functions or constraints such as safety, performance, and security. Some of these objective functions may be conflicting as an improvement of one objective leads to a decrease of another. For example, improving safety by active redundancy may decrease performance. In this paper, we investigate the conflicting aspect between schedulability and security in Time and Space Partitioning (TSP) systems. Many researches have shown that enforcing the security of a system results in an overhead affecting its schedulability. We formulate a design space exploration (DSE) process with a meta-heuristic to explore solutions defined by the tasks to partitions assignment according to security requirements and timing constraints. Experiments are conducted with the Cheddar scheduling analyzer to characterize applications that are concerned by this conflicting issue and to evaluate the tradeoffs between schedulability and security.

Pangaea is a wide-area file system that supports data sharing among a community of widely distributed users. It is built on a symmetrically decentralized infrastructure that consists of commodity computers provided by the end users. Computers act autonomously to serve data to their local users. When possible, they exchange data with nearby peers to improve the system's overall performance, availability, and network economy. This approach is realized by aggressively creating a replica of a file whenever and wherever it is accessed.This paper presents the design, implementation, and evaluation of the Pangaea file system. Pangaea offers efficient, randomized algorithms to manage highly dynamic and potentially large groups of file replicas. It applies optimistic consistency semantics to replica contents, but it also offers stronger guarantees when required by the users. The evaluation demonstrates that Pangaea outperforms existing distributed file systems in large heterogeneous environments, typical of the Internet and of large corporate intranets.

A global and parallel file system for grids

We present Confluence, an enhancement to a desktop file search tool called Confluence which extracts conceptual relationships between files by their temporal access patterns in the file system. A limitation of a purely file-based approach is that as file operations are increasingly abstracted by applications, their correlation to a user's activity weakens and thereby reduces the applicability of their temporal patterns. To deal with this problem, we augment the file event stream with a stream of window focus events from the UI layer. We present 3 algorithms that analyze this new stream, extracting the user's task information which informs the existing Confluence algorithms. We present results and conclusions from a preliminary user study on Confluence.

This paper presents a design of effective cryptographic file system (ECFS) for storage services. ECFS can be used to provide the data security in existing network and p2p file system such as NFS, CIFS, OceanStore and Yahoo! Briefcase. We assume that the storage system isn't trusted entirely and end user must be able to control some security functions. Adding security to storage systems leads to performance degradation. In order to achieve good performance, ECFS uses a key management using symmetric cipher instead of public-key approach. ECFS also uses two-phases revocation to decrease the overhead of key revocation. We evaluate the ECFS security based on the set of criteria for evaluating a storage solution.