Bonsai: Balanced Lineage Authentication

Dr Paulino Ruiz-de-Clavijo from Universidad de Sevilla in Spain talks about the research behind his Letter ‘NanoFS: hardware-oriented file system’ on page 1216. Dr Paulino Ruiz-de-Clavijo During my higher education as a computer engineer I have always been interested in low-level programming and digital electronic circuits. In 1999 I started my PhD research at the University of Seville and the National Microelectronics Centre - CSIC, which I completed at the ID2 research group of the University of Seville. That period allowed me to start working with reprogrammable devices; FPGAs in particular. This technology fascinated me because it opened up the possibility to reuse my knowledge and experience in software programming to develop hardware using hardware description languages (HDLs), and I found the hardware-software codesign a great field to explore. The research in our group involves the wide field of hardware-software codesign and reconfigurable hardware. Our work is mainly focused on the design of digital embedded systems and SoC (system on chip) on FPGAs and the design of application-specific peripherals for SoC. Relating to SoCs, we have worked with open platforms like the LEON and OpenRISC soft-core microprocessors. In our group we have also developed hardware for industrial projects like a SNTP synchronisation platform based on FPGA and a frequency synchroniser for OFDM systems. We have developed a new file system called ‘nano file system’ (NanoFS) which is specially tailored to be implemented in hardware. It has been designed from scratch keeping in mind the hardware implementation limitations. The main objective is to be able to do file system operations just with digital circuits while keeping a minimum hardware footprint. The approach keeps the high level file system features and it can be used in both low level embedded systems and high-level operating systems. An all-hardware file system reader core is presented in our Letter which has been successfully implemented and tested in a low-range FPGA device. It can be included in custom designs to read data from files stored in external storage media. Most file systems are designed with two overall objectives: to be optimal for the storage medium and/or to be optimal when implemented in software. Our proposal is a hardware-friendly file system suitable for embedded systems. The file system can be used without an operating system or before the operating system is booted. File systems like FAT or EXT2 are widely used in electronic devices and SoC. The layouts of these traditional file systems are intended to be managed with software algorithms. When these algorithms are translated to HDL, a large amount of resources are necessary. There have been a few initiatives to develop file systems for embedded systems to solve these problems that simplifies data structures and improves some software routines. These file systems are written in C and supply an API for some microcontrolers but are not designed to achieve a hardware implementation. Some custom designs in hardware require dynamic configuration, such as the network configuration, calibration parameters, etc. In previous work we developed a SNTP synchronisation server designed as a full hardware solution to improve the precision. To try different configurations of these servers, we had to reprogram the FPGA device. We thought that a data file in a file system that could be accessed directly from the device hardware would be a faster and more flexible method to achieve that. We created NanoFS to solve the problems we found when we tried to access other file systems from hardware. Currently, from a software perspective, a dedicated file manager application is used to access the file system. We are working to improve a Linux native driver for NanoFS so we can access the files right from the operating system. At this moment the driver works in read-only mode. We are also migrating the file manager to the MS Windows platform. From the hardware perspective, we are developing some new hardware cores with write support. We plan to test the write support module with current solid state disks to measure the performance. Further applications we have planned to use the new file system are: data acquisition systems; boot loaders for embedded systems; and FPGA dynamic reconfiguration.

File system operation in a transparently fault-tolerant system that uses checkpointing and message logging is discussed. Logging messages to disk is one of the primary performance costs of such systems. The author has measured the file system operations performed on large timesharing systems running Unix in terms of the level of concurrency (number of consecutive operations that do not change the state of the file system). By performing much of the data analysis online within a modified Unix kernel, statistics were collected over a long period of time with a substantial variation in system load. Using this data, it is demonstrated that a technique called null logging can reduce the number of messages logged to disk by a factor of 10 to 25, depending on the workload. This reduces the overhead of the fault-tolerance mechanism and allows a large fraction of file system operations to commit instantaneously. >

This paper describes the foundation, design, implementation, and evaluation of DualFS, a new high-performance journaling file system which has the same consistency guarantees as traditional journaling file systems but a greater performance. DualFS places data and metadata in different devices (usually, two partitions of the same storage device) and manages them in very different ways. The metadata device is organized as a log-structured file system, whereas the data device is organized as groups. The new design allows DualFS not only to recover the consistency quickly after a system crash, but also to improve the overall file system performance. We have evaluated DualFS and we have found that it greatly reduces the total I/O time taken by the file system in most workloads as compared to other file systems (Ext2, Ext3, ReiserFS, XFS, and JFS). The work carried out has also allowed us to draw some lessons which ought to be taken into account when implementing new file systems

Existing nonvolatile memory (NVM)-based file systems can fully leverage the characteristics of NVM to obtain better performance than traditional disk-based file systems. It has the potential capacity to efficiently manage metadata and perform fast metadata operations. However, most NVM-based file systems mainly focus on managing file metadata (inode), while pay little attention to directory metadata (dentry), which also has a noticeable impact on the file system performance. Besides, the traditional journaling technique that guarantees metadata consistency may not yield satisfactory performance on NVM-based file systems. To solve these problems, in this article we propose a fast and low overhead metadata operation mechanism, called FLOMO. It first adopts a novel slotted-paging structure in NVM to reorganize dentry for efficiently performing dentry operations, and utilizes the red–black tree in DRAM to accelerate dentry lookup and the search process of dentry deletion. Moreover, FLOMO presents a selective journaling scheme for metadata updates, which partially logs the changes related to dentry in the proposed slotted page, thereby, mitigating the redundant journaling overhead. To verify FLOMO, we implement it in a typical NVM-based file system, the persistent memory file system (PMFS). Experimental results show that FLOMO accelerates the metadata operations in PMFS by 34.4% <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\sim 59$ </tex-math></inline-formula> %, and notably reduces the journaling overhead for metadata, shortening the latency by 59% on average. For real-world applications, FLOMO has higher throughput compared with PMFS, PMFS without journal, and NOVA, achieving up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.1\times $ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.1\times $ </tex-math></inline-formula> , and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.3\times $ </tex-math></inline-formula> performance improvement, respectively.

Despite continual improvements in the performance and reliability of large scale file systems, the management of user-defined file system metadata has changed little in the past decade. The mismatch between the size and complexity of large scale data stores and their ability to organize and query their metadata has led to a de facto standard in which raw data is stored in traditional file systems, while related, application-specific metadata is stored in relational databases. This separation of data and semantic metadata requires considerable effort to maintain consistency and can result in complex, slow, and inflexible system operation. To address these problems, we have developed the Quasar File System (QFS), a metadata-rich file system in which files, user-defined attributes, and file relationships are all first class objects. In contrast to hierarchical file systems and relational databases, QFS defines a graph data model composed of files and their relationships. QFS incorporates Quasar, an XPATH-extended query language for searching the file system. Results from our QFS prototype show the effectiveness of this approach. Compared to the de facto standard, the QFS prototype shows superior ingest performance and comparable query performance on user metadata-intensive operations and superior performance on normal file metadata operations.

Despite continual improvements in the performance and reliability of large scale file systems, the management of file system metadata has changed little in the past decade. The mismatch between the size and complexity of large scale data stores and their ability to organize and query their metadata has led to a de facto standard in which raw data is stored in traditional file systems, while related, application-specific metadata is stored in relational databases. This separation of data and metadata requires considerable effort to maintain consistency and can result in complex, slow, and inflexible system operation. To address these problems, we have developed the Quasar File System (QFS), a metadata-rich file system in which files, metadata, and file relationships are all first class objects. In contrast to hierarchical file systems and relational databases, QFS defines a graph data model composed of files and their relationships. QFS includes Quasar, an XPATH-extended query language for searching the file system. Results from our QFS prototype show the effectiveness of this approach. Compared to the defacto standard, the QFS prototype shows superior ingest performance and comparable query performance on user metadata-intensive operations and superior performance on normal file metadata operations.

In storage using Non-Volatile Memory (NVM), file placements affect the performance of processor cache. However, the impact of file system policies on processor cache has not been well studied. In particular, sequential block allocation of file systems may incur cache pollution that evicts hot data from cache. To prevent cache pollution, we propose Cache-Aware Block Allocation (CABA) scheme for NVM storage. CABA assigns colors to data blocks in NVM storage such that blocks with the same color compete for cache space. It allocates blocks with specific colors to a file in order to restrict cache space occupied by the file data. A naive color-based block allocation, though, tends to allocate scattered blocks to a file, leading to performance degradation of file systems. To reduce the performance penalty, CABA defines a color group with some consecutive blocks and allocates consecutive blocks in the color group to a file. By so doing, CABA balances cache performance and file system performance. We implement CABA in the Ext4 and NOVA file systems and, through experiments, show performance improvements for memory-intensive programs.

The z/OS operating system offers a strong and well- organized dataset management System that enables the efficient storage, retrieval, and processing of data through high-volume andmission - critical workloads by large corporations. This paper delves into the intricacy of the z/OS operating system's file management, moreso its unique but most sophisticated dataset structures and UNIX System Services (USS) file system. This paper explores various dataset organizations, record structures, and accessmethods native to z/OS. This paper also talks abouthowthetraditional z/OS datasetsintegrate with the hierarchical file system of USS,with a focus oncoexistence and interoperation between the two environments. This paper aims to comprehensively understand the z/OS file systems and datasets by comparing similar technologies outside of zOS and using coding examples.

Installation and Configuration of Tru64 UNIX

Modern file systems associate the deletion of a file with the immediate release of storage, and file writes with the irrevocable change of file contents. We argue that this behavior is a relic of the past, when disk storage was a scarce resource. Today, large cheap disks make it possible for the file system to protect valuable data from accidental delete or overwrite.This paper describes the design, implementation, and performance of the Elephant file system, which automatically retains all important versions of user files. Users name previous file versions by combining a traditional pathname with a time when the desired version of a file or directory existed. Storage in Elephant is managed by the system using file-grain user-specified retention policies. This approach contrasts with checkpointing file systems such as Plan-9, AFS, and WAFL that periodically generate efficient checkpoints of entire file systems and thus restrict retention to be guided by a single policy for all files within that file system.Elephant is implemented as a new Virtual File System in the FreeBSD kernel.

This paper describes a simple distributed mechanism for caching files among a networked collection of workstations. We have implemented it as part of Sprite, a new operating system being implemented at the University of California at Berkeley. A preliminary version of Sprite is currently running on Sun-2 and Sun-3 workstations, which have about 1-2 MIPS processing power and 4-16 Mbytes of main memory. The system is targeted for workstations like these and newer models likely to become available in the near future; we expect the future machines to have at least five to ten times the processing power and main memory of our current machines, as well as small degrees of multiprocessing. We hope that Sprite will be suitable for networks of up to a few hundred of these workstations. Because of economic and environmental factors, most workstations will not have local disks; instead, large fast disks will be concentrated on a few server machines.

Parallel and distributed file systems are widely used to provide high throughput in high-performance computing and Cloud computing systems. To increase the parallelism, I/O requests are partitioned into multiple sub-requests (or `flows') and distributed across different data nodes. The performance of file systems is extremely poor if data nodes have highly unbalanced response time. Client-side caching offers a promising direction for addressing this issue. However, current work has primarily used client-side memory as a read cache and employed a write-through policy which requires synchronous update for every write and significantly under-utilizes the client-side cache when the applications are write-intensive. Realizing that the cost of an I/O request depends on the struggler sub-requests, we propose a cost-aware client-side file caching (CCFC) strategy, that is designed to cache the sub-requests with high I/O cost on the client end. This caching policy enables a new trade-off across write performance, consistency guarantee and cache size dimensions. Using benchmark workloads MADbench2, we evaluate our new cache policy alongside conventional write-through. We find that the proposed CCFC strategy can achieve up to 110% throughput improvement compared to the conventional write-through policies with the same cache size on an 85-node cluster.

Input/Output (I/O) operations can represent a significant proportion of the run-time of parallel scientific computing applications. Although there have been several advances in file format libraries, file system design and I/O hardware, a growing divergence exists between the performance of parallel file systems and the compute clusters that they support. In this paper, we document the design and application of the RIOT I/O toolkit (RIOT) being developed at the University of Warwick with our industrial partners at the Atomic Weapons Establishment and Sandia National Laboratories. We use the toolkit to assess the performance of three industry-standard I/O benchmarks on three contrasting supercomputers, ranging from a mid-sized commodity cluster to a large-scale proprietary IBM BlueGene/P system. RIOT provides a powerful framework in which to analyse I/O and parallel file system behaviour—we demonstrate, for example, the large file locking overhead of IBM's General Parallel File System, which can consume nearly 30% of the total write time in the FLASH-IO benchmark. Through I/O trace analysis, we also assess the performance of HDF-5 in its default configuration, identifying a bottleneck created by the use of suboptimal Message Passing Interface hints. Furthermore, we investigate the performance gains attributed to the Parallel Log-structured File System (PLFS) being developed by EMC Corporation and the Los Alamos National Laboratory. Our evaluation of PLFS involves two high-performance computing systems with contrasting I/O backplanes and illustrates the varied improvements to I/O that result from the deployment of PLFS (ranging from up to 25× speed-up in I/O performance on a large I/O installation to 2× speed-up on the much smaller installation at the University of Warwick).

The hybrid file system is designed to optimize the latency of the response of File System I/Os and extend the capacity of the local file system to cloud by taking the advantage of Internet. Our hybrid file system is consist of SSD, HDD and Amazon S3 cloud file system. We store small files, directory tree and metadata of all the files in SSD, because SSD has a good performance for the response of small and random I/Os. HDD is good at responding big and sequential I/Os, so we use it just like a warehouse to store big files which are linked by the symbolic files in the SSD. We also extend the local file system to cloud in order to enlarge its capacity. In this paper we describe test data of our hybrid file system and also its design and implement details.

The File Allocation Table (FAT) file system is widely used file system in tablet personal computers, mobile phones, digital cameras and other embedded devices for data storage and multi-media applications such as video imaging, audio/video playback and recording. The FAT file system is not power fail-safe. This means that, the uncontrolled power loss or abrupt removal of storage device from computer/embedded system causes the file system corruption. The TFAT (Transaction safe FAT) file system is an extension of FAT file system to provide power fail-safe feature to the FAT file system. This paper explores the design methodologies of cluster allocation algorithms of TFAT file system by conducting various combinations of file system operations in Windows CE (Compact Embedded) 6.0 Operating System (OS). This paper also records the performance bench-marking of TFAT file system in comparison with FAT File system.