CA-NFS

This paper describes a simple distributed mechanism for caching files among a networked collection of workstations. We have implemented it as part of Sprite, a new operating system being implemented at the University of California at Berkeley. A preliminary version of Sprite is currently running on Sun-2 and Sun-3 workstations, which have about 1-2 MIPS processing power and 4-16 Mbytes of main memory. The system is targeted for workstations like these and newer models likely to become available in the near future; we expect the future machines to have at least five to ten times the processing power and main memory of our current machines, as well as small degrees of multiprocessing. We hope that Sprite will be suitable for networks of up to a few hundred of these workstations. Because of economic and environmental factors, most workstations will not have local disks; instead, large fast disks will be concentrated on a few server machines.

Peer-to-peer (P2P) systems are, in contrast to client-server (C/S) systems, fault-tolerant, robust, and scalable. While C/S distributed file systems, such as NFS (network file system) or SMB (server message block), do not scale with respect to the number of clients and exhibit a single point of failure, P2P file systems have the potential to cope with an increasing number of participants. Thus, this paper presents DRFS (distributed reliable file system), a P2P file system for cooperative environments. DRFS uses random, content-independent identifiers for data storage, while maintaining high performance and low overhead with many concurrent reads and writes. A dynamic replication mechanism ensures data availability, even under high churn. The application scenario considers an office environment, where DRFS is installed on employees' machines, who store and request files. DRFS has been implemented using the Filesystem in Userspace (FUSE) interface, in order to provide users with transparent read and write operations. Experiments show the benefits of such a peer-to-peer architecture, when a small number of peers reads or writes in parallel: DRFS performs better than NFS, as soon as 6 peers read or write in parallel a 32 MB file. For unpopular files, it is also more reliable than IgorFS.

Network File System is a widely used distributed file system that allows the user to access and manipulate storage on remote computers, as if they were a part of the local machine. Network File System is notoriously slow in its default configuration and if more clients connects to the NFS environment, it merely accentuates the delay. When configured to deliver faster speeds, the system suffers from higher risk of data corruption and loss. This study proposes a number of modifications to the Network File System, enabling it to provide elevated system performance, while containing the risk of data loss and corruption. Further, the proposed system behaves better in congested networks by consuming less bandwidth, ensuring decent speeds, even during periods of heavy network traffic.

Distributed file systems (DFSs) play important roles in datacenters. Recent advances in persistent memory (PM) and remote direct memory access (RDMA) technologies provide opportunities in enhancing distributed file systems. However, state-of-the-art distributed PM file systems (DPMFSs) still suffer from a duplication problem and a fixed transmission problem, leading to high network latency and low transmission throughput. To tackle these two problems, we propose Zebra, an efficient RDMA-enabled distributed PM file system—Zebra uses a replication group design for alleviating the heavy replication overhead, and leverages a novel transmission protocol for adaptively transmitting file replications among nodes, eliminating the fixed transmission problem. We implement Zebra and evaluate its performance against state-of-the-art distributed file systems on an Intel Optane DC PM platform. The evaluation results show that Zebra outperforms CephFS, GlusterFS, and NFS by 4.38\(\times \), 5.61\(\times \), and 2.71\(\times \) on average in throughput, respectively. KeywordsRDMAAdaptive transmissionPersistent memoryDistributed file system

Analysis of NGS (Next Generation Sequencing) data is a computationally demanding task requiring large amounts of CPU, memory, and disk space. There is also a requirement for high performance data storage systems, resilient to hardware failure, to be connected directly to the computing infrastructure (typically a multi-node cluster) to store large quantities of NGS data reliably. Traditional shared file systems such as NFS (Network File System) do not offer the performance, scalability or cache coherence required by modern NGS data analysis, so alternatives including GlusterFS, Ceph, and Lustre have been developed. However, there is a trade-off between data safety on replicated local storage and degradation of performance across distributed storage. Resilience to hardware failure is typically provided by RAID (Redundant Array of Independent Disks) and redundant storage nodes. Here we describe the evaluation of an alternative file system, RozoFS ( https://github.com/rozofs/rozofs ) for use with demanding NGS data analysis workloads. We used a synthetic data set (DREAM-TCGA data set 3) to run a complete tumor-normal analysis pipeline (“bcbio”, https://github.com/chapmanb/bcbio-nextgen ), including base quality recalibration, local indel realignment, somatic variant calling, and structural variants as a benchmark to compare RozoFS with a traditional shared file system (NFS) on two different HPC (High Performance Computing; Cloud4CaRE project) clusters. Our results show high reliability and good performance of RozoFS compared to NFS, in particular, during heavy I/O workloads. These findings indicate that the reliability and robustness of RozoFS's make it a good candidate for demanding NGS analysis workloads.

Users rarely consider running network file systems over slow or wide-area networks, as the performance would be unacceptable and the bandwidth consumption too high. Nonetheless, efficient remote file access would often be desirable over such networks---particularly when high latency makes remote login sessions unresponsive. Rather than run interactive programs such as editors remotely, users could run the programs locally and manipulate remote files through the file system. To do so, however, would require a network file system that consumes less bandwidth than most current file systems.This paper presents LBFS, a network file system designed for low-bandwidth networks. LBFS exploits similarities between files or versions of the same file to save bandwidth. It avoids sending data over the network when the same data can already be found in the server's file system or the client's cache. Using this technique in conjunction with conventional compression and caching, LBFS consumes over an order of magnitude less bandwidth than traditional network file systems on common workloads.

This paper presents Kosha, a peer-to-peer (p2p) enhancement for the widely-used Network File System (NFS). Kosha harvests redundant storage space on cluster nodes and user desktops to provide a reliable, shared file system that acts as a large storage with normal NFS semantics. P2p storage systems provide location transparency, mobility transparency, load balancing, and file replication - features that are not available in NFS. On the other hand, NFS provides hierarchical file organization, directory listings, and file permissions, which are missing from p2p storage systems. By blending the strengths of NFS and p2p storage systems, Kosha provides a low overhead storage solution. Our experiments show that compared to unmodified NFS, Kosha introduces a 4.1% fixed overhead and 1.5% additional overhead as nodes are increased from one to eight. For larger number of nodes, the additional overhead increases slowly. Kosha achieves load balancing in distributed directories, and guarantees 99.99% or better file availability.

This research paper is a survey of the present state of design and implementation of distributed file systems. In a distributed file system, one or more central servers store files that can be accessed, with proper authorization rights. It consists of four parts: an Introduction of Distributed file system, implementation, architecture and component of distributed file system and their design goals for good distributed file system i.e. transparency – the main goal of good DFS. In this paper we also discuss some distributed file systems like NFS, CIFS, OwFS and HDFS. We explain features, architecture of above file system. This paper also include the criteria for choosing right Distributed file system of our need and recommendation foe right Distributed File System .The above file systems are also compared on the basis of File size, Analytic functionality, TCO and their selection criteria. This research also included the future aspect of distributed file systems like the problem we based in current DFS like transparency and at last the conclusion of the whole paper on the basis of above survey.

Distributed file systems have become popular in recent years. However, they still lack functions for doing fast arbitrary data insertion and truncation. To solve the problem, we present Wofs, an object-based distributed network file system which supports fast arbitrary data insertion and truncation. Wofs splits a file into many small objects, stores these objects in remote file servers, and uses a special B+tree to manage the metadata of these objects. Besides, Wofs uses the object-range locking policy to avoid data incoherence and improve performance.

Building a Portable File System for Heterogeneous Clusters

Handling of Big Data and cloud computing are the two important prime concerns which have become more and more popular in recent years. Due to tremendous hike in data production, the need for the efficient processing, transaction storage or retrieval, and management of the structured and unstructured data have become one of the important issues of the IT industry. In this paper, we examine the concept of evolution of various distributed file systems, advantages, and limitations with respect to the cloud computing paradigm. Distributed File System is a client-based application which permits its users to access, process and modify the data which is stored on a remote server as if it existed on their local systems. The Google file system (GFS), a proprietary distributed file system was developed by Google to meet its own growing big data need, which involved the utilization of commodity hardware in managing the massive generation of raw data. Inspired by GFS, Hadoop Distributed File System (HDFS) was developed, which is an open-source community project. It is designed to store large chunks of data sets consistently as well as providing high bandwidth sets for streaming data on client based applications. In this paper, we describe the GFS and HDFS models in detail and relate these two distributed file systems on the basis of their properties and characteristic behavior in different environments. We also discuss several techniques and modifications to prevent and mitigate the limitations of these file systems.

The distributed file system, HDFS, is widely deployed as the bedrock for many parallel big data analysis. However, when running multiple parallel applications over the shared file system, the data requests from different processes/executors will unfortunately be served in a surprisingly imbalanced fashion on the distributed storage servers. These imbalanced access patterns among storage nodes are caused because a). unlike conventional parallel file system using striping policies to evenly distribute data among storage nodes, data-intensive file system such as HDFS store each data unit, referred to as chunk file, with several copies based on a relative random policy, which can result in an uneven data distribution among storage nodes; b). based on the data retrieval policy in HDFS, the more data a storage node contains, the higher probability the storage node could be selected to serve the data. Therefore, on the nodes serving multiple chunk files, the data requests from different processes/executors will compete for shared resources such as hard disk head and networkbandwidth, resulting in a degraded I/O performance. In this paper, we first conduct a complete analysis on how remote and imbalanced read/write patterns occur and how they are affected by the size of the cluster. We then propose novel methods, referred to as Opass, to optimize parallel data reads, as well as to reduce the imbalance of parallel writes on distributed file systems. Our proposed methods can benefit parallel data-intensive analysis with various parallel data access strategies. Opass adopts new matching-based algorithms to match processes to data so as to compute the maximum degree of data locality and balanced data access. Furthermore, to reduce the imbalance of parallel writes, Opass employs a heatmap for monitoring the I/O statuses of storage nodes and performs HM-LRU policy to select a local optimal storage node for serving write requests. Experiments are conducted on PRObE's Marmot 128-node cluster testbed and the results from both benchmark and well-known parallel applications show the performance benefits and scalability of Opass.

In this paper, we propose a new paradigm for network file system design, serverless network file systems. While traditional network file systems rely on a central server machine, a serverless system utilizes workstations cooperating as peers to provide all file system services. Any machine in the system can store, cache, or control any block of data. Our approach uses this location independence, in combination with fast local area networks, to provide better performance and scalability than traditional file systems. Further, because any machine in the system can assume the responsibilities of a failed component, our serverless design also provides high availability via redundant data storage. To demonstrate our approach, we have implemented a prototype serverless network file system called xFS. Preliminary performance measurements suggest that our architecture achieves its goal of scalability. For instance, in a 32-node xFS system with 32 active clients, each client receives nearly as much read or write throughput as it would see if it were the only active client.

The issues that must be considered in structuring file storage for very large distributed systems are discussed. The authors describe the distributed file system they are developing, which is called FACE (file system for ACE). The first prototype of FACE consists of a series of enhancements to Sun's network file system (NFS) in order to add a stashing capability. User processes interact with the system call interface to perform file and stashing operations. These operations may be invoked directly by user processes or by the FACE user-level routines. The FACE user-level routines implement diverse facilities to allow users to select the files that will be stashed (or dropped from the stash). The first FACE prototype is being implemented on a Sun 3/50 computer running Suri's Unix 4.2 release 3.3. >

Delayed write in a multilevel file system cache hierarchy offers a way to improve performance in diverse client/server scenarios, such as integrating mass store into a distributed file system or providing distributed file system access over low-speed links. Using file system traces and cache simulations, we explore extensions and modifications to the traditional client-caching model employed in such file systems as AFS, Sprite, and DFS. High cache hit rates at an intermediate cache server-a machine logically interposed between clients and servers that provides cached file service to the clients-combined with high client cache hit rates lend practicality to an integrated mass storage file system. In such a system, magnetic tape or optical-based mass storage devices may be used as a first-class data repository, fronted by disk and RAM caches to offer acceptable access times to the large, but slow, mass storage system. Similarly, a high cache hit rate is necessary for users accessing file systems via low-speed links, where a delayed write intermediate caching server can mediate traffic to make better use of available bandwidth. In an example taken from mobile computing, an intermediate server might be used as a docking station at a user's home. This arrangement would be convenient for users of mobile computers who upload large amounts of data generated while operating in disconnected mode. Simulations of delayed write caching strategies are applicable to both the mass storage and low-speed link scenarios.