Distributed operating systems based on a protected global virtual address space

No abstract available.

In the effort for exascale performance, current High-Performance Computing (HPC) systems need to be improved. Simple hardware scaling is not a feasible solution due to the increasing utility costs and power consumption limitations. Hence, apart from improvements in implementation technology, what is needed is to refine the HPC application development flow as well as the system architecture of future HPC systems. ECOSCALE tackles precisely these challenges by proposing a scalable programming environment and architecture, aiming to substantially reduce energy consumption as well as data traffic and latency. Furthermore, ECOSCALE introduces a novel heterogeneous energy-efficient hierarchical architecture, as well as a hybrid many-core+OpenCL programming environment and runtime system. The ECOSCALE approach is hierarchical and is expected to scale well by partitioning the physical system into multiple independent Workers. Workers are interconnected in a tree-like fashion and define a contiguous global address space that can be viewed either as a set of partitions in a Partitioned Global Address Space (PGAS), or as a set of nodes hierarchically interconnected via an MPI-like protocol. To further increase energy efficiency, as well as to provide resilience, the Workers employ reconfigurable accelerators mapped onto the virtual address space by utilising a dual-stage system memory management unit with coherent memory access. The architecture supports shared partitioned reconfigurable resources accessed by any Worker in a PGAS partition, as well as automated hardware synthesis of these resources from an OpenCL-based programming model.

Presents a paradigm that combines the speed of single address spaces with the traditional protection mechanisms found in multiple address space systems such as UNIX. Multiple address spaces are common today for providing safety and security in operating systems. A process in one address space cannot corrupt or compromise memory locations an another process's address space. However, inter-process communication is slow because it is expensive to context switch between address spaces. Single address space systems provide a solution to the problem of slow context switches. In a single address space system, which is typically implemented in a 64-bit address space, all processes shape a common address space. Hence, the need to move between address spaces during inter-process communication is avoided. It is our position that it is not strictly necessary to use a single address space in order to achieve fast inter-process communication. >

The Partitioned Global Address Space (PGAS) programming paradigm offers ease-of-use in expressing parallelism through a global shared address space while emphasizing performance by providing locality awareness through the partitioning of this address space. Therefore, the interest in PGAS programming languages is growing and many new languages have emerged and are becoming ubiquitously available on nearly all modern parallel architectures. Recently, new parallel machines with multiple cores are designed for targeting high performance applications. Most of the efforts have gone into benchmarking but there are a few examples of real high performance applications running on multicore machines. In this paper, we present and evaluate a parallelization technique for implementing a local DNA sequence alignment algorithm using a PGAS based language, UPC (Unified Parallel C) on a chip multithreading architecture, the UltraSPARC T1.

Memory-centric computing demands careful organization of the virtual address space, but traditional methods for doing so are inflexible and inefficient. If an application wishes to address larger physical memory than virtual address bits allow, if it wishes to maintain pointer-based data structures beyond process lifetimes, or if it wishes to share large amounts of memory across simultaneously executing processes, legacy interfaces for managing the address space are cumbersome and often incur excessive overheads. We propose a new operating system design that promotes virtual address spaces to first-class citizens, enabling process threads to attach to, detach from, and switch between multiple virtual address spaces. Our work enables data-centric applications to utilize vast physical memory beyond the virtual range, represent persistent pointer-rich data structures without special pointer representations, and share large amounts of memory between processes efficiently.

Distributed memory parallel machines are increasingly being used by the scientific community to provide computational power for solving problems. Building a distributed memory parallel machine offers a tremendous advantage in terms of cost and scalability over building a shared memory parallel machine. Unfortunately, distributed memory machines are much more difficult to program, mainly due to the lack of a single global address space. Communication between processors in a distributed memory machine is accomplished by message passing. The time cost for acquiring data from another processor's memory is significantly greater than the time cost of acquiring data from local memory. To assist the programmer in using a distributed memory machine, compilers have been developed which alleviate the need for the programmer to manage the lack of a global address space. These compilers will generate semantically correct code containing communication routines to produce a logically shared memory machine on a physically distributed memory machine. However, the runtime performance of the code emitted from some of these compilers can be further improved. This thesis is primarily concerned with advanced compilation techniques which will provide better runtime performance of programs written in imperative languages for regular scientific programs. The optimization of communication generation for distributed memory machines is used to produce this result. We look at two "state of the art" compilation systems and find that improvement in the communication code generated by these compilers is still possible. We have simplified existing descriptions of the various formalisms found in these compilers, while still providing enough formalism to study the problems in depth. We have extended some existing optimization techniques by enhancing the elimination of masks, by extending the notion of relaxing the owner computes rule, by extending the notion of replicating computation, and by extending the notion of replicating data. This thesis has discussed a novel technique to split loops so that they do not contain any guarded statements. Finally, we have suggested avenues for future research, as it is clear from this thesis that more work is required before compilation systems for distributed memory parallel machines will be widely used for general purpose computing.

Mobile ad hoc networks (MANETs) are networks with self-organizing capabilities and without a fixed infrastructure. Wireless nodes communicate among themselves using multi-hop radio relaying, without requiring the packets to pass through a central access point or a base station. Routing is a function of speed and stability with which the nodes are able to acquire addresses. Efficient routing forms the basis of a fast and reliable communication network. In a highly mobile and infrastructure-less scenario, pre-configuration of addresses is not possible. Therefore node addresses need to be configured dynamically with minimum delay and packet loss. The main task of an address auto-configuration protocol is to manage the resource address space. It must be able to select, allocate, and assign a unique network address to an un-configured node. This paper proposes a new address auto-configuration protocol for mobile ad hoc networks. The scheme uses virtual address space for addressing new nodes joining a network. The aim is to map one point from virtual address sheet to exactly one new node. The reason for using the term "virtual" is that the whole corresponding address space is a 2D flat sheet and each point of this sheet is virtually mapped to a node in MANET. The protocol uses coordinate values for generating addresses.

Data parallel languages designed for distributed memory computing environments provide a single global address space to the programmer. The mapping from this global address space to the distributed local address space is performed by a compiler, which does this mapping based on the array distribution format. Thus, each array in a data parallel language program has its own distribution format. The Reshape function changes the array distribution format as well as the array shape. However, the changed distribution format cannot be represented by any distribution format supported in current languages. Because there is no suitable distribution format, it is necessary to change the reshaped distribution format to an existing distribution format, with heavy overhead due to the redistribution function. To eliminate the redistribution step in Reshape function, we have proposed a new distribution format, HIER-CYCLIC, which can represent the reshaped distribution format. We have also proposed a language syntax to use HIER-CYCLIC and a compiling mechanism. Finally, we performed an experiment on an IBM-SP2 machine using a shift function.

Some aspects of research into distributed shared memory (DSM) systems which concern reliability are presented. The research is concerned with applications that use shared memory in single-site systems, and their extension to operation in a distributed environment. A DSM system consists of a shared address space that is accessed by logical operations. In the author's model processes create shared memory by using a segment; the segment's size, name, and access protection are defined when the memory is created. Processes locate and attach the segment into their virtual memory address space by name. When a process is finished with the segment, it may be detached. The last detach of a segment destroys the data. The shared portion of a process's address space is partitioned into data pages, which are replicated. System implementation, distributed architecture, and issues affecting DSM reliability are discussed. >

Irregular memory access patterns pose performance and user productivity challenges on distributed-memory systems. They can lead to fine-grained remote communication and the data access patterns are often not known until runtime. The Partitioned Global Address Space (PGAS) programming model addresses these challenges by providing users with a view of a distributed-memory system that resembles a single shared address space. However, this view often leads programmers to write code that causes fine-grained remote communication, which can result in poor performance. Prior work has shown that the performance of irregular applications written in Chapel, a high-level PGAS language, can be improved by manually applying optimizations. However, applying such optimizations by hand reduces the productivity advantages provided by Chapel and the PGAS model. We present an inspector-executor based compiler optimization for Chapel programs that automatically performs remote data replication. While there have been similar compiler optimizations implemented for other PGAS languages, high-level features in Chapel such as implicit processor affinity lead to new challenges for compiler optimization. We evaluate the performance of our optimization across two irregular applications. Our results show that the total runtime can be improved by as much as 52x on a Cray XC system with a low-latency interconnect and 364x on a standard Linux cluster with an Infiniband interconnect, demonstrating that significant performance gains can be achieved without sacrificing user productivity.

Partitioned Global Address Space (PGAS) parallel programming models can provide an efficient mechanism for managing shared data stored across multiple nodes in a distributed memory system. However, these models are traditionally directly addressed and, for applications with loosely-structured or sparse data, determining the location of a given data element within a PGAS can incur significant overheads. Applications incur additional overhead from the network latency of lookups from remote location resolution structures. Further, for large data, caching such structures locally incurs space and coherence overheads that can limit scaling. We observe that the matching structures used by implementations of the Message Passing Interface (MPI) establish a separation between incoming data writes and the location where data will be stored. In this work, we investigate extending such structures to add a layer of indirection between incoming data reads and the location from which data will be read, effectively extending PGAS models with logical addressing.

Memory consistency models, or memory models, allow both programmers and program language implementers to reason about concurrent accesses to one or more memory locations. Memory model specifications balance the often conflicting needs for precise semantics, implementation flexibility, and ease of understanding. Toward that end, popular programming languages like Java, C, and C++ have adopted memory models built on the conceptual foundation of Sequential Consistency for Data-Race-Free programs (SC for DRF). These SC for DRF languages were created with general-purpose homogeneous CPU systems in mind, and all assume a single, global memory address space. Such a uniform address space is usually power and performance prohibitive in heterogeneous Systems on Chips (SoCs), and for that reason most heterogeneous languages have adopted split address spaces and operations with nonglobal visibility. There have recently been two attempts to bridge the disconnect between the CPU-centric assumptions of the SC for DRF framework and the realities of heterogeneous SoC architectures. Hower et al. proposed a class of Heterogeneous-Race-Free (HRF) memory models that provide a foundation for understanding many of the issues in heterogeneous memory models. At the same time, the Khronos Group developed the OpenCL 2.0 memory model that builds on the C++ memory model. The OpenCL 2.0 model includes features not addressed by HRF: primarily support for relaxed atomics and a property referred to as scope inclusion. In this article, we generalize HRF to allow formalization of and reasoning about more complicated models using OpenCL 2.0 as a point of reference. With that generalization, we (1) make the OpenCL 2.0 memory model more accessible by introducing a platform for feature comparisons to other models, (2) consider a number of shortcomings in the current OpenCL 2.0 model, and (3) propose changes that could be adopted by future OpenCL 2.0 revisions or by other, related, models.

An evaluation is made of a way to reduce the cost of program restructuring by having a compiler determine the program's packing in virtual address space from an analysis of its source code. Two features of this method are the duplication of code modules in virtual address space and the inline substitution of the code for a called procedure. This compile-time restructuring algorithm is evaluated using the instruction-only address traces from a collection of programs. In a simulation of a virtual memory system using disks as secondary storage devices, the method is not successful, since it leads to a higher optimum space-time execution cost than that of the unrestructured program. The algorithm did reduce program space-time execution cost for some arbitrarily chosen memory allocations smaller than the optimum. This could be useful in a multiuser, multiprogrammed environment.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

The authors propose an information management system providing distribution and persistency. By separating context from virtual address space, the system has a unified approach for both distribution and persistency. The former is achieved by moving contents between sites and the latter by moving contents between virtual address space and persistent storage. Contents include any information including data, program, and even the state of execution of a program. Contents are stored persistently in a logical space termed the distributed shared repository (DSR). A programming model for the DSR is proposed. Using the model, persistency, fine-grain mobility of information, and the passing of various distributed parameters can be obtained. The implementation anti experimental performance of the system are also presented. >

Andries van DamThe panel will examine the evolution of the professional workstation from a time shared terminal to powerful graphics-based personal computer connected to a resource-sharing local network. The panelists will speculate on the future evolution of both the hardware/software architecture and end user environment.Workstation development at StanfordJames H. ClarkWorkstation development at Stanford is closely linked with graphics, distributed systems and networking. Two distinct systems have evolved over the last 3 years: the SUN system and the IRIS system. The SUN is a low-cost system based upon an efficient MC68000 processor/memory design and a relatively low-performance, high-resolution bit-map display. The IRIS (Integrated Raster Imaging System) is based upon the same MC68000 design, but the graphics part of the system is a modest- to low-cost, high-performance, high-resolution, color or black and white system that uses the Geometry Engine and several other custom IC parts. Both SUN and IRIS interface to the Stanford Ethernet network, and both are being used for distributed systems research, VLSI design stations, and graphics research. In addition, IRIS will probably be used for mechanical CAD research by the mechanical engineering department and in situations where high-performance graphics is important.Improvement Goals for Workstation FacilitiesRobert M. DunnThree major areas of improvement are needed: interaction techniques that are simpler, provide faster reaction, are useful for higher-level inputs, and have user-style-oriented alternatives. The second area is for image rendering based on local capabilities, in varying degrees of image quality as a function of desired “grade of service”. The third area is to provide support to incremental model construction and approximate design evaluation. There must be evaluation techniques that can work on partial models and give one approximate results. Implications of these criteria for workstation architecture will be discussed.The application of network workstations to large-scale engineeringHarvey KriloffThe development of computerized analysis procedures during the last twenty years has been largely oriented toward providing increased analytic function. This has meant that considerations of user access to specific capabilities or ease of use of these mammoth programs are only now becoming user concerns. The emerging new technologies for display, input and their interaction, when applied to the professional workstation, will be playing an increasingly important role in satisfying these concerns. A professional workstation can be used both to improve the efficiency by which data is collected for an existing analysis program and to assist the user in the preparation of data for a formal presentation or report. This performance improvement can be accomplished through the development of user-adapted “macro procedures” for data entry, the execution of processes to check program input data for accuracy and consistency, and workstation assistance in the training of both new and existing users. At the output stage of analysis, the workstation can be used to select and reformat information prepared by the analysis program, explore the interrelationship of results from several different analyses, and derive the data for a succeeding analysis in an iterative design mode.At Boeing Computer Services, we have been exploring the benefits realized by the development of such a workstation when applied to the field of structural engineering. An interaction-rich workstation with a human-engineered executive that controls and integrates the user interface, connected to our national network of IBM, CDC and CRAY computers with a broad variety of engineering software, supplies the engineer with a full range of tools for analysis, data preparation, report writing and data relationship exploration that are only beginning to be appreciated. In the next few years it is expected that this workstation (distributed) methodology for doing engineering and other quantitative professional functions will radically change the way these analytical processes are performed.Personal Workstations in a Local NetworkDavid NelsonFor many applications, personal workstations provide a superior form of computing compared to timesharing. The requirements of workstations, however, go well beyond the facilities provided by the local computer; they must include the equivalent advantages of timesharing such as user-to-user communications, shared programs and data files, shared peripherals, etc. The preferred way to implement these functions is to interconnect all workstations by means of a high-speed local network, controlled by a distributed operating system which provides transparent access to all network resources through a network-wide virtual memory system. Additional workstation functions such as large virtual address space and a concurrent multiple-window display system significantly increase user productivity. A high resolution bit-map display system with hardware support for dynamics is an essential component. For the future we look towards both significant cost reductions and improvement in performance, as well as significantly higher level software to better implement the user-computer interface.