Neighborhood mutual remainder: self-stabilizing distributed implementation and applications

Distribution has become a necessity for the majority of computing domains, but developing distributed applications remains a highly delicate and complex task. Several emerging distributed computing and application domains, including cloud computing, service-oriented computing, stream processing, sensor networks, and context-aware computing, pose unprecedented challenges to the programmer. Applications in these domains can deliver tangible benefits to the user once they enter the mainstream of industrial software development. What is hindering the wide adoption and use of these applications is the prevalence of ad-hoc programming practices in their software development process. The goal of this workshop is to explore programming support innovations that can address the incongruence between the advanced programming requirements of emerging distributed applications and the current state of the art of their programming support. To that end, the workshop will provide a venue for free and open discussions among academic researchers and industry practitioners of distributed applications.

It is generally accepted that the ability to develop large-scale distributed applications that are extensible and independent of infrastructure details has lagged seriously behind other developments in cyberinfrastructure. As the sophistication and scale of distributed infrastructure increases, the complexity of successfully developing and deploying distributed applications increases both quantitatively and in qualitatively newer ways. In this paper we trace the evolution of a representative set of “state-of-the-art” distributed applications and production infrastructure; in doing so we aim to provide insight into the evolving sophistication of distributed applications — from simple generalizations of legacy static high-performance to applications composed of multiple loosely-coupled and dynamic components. The ultimate aim of this work is to highlight that even accounting for the fact that developing applications for distributed infrastructure is a difficult undertaking, there are suspiciously few novel and interesting distributed applications that utilize production Grid infrastructure. Along the way, we aim to provide an appreciation for the fact that developing distributed applications and the theory and practise of production Grid infrastructure have often not progressed in phase. Progress in the next phase and generation of distributed applications will require stronger coupling between the design and implementation of production infrastructure and the theory of distributed applications, including but not limited to explicit support for distributed application usage modes and advances that enable distributed applications to scale-out.

This paper describes the IBM SNA Distribution Services (SNADS). Heretofore, SNA has focused on synchronous data distribution. Along with the advent of office systems and other distributed applications has come the requirement to provide a common architecture for interchanging data asynchronously among diverse systems and products. SNA Distribution Services provides a general asynchronous (delayed delivery) data distribution facility for SNA applications. The initial implementations are for office systems applications. Discussed are objectives for an asynchronous data distribution service, key architectural concepts, the relationship between SNADS and SNA synchronous communication architecture, and the interface between the distribution service and application transaction programs.

A distributed application may comprise a range of processors, from two to thousands. For the processors of an application comprising many different machines to work together, they must often coordinate. Coordination may be as simple as agreeing upon a basic configuration, e.g., a list of server addresses and ports for example. In the context of distributed systems, even agreeing upon a basic configuration can be complicated: servers may be down during configuration changes, servers may need to adapt quickly to dynamic configuration changes, and during changes servers may have conflicting configurations. Most distributed applications also require more sophisticated coordination primitives such as leader election, group membership, and rendezvous.

This paper presents a novel approach, called WebGOP, for architecture modeling and programming of web-based distributed applications. WebGOP uses the graph-oriented programming (GOP) mode, under which the components of a distributed program are configured as a logical graph and implemented using a set of operations defined over the graph. WebGOP extends the application of GOP to the World Wide Web environment and provides more powerful architectural support. In WebGOP, the architecture graph is reified as an explicit object which itself is distributed over the network providing a graph-oriented context for the execution of distributed applications. The programmer can specialize the type of a graph to represent a particular architecture style tailored for an application. WebGOP also has built-in support for flexible and dynamic architectures, including dynamic reconfiguration. We describe the WebGOP framework, a prototypical implementation of the framework on top of SOAP, and performance evaluation of the prototype. Results of the performance evaluation showed that the overhead introduced by WebGOP over SOAP is reasonable and acceptable.

Although distributed applications such as grids and peer-to-peer technologies are quickly evolving, their deployment across heterogeneous networks such as non-globally addressable networks has not received much attention. The lack of IP addressing space for instance has created an explosion of network address translators and thus a large amount of non-globally addressable hosts. Whereas these boxes do not disturb much the behavior of a client-server communication, this is no longer true in a distributed application built on the peer-to-peer model where any entity can play both roles and thus needs a globally known address. Application level gateways are proposed to solve this problem but there are usually specific to a given distributed application. In this paper we propose an application level addressing scheme and a routing mechanism in order to overcome the non-global addressing limitation of heterogeneous networks such as those brought by network address translation technologies. Our architecture is designed to bring increased scalability and flexibility for communications between members of a distributed application. We give simulation results concerning the efficiency of the addressing and routing mechanisms.KeywordsHeterogeneous NetworkZone SizeAverage Path LengthNetwork Address TranslatorMersenne TwisterThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The goal of a management system in a distributed computing environment is to provide a centralized and coordinated view of an otherwise distributed and heterogeneous collection of hardware and software resources. Management systems monitor, analyse and control network resources, system resources, and distributed application programs. Many organizations currently depend on mission-critical distributed applications, a trend that will increase as software engineering tools emerge that make it easier to construct distributed applications. We believe that manageability must be built in to distributed applications from the beginning rather than added in an ad hoc fashion after they have been developed. Just as designing software for usability, testability and maintenance are being addressed in the development process, so must designing for manageability. Application manageability is a research issue of particular interest to us. The work described in this paper focuses on instrumenting processes to allow them to respond to management requests, generate management reports, and maintain information required by the management system. We present an instrumentation architecture to support this, a prototype implementation which includes a class library of standard instrumentation, and a methodology for instrumentation.

Although a great deal of research has been carried out in the field of distributed computing, a relatively small fraction of the work has concentrated on the area of distributed applicaitons. This paper is an account of one attempt to explore some of the problems which arise in actually implementing a substantial distributed application; in this case the board game Scrabble. Particular emphasis is placed on continunity, i.e. the ability of the applicaion to continue executing despite partial failure. Our main conclusion from this experience is that much better support should be provided at both the operating system and programming environment level to handle the development and debugging of reliable distributed applications. A number of suggestions are made as to how this can be achieved.

Distributed applications are difficult to develop and manage due to their inherent dynamics and heterogeneity of component technologies and the possibility of different network protocols. Distributed component-based software applications consist of a collection of software components that communicate via a distributed middleware to provide distributed application services. The distributed middleware, or simply middleware plays a crucial role by providing APIs and support functions to bridge the gap between the network operating system and distributed components. Component and middleware technologies dramatically simplify the development of distributed applications, but it still difficult to assemble and manage such applications at runtime. In order to address this problem, we present the early findings of our work, which is concerned with the development of a middleware-based framework for the "on-demand" software assembly and subsequent management of distributed applications.

This report presents the design and implementation of a middleware for building large-scale, dynamic, and self-organizing distributed applications for the Internet. First, we identify the challenges that are faced when building this type of applications and the constraints imposed on the middleware that is to support them. We derive a set of essential services that are to be provided by our middleware in order to facilitate the development of distributed applications. These services include scalable communication, failure detection, name-based overlay routing, group communication and a distributed hash table abstraction. We present the event-based component-oriented architecture of the system, discussing the design choices that we made in order to meet the aforementioned challenges and constraints while providing the essential services for distributed applications. We describe in detail the event scheduling mechanism, the communication and failure detection, as well as the interface to applications and other miscellaneous services.

Dynamic optical interconnects using fast Optical Circuit Switches (OCS) are emerging as a scalable and energy efficient alternative to increasing network demands. Initial concerns regarding slow switching speeds of OCSs were recently overcome, with prototypes enabling circuit setup in a few microseconds. This can potentially broaden the classes of traffic patterns that can be carried efficiently by an alloptical interconnect. However, application performance on such newer interconnects has not been fully understood yet. In this paper, we explore the the gap between advances in faster OCS hardware and the potential success of such newer technologies in terms of application performance and cluster energy efficiency. We evaluate the performance of latencysensitive distributed applications running on a fast OCS environment, analyzing its impact to overall server and network energy efficiency. We also discuss scheduling inefficiencies of current fast OCSs and evaluate ideas to solve them. We find that while some distributed applications suffer minimal performance penalty when running on fast OCSs, more flexible schedulers, like the ones outlined in this paper, improve application performance and OCS efficiency by up to 2.44 times compared to the strategies in the literature. Dynamic optical interconnects using fast Optical Circuit Switches (OCS) are emerging as a scalable and energy efficient alternative to increasing network demands. Initial concerns regarding slow switching speeds of OCSs were recently overcome, with prototypes enabling circuit setup in a few microseconds. This can potentially broaden the classes of traffic patterns that can be carried efficiently by an all-optical interconnect. However, application performance on such newer interconnects has not been fully understood yet. In this paper, we explore the the gap between advances in faster OCS hardware and the potential success of such newer technologies in terms of application performance and cluster energy efficiency. We evaluate the performance of latency-sensitive distributed applications running on a fast OCS environment, analyzing its impact to overall server and network energy efficiency. We also discuss scheduling inefficiencies of current fast OCSs and evaluate ideas to solve them. We find that while some distributed applications suffer minimal performance penalty when running on fast OCSs, more exible schedulers, like the ones outlined in this paper, improve application performance and OCS efficiency by up to 2.44 times compared to the strategies in the literature.

The FuzzyLog is a partially ordered shared log abstraction. Distributed applications can concurrently append to the partial order and play it back. FuzzyLog applications obtain the benefits of an underlying shared log - extracting strong consistency, durability, and failure atomicity in simple ways - without suffering from its drawbacks. By exposing a partial order, the FuzzyLog enables three key capabilities for applications: linear scaling for throughput and capacity (without sacrificing atomicity), weaker consistency guarantees, and tolerance to network partitions. We present Dapple, a distributed implementation of the FuzzyLog abstraction that stores the partial order compactly and supports efficient appends / playback via a new ordering protocol. We implement several data structures and applications over the FuzzyLog, including several map variants as well as a ZooKeeper implementation. Our evaluation shows that these applications are compact, fast, and flexible: they retain the simplicity (100s of lines of code) and strong semantics (durability and failure atomicity) of a shared log design while exploiting the partial order of the FuzzyLog for linear scalability, flexible consistency guarantees (e.g., causal+ consistency), and network partition tolerance. On a 6-node Dapple deployment, our FuzzyLog-based ZooKeeper supports 3M/sec single-key writes, and 150K/sec atomic cross-shard renames.

VRML —the Virtual Reality Modeling Language— has estab-lished as the standard description language for 3D worlds on theInternet. Although it recently has been extended by features inorder to support behaviors and user interactions, enablingauthors to realize interactive virtual worlds, there is still a lack ofsupport to share such worlds with other users.In this paper we will present our approach on supportinglarge-scaled shared virtual worlds based on VRML by an appro-priate network infrastructure as well as mechanisms to partitionthese worlds. We will show how virtual worlds can be populatedby world-wide distributed users and applications, and the re-quirements to share behaviors and interactions between those.CR Categories and Subject Descriptors: C.2.2 [ComputerCommunication Networks]: Network Protocols; C.2.4 [Com-puter Communication Networks]: Distributed Systems -Dis-tributed applications; H.5.1 [Information Interfaces and Pre-sentation] Multimedia Information Systems - ArtificialRealities; I.3.2 [Computer Graphics]: Graphics Systems -Dis-tributed/network graphics;I.3.6 [Computer Graphics]: Meth-odology and Techniques - Interaction Techniques; I.3.7. [Com-puter Graphics]: Three-Dimensional Graphics and Realism -Virtual Reality;Additional Keywords Virtual reality modeling language(VRML), multiuser environments, subdivision of shared virtualworlds, multicasting.

Research projects have started working on cloud computing platforms to help cloud applications to be developed in an easiest manner, from year 2000 on. Studies about cloud computing elasticity solutions showed many works were focusing in processing and storage resource management for client-server applications. However, only a small number of research works explore the potential of application contexts regarding network resource management (e.g., content distribution applications). Therefore it is interesting to develop an open platform for distributed applications development helping to manage resources and elasticity in clouds. This dissertation presents Trade Wind, a solution to help the development of distributed applications and services for cloud computing resource and elasticity management. The solution is composed by an elastic application development model, an application compostion model from features and services development, an architecture and a middleware. In order to evaluate and validate the suggested solution, it was developed a test prototype implementing an application for real time video streams distribution utilizing an automatic redundant streams reduction feature. The results collected from the test executions validate Trade Wind solution running the adapted proof of concept application. The tests also showed the multichannel feature added working in a adequate manner. The redundant streams reduction application has proven to reduce bandwidth consumption by the half in the configured test scenarios. And it also has potential to save more bandwidth resources in a scenario with higher number of redundant video streams.

Large-scale service-oriented computing is based on the idea that services from various servers are combined into one distributed application. Referring to a collection of services on one server as a “service cloud”, the problem investigated in this paper is to define formal high-level specifications of such distributed applications and to enable the location of suitable services for them. Based on the language-independent model of Abstract State Services (AS2s), which serves as a universal integrated model for data and software as services, we extend AS2s by high-level action schemes called “plots” as a means to specify permitted sequences of service operations. On these grounds, we develop a model for service mediators, that is, specifications of composed services in which service slots have to be filled by actual services, and investigate matching conditions for slots of mediators and services. For a services to match a slot in a mediator, a (generalised) projection of the mediator must comply with the plot of the service. Furthermore, the service must be semantically adequate, which requires the use of a service ontology.