Replication Techniques in Distributed Systems

Heterogeneous Distributed Computing (HDC) system consists of Workstations (WSs) and Personal Computers (PCs). In HDC system, each WS/PC may have different processor and performance. In order to take advantage of this diversity of processing power of a system, an effective task partitioning, scheduling, and load balancing are needed to get the optimum performance. This paper examines the effectiveness of task partitioning and scheduling strategies for image (raytracing) processing application on HDC system. The static and dynamic/Run-time Task Scheduling (RTS) strategies are shown inadequate for balancing the load of HDC system. Two adaptive tasks scheduling strategies are proposed for HDC image computing system. These adaptive strategies are: i) Master Initiate Sub-task size (MIS) strategy (based on centralized resources management approach), and ii) Worker Initiated Sub-task size (WIS) strategy (based on semi-decentralized resources management approach). The measured results show that the adaptive (MIS & WIS) strategies dramatically improve the performance of HDC raytracing system and remedy the defects of static and RTS strategies. Performances of the all investigated strategies are evaluated on manager/master and workers model of HDC system. In performance comparisons of MIS and WIS strategies, the WIS strategy shows a slightly better speedup and scalability over MIS strategy.

Distributed problems raise privacy issues. The user would like to specify securely his constraints (desires, availability, money) on his computer once. The computer is expected to compute and communicate for searching an acceptable solution while maintaining the privacy of the user. Even without computers infested with spy viruses that capture the interaction with the user, most agent based approaches reveal parts of one agent's secret data to its partners in distributed computations [Using privacy loss to guide decisions in distributed CSP search]. Some cryptographic multi-party computation protocols [Completeness theorems for non-cryptographic fault-tolerant distributed computating] succeed to avoid leaking secrets at the computation of some functions with private inputs. They have been applied to find the set of all solutions for the meeting scheduling problem [On securely scheduling a meeting]. However, nobody yet succeeded to apply those techniques for finding a random solution to the meeting scheduling problem. Note that revealing all solutions, when you only need a single one, leaks a lot of data about when others are, or are not, available. Some answers were proposed in our previous approaches to distributed constraint problems [Solving a distributed CSP with cryptographic multi-party computations, without revealing constraints and without involving trusted servers]. They guarantee that no agent can infer with certitude a secret from the identity of the solution of the problem (other than the acceptance of the solution), but guarantee nothing about inference of probabilistic information about secrets. Our new technique answers this problem, too.

Computation scheduling is very important in the design of distributed information processing and control systems. Effective scheduling algorithms allow developer to find technical solutions that are adequate to the existing constraints. This is especially important for computers located on autonomous carriers, such as unmanned aerial vehicles, autonomous underwater vehicles, and other vehicles. Scheduling algorithms for tasks in the distributed non-deterministic computing system, when the task execution time is known inaccurately and described as time interval, are proposed and researched. The solution of the problem is achieved by reducing it to a known problem of flow shop scheduling with subsequent application of the formalism of solvable classes of distributed computing systems. Authors propose two algorithms for scheduling tasks for a non-deterministic distributed computing system. Algorithms allow the absence of isomorphisms between task graphs and the graph of interprocessor communications for the system, and the presence of many information outputs and branches between tasks. In these conditions, it is impossible to use known algorithms of flow shop scheduling. Proposed algorithms assume preliminary reduction of the considered system to the required form and base on the provisions of interval analysis and the concept of a solvable class of distributed computing systems. Optimality criterion for proposed algorithms is the criterion of minimum average time a task remains in the system. Additionally, the criterion of minimum of maximum deviation from directive terms is used. For the introduced solvable classes of systems, optimal scheduling algorithms of polynomial complexity are proposed. These algorithms allow us to schedule computations in real distributed computing systems when the system deviates from the canonical form and when the durations of the tasks are not precisely known. Proposed algorithms can be applied when scheduling computations in real distributed computing systems and solving tasks with not precisely known durations and also, for example, in scheduling economic processes.

At first thought, one may not realize the need for a special issue of Parallel and Distributed Computing Practices focused on Cluster Computing when our community has already invested years of research and spent a wealth of resources on traditional Big Iron supercomputers, for instance, the SGI/Cray Origin and IBM SP. Academia, industry, and national laboratories are still using, and for the foreseeable future, will continue to use, supercomputers to solve both grand-challenge scale and high-throughput applications. Then why the interest and new popularity of research into clusters for high-performance parallel and distributed applications? Clusters incorporate the tremendous successes that are attributable to our parallel and distributed processing community. Over the next decade, clusters will span the entire range of high-performance computing platforms. Instead of being restricted solely to supercomputers for achieving high-performance computing, the spectrum of available platforms will spread out to Beowulf-class clusters, to superclusters, and on to supercomputers. Moving along this continuum represents increasing computational resources, which in turn allows the capability for solving larger classes of applications or instances of problems. Compute-bound applications on workstations often may take advantage of the extra horsepower provided by symmetric multiprocessors. Applications with even larger computational requirements and a high-degree of concurrency may use Beowulf clusters, that is, clusters that are home-built from mass-market, commercial off-the-shelf workstations and networks, for instance, PC's interconnected by Fast Ethernet. A supercluster may contain commodity SMP nodes interconnected with scalable, low-latency, high-bandwidth networks, and is integrated by a system vendor rather than the computational scientist. Applications with greater complexity, for example, grand challenges from the sciences, demand scalable systems with high-performance, interconnects, perhaps coupled with terascale I/O subsystems, and thus, are appropriate for these superclusters. Finally, supercomputers, the largest of the clusters, may provide the ultimate platform for the highest-fidelity models and simulations and data-intensive applications. At each step up this ladder of cluster architectures, the system cost increases approximately by an order of magnitude, but having the high-performance capability to solve the problem of interest may require an entry point at any of these levels, the Beowulf, or supercluster, or even supercomputer platform. Cluster computing with Beowulf- or superclusters, thus, may be one or two orders of magnitude less expensive than a supercomputer, and provide a cost-effective solution and capability that was not available on a workstation. But this cost comparison measures just the basic system price. What about the applications? As recently as a few years ago, the lack of a canonical high-performance architectural paradigm meant that migrating to a new computer system, even from the same vendor, typically required redesigning applications to efficiently use the high-performance system. Thus, the application must be redesigned with parallel algorithms and libraries that are optimized for each high-performance system. This process of porting, redesigning, optimizing, debugging, and analyzing, the application may in fact be prohibitively expensive, and certainly frustrating when the process is completed just in time to greet the next generation system! That was then, this is now. With the successful standardization of system-level interfaces, many of these issues no longer hinder our expedited use of these new, high-performance clusters. The message-passing interface (MPI) is a clear example of a standard that has contributed significantly to the success of cluster computing. Supercomputer vendors working together with the high-performance computing community embarked on a process to standardize the user-level interface and behavior for passing messages between concurrent tasks. Now, using ordinary sequential compilers and vendor-supplied or freely-available implementations of this message-passing interface, applications can be implemented efficiently on one supercomputer and ported with little or no effort to other parallel machines. The message-passing paradigm, thus, provides a reasonably good canonical high-performance model and methodology for implementing high-performance applications. An algorithm that exhibits efficient execution and scaling on one machine then is likely to perform well on another with little or no additional programming effort. Using this cluster computing paradigm, algorithms typically are optimized by minimizing the number of remote memory accesses (reducing the amount of messages passed), and carefully organizing the data placement to increase memory locality. Thus, high-performance algorithms and applications that have been optimized for supercomputers, using the same cluster paradigm, may be appropriate for Beowulf and superclusters. Cluster computing, realizing the importance of a standard high-performance programming environment that applies equally well to both ends of the architectural spectrum, highly leverages these and other successes from parallel and distributed processing research and development. This is in an exciting, new era for high-performance computing where the canonical architectural paradigm stretches from home-built Beowulf clusters, to superclusters, and on to supercomputers. But cluster computing faces some challenges ahead of it. Continued research is required for designing 1) scalable, high-performance algorithms, applications, and methodologies, that can tolerate the communication overheads, and adapt to hybrid, SMP, and heterogeneous, nodes; 2) high-performance computer architectures, scalable interconnection networks, and I/O subsystems; and 3) scalable systems software, compilers, and monitoring tools. Cluster computing has come of age, and it is high time for the parallel and distributed processing community to lay the groundwork for our own legacy of success stories. David A. Bader University of New Mexico

The Hadoop Distributed File System (HDFS) is designed to store, analysis, transfer larg datasets reliably, and stream it at high bandwidth to the user applications. HDFS is a variant of the Google File System (GFS). It handles fault tolerance by using data replication, where each data block is replicated and stored on multiple DataNodes. Therefore, the HDFS supports availability and reliability. The existed implementation of the HDFS in Hadoop performs replication in a pipelined manner that takes much time for replication. In this paper, an alternative technique for efficient replica placement, called Enhancement Lazy replication technique, has been proposed. The main principle of this technique is that, the client allows to write a block to two DataNodes in parallel, which store the packet. which will send acknowledgement directly to the client without waiting of receiving acknowledgement from other DataNodes. The experiment has been performed to evaluate the performance of the proposed HDFS replication technique with the default pipelined replication technique and the existed replication techniques; using TestDFSIO benchmark. According to the experimental results (i.e., the execution time and throughput), it is found that the HDFS availability has been improved in the proposed replication technique.

This research work focus on distributed system based network issues. The networked servers and computers organize their actions only by passing the proper messages. This networked system having especially significant characteristics like concurrency components and independent failures components. While we share the files through by passing message, there will be security problems because of connecting the entire computers in network. For High performance in data accessing, data surviving and also secure storage designing, the data partitioning and replication techniques can be combined. In this work, two heuristic algorithms developed for two sub problem in which the Authentication of the data share can be done using network manager to improve security. The replicated data are authenticated and send dynamically in the distributed system. The newly developed algorithms provide maximum performances in data sharing through distributed system and in achieving maximum security.

In 1998, Ian Foster and Carl Kesselman, together with thirty distinguished experts in high-performance computing and networking, laid the foundations of a new computing model called GRID. Their vision was introduced in the book, The GRID: Blueprint for a New Computing Infrastructure (see review, SCPE Vol. 3, No. 3 ). The editors wrote: … A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities… allowing new classes of applications to emerge… as yet, we have only a preliminary understanding of what these new applications will look like. Over the next five years, Grid computing became the hottest R&D field. Academia and national labs built experimental infrastructures, new startups were established to develop Grid technologies, and the HPC giant vendors joined the party. Each one pushed in a different direction based on what and how he believed Grid computing should look. It was only a matter of time until people started to ask what the Grid is all about. A survey of ten Grid experts produced ten different answers. The question remains without a clear answer, leaving many people confused. Ian Foster picked up the gloves. In his article, What is the GRID? A three point checklist (GRIDtoday, July 22, 2002: Vol. 1, No. 6), he redefines the Grid model to reflect the trades and changes in the field during the past five years. According to Foster, a Grid is a system that: Coordinates resources that are not subject to centralized control. Uses standard, open, general-purpose protocols and interfaces. Delivers a nontrivial quality of service. But Foster's new definition could not encompass all of the Grid's approaches. Moreover, Foster explained by example what the Grid is not: A cluster management system such as Sun's Sun Grid Engine, Platform's Load Sharing Facility, or Viridian's Portable Batch System can, when installed on a parallel computer or local area network, deliver quality of service guarantees and thus constitutes a powerful Grid resource. However, such a system is not a Grid itself, due to its centralized control of the hosts that it manages Within a week the responses started to come in. The first one was from Wolfgang Gentzsch, Director of Grid computing at Sun Microsystems (GRIDtoday, August 5, 2002: Vol. 1, No. 8). Gentzsch wrote: I very much like Ian's two earlier definitions. Combined, they may provide a very generic Grid definition. The new checklist, however, reduces the potential of a wide and rich variety of Grids, with different architectures, operational models, and applications, and may miss the chance to becoming widely accepted as a standard definition of the Grid. Why do we need a definition for the Grid? Can the Grid be defined? Is a definition is necessary to the Grid's development? The Grid is an open vision and thus cannot be defined. An attempt to define the Grid can only reach the foreseen technology horizon, and this horizon is only one to two years ahead. Grid computing is a paradigm in shift. New architectures and standards are changing the shape and direction of the Grid's development. In the past year one such prominent architecture was Open Grid Services Architecture (OGSA). The OGSA is the first attempt to bridge the gap between two cyberspace worlds: the Internet and the Grid. The Internet and the Grid are two different computing models. There are people who believe that the Grid is the next generation of the Internet and the World Wide Web, saying that the Great Global Grid is still many years away. Others state that the Grid will complement, not replace, the Internet as we know it. In 1998, when the Grid blueprint was published, a new Internet language appeared called eXtensible Markup Language (XML). XML is a development tool for a new Internet computing model known as Web Services. At that time, nobody understood that there is a relationship between Web Services and the Grid. Three years past before Steve Tuecke from Argonne National Laboratory started to write a spec for Grid Services. Later, people from IBM joined the mission and together they created OGSA. The OGSA specification outlines interfaces to grid computing software that comply with Web Services standards. If adopted, Grid services such as job scheduling, authentication, failure-detection, staging of applications and data, and migration of applications and data, will all be accessed through standard Web Services architecture. OGSA is basically where Grid meets Web Services and maybe the first step towards full integration of the Internet and the Grid. The Internet appeared a decade ago. In the beginning, the Net offered primitive applications: a mailing system, ftp services, and a web of home pages. Nobody asked, What is the Internet? The years passed and new emerging Internet technologies were developed. Today we can talk on the phone over the Net, chat with friends, and video conference with colleagues on the other side of the world. New computing and business models were developed: e-commerce, e-learning, e-science, and e-you-name-it. And nobody asks what the Internet is. The Grid is a super-model, a bag of networking models associated with many types of architectures, software packages, applications, and standards. Today, when people talk about the Grid they mention Data Grids, Science Grids, and Campus Grids. And at the same time they mention Farm computing, Peer-to-peer computing, and Utility computing, among others. Ten years from now many of today's technologies will disappear and new ones will be developed. I don't know how Grid computing will look a decade from now, but I know that we will have g-commerce, g-learning, g-science, and g-chat. Nobody will ask what the Grid is. It will be obvious, just as the Internet is today. Ami Marowka The Computer Aided Design Laboratory School of Engineering and Computer Science Hebrew University of Jerusalem

A rapid updating system of three-dimensional road traffic noise maps is presented. The system described in this paper primarily consists of two parts. The first part is an optimized noise update algorithm, which can significantly improve the update speed for large scale noise maps by reducing the amount of calculation work. To make full use of scattered computing resources, the second part is a distributed computation system with high simplicity and extensibility and the ability of self-balance. For traffic flow data gathered in large cities, a model that uses a speed-density relationship is developed to estimate the traffic volume from the Global Positioning System (GPS) data collected by floating cars. Finally, an experiment was performed to analyze the efficiency of both the distributed computation system and the optimized algorithm. The outcomes show that the distributed computation system has a near-linear speedup and the optimized algorithm can update a noise map of a 5.25 km2 area with 446,367 noise receiver points using about 96 s. Thus, for a large city with an area of approximately 500 km2, such as the central area of Guangzhou, its noise map can be updated hourly using only 3 personal computers.

As a main distributed computing system, Spark has been used to solve problems with more and more complex tasks. However, the native scheduling strategy of Spark assumes it works on a homogenized cluster, which is not so effective when it comes to heterogeneous cluster. The aim of this study is looking for a more effective strategy to schedule tasks and adding it to the source code of Spark. After investigating Spark scheduling principles and mechanisms, we developed a stratifying algorithm and a node scheduling algorithm are proposed in this paper to optimize the native scheduling strategy of Spark. In this new strategy, the static level of nodes is calculated, the dynamic factors such as the length of running tasks, and CPU usage of work nodes are considered comprehensively. And through a series of comparative experiments in alienation cluster, the new strategy costs less running time and lower CPU usage rate than the original Spark strategy, which verifies that the new schedule strategy is more effective one.

New Method to Use Idle Personal Computers for Solving Coherent Tasks

Recently, more and more researches and applications exploit grid computing systems to deal with high performance computing. However, the mass data transmissions across different grid sites affect overall computing performance. Therefore, grid systems start to integrate with the P2P technology to support the high performance distributed computing. The new distributed computing system is named the P2P Grid computing system. Although the P2P Grid computing system combines the advantages of the grid computing system and the P2P technology, some issues are still needed to be solved. For example, the highly variable resource usage and the heterogeneity of resources could intensely affect the P2P Grid system performance. In this case, the computing performance depends on the resource management policy. Therefore, this study proposes a distributed dynamic load balance policy to manage resources more effectively and to further improve the resource utilization. The prototype is implemented on the sites of the Taiwan Uni Grid, and the P2P grid sites exchange information by JXTA advertisements. Experimental results show that the proposed algorithm could efficiently distribute the workload for execution, that is, it not only can minimize the job execution time, but also maximize the resource utilization.

Loosely coupled applications composed of a potentially very large number (from tens of thousands to even billions) of tasks are commonly used in high-throughput computing and many-task computing paradigms. To efficiently execute large-scale computations which can exceed the capability in a single type of computing resources within expected time, we should be able to effectively integrate resources from heterogeneous distributed computing (HDC) systems such as clusters, grids, and clouds. In this paper, we quantitatively analyze the performance of three different real scientific applications consisting of many tasks on top of HDC systems based on a partnership of distributed computing clusters, grids, and clouds to understand the application and resource characteristics, and show practical issues that normal scientific users can face during the course of leveraging these systems. Our experimental study shows that the performance of a loosely coupled application can be significantly affected by the characteristics of a HDC system, along with hardware specification of a node, and their impacts on the performance can vary widely depending on the resource usage pattern of each application. We then devise a preference-based scheduling algorithm that can reflect characteristics and resource usage patterns of various loosely coupled applications running on top of HDC systems from our experimental study. Our preference-based scheduling algorithm can allocate the resources from different HDC systems to loosely coupled applications based on the preferences of the applications for the HDC systems. We evaluate the overall system performance over various preference types, using trace-based simulations, which can be determined based on different factors such as CPU specifications and application throughputs. Our simulation results demonstrate the importance of understanding the application and resource characteristics on effective scheduling of loosely coupled applications on the HDC systems.

Robustness is a fundamental and timeless issue, and it remains vital to all aspects of computation systems, regardless of specific computation platforms, architectures, and algorithm design. The issue is also timely: modern computing systems are increasingly built on unreliable substrates. This thesis designs reliable computing techniques for distributed systems, circuits and networks.We primarily study techniques inspired from coding theory to address the robustness issues such as system elasticity, stragglers (slow workers), machine failures and soft errors, by carefully weaving redundancy into the data andthe design of the algorithm. We primarily focus on three aspects of coding-based computation techniques.The first aspect is to design adaptive computing techniques in largescale systems that are elastic, i.e., the number of machines can change with time. The second is to make the coding-based computation schemes cater to the specific needs of iterative and multi-stage algorithms, such as power iterations and gradient descent that are essential for today’s data analytics. The third aspect is to study the fundamental limits of information propagation in computation networks, provide information-theoretic outer bounds on erroraccumulation, and design in-network computing schemes to compensate for this error accumulation, e.g., in a circuit network where each computationcomponent can be unreliable. This thesis presents theoretical results that are fundamental to the understandingof computation systems, and opportunities for radical improvements, e.g. in computation load, communication overhead, and storage cost. We also present real-system implementations and real-data experiments and show our progress on taking these theoretical results closer to practice. The academic results presented in this thesis advance on classical results in the historical field of reliable distributed computing, while simultaneously addressing timely issues arising in today’s computing systems.

The data grid is one of the infrastructures for data and storage resource management. Data replication can improve data access performance, fault tolerance, and reduce network transmission bandwidth. As the site storage space is limited in the grid, how to effectively use grid resources become an important challenge. This paper introduces a dynamic grid replication algorithm based on popularity support and confidence (BPSC). Through the algorithm, data and its associated copy can be placed on a suitable site, with reduced access latency. Through Optorsim, simulation results show that the algorithm can provide better performance compared with other algorithms Grid computing is an important branch of distributed computing. Grid computing constructs a virtual computer system through using a different location of the large number of heterogeneous resources (CPU and storage), provides the model to solve the problem of large scale computing and storage. The data grid is mainly the grid for data storage management. The data grid is defined as an effective combination of data and computing resources of distributed systems by Data Grid Group. The large-scale data- intensive applications, such as bioinformatics, high energy physics, space information generated vast amounts of data, these data may be TB scale, even up to the scale of PB. It is not practical to maintain large scale data at each site, and provide computing resources for visitors. At the same time, it will lead to increased access latency through a centralized way to storing and accessing these data from each site. Data replication technology is an effective method to reduce access latency in the data grid system. Data replication is one of the basic methods of distributed systems to improve the efficiency of system access, maintenance, system high availability. Data replication is to store data in the appropriate site for users to effective access to massive data. Data replication can reduce data access latency, reduce network bandwidth consumption, avoid congestion, adjust the load balancing in server-side, improve system reliability. Many data replication algorithm can reduce data access latency, improve system reliability, but which data will be copied, when the data will be copied at, where the data will be copied are the basic means of determining data replication algorithm. In this paper, we introduced a real-time dynamic data replication algorithm. The algorithm is based on the popularity of file dynamic to replicate data, and consider the storage space limitations of the site. Through optorsim simulation verified the algorithm. This paper is organized as follows: Section 2 is about related works on replication in data grids. In Section 3, system model and component in system are presented. The file

This work describes modern methods of parallelizing time consuming problems. Commonly used parallelization methods, such as OpenMP, MPI and GPUs, aimed to accelerate the solution of electromagnetic forward problems are revisited. Comparison of these technologies, their constraints and benefits for complex three-dimensional geoelectromagnetic field modeling are provided. An approach to configure the distributed computing system to solve most resource-intensive geoelectromagnetic problems is proposed. The problem of load balancing between computational nodes of the distributed computing system, advantages and disadvantages of different task orchestration approaches are considered. Different approaches on building fault tolerant systems to create a reliable one that will be capable of carrying out complex three-dimensional time consuming problems are discussed. The efficiency of the proposed method of distributed computing is shown on the example of solving 3D inverse problems for geoelectromagnetic survey with geoelectrical imaging of a medium with a complex structure. The structure and conductivity of the three layers located at different depths with varying lateral depth, thickness and conductivity was restored. Results of computational experiments show high computational efficiency of the proposed system and fault tolerance to both equipment failures and errors of computing software.