A scalable real-time synchronization protocol for distributed systems

The scheduling of real-time tasks with primary-backup based fault-tolerant requirements has been an important problem for several years. Most of the known scheduling schemes are non-adaptive in nature meaning that they do not adapt to the dynamics of faults and task’s parameters in the system. In this paper,we propose an adaptive faulttolerant scheduling scheme that has a mechanism to control the overlap interval between the primary and backup versions of tasks such that the overall performance of the system is improved. The overlap interval is determined based on the estimated primary fault probability and task’s soft laxity. We also propose a new performance index that integrates schedulability (S) and reliability (R) into a single metric, called SR index. To evaluate the proposed scheme, we have conducted analytical and simulation studies under different fault and deadline scenarios, and found that the proposed adaptive scheme adapts to system dynamics and offers better SR index than that of the non-adaptive schemes.KeywordsSchedule SchemeProcessor UtilizationFault RateFault ProbabilityExecution IntervalThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Efficient maximum reaction time analysis for data chains of real-time tasks in multiprocessor systems

Scheduling tasks in multiprocessor systems is a difficult problem. The paper describes a method based upon evolutionary strategies (using genetic algorithms) to aid in finding good task assignments. The technique is illustrated by scheduling a digital signal processing algorithm on a two processor distributed system. >

The increasing need for data sharing in large-scale distributed systems may place a heavy burden on critical resources such as file servers and networks. Our examination of the workload in one large commercial engineering environment shows that wide-spread sharing of unstable files among tens to hundreds of users is common. Traditional client-based file cacheing techniques are not scalable in such environments.We propose Frolic, a scheme for cluster-based file replication in large-scale distributed file systems. A cluster is a group of workstations and one or more file servers on a local area network. Large distributed systems may have tens or hundreds of clusters connected by a backbone network. By dynamically creating and maintaining replicas of shared files on the file servers in the clusters using those files, we effectively reduce reliance on central servers supporting such files, as well as reduce the distances between the accessing sites and data. We propose and study algorithms for the two main issues in Frolic, 1) locating a valid file replica, and 2) maintaining consistency among replicas. Our simulation experiments using a statistical workload model based upon measurement data and real workload characteristics show that cluster-based file replication can significantly reduce file access delays and server and backbone network utilizations in large-scale distributed systems over a wide range of workload conditions. The workload characteristics most critical to replication performance are: the size of shared files, the number of clusters that modify a file, and the number of consecutive accesses to files from a particular cluster.

A coherent global address space in a distributed system enables shared memory programming in a much larger scale than a single multicore or a single SMP. Without dedicated hardware support at this scale, the solution is a software distributed shared memory (DSM) system. However, traditional approaches to coherence (centralized via active home-node directories) and critical-section execution (distributed across nodes and cores) are inherently unfit for such a scenario. Instead, it is crucial to make decisions locally and avoid the long latencies imposed by both network and software message handlers. Likewise, synchronization is fast if it rarely involves communication with distant nodes (or even other sockets). To minimize the amount of long-latency communication required in both coherence and critical section execution, we propose a DSM system with a novel coherence protocol, and a novel hierarchical queue delegation locking approach. More specifically, we propose an approach, suitable for Data-Race-Free programs, based on self-invalidation, self-downgrade, and passive data classification directories that require no message handlers, thereby incurring no extra latency. For fast synchronization we extend Queue Delegation Locking to execute critical sections in large batches on a single core before passing execution along to other cores, sockets, or nodes, in that hierarchical order. The result is a software DSM system called Argo which localizes as many decisions as possible and allows high parallel performance with little overhead on synchronization when compared to prior DSM implementations.

This paper describes a real-time robotic system. It runs on a heterogeneous multiprocessor system and the system communicates with the robot via RS232 link or infrared link. The multiprocessing system is a IBM personal computer and a i860 add-on card. There are several tasks required for a high performance real-time robotics system: (1) An efficient decision-making task to fulfil the real-time requirement such as supervision, coordination and local control of the robot (2) A real-time graphics task to display the robot's location and its environment (3) A real-time alerting task to alert the system software to perform efficient recovery when the robot is not in its controlling environment (4) Other nonreal-time tasks such as auditing and scheduling (5) Tasks for communication between the robot and the host. In this paper, a multiprocessing distributed system is designed such that the tasks 1 to 4 is handled on the IBM, and the graphics display system is handled by the i860 because of its extremely good graphics performance. The software and hardware design for the system to have a good real-time performance is also discussed. >

The authors present a dynamic token-based mutual exclusion algorithm for distributed systems. In the algorithm, a site invoking mutual exclusion sends token request messages to a set of sites possibly holding the token as opposed to all the sites as in the algorithm proposed by I. Suzuki and T. Kasami (1985). A request set is used to record the identifiers of such a set of sites in the system. The algorithm is dynamic because the request set is dynamically changed as the algorithm is executed. As the size of the request set is reduced, the number of the messages exchanged per critical section (CS) execution is reduced. Compared with Suzuki and Kasami's algorithm, the proposed algorithm can reduce the number of the messages exchanged by 40% when the traffic is light. Finally, the authors discuss relationship between the token-based and non-token-based mutual exclusion algorithms. This relationship motivated the authors to design such a dynamic token-based algorithm. >

Today's oil and gas drilling operations often face significant technical challenges, especially in remote locations with increasingly difficult geological settings. Stuck pipe incidents have become a major operational challenge for the exploration and production industry, with events typically resulting in substantial amounts of lost time and associated costs. Real-time monitoring has emerged as an important tool to achieve drilling optimization in avoiding downtime, particularly stuck pipe incidents. With the addition of a predictive monitoring system, this process becomes much more effective and competent. Predictive monitoring is used for advanced real-time monitoring in Wells Real Time Center (WRTC) and operational workflows to aid in the drilling execution of complex or critical well sections. The emphasis will be on reducing the complexity of real-time data analysis by utilizing trends and deviations between modelled and actual data to monitor wellbore conditions. This monitoring system and trend-based predictive capability enable drilling teams to detect borehole changes and take preventive action up to several hours in advance. By maximizing productive time, it improves operational efficiency. Predictive monitoring can provide early warning of stuck pipe symptoms, allowing the rig and operations team to take corrective and step-by-step actions. In raw drilling data, the conditions that lead to the stuck pipe can be difficult to read and detect. Various factors may indicate potential problems, but these are frequently missed until the situation has progressed to the point where the drill string becomes stuck. This system could have provided the rig crew with advance notice of changes in downhole conditions, in this case, avoiding the stuck pipe situation. We will look into predictive monitoring adoption in Field B operation as an example. Well E is a highly deviated extended reach well (ERD), with a 12,000ft long horizontal section, exceptionally challenging in terms of geomechanics perspective as well as the well design. When original Well E was drilled, a stuck pipe was encountered which caused the wellbore to be sidetracked. Predictive monitoring was implemented to assist drilling operation for the sidetracked well, and it had been completed successfully with minor hole condition issues. The predictive monitoring system is built around a trio of tightly coupled real-time dynamic models consisting of hydraulic, mechanical, and thermodynamic that simulate the wellbore state and physical processes during drilling operations. These models work together continuously to assess drilling performance, borehole conditions, and any other associated risks. It uses dynamic modelling to accurately model key drilling parameters and variables such as hook load, surface torque, cuttings transport, tank volumes, standpipe pressure, and equivalent circulating density (ECD) in real-time.

One notable advantage of Model-Driven Architecture (MDA) method is that software developers could do sufficient analysis and tests on software models in the design phase, which helps construct high confidence on the expected software behaviors and performance, especially for safety-critical real-time software. Most existing literature of reliability analysis ignores the effects from those deadline requirements of tasks which are critical properties for real-time software and thus cannot be ignored. Considering the contradictory relationship between the deadline requirements and time costs of fault tolerance in real-time tasks, in this paper, we present a novel reliability model, which takes schedulability as one of the major factors affecting the reliability, to analyze reliability of the task execution model in real-time software design phase. The tasks in this reliability model has no restrictions on their distributions and thus could be distributed on a multiprocessor or on a distributed system. Furthermore, the tasks also define arrival rates of faults and fault-tolerant mechanisms to model the occurrences of non-permanent faults and the corresponding time costs of fault handling. By analyzing the probability of tasks still being schedulable in the worst-case execution scenario with faults occurring, reliability and schedulability are combined into an unified analysis framework, and two algorithms for reliability analysis are given. To make this reliability model more pragmatic, we also present an estimation technique for estimating the fault arrival rate of each task. We show through two case studies respectively the detailed derivation process under static-priority scheduling in a multiprocessor system and in the design process of avionics software, and then analyze the factors affecting the reliability analysis by setting up simulation experiments. When no assumptions of fault occurrences made on the task model, this reliability model regresses to a generic schedulability model.

A note on Raymond's tree based algorithm for distributed mutual exclusion

Different task models have been proposed to represent the parallel structure of real-time tasks executing on manycore platforms: fork/join, synchronous parallel, DAG-based, etc. Despite different schedulability tests and resource augmentation bounds are available for these task systems, we experience difficulties in applying such results to real application scenarios, where the execution flow of parallel tasks is characterized by multiple (and nested) conditional structures. When a conditional branch drives the number and size of sub-jobs to spawn, it is hard to decide which execution path to select for modeling the worst-case scenario. To circumvent this problem, we integrate control flow information in the task model, considering conditional parallel tasks (cp-tasks) represented by DAGs composed of both precedence and conditional edges. For this task model, we identify meaningful parameters that characterize the schedulability of the system, and derive efficient algorithms to compute them. A response time analysis based on these parameters is then presented for different scheduling policies. A set of simulations shows that the proposed approach allows efficiently checking the schedulability of the addressed systems, and that it significantly tightens the schedulability analysis of non-conditional (e.g., Classic DAG) tasks over existing approaches.

Distributed systems have experienced a shift of scale in the past few years. This evolution has generated an interest in peer-to-peer systems and resulted in much interesting work. Peer-to-peer systems are characterized by their potential to scale due to their fully decentralized nature. They are self-organizing, adapting automatically to peer arrivals and departures, and are highly resilient to failures. They rely on a symmetric communication model where peers act both as servers and clients. As the peer-to-peer concepts and technologies become more mature, many distributed services and applications relying on this model are envisaged in the context of large-scale distributed and parallel systems. This topic examines peer-to-peer technologies, applications, and systems, and also identifies key research issues and challenges. Twenty-six papers were submitted to the track and we accepted six. We organized two sessions, the first devoted to the problem of query management in structured and unstructured overlay networks, the second containing a broader selection of topics.

In distributed systems,cooperating process essh are both local and remoter sources. Chance is very high that multiple processes makesimultaneous requests to the same resource. If the resourcerequires mutually exclusive access (critical section - CS), then some regulation is needed to access it for ensuring synchronizedaccess of the resources thatonly one process could use the resource at a given time. This is the distributed mutual exclusion problem. The problem of coordinating the execution of critical sections by eachprocessissolved by providing mutually exclusive access to the CS. Mutual exclusion ensures that concurrent processes make a serialized access to shared resources.Quorum-based algorithms offer the advantage of protocol symmetry, spreading effort and responsibility uniformly across the distributed systems. In thispaper, we have proposed a permission baseddistributedmutual exclusion algorithmwhichis an improvement of Maekawa’s algorithm. The number of messages required by the improvisedalgorithmis in the range 3Mto 5Mper critical section invocation whereMis the number of intersection nodes1 in the system. A reduction in number of message by restricting the communication of anynodewith the intersection nodes of the quorums, withoutany modification of the basic structure of the algorithm.

In this paper, we propose Elastic-RAID, a new RAID architecture to achieve high performance and high reliability for large-scale distributed and parallel storage systems. The key idea behind Elastic-RAID is to smartly utilize the free space existing in parity-based disk arrays to store additional mirroring data. This additional mirroring data redundancy, when strategically and judiciously activated and exploited in a RAID system, enables improved system I/O performance, fault tolerance and recovery. Depending on the amount of free space available and whether the emphasis is on performance or reliability, the elasticity in Elastic-RAID is manifested in how each design objective is achieved. For the performance objective, Elastic-RAID improves small-write performance by writing original and mirroring data synchronously and leaving the costly parity update in the background at a later idle/lightly-loaded time. For the reliability objective, at least two concurrent disk failures can be tolerated when Elastic-RAID is employed in a RAID5 system that has 50 percent or more free space. Higher reliability is provided for important data when free space is less than 50 percent. To achieve the design goal of elasticity, we introduce a novel data layout and addressing scheme. Our extensive trace-driven evaluations on an Elastic-RAID prototype in the typical configurations of RAID5 show that Elastic-RAID boosts the small-write performance in the normal operational state by at least 40 percent, improves the user I/O performance in the reconstruction state by at least 30 percent and shortens the recovery time by at least 40 percent.

In real-time operating systems, timer interrupts are usually used for indicating when a real-time task should be started. Critical sections with interrupts disabled can, however, cause an unacceptable delay in the execution of these tasks. Existing approaches for overcoming this issue either require modifications to many places in the kernel source or introduce a new structural design. In this paper, we propose a simple interrupt management technique for reducing the latency of timer interrupts. The locking mechanism and the timer interrupt handler have been modified so that timer interrupts are not disabled during critical sections, allowing the execution of emergency jobs requested by real-time tasks. We implemented a prototype system in Linux 2.6.20. Experimental results showed that the timer interrupt latency is reduced to 2.2\% of the original Linux kernel and the latency time for executing an emergency real-time job is 27.9$\mu$s in the worst case. Comparing to existing real-time systems, the results showed that proposed technique is efficient enough for a future real-time operating system.