Adaptive fair resource management with an arbiter for multi-tier computing systems

Cronus [1-7] is an object oriented distributed system which operates in a heterogeneous computer environment of interconnected local area networks. As a distributed system architecture. Cronus faces a number of resource management issues not present in non-distributed architectures. Strategies for effectively controlling the redundancy and reconfigurability inherent in Cronus are needed to take advantage of the distributed system environment. These strategies for resource management are often conveniently separated into policies and mechanisms. A policy is a goal or guideline set by a system administrator or component designer constraining the decisions made by a resource allocator. A mechanism is an internal system structure designed to implement a class of policies.In the Cronus distributed system model, there are currently two general aspects of resource allocation mechanisms which are particular to the network environment and must be effectively managed. One of these is the binding of a client request to a particular resource manager for those resources which are available redundantly. Redundancy comes in two forms: replicated objects, (e.g., a multi-copy file) and replicated managers (any manager for a type can create a new instance of that object type). For both forms the selection of an object manager to provide the given service is an important resource management policy decision. The other important aspect of resource management mechanism is the ability to reconfigure parts of the system by dynamically migrating objects. Object migration is a powerful tool for matching system resources to tasks in a manner that attempts to maximize some measure of system performance, reliability, or survivability. Both static reconfiguration (e.g., choosing or amending the placement of object managers and their associated objects), and dynamic reconfiguration (e.g., moving an individual object in direct response to demand for its use possible in the Cronus architecture and design.The general approach to resource management in Cronus is to individually control the management of the classes of objects which make up the system. This approach extends Cronus resource management concepts beyond system resources to the abstract resources developed by applications. Resource management for an individual abstract resource (type) and for groups of related resource types (services) in Cronus is based on combining a number of carefully planned mechanisms which are part of the system architecture. In addition to resource management by resource type, a client specific policy which considers collections of object types used in a specific context can be constructed.In Cronus we achieve system wide and easily controllable resource management by requiring the object managers to cooperate in enforcing a resource management policy for their resource type. An object manager can redirect operations to a peer manager on another processor on the basis of current resource status. In the case for creation of a file to an alternate file manager which may have more storage available. Part of the basis for decisions to redirect requests are parameters, settable dynamically by system administrators through monitoring and control functions, which control the resource management strategy. The creation of objects and resource management in general thus becomes a responsibility that is decentralized among object managers on each processor based on a global allocation policy, both in terms of sharing current status and possible redirection of operations between managers.The Cronus resource management model is based on the integration of the following set of primitive mechanisms: the kernel routes requests for a given type to any available manager of that type, using the Locate mechanismCronus managers redirect requests to a more appropriate peer to accomplish resource management objectivesmanagers periodically collect current status of their peers, via a standard mechanism (ReportStatus), to be used as a basis for selecting a site for redirecting an operationusers and applications optionally indicate specific location preferences with requestsan operator can monitor and regulate the functioning of the resource management policyapplications can use the basic system mechanisms (e.g. broadcast or multicast, ReportStatus operations) to build special-purpose resource management strategies tailored to their needThere is a hierarchy of desirable locations for implementing resource management policies: object managers, shared libraries, and finally client programs or users themselves. Managers are the most desirable because they are readily identifiable with a limited set of object types which are likely to have similar policy goals. Also, managers are addressable as a group for administrative control to regulate the policy parameters, and for peer manager cooperation to implement the policy. Many decisions are negotiated between managers based upon information periodically obtained via the ReportStatus operation. The Cronus Monitoring and Control System uses the same ReportStatus operation to present the operator with a global view of how well the resource management policy is proceeding.In this paper we elaborate on these approaches to distributed resource management in Cronus, including appropriate background, detailed system design, and experiences with incorporating these ideas into three different resource managers.

Peer-to-peer (P2P) network is a high level logical network architecture build over end-user nodes interconnected by a physical network infrastructure. The main interest of their distributed structure is to avoid any centralized point. The characteristics of P2P networks make them a better choice for multimedia content sharing/streaming over IP networks. This kind of applications demands a QoS (quality of service) level. However, different customers are believed to have different QoS perception levels for the same application. Some customers may content to pay less in exchange for a lower QoS. On the other hand, in order to provide users the required QoS level, an incentive mechanism to motivate peers to contribute is required. In this paper we focus the problem of charging for real-time media streaming of IP packet over P2P networks. We consider a model of multiple senders to a single receiver. We define an incentive mechanism that allows peers generating revenues for their contributions. The incentive mechanism is based on competitive market model. Therefore, a charging model based on the QoS level is defined. Finally, the proposed model defines a sender peers selection mechanism based on the competitive market model and the user's QoS satisfaction level. (8 pages)

Managers of natural resources are faced with increasingly complex issues involving resource management concerns and the emerging importance of the socio-cultural context of resource use. The resource manager has been forced to respond and react to new social, cultural, ecological, and economic realities. Educational institutions have often been slow to respond to the changing work environment, even though a growing number of practitioners and educators believe that traditional approaches to training resource managers are inadequate and increasingly, ineffectual. This paper outlines the development of a new curriculum attempting to respond to the new challenges faced by contemporary resource managers. The concepts of interdisciplinary and cross-disciplinary study are described, and the need for a new educational approach is argued. Finally, one institution's response to the changing needs and skills of resource managers is reviewed.

While the differentiated services (DiftServ) infrastructure is scalable and robust in providing network quality of service (QoS), there are serious drawbacks with the services provided by DiffServ: (1) the services are coarse-grained and one-way only; (2) no service differentiation and resource isolation are provided to meta-data packets such as TCP SYN and ACK packets. Moreover the coarse-grained service differentiation and the lack of resource isolation at IP routers exposes its vulnerability to distributed denial of service (DDoS) attacks. Based on the concept of layer-4 service differentiation and resource isolation, where the transport-layer information is inferred from the IP headers and used for packet classification and resource management, we present a scalable fine-grained DiffServ (sf-DiffServ) architecture that provides fine-grained service differentiation and resource isolation among thinner behavior aggregates (BAs). The sf-DiffServ architecture consists of a fine-grained QoS classifier and an adaptive weight-based resource manager at IP routers. A two-stage packet classification mechanism is devised to decouple the fine-grained QoS lookup from the routing lookup at core routers. Due to its scalable QoS support for TCP control segments, sf-DiffServ supports bi-directional differentiated services for TCP sessions. Most importantly, the fine-grained resource isolation provided inside the sf-DiffServ is a powerful built-in protection mechanism to counter DDoS attacks, reducing the vulnerability of the Internet to DDoS attacks.

Real-time and embedded systems have traditionally been designed for closed environments where operating conditions, input workloads, and resource availability are known a priori and are subject to little or no change at runtime. There is an increasing demand, however, for autonomous capabilities in open distributed real-time and embedded (DRE) systems that execute in environments where input workload and resource availability cannot be accurately characterized a priori. These systems can benefit from autonomic computing capabilities, such as self-(re)configuration and self-optimization, that enable autonomous adaptation under varying-even unpredictable-operational conditions. A challenging problem faced by researchers and developers in enabling autonomic computing capabilities to open DRE systems involves devising adaptive planning and resource management strategies that can meet mission objectives and end-to-end quality of service (QoS) requirements of applications. To address this challenge, this paper presents the integrated planning, allocation, and control (IPAC) framework, which provides decision-theoretic planning, dynamic resource allocation, and runtime system control to provide coordinated system adaptation and enable the autonomous operation of open DRE systems. This paper presents two contributions to research on autonomic computing for open DRE systems. First, we describe the design of IPAC and show how IPAC resolves the challenges associated with the autonomous operation of a representative open DRE system case study. Second, we empirically evaluate the planning and adaptive resource management capabilities of IPAC in the context of our case study. Our experimental results demonstrate that IPAC enables the autonomous operation of open DRE systems by performing adaptive planning and management of system resources.

Resource management strategy in adaptive cloud provisions the needed resources dynamically to the end-users. To improve the runtime performance of adaptive cloud for service-based applications, two aspects of technical issues are required to be addressed. The first one is the balancing of a large amount of data on existing resources and the second is resource provisioning which can adjust the number of resources optimally to adapt the time-varying workload. As the growth of data is increasing tremendously, efficient resource management is the need in cloud computing. We build a cloud framework to process data in automation with adaptive resource and workload management strategy. Numbers of approaches are reviewed and applied for workload prediction. We developed the model Auto-Regressive Integrated Moving Average-workload Prediction for Efficient Resource Provisioning (ARIMA-PERP) and evaluated the results that can satisfy the on-demand need of end-users for efficient resource utilization. To serve the maximum number of user requests, performance metrics of the proposed approach are evaluated. It is observed that our evaluated results achieved an accurate prediction by 91.11%, which meets the efficient resource utilization for the demanded workload. As compared with the exiting approach, we achieved better performance by 0.11% for accurate prediction. The proposed architecture is intended to provide the resources dynamically and efficiently satisfying the demands of the user. To achieve this objective of efficient resource provisioning, algorithms are developed for workload prediction which helps in deciding optimum resource provisioning. Our system uses proactive approach resource management and deployment of the adaptive cloud system. In traditional systems, resources are managed based on demand, availability and the strategy of scheduling which results in delayed response time at large. We configured the ARIMA model to predict the future workload for provisioning the resources dynamically and remove the problem of over-provisioning and under-provisioning in a cloud environment.

This paper describes ongoing investigations at Microsoft Research into algorithms for adaptive real-time scheduling and resource management. We are implementing a system which will allow sets of independently authored real-time applications to be scheduled, providing both initial resource negotiation and dynamic feedback to applications about their resource consumption. Instead of relying on human beings to pre-compute scheduling and resource requirements, applications will be able to dynamically adapt their run-time behavior to the resources available. Under overload conditions, user preferences and policies will be applied in order to bring the total resource requirements within available resources. This will permit chosen applications to continue executing perfectly even while others execute with dynamically renegotiated resource grants or are gracefully shut down, as per user preferences. We believe that this approach will lead to real-time scheduling and resource management algorithms which are appropriate for workstation and home multi-media environments.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Pricing plays an important role to efficiently manage the cellular network resources, maintain user required QoS (Quality of Service) levels and to optimize the operator's revenue. In this paper, we analyze some previous pricing schemes proposed for general wireless multimedia systems and for WiMAX and adapt these methods within LTE (Long Term Evolution) networks. We also propose a dynamic pricing strategy for LTE. Most of these methods (including ours) are based on users' price categories: Gold (or high), Silver (or medium) and Bronze (or low). The price charged to these users may also vary with the QoS level. We compare these methods considering the LTE network context. Our proposed strategy provides high revenue for the operator as well as a smaller average price paid for the user, with regard to the flat-rate pricing strategy.

The work described in this paper began with a desire to include some linguistic concept of a resource manager within a dataflow language we have been designing [AGP76]. In doing so, we discovered that dataflow monitors (resource managers) provide a natural way of thinking about resources and especially their scheduling. Dataflow semantics are based upon a program composed of asynchronous operators interconnected by lines along which data tokens (messages) flow, such that when all of the input tokens for a given operator have arrived then that operator may fire (execute) by absorbing the input tokens, computing, and producing an output token as its result. These operations closely match one's intuitive model of resource managers (operators) passing signals (tokens) to one another for the purpose of synchronizing and scheduling resource usage. Previously though, dataflow languages [D73, K73, W75] have dealt only with the expression of highly asynchronous yet determinate computations; however, resource management characteristically involves indeterminate computation. The introduction here of dataflow monitors and an explicit nondeterministic merge operator for dataflow streams makes dataflow very well suited for expressing interprocess communication and operations on resources.

Support for intelligent, autonomous, adaptive and distributed resource management is a key to the success of scalable and dynamic wireless sensor network applications. Distributed independent reinforcement learning (DIRL) is a micro-learning framework that enables distributed, adaptive resource management using only local information at individual sensor nodes. In this paper we propose COllective INtelligence (COIN) a macro-learning paradigm that aims to specifically address the problem of designing utility functions for individual agents in order to achieve higher system wide utility. We extend DIRL, by combining it with COIN based macro-learning paradigm to steer the system towards global optimization, improving performance with minimal communication overheads. We present results of simulation to compare our approach with other existing approaches such as team game and DIRL in an example object tracking application. Simulation results demonstrate that a combination of micro- and macro-learners is two times more energy-efficient than micro-learners (DIRL) alone and four times more energy-efficient than macro-learners (TEAM game) alone.

In real-time embedded systems, there is requirement for adapting both energy consumption and Quality of Services (QoS) of tasks according to their importance. This paper proposes an adaptive power-aware resource allocation method to resolve a trade-off between the energy consumption and QoS levels according to their importance with guaranteeing fairness. The proposed resource allocator consists of two components: the total resource optimizer to search for the optimal total resource and QoS-fairness-based allocator to allocate resource to tasks guaranteeing the fairness. These components adaptively achieve the optimal resource allocation formulated by a nonlinear optimization problem with the time complexity O(n) for the number of tasks n even if tasks' characteristics cannot be identified precisely. The simulation result shows that the rapidness of the convergence of the resource allocation to the optimal one is suitable for real-time systems with large number of tasks.

Recently, the evaluation on quality of the air interface in a wireless network has shifted from emphasis on QoS (quality of service) only to incorporation of QoE (quality of experience). However, it seems that QoE metrics have not been related to the surroundings of the served users so far, which may have a great influence on users' expected QoE. For instance, with a high level of QoS, a user's expectation of QoE could still be terrible because of the serious interference from the surroundings such as noise, ambient light or shaking of terminals due to mobility. From another perspective, it means that in such a case a high level of QoS may be helpless to enhance users' QoE and the requirement could be relaxed accordingly. In this paper, an approach called surrounding-adaptive QoE adjustment (SAQA) is proposed to optimize the capacity and the quality of air interface with voice service as an example. The basic idea of the approach is that user equipment (UE) detects the surroundings interference which may impact users' QoE and then adjusts QoS according to the detected results: a lower level of QoS is chosen in a bad environment to enhance the capacity, while a higher level of QoS is selected to guarantee the users' QoE in a better circumstance. Simulation results show that this method can remarkably enhance the voice capacity of the communication systems as well as guarantee users' QoE.

We consider a network design problem, where applications require various levels of quality-of-service (QoS) while connections have limited performance. Suppose that a source needs to send a message to a heterogeneous set of receivers. The objective is to design a low cost multicast tree from the source that would provide the QoS levels (e.g., bandwidth) requested by the receivers. We assume that the QoS level required on a link is the maximum among the QoS levels of the receivers that are connected to the source through the link. In accordance, we define the cost of a link to be a function of the QoS level that it provides. This definition of cost makes this optimization problem more general than the classical Steiner tree problem. We consider several variants of this problem all of which are proved to be NP-hard. For the variant where QoS levels of a link can vary arbitrarily and the cost function is linear in its QoS level, we give a heuristic that achieves a multicast tree with cost at most a constant times the cost of an optimal multicast tree. The constant depends on the best constant approximation ratio of the classical Steiner tree problem. For the more general variant, where each link has a given QoS level and cost we present a heuristic that generates a multicast tree with cost O(min{logr,k}) times the cost of an optimal tree, where r denotes the number of receivers, and k denotes the number of different levels of QoS required. We generalize this result to hold for the case of many multicast groups.

Chapter 8 - Two-Phase Commit

Purpose of ReviewTheoretical frameworks for adaptive natural resource management are quite common, whereas documented examples showing successful implementation of adaptive management and learning through multiple time intervals have remained uncommon. Measures of quality of adaptive natural resource management processes are needed to examine potential factors driving the successful implementation. To address this gap, we developed a multimetric index composed of 22 metrics to assess quality of case studies using quantitative decision support (QDS) to inform adaptive forest management (AFM). Metrics represented three main tasks, including conceptual setup, modeling, and application. We further distinguished these into subtasks: definition of objectives and management options (setup); specifying uncertainty, prediction, and optimization (modeling); and stakeholder involvement along with practice and learning (application). We used a multimetric index to examine temporal and geographic variation in quality of reviewed case studies using QDS to inform AFM. We then conducted a structured literature review of 179 articles, wherein 34 case studies met a priori criteria.Recent FindingsWhen applying the multimetric index to these case studies, we found that over the past decade the index has been intermediate and annual average scores declined by 33% from 4.5 to 3.0 of 10 (where 10 is the highest possible quality score). Aligning with reviews of adaptive natural resource management, reported on-ground application of QDS to inform AFM was rare (n = 2). We also confirmed the expectation that there has been a substantial lack of stakeholder engagement during QDS development tasks.SummaryOur multimetric index provides a novel tool to examine gaps in the use of QDS for adaptive management in diverse domains including but not limited to forests.