Using artificial life techniques for distributed grid job scheduling

Dear SCPE Reader, We present a selection of papers which are extensions of papers presented at the 7-th International Symposium on Parallel and Distributed Computing, 1–5 July 2008, in Krakow, Poland. The motivation for publishing the selection in the SCPE Journal was, on the one hand, to present the flavour of the research reported at the conference and on the other hand to present some of the most relevant topics currently focused on the research on parallel and distributed computing in general. The selection contains only 6 papers out of about 60 presented at the conference, and thus, is far from covering all relevant topics represented at the ISPDC 2008. This is because not all of the invited authors were patient enough to accept a fairly long paper publishing process. Nevertheless, we hope that the presented papers will bring you closer to the research covered by the ISPDC conferences and will encourage you to participate in future ISPDC editions. The first paper ``The Impact of Workload Variability on Load Balancing Algorithms'' is by Marta Beltran and Antonio Guzman from King Juan Carlos University in Spain. It concerns an important topic of load balancing in cluster systems, namely adaptativity of the load balancing algorithms to changes of the workload in the system. Adequate accounting for additional load in the hosting system is of great relevance for correct optimization effects. The paper presents a thorough formal analysis of the workload variability metrics and their influence on the quality of load balancing algorithms. Four basic activities appearing in load balancing algorithms are identified, and based on them some algorithmic solutions are proposed to correctly deal with workload variability in system load balancing. The problem of dynamic load balancing algorithms robustness has been discussed. Two different robustness metrics sensitive to the applied type of opimization: local task-oriented or a global one enable selecting task remote execution or migration as load balancing operations. The proposed approach is illustrated with experiments. The second paper ``Model-Driven Engineering and Formal Validation of High-Performance Embedded Systems'' is by Abdoulaye Gamatie, Eric Rutten, Huafeng Yu, Pierre Boulet, Jean-Luc Dekeyser, from University of Lille and INRIA in France. The paper is concerned with a very advanced methodology of designing correct parallel embedded systems for intensive data-parallel computing. In their previous research, the authors of the paper designed the GASPARD embedded system design framework. It is based on the hardware/software co-design approach through model-driven engineering. The framework is based on an UML-like model specification language in which hardware and software elements are modelled using a component approach with special mechanisms for repetitive structures. This paper tries to combine the modelling framework of GASPARD with the mechanisms of synchronous languages to achieve design verifiability provided for such languages. The paper shows how GASPARD models can be translated into synchronous models based on data flow equations in order to formally check their correctness. The proposed approach is illustrated with an example of a video processing system. The third paper ``Relations Between Several Parallel Computational Models'' is by Stephan Bruda and Yuanqiao Zhang from Bishop’s University in Canada. The paper is concerned with theoretical aspects of shared memory systems described by the parallel random access machine PRAM model and aims in studying performance properties of different types of PRAM systems. The attention is focussed on analysing the computational power of two more sophisticated PRAM models (Combining CRCW and Broadcast Selective Reduction), which include data reduction in case of concurrent writes. The paper shows that these two models have equivalent computational power, which is a new result comparing the existing literature. The performance of both models applied to reconfigurable multiple bus machines was studied as a possible architectural solution for current VLSI processor implementations. It was shown that in such systems under reasonable assumptions concurrent-write does not enhance performance comparing the exclusive-write model. Another result important for the VLSI technology is that the Combining CRCW PRAM model (in which data of concurrent writes are arthmetically or logically combined before write) and the exclusive-write on directed reconfigurable busses perform in equivalent way under strong real-time requirements. The fourth paper ``Experiences with Mesh-Like Computations Using Prediction Binary Trees'' is by Gennaro Cordasco, Biagio Cosenza, Rosario de Chiara, Ugo Erra and Vittorio Scarano from the University ``degli Studi'' of Salerno and the University ``degli Studi della Biasilicata'' of Potenza in Italy. The paper concerns optimization methods for mesh-like computations in clusters of processors. The computations are perfomed assuming a phase-like program execution control using a tiling approach which reduces inter-processor communication. A temporal coherence is also assumed, which means that task sizes provide similar execution times in consecutive phases. Temporary coherent computations are structured in a Prediction Binary Tree, in which leaves represent computing tiles to be mapped to processors. A phase-by-phase semi-static load balancing is introduced to the scheduling algorithm. The scheduling algorithm is equipped with a predictor, which estimates the computation time of next phase tiles based on previous execution times and modifies the tiles to achieve balanced execution in phases. For this, two heuristics are used to leverage on data locality in processors. The proposed approach is illustrated by the example of interactive rendering with Parallel Ray Tracing algorithm. The fifth paper ``The Influence of the IBM pSeries Servers Virtualization Mechanism on Dynamic Resource Allocation in AIX 5L'' is by Maciej Mlynski from ASpartner Limited in Poland. The paper concerns a very up-to-date problem of system virtualization and presents the results of research carried on IBM pSeries servers. IBM is strongly developing the virtualization technique especially on IBM pSeries servers enabling an improved and flexible sharing of system resources between applications. The paper investigates novel facilities for dynamic resource management such as micro-partitioning and partition load manager. They enable dynamic creation of workload logical partitions of system resources and their dynamic mangement. It includes run-time resource re-alocation between logical partitions including setting of sharing specifications as well as run-time adding/removing/setting parameters of resources in the system. It remains an open question how to properly tune parameters of the operating system using the provided virtualization facilities to obtain the best efficiency for a given application program. The paper presents the results of experiments which study the effects of tuning the disk subsystem parameters under the IBM AIX 5L operating system with the use of the provided virtualization facilities on the resulting application execution performance. The results show that even small deterioration in the resource pool status requires an immediate adaptation of the operating system parameters to maintain the required performance. The sixth paper ``HeteroPBLAS: A Set of Parallel Basic Linear Algebra Subprograms Optimized for Heterogeneous Computational Clusters'' is by Ravi Reddy, Alexey Lastovetsky and Pedro Alonso from University College Dublin in Ireland and Polytechnic University of Valencia in Spain. The paper concerns the methodology for parallelization of linear algebra computations for execution in heterogeneous cluster environments. The design of the HeteroPBLAS library (Parallel Basic Linear Algebra Subprograms) for heterogeneous computational clusters is presented. The main contribution of the paper is the automation of the parallelization and optimization of the PBLAS, which is done by means of a special user interface and the underlying set of functions. An important element is here a performance model that is based on program code instrumentation, which determines parameters of the application and the executive heterogeneous platform relevant for execution performance of parallel code. The parameter values specified for or returned by execution of the performance model functions are next used for generation and optimal mapping of the parallel code of the library subroutines. The proposed approach is illustrated by experimental results of execution of optimized HeteroPBLAS programs on homogeneous and heterogeneous computing clusters. Marek Tudruj

Load balancing algorithms are designed to achieve efficient resource utilisation in large-scale distributed systems. The primary goal of designing load balancing algorithms is to balance the overall workload among all the nodes in distributed systems to improve the performance. This paper presents an in-depth study and detailed comparative analysis of different load monitoring and balancing algorithms employing fuzzy logic and mobile agents. A detailed qualitative as well as quantitative analysis of algorithmic performances is included. The analytical survey presented in this paper would enable understanding of applications of different algorithms under different computing environments. Furthermore, this paper proposes a hybrid architecture combining fuzzy logic and mobile agents for load balancing and monitoring in distributed systems.

Nginx is a commonly used and free open-source web server that is used as a reverse proxy server, load balancer and HTTP cache. It consumes less memory and can handle more clients with less number of processes. Nginx provides users with five predefined load balancing algorithms. However, most of these algorithms are static and some of the load balancing rules are inefficient. In order to make the load of a cluster more stable under high concurrent requests, we developed a Dynamic Load Balancing (DLB) algorithm that uses Nginx as a network security control panel to provide load balancing for a cluster of backend servers. The DLB algorithm is based on the weighted round robin module of Nginx, Logistic Regression and Maximum Likelihood Estimation (MLE) algorithm. It handles the situation of high concurrent requests and reduces the probability of omitted or under-reported incident and status. We also propose a Hybrid Load Balance Method (HLBM) that incorporates the DLB algorithm and evolutionary computing to further improve the performance. We have conducted limited experiment by using dynamic load balancing algorithm. We will complete to develop the Hybrid Load Balance method and conduct experiments for HLBM.

This article is about the ways of load balancing for server stations. The article contains most popular approaches and algorithms of load balancing in modern applications. The algorithm of round load balancing was described. This algorithm was chosen as the base for simulation model. This model was created by software tools MATLAB/SimulinkiSimEvents/Stateflow. The model takes into account the random nature of requests appearance and the variable time delay between the receiving of tasks. The process of model simulation provides numerical result of load balancing. It is possible to compare the modeling result with a real load balancing system. Finally, in this article the authors provided the idea of updating a round balancing algorithm for a simulation model as the way of future investigation.

Incorporating job sizes in distributed load balancing

Fuzzy systems (FSs) have been extensively applied to automatic control, pattern recognition, and decision analysis. However, a common bottleneck is encountered in the derivation of fuzzy rules, which is often difficult, time consuming, and relies on expert knowledge. To automate the design of FSs, many metaheuristic learning algorithms have been proposed. One major optimization category uses Swarm Intelligence (SI) model (Kennedy et al., 2001). The SI technique studies collective behavior in decentralized systems. Its development was based on mimicking the social behavior of animals or insects in an effort to find the optima in the problem space. SI models are initialized with a population of random solutions. One well-known SI model is particle swarm optimization (PSO) (Kennedy & Eberhart, 1995). Many modified PSO models have been proposed and successfully applied to different optimization problems (Clerc & Kennedy, 2002; Bergh & Engelbrecht, 2004; Ratnaweera et al., 2004; Juang, 2004; Kennedy & Mendes, 2006; Parrott & Li, 2006; Chen & Li, 2007). FS design using PSO has also been proposed in several studies (Juang, 2004; Chatterjee et al., 2005; Juang et al., 2007; Araujo & Coelho, 2008; Sharma et al., 2009). Another well-known SI is ant colony optimization (ACO) (Dorigo & Stutzle, 2004). The ACO technique is inspired by real ant colony observations. It is a multi-agent approach that was originally proposed to solve difficult discrete combinatorial optimization problems, such as the traveling salesman problem (TSP) (Dorigo et al., 1996; Dorigo & Gambardella, 1997). In the original ACO meta-heuristic, artificial ant colonies cooperate to find good solutions for difficult discrete optimization problems. Different ACO models have been applied to FS design problems (Cassillas et al., 2000; Cassillas et al., 2005; Mucientes & Casillas; 2007; Juang & Lo, 2007; Juang et al., 2008; Juang & Lu; 2009). In (Cassillas et al.,2000; Mucientes & Casillas; 2007; Juang et al., 2008; Juang & Lu; 2009), the FS input space was partitioned in grid type with antecedent part parameters of an FS manually assigned in advance. In (Juang & Lo, 2007), the FS input space was flexibly partitioned using a fuzzy clustering-like algorithm in order to reduce the total number of rules. For all of these studies, the consequent part parameters were optimized in discrete space using ACO. Since only the consequent part parameters are optimized, and the optimization space is restricted to be discrete, the designed FSs are unsuitable for problems where high accuracy is a major concern. Source: Fuzzy Systems, Book edited by: Ahmad Taher Azar, ISBN 978-953-7619-92-3, pp. 216, February 2010, INTECH, Croatia, downloaded from SCIYO.COM

Workload information management and resource management are two key aspects in grid computing to provide the better services to grid environment users. Grid computing also faced other challenging areas like heterogeneous nature of resources, huge number of computing elements, independency of computing resources, different processing capacities of the nodes, different types of load conditions, overloading of jobs at computing nodes, underutilized of resources, etc. A distributed load balancing algorithm is proposed that can handle any kind of grid structure. The proposed algorithm is divided into two steps: To reduce the time taken for executing the jobs or we can say reduce the response time and communication cost between transferring the jobs from one computing node to another in a grid architecture. In proposed algorithm it is shown that the load will be primarily balanced at local level to reduce the communication cost.

An algorithm for load balancing in multiprocessor systems

Packet switching remains one of the bottlenecks in building fast Internet routers. Load balancing and switch scheduling are two important algorithms in the effort to maximize the throughput and minimize the latency of these packet switches. A load balancing algorithm regulates the traffic to conform to the service rates while a switch scheduling algorithm allocates the service rates adaptive to the arrival patterns. Many existing load balancing and switch scheduling algorithms are very similar. We show that load balancing and switch scheduling systems are dual systems based on the linear queue dynamics approximation. This allows us to cast a load balancing problem as a scheduling problem, and vice versa. We further show an example of designing a new algorithm for load balancing using an existing scheduling algorithm based on the duality. The duality perspective also allows us to solve unknown problems. We find the entropy rate of the randomized bandwidth allocation system with linear queue dynamics based on the knowledge of the entropy rate of the randomized load balancing system. For the general case, we find both an upper and a lower bound on the entropy rate. The joint use of dual load balancing and switch scheduling algorithms leads to performance gains as we show using mean field analysis.

Past approaches to routing in mesh networks either (i) do not account for the MAC-layer interactions between the links in a tractable manner, or (ii) are agnostic to load-balancing across gateways. Our answer to these problems is MaLB (MAC-aware and Load Balanced routing algorithm), a greedy, tractable, and distributed mesh routing algorithm. Since the underlying objective function has high combinatorial complexity, MaLB uses a greedy approach. MaLB finds an optimum routing forest (union of trees rooted at the gateways) by taking into account MAC-layer interaction between links, as well as optimum multi-hop association of mesh nodes to gateways. MaLB builds on top of ETP (Expected Through-Put), a recently proposed MAC-aware routing metric. We also propose a low complexity variant of MaLB called LB (Load Balanced routing algorithm) which performs load balancing in a MAC-agnostic manner. MaLB performs especially well in networks with skewed topologies that result from unplanned mesh network deployment, as well as in the presence of gateway failures. Simulations with an enhanced version of ns-2 show that MaLB results in up to 60% higher throughput than a shortest path algorithm with ETX (Expected Transmission Count). Furthermore, MaLB results in up to 30% improvement over the LB algorithm, as well as a shortest path algorithm with ETT (Expected Transmission Time).

Cloud computing is increasing rapidly as a successful paradigm presenting on-demand infrastructure, platform, and software services to clients. Load balancing is one of the important issues in cloud computing to distribute the dynamic workload equally among all the nodes to avoid the status that some nodes are overloaded while others are underloaded. Many algorithms have been suggested to perform this task. Recently, worldview is turning into a new paradigm for optimization search by applying the osmosis theory from chemistry science to form osmotic computing. Osmotic computing is aimed to achieve balance in highly distributed environments. The main goal of this paper is to propose a hybrid metaheuristics technique which combines the osmotic behavior with bio-inspired load balancing algorithms. The osmotic behavior enables the automatic deployment of virtual machines (VMs) that are migrated through cloud infrastructures. Since the hybrid artificial bee colony and ant colony optimization proved its efficiency in the dynamic environment in cloud computing, the paper then exploits the advantages of these bio-inspired algorithms to form an osmotic hybrid artificial bee and ant colony (OH_BAC) optimization load balancing algorithm. It overcomes the drawbacks of the existing bio-inspired algorithms in achieving load balancing between physical machines. The simulation results show that OH_BAC decreases energy consumption, the number of VMs migrations and the number of shutdown hosts compared to existing algorithms. In addition, it enhances the quality of services (QoSs) which is measured by service level agreement violation (SLAV) and performance degradation due to migrations (PDMs).

The exponentially extensive and emerging expansion of the cloud computing network (CCN) and its prolific application result in an ample number of users/clients. The frugal superintend of the client’s requests for resources is troublesome due to a precise number of resources and to actualize the engrossment of Quality of Service (QOS). In aisle to achieve QOS, it is mandatory to maximize the efficiency and reliability of CNN resources. The zero-ground affair is to find an efficient load balancing (LB) algorithm (LBA) that obliged to parcel out data traffic amidst multitudinous resources. Here, in this paper, a multitudinous proposed scheme for QOS has been analyzed.

This article introduces a new load balancing algorithm, called diffusive load balancing, and compares its performance with three other load balancing algorithms: static, round robin, and shortest queue load balancing. The comparison of load balancing algorithms is made in three simulated client-server environments: a small-scale, intranet, and Internet environment. Experimental results of performance modeling show that diffusive load balancing is better than round robin and static load balancing in a dynamic environment, which manifest in frequent clients' object creation requests and in short objects' lifetimes. In this research, the diffusive load balancing algorithm is discussed in juxtaposition with the distributed client-server architectures.

We specify a load balancing algorithm using action systems with procedures as described in Chapter 5. A process network is considered to be associated with an action system assigning each variable to a process. Messages are passed between the processes by explicit communication. In a distributed action system each action and procedure is local to some process referring only to variables of that process.

We give a general model of partially asynchronous, distributed load-balancing algorithms for the discrete load model in parallel computers, where the processor loads are treated as non-negative integers. We prove that all load-balancing algorithms in this model are finite. This means that all load-balancing algorithms based on this model are guaranteed to reach a stable situation at a certain time (which depends on the particular algorithm) at which no load will be sent from one processor to another. With an additional assumption, we prove that the largest load difference between any two processors, in the final stable situation of the load-balancing algorithms in this model, is upper-bounded by the diameter of the topology.