Dynamic load balancing with enhanced shared-memory parallelism for particle-in-cell codes

In recent years, cloud computing provides a spectacular platform for numerous users with persistent and alternative varying requirements. In the cloud environment, security and service availability are the two most significant factors during the data encryption process. For providing optimal service availability, it is necessary to establish a load balancing technique that is capable of balancing the request from diverse nodes present in the cloud. This paper aims in establishing a dynamic load balancing technique using the APMG approach. Here in this paper, we integrated adaptive neuro-fuzzy interference system-polynomial neural network as well as memory-based grey wolf optimization algorithm for optimal load balancing. The memory-based grey wolf optimization algorithm is employed to enhance the precision of ANFIS-PNN and to maximize the locations of the membership functions respectively. Also, two significant factors namely the turnaround time and CPU utilization involved in optimal load balancing scheme are evaluated. Finally, the performance evaluation of the proposed MG-ANFIS based dynamic load balancing approach is compared with various other load balancing approaches to determine the system performances.

Nginx is a commonly used and free open-source web server that is used as a reverse proxy server, load balancer and HTTP cache. It consumes less memory and can handle more clients with less number of processes. Nginx provides users with five predefined load balancing algorithms. However, most of these algorithms are static and some of the load balancing rules are inefficient. In order to make the load of a cluster more stable under high concurrent requests, we developed a Dynamic Load Balancing (DLB) algorithm that uses Nginx as a network security control panel to provide load balancing for a cluster of backend servers. The DLB algorithm is based on the weighted round robin module of Nginx, Logistic Regression and Maximum Likelihood Estimation (MLE) algorithm. It handles the situation of high concurrent requests and reduces the probability of omitted or under-reported incident and status. We also propose a Hybrid Load Balance Method (HLBM) that incorporates the DLB algorithm and evolutionary computing to further improve the performance. We have conducted limited experiment by using dynamic load balancing algorithm. We will complete to develop the Hybrid Load Balance method and conduct experiments for HLBM.

Dear SCPE Reader, We present a selection of papers which are extensions of papers presented at the 7-th International Symposium on Parallel and Distributed Computing, 1–5 July 2008, in Krakow, Poland. The motivation for publishing the selection in the SCPE Journal was, on the one hand, to present the flavour of the research reported at the conference and on the other hand to present some of the most relevant topics currently focused on the research on parallel and distributed computing in general. The selection contains only 6 papers out of about 60 presented at the conference, and thus, is far from covering all relevant topics represented at the ISPDC 2008. This is because not all of the invited authors were patient enough to accept a fairly long paper publishing process. Nevertheless, we hope that the presented papers will bring you closer to the research covered by the ISPDC conferences and will encourage you to participate in future ISPDC editions. The first paper ``The Impact of Workload Variability on Load Balancing Algorithms'' is by Marta Beltran and Antonio Guzman from King Juan Carlos University in Spain. It concerns an important topic of load balancing in cluster systems, namely adaptativity of the load balancing algorithms to changes of the workload in the system. Adequate accounting for additional load in the hosting system is of great relevance for correct optimization effects. The paper presents a thorough formal analysis of the workload variability metrics and their influence on the quality of load balancing algorithms. Four basic activities appearing in load balancing algorithms are identified, and based on them some algorithmic solutions are proposed to correctly deal with workload variability in system load balancing. The problem of dynamic load balancing algorithms robustness has been discussed. Two different robustness metrics sensitive to the applied type of opimization: local task-oriented or a global one enable selecting task remote execution or migration as load balancing operations. The proposed approach is illustrated with experiments. The second paper ``Model-Driven Engineering and Formal Validation of High-Performance Embedded Systems'' is by Abdoulaye Gamatie, Eric Rutten, Huafeng Yu, Pierre Boulet, Jean-Luc Dekeyser, from University of Lille and INRIA in France. The paper is concerned with a very advanced methodology of designing correct parallel embedded systems for intensive data-parallel computing. In their previous research, the authors of the paper designed the GASPARD embedded system design framework. It is based on the hardware/software co-design approach through model-driven engineering. The framework is based on an UML-like model specification language in which hardware and software elements are modelled using a component approach with special mechanisms for repetitive structures. This paper tries to combine the modelling framework of GASPARD with the mechanisms of synchronous languages to achieve design verifiability provided for such languages. The paper shows how GASPARD models can be translated into synchronous models based on data flow equations in order to formally check their correctness. The proposed approach is illustrated with an example of a video processing system. The third paper ``Relations Between Several Parallel Computational Models'' is by Stephan Bruda and Yuanqiao Zhang from Bishop’s University in Canada. The paper is concerned with theoretical aspects of shared memory systems described by the parallel random access machine PRAM model and aims in studying performance properties of different types of PRAM systems. The attention is focussed on analysing the computational power of two more sophisticated PRAM models (Combining CRCW and Broadcast Selective Reduction), which include data reduction in case of concurrent writes. The paper shows that these two models have equivalent computational power, which is a new result comparing the existing literature. The performance of both models applied to reconfigurable multiple bus machines was studied as a possible architectural solution for current VLSI processor implementations. It was shown that in such systems under reasonable assumptions concurrent-write does not enhance performance comparing the exclusive-write model. Another result important for the VLSI technology is that the Combining CRCW PRAM model (in which data of concurrent writes are arthmetically or logically combined before write) and the exclusive-write on directed reconfigurable busses perform in equivalent way under strong real-time requirements. The fourth paper ``Experiences with Mesh-Like Computations Using Prediction Binary Trees'' is by Gennaro Cordasco, Biagio Cosenza, Rosario de Chiara, Ugo Erra and Vittorio Scarano from the University ``degli Studi'' of Salerno and the University ``degli Studi della Biasilicata'' of Potenza in Italy. The paper concerns optimization methods for mesh-like computations in clusters of processors. The computations are perfomed assuming a phase-like program execution control using a tiling approach which reduces inter-processor communication. A temporal coherence is also assumed, which means that task sizes provide similar execution times in consecutive phases. Temporary coherent computations are structured in a Prediction Binary Tree, in which leaves represent computing tiles to be mapped to processors. A phase-by-phase semi-static load balancing is introduced to the scheduling algorithm. The scheduling algorithm is equipped with a predictor, which estimates the computation time of next phase tiles based on previous execution times and modifies the tiles to achieve balanced execution in phases. For this, two heuristics are used to leverage on data locality in processors. The proposed approach is illustrated by the example of interactive rendering with Parallel Ray Tracing algorithm. The fifth paper ``The Influence of the IBM pSeries Servers Virtualization Mechanism on Dynamic Resource Allocation in AIX 5L'' is by Maciej Mlynski from ASpartner Limited in Poland. The paper concerns a very up-to-date problem of system virtualization and presents the results of research carried on IBM pSeries servers. IBM is strongly developing the virtualization technique especially on IBM pSeries servers enabling an improved and flexible sharing of system resources between applications. The paper investigates novel facilities for dynamic resource management such as micro-partitioning and partition load manager. They enable dynamic creation of workload logical partitions of system resources and their dynamic mangement. It includes run-time resource re-alocation between logical partitions including setting of sharing specifications as well as run-time adding/removing/setting parameters of resources in the system. It remains an open question how to properly tune parameters of the operating system using the provided virtualization facilities to obtain the best efficiency for a given application program. The paper presents the results of experiments which study the effects of tuning the disk subsystem parameters under the IBM AIX 5L operating system with the use of the provided virtualization facilities on the resulting application execution performance. The results show that even small deterioration in the resource pool status requires an immediate adaptation of the operating system parameters to maintain the required performance. The sixth paper ``HeteroPBLAS: A Set of Parallel Basic Linear Algebra Subprograms Optimized for Heterogeneous Computational Clusters'' is by Ravi Reddy, Alexey Lastovetsky and Pedro Alonso from University College Dublin in Ireland and Polytechnic University of Valencia in Spain. The paper concerns the methodology for parallelization of linear algebra computations for execution in heterogeneous cluster environments. The design of the HeteroPBLAS library (Parallel Basic Linear Algebra Subprograms) for heterogeneous computational clusters is presented. The main contribution of the paper is the automation of the parallelization and optimization of the PBLAS, which is done by means of a special user interface and the underlying set of functions. An important element is here a performance model that is based on program code instrumentation, which determines parameters of the application and the executive heterogeneous platform relevant for execution performance of parallel code. The parameter values specified for or returned by execution of the performance model functions are next used for generation and optimal mapping of the parallel code of the library subroutines. The proposed approach is illustrated by experimental results of execution of optimized HeteroPBLAS programs on homogeneous and heterogeneous computing clusters. Marek Tudruj

A Three-Dimensional Finite Volume Arbitrary Lagrangian-Eulerian simulation code was developed to study different plasma physics problems in 3D+t. The code is based on a complex multi-component species program with transport and radiation terms written and applied to plasma and fusion physics problems. The integration domain is represented with a structured irregular mesh, with fixed connectivity made of hexahedral cells. Coordinates and velocities are assigned to cell vertices. After each calculation cycle, mesh vertices are moved arbitrary over the fluid. The adaptive method consists of shifting mesh vertices over the fluid in order to keep a reasonable mesh structure and increase the spatial resolution where the physical solution demands. The code was a valuable tool for parametric study of different physical problems, mainly optimization of Plasma Focus Machine, detonation and propagation of thermonuclear reactions, and Kelvin-Helmholtz instabilities in the boundary layer of the terrestrial magnetopause.

An emerging method of digital computing known as "cloud computing" has recently gained immense popularity. While there are many benefits to using the cloud over the internet, there are also serious challenges that must be addressed in order to boost the efficiency of this method. Challenges to cloud computing via the internet include load balancing, work scheduling, fault tolerance, and several security concerns. The effectiveness of the cloud may be enhanced by fixing a number of problems, one of the most pressing being load balancing. To prevent any one node from being too overburdened or underused, a system employs a technique known as load balancing. In order to increase utilization and minimize overall task execution time, load balancing algorithms are developed to distribute work fairly across available resources. In this article, a review and comparison of different load-balancing algorithms is provided. This paper provides a basis of the application of different load-balancing techniques, which utilize the approach of soft computing within the cloud computing environment.

In a cluster environment, the identification and the classification of the requested task is a prerequisite for choosing the appropriate load balancing algorithm and improving the processing capacity of the network. As the existing load balancing strategies cannot relate tasks and load balancing algorithm intelligently, this paper present an Agent-based dynamic adaptive load balancing model. Through the identification and classification of the requested task, the model selects corresponding load balancing algorithm to reach the load balancing in cluster. This model has a distinct advantage over other load balancing schedule strategies on response time. DOI : http://dx.doi.org/10.11591/telkomnika.v12i3.4568 Full Text: PDF

Load balancing algorithms are designed to achieve efficient resource utilisation in large-scale distributed systems. The primary goal of designing load balancing algorithms is to balance the overall workload among all the nodes in distributed systems to improve the performance. This paper presents an in-depth study and detailed comparative analysis of different load monitoring and balancing algorithms employing fuzzy logic and mobile agents. A detailed qualitative as well as quantitative analysis of algorithmic performances is included. The analytical survey presented in this paper would enable understanding of applications of different algorithms under different computing environments. Furthermore, this paper proposes a hybrid architecture combining fuzzy logic and mobile agents for load balancing and monitoring in distributed systems.

Throughput maximization in a network is one of the major key challenges in wireless LANs (WLANs) and it could be achieved when the load of access points (APs) is balanced. The AP load in WLANs is very often found to be uneven. To overcome such imbalance of load, a load balancing technique have been proposed. The load balancing techniques commonly require proprietary software or hardware at the user side for controlling the user-AP association. A new load balancing technique by controlling the size of WLAN cells (i.e., AP’s coverage range), which is conceptually similar to cell breathing in cellular networks. The proposed scheme does not require any modification from the user side nor may any changes be required to the IEEE 802.11 standard [1]. It only requires the ability of dynamically changing the transmission power of the AP beacon messages. We develop a newer load balancing algorithm that will find the optimal beacon power settings which will thereon help to minimize the load of the most congested AP. The load may become shared and this may be performed with minimal delay and also maximized throughput is to be achieved. Keywords: Network throughput, Access points, WLAN, Beacon Messages, Load balancing, Throughput, Wireless LANs.

Objectives: In this paper various task scheduling algorithm along with load distribution techniques investigated to ensure efficient mapping of tasks to resources and for coherent resource utilization in heterogeneous environment. Statistical Analysis: A detailed comparative approach is conferred for different job scheduling and load balancing algorithm available in the literature. Considered parameters for comparisons are scheduling approaches, techniques, findings, benefits, pros and cons. This study facilitates to identify the efficient scheduling and load balancing approach which optimizes resource use, minimizes completion time as well as task waiting time and balanced load among resources. Findings: Make span time and flow time are two important parameters to compute and minimize job execution time. This paper explores an intense survey of minimization of task execution time with minimum make span time and flow time. Apart from this a joint review on job scheduling, load balancing and minimizing job completion time using Expected Time to Compute (ETC) matrix is presented. At the end of this paper a research model is proposed which is a novel and hybrid approach with a combination of the efficiency of scheduling and load balancing. Application: The Load Balancing (LB) is an important feature of the grid environment. The efficient LB algorithm assures cost effective resource usage by the provisioning of resources to users on demand schedule in pay-as-you-say-manner.

In the contemporary realm of operation in the World Wide Web, the cloud applications have to meet the high availability, operations on dynamic loads and must function at a very high quality. Load balancing is a proper technique to achieve above goals to distribute the loads across the server or to the various resources. This distribution optimizes performance, scalability, reliability and resources to be used, hence making the network to be appealing to bigger organizations to adopt it. The advantages because of load balancing are reduced response time, increased throughput and fault tolerance because the network traffic load is spread evenly among different servers thereby avoiding the occurrence of a bottleneck server. Some of these ways include Round Robin, Least Connection, and Weighted Distribution, help in the process by making it possible for systems to horizontally scale up for more traffic. As cloud services becomes popular, it becomes very important to manage the load since some applications or services may be more resource intensive than others. It distributes the incoming requests following the current loads on various servers and consequently, no server is too busy or idle. Also, load balancers provide for fault tolerance in the sense that traffic will be handled by other healthy servers in the occurrence of failures hence the solutions are highly available and reliable. Load balancing algorithms such from the basic rule-dependent to complex reinforcement learning algorithms are very crucial with respect to allocation of resources, energy management, and stability of the system. These algorithms will be increasingly important as cloud environments develop further in the efficiency of delivering services and the managing of resources needed, benefiting both the economy and the environment. INDEX TERMS: Performance, Scalability, Response Times, Latency, Horizontal Scaling, Load Balancing

A multiprocessor system is a computer with two or more central processing units (CPUs) with each one sharing the common main memory as well as the peripherals. Multiprocessor system is either homogeneous or heterogeneous system. A homogeneous system is a cluster of processors joined to a high speed network for accomplishing the required task; also it is defined as parallel computing system. Homogeneous is a technique of parallel computing system. A heterogeneous system can be defined as the interconnection of a number of processors, having dissimilar computational speed. Load balance is a method of distributing work between the processors fairly in order to get optimal response time, resource utilization, and throughput. Load balancing is either static or dynamic. In static load balancing, work is distributed among all processors before the execution of the algorithm. In dynamic load balancing, work is distributed among all processors during execution of the algorithm. So problems arise when it cannot statistically divide the tasks among the processors. To use multiprocessor systems efficiently, several load balancing algorithms have been adopted widely. This paper proposes an efficient load balance algorithm which addresses common overheads that may decrease the efficiency of a multiprocessor system. Such overheads are synchronization, data communication, response time, and throughput.

Cloud computing has become popular due to its attractive features. The load on the cloud is increasing tremendously with the development of new applications. Load balancing is an important part of cloud computing environment which ensures that all devices or processors perform same amount of work in equal amount of time. Different models and algorithms for load balancing in cloud computing has been developed with the aim to make cloud resources accessible to the end users with ease and convenience. In this paper, we aim to provide a structured and comprehensive overview of the research on load balancing algorithms in cloud computing. This paper surveys the state of the art load balancing tools and techniques over the period of 2004–2015. We group existing approaches aimed at providing load balancing in a fair manner. With this categorization we provide an easy and concise view of the underlying model adopted by each approach.

Dynamic load balancing is a key problem for the efficient use of parallel systems when solving applications with unpredictable load estimates. However, depending on the underlying programming paradigm Single Program Multiple Data (SPMD) or Multiple Program Multiple Data (MPMD) the balancing requirements vary. In SPMD scenarios, a perfect load balance is desired, whereas in MPMD scenarios it might be better to quickly obtain a large reduction in load imbalance in a short period of time. We propose extending the local domain of a given processor in the load-balancing algorithms to find a better scope for each paradigm. For that purpose, we present a generalised version of the Diffusion Algorithm Searching Unbalanced Domains (called ds-DASUD), which extends the local domain of each processor beyond its immediate neighbour. ds-DASUD belongs to the iterative distributed load-balancing (IDLB) class and, in its original formulation, operates in a diffusion scheme where a processor balances its load with all its immediate neighbours (ds=1). We evaluate this algorithm for the two programming paradigms varying the domain size. The evaluation was carried out using two simulators (load-balancing and network simulators) for a large set of load distributions that exhibit different degrees of initial workload unbalancing. These distributions were applied to torus and hypercube topologies, and the number of processors ranged from 8 to 128. From these experiments, we conclude that the 1-DASUD fits well for SPMD scenarios, whereas for MPMD 3-DASUD and ((d/2)+1)-DASUD for hypercube and torus topologies, respectively, obtain the best trade-off between the imbalance reduction (up to 85%) and the cost incurred in reaching it.

Modern standards in High Performance Computing (HPC) have started to consider energy consumption and power draw as a limiting factor. New and more complex architectures have been introduced in HPC systems to afford these new restrictions, and include coprocessors such as GPGPUs for intensive computational tasks. As systems increase in heterogeneity, workload distribution becomes a more core problem to achieve the maximum efficiency in every computational component. We present a Multi-Objective Dynamic Load Balancing (DLB) approach where several objectives can be applied to tune an application. These objectives can be dynamically exchanged during the execution of an algorithm to better adapt to the resources available in a system. We have implemented the Multi-Objective DLB together with a generic heuristic engine, designed to perform multiple strategies for DLB in iterative problems. We also present Ull Multiobjective Framework (UllMF), an open-source tool that implements the Multi-Objective generic approach. UllMF separates metric gathering, objective functions to be optimized and load balancing algorithms, and improves code portability using a simple interface to reduce the costs of new implementations. We illustrate how performance and energy consumption are improved for the implemented techniques, and analyze their quality using different DLB techniques from the literature.

This paper reports on the results of a preliminary study in dynamic load balancing on an Intel Hypercube. The purpose of this research is to provide experimental data in how parallel algorithms should be constructed to obtain maximal utilization of a parallel architecture. This study is one aspect of an ongoing research project into the construction of an automated parallelization tool. This tool will take FORTRAN source as input, and construct a parallel algorithm that will produce the same results as the original serial input. The focus of this paper is on the load balancing aspect of that project. The basic idea is to reserve a certain percentage of the computation task, subdivide that percentage into arbitrarily fine tasks, and dole those small tasks out to nodes on request. Ij” the percentage is chosen correctly, then a minority of nodes should be involved in consuming the filler tasks, and the overall throughput of the job should increase as a result of the individual node efJciencies having increased. This paper will outline our approach to performing dynamic load balancing on an Intel iPSC/2. We take the view that the problem of load balancing is really a problem of dividing a “computational task” into smaller components, each of roughly equal complexity, and each an independent event. After this is done, the components of the task can be sent to a node for execution. The key to an optimally balanced load across all computational nodes is the ability to form a statistical profile of the individual components of each computational task. This statistical profile will determine an initial sequence of execution. Our experience indicates that a speedup on the order of 80% is achievable with the judicious use of profiled load balancing. During the process of execution, the initial profile will be altered according to the actual behavior exhibited by the nodes. The difference between the actual and expected performance will be used to determine how much additional time should be devoted to altering the current execution schedule. Currently, our work involves statically setting the load balancing parameters. Our load balancing system determines the execution schedule