Load Balancing in SPMD Applications: Concepts and Experiments

The performance of SPMD programs is strongly affected by dynamic load imbalancing factors. The use of a suitable load balancing algorithm is essential for overcoming the effects of these imbalancing factors. In this work, we evaluate the performance of a scientific SPMD parallel application when executed on a computational grid, with different kinds of load balancing strategies. The developed SPMD application computes the macroscopic thermal dispersion in porous media. A set of experiments was conducted on a computational grid composed by two geographically separated clusters. The main contribution of this work is the performance evaluation and comparison of a large variety of load balancing techniques under dynamic environment conditions. The experimental results showed the importance of choosing appropriate load balancing strategies when developing SPMD applications on a grid environment.

The signal intensity based wireless terminal access mechanism is the key cause of imbalanced load of WLAN. In the environment where users are densely packed, such as banks, office buildings and malls, load imbalance will severely decrease the quality and source utility rate of WLAN, thus the node load balance strategy becomes increasingly in demand to solve the problem. First, this paper studied the mainstream node load balance technologies and proposed an evaluation model which identifies and measures the level of load; next, the load level gain calculation model was designed and the prediction-based load balance and resource adjustment strategy was put forward; at last, the load balance mechanism ST-WLB was enabled and the test result shows that it can improve the performance by 10% without excessive costs.

Routing and wavelength assignment (RWA) is one of the key issues in the wavelength-routed optical networks. Although some RWA algorithms have been well performed to meet the need of certain networks requirement, they usually neglect the performance of the whole networks, especially the load balancing of the whole networks. This is quite likely to lead to some links bearing excessive ligthpaths and traffic load, while other links being at an idle state. In this paper, the load distribution vector ( LDV ) is introduced to describe the links load of the networks firstly. Then by means of minimizing the LDV of the networks, the load balancing of the whole networks is tried to improve. Based on this, a heuristic load balancing (HLB) strategy is presented. Moreover, a novel RWA algorithm adopting the heuristic load balancing strategy is developed, as well as two other RWA algorithms adopting other load balancing strategies. At last, the simulations of the three RWA algorithms with different load balancing strategies are done for comparison on the basis of both the regular topology and the irregular topology networks. The simulation results show that the key performance parameters such as the average variance of links, the maximum link load and the number of established lightpath are improved by means of our novel RWA algorithm with the heuristic load balancing strategy.

A pore scale analysis is implemented in this numerical study to investigate the behavior of microscopic inertia and thermal dispersion in a porous medium with a periodic structure. The macroscopic characteristics of the transport phenomena are evaluated with an averaging technique of the controlling variables at a pore scale level in an elementary cell of the porous structure. The Darcy–Forchheimer model describes the fluid motion through the porous medium while the continuity and Navier–Stokes equations are applied within the unit cell. An average energy equation is employed for the thermal part of the porous medium. The macroscopic pressure loss is computed in order to evaluate the dominant microscopic inertial effects. Local fluctuations of velocity and temperature at the pore scale are instrumental in the quantification of the thermal dispersion through the total effective thermal diffusivity. The numerical results demonstrate that microscopic inertia contributes significantly to the magnitude of the macroscopic pressure loss, in some instances with as much as 70%. Depending on the nature of the porous medium, the thermal dispersion may have a marked bearing on the heat transfer, particularly in the streamwise direction for a highly conducting fluid and certain values of the Peclet number.

Transient convective heat transfer is a type of heat transfer that occurs when fluid convection changes in temperature, heat flux, or velocity over time. However, the heat transfer phenomenon in porous media under transient situations has not been extensively examined in current research, and only a few studies have focused on triangle enclosures. Thus, a novel Dull Cornered Triangular Enclosure with Bottom separated Open-Cell Aluminum Metal Foam and Hybrid Nano Fluid is introduced. Neglecting viscous dissipation and thermal dispersion in porous media in the prevailing studies leads to underestimated temperature rises. Hence, a novel DMW-GO-MWCNT nanofluid is used as a coolant, featuring 2% graphene oxide (GO) and multi-walled carbon nanotubes (MWCNTs), enhancing convective heat transmission. Including viscous dissipation and thermal dispersion in transient simulations accurately models temperature rises, avoiding suboptimal designs. The sharp lower corners in traditional triangle enclosures create stagnation points, disrupting uniform heating or cooling and reducing heat transfer efficiency. Thus, a Dull Cornered Triangular Enclosure is designed to eliminate recirculation zones by rounding the bottom corners, improving heat transfer efficiency. The placement of porous media in contact with heating surface enclosures hinders transient convective heat transfer, leading to dominant conductive heat transfer, increased cavitation risk, reduced Nusselt number, and impaired overall convective heat transfer efficiency. Therefore, incorporating Bottom Separated Open-Cell Aluminum Metal Foam in the proposed triangular enclosure lowers pressure drop and increases the Nusselt number with increased convective heat transfer efficiency. The results show that the proposed model has improved the overall transient convective heat transfer performance with a high Nusselt number, thermal conductivity, thermal expansion coefficient, effectiveness, heat transfer coefficient, heat transfer rate; and low deviation time, skin friction coefficient, and pressure drop.

Thermal dispersion in porous media: one-equation model

We investigate proactive dynamic load balancing on multicore systems, in which threads are continually migrated to reduce the impact of processor/thread mismatches. Our goal is to enhance the flexibility of the SPMD-style programming model and enable SPMD applications to run efficiently in multiprogrammed environments. We present Juggle, a practical decentralized, user-space implementation of a proactive load balancer that emphasizes portability and usability. In this paper we assume perfect intrinsic load balance and focus on extrinsic imbalances caused by OS noise, multiprogramming and mismatches of threads to hardware parallelism. Juggle shows performance improvements of up to 80 % over static load balancing for oversubscribed UPC, OpenMP, and pthreads benchmarks. We also show that Juggle is effective in unpredictable, multiprogrammed environments, with up to a 50 % performance improvement over the Linux load balancer and a 25 % reduction in performance variation. We analyze the impact of Juggle on parallel applications and derive lower bounds and approximations for thread completion times. We show that results from Juggle closely match theoretical predictions across a variety of architectures, including NUMA and hyper-threaded systems.

Most cluster schemes select cluster head (CH) randomly without considering load balancing and security in Wireless Sensor Networks (WSNs). Our research on relevant literature shows that the existing authentication schemes do not fully consider the load balancing of CHs, while the load balancing schemes ignore the security authentication of CHs. Therefore, this paper effectively combines load balancing and security verification, and proposes a secure load balancing and authentication scheme (SLEB) based on Clustered Wireless Sensor networks. SLEB not only effectively maintains the energy balance of the whole network, but also successfully improves packet forwarding rate and prolongs the network lifetime. The simulation shows that SLEB is energy-saving, safe and sustainable compared with other similar schemes.

High-speed routers and high-performance data centers share a common system-level architecture in which multiple processing nodes are connected by an interconnection network for high-speed communications. Load balancing is an important technique for maximizing throughput and minimizing delay of the interconnection network. In this thesis, efficient load balancing schemes are designed and analyzed for next-generation routers and data centers.
\n
\nIn high-speed router design, two preferred switch architectures are input-queued switch and load-balanced switch. In an input-queued switch, time-domain load balancing can be carried out by an iterative algorithm that schedules packets for sending in different time slots. The complexity of an iterative algorithm increases rapidly with the number of scheduling iterations. To address this problem, a single-iteration scheduling algorithm called D-LQF is designed, in which exhaustive service policy is adopted for reusing the matched input-output pairs in the previous time slots to grow the match size. 
\n
\nUnlike an input-queued switch, a load-balanced switch consists of two stages of crossbar switch fabrics, where load balancing is carried out in both time and space domains. Among various load-balanced switches, the feedback-based switch gives the best delay-throughput performance. In this thesis, the feedback-based switch is enhanced in three aspects. Firstly, we focus on reducing its switch fabric complexity. Instead of using crossbars, a dual-banyan network is proposed. The complexity of dual-banyan can be further reduced by merging the two banyans to form a Clos network, resulting in a Clos-banyan network. Secondly, we target at improving the delay performance of the feedback-based switch. A Clos-feedback switch architecture is devised where each switch module in the Clos network is a small feedback-based switch. With application-flow based load balancing, packet order is ensured and the average packet delay is reduced from O(N) to O(n), where N and n are the switch and switch module sizes, respectively. Thirdly, we extend the feedback-based switch to support multicast traffic. Based on the notion of pointer-based multicast VOQ, an efficient multicast scheduling algorithm with packet replication at the middle-stage ports only is proposed. In order to provide close-to-100% throughput for any admissible multicast traffic patterns, a three-stage implementation of feedback-based switch is also designed.
\n
\nIn designing load balancing schemes for data centers, we focus on the most popular fat-tree based data centers. Notably, packet-based load balancing is widely considered infeasible for data centers. This is because the associated packet out-of-order problem will cause unnecessary TCP fast retransmits, and as a result, severely undermine TCP performance. In this thesis, we show that if packet-based load balancing is performed properly, the packet out-of-order problem can be easily addressed by slightly increasing the number of duplicate ACKs required for triggering fast retransmit. Admittedly, in case of a real packet loss, the loss recovery time will be increased. But our simulation results show that such an increase is far less than the reduction in the network queueing delay (due to a better load-balanced network). As compared to a flow-based load balancing scheme, our packet-based scheme consistently provides significantly higher goodput and noticeably smaller delay.

Load balancing for clusters has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, previous CPU- or memory-centric load balancing schemes suffer significant performance drop under I/O-intensive workloads due to the imbalance of I/O load. To solve this problem, we propose two simple yet effective I/O-aware load-balancing schemes for two types of clusters: (1) homogeneous clusters where nodes are identical and (2) heterogeneous clusters, which are comprised of a variety of nodes with different performance characteristics in computing power, memory capacity, and disk speed. In addition to assigning I/O-intensive sequential and parallel jobs to nodes with light I/O loads, the proposed schemes judiciously take into account both CPU and memory load sharing in the system. Therefore, our schemes are able to maintain high performance for a wide spectrum of workloads. We develop analytic models to study mean slowdowns, task arrival, and transfer processes in system levels. Using a set of real I/O-intensive parallel applications and synthetic parallel jobs with various I/O characteristics, we show that our proposed schemes consistently improve the performance over existing non-I/O-aware load-balancing schemes, including CPU- and Memory-aware schemes and a PBS-like batch scheduler for parallel and sequential jobs, for a diverse set of workload conditions. Importantly, this performance improvement becomes much more pronounced when the applications are I/O-intensive. For example, the proposed approaches deliver 23.6--88.0 % performance improvements for I/O-intensive applications such as LU decomposition, Sparse Cholesky, Titan, Parallel text searching, and Data Mining. When I/O load is low or well balanced, the proposed schemes are capable of maintaining the same level of performance as the existing non-I/O-aware schemes.

Developing SPMD applications with load balancing

Load balancing (LB) aims at tackling the hot-spot cell problem caused by uneven load distribution and resources shortage. Traditional LB schemes mainly focus on the load reduction of the hot-spot cell, while some partner cells might suffer from aggravating load after receiving inappropriate traffic, which triggers ripple LB and reduces the LB efficiency. To deal with this problem, a self- optimizing LB scheme in fixed relay station (RS) cellular networks is proposed. In the scheme, partner cells share the load jointly to guarantee each partner has similar load after LB. Meanwhile, the partner response mechanism considers the resources usage of partner's RSs to further reduce the ripple LB. Simulation results show that the proposed scheme can effectively reduce the number of ripple LB and reduce the call blocking rate.

A periodic load balancing method is proposed according to the load imbalance of network server in a clustered system of high concurrency. It is divided into different load periods based on the load condition of server node. Corresponding load balancing strategy is adopted for each period that the strategy of fast select is used in the period of small load while the server node with stable response time is first adopted in the period of large load. Periodic load strategy avoids the load imbalance caused by the single load strategy. It can be known from the experimental comparison that the improved periodic load balancing strategy is superior to the load balancing strategy of weighted least connection.

Load balancing and task partitioning are important components of distributed computing. The optimum performance from the distributed computing system is achieved by using effective scheduling and load balancing strategy. Researchers have well explored CPU, memory, and I/O-intensive tasks scheduling, and load balancing techniques. But one of the main obstacles of the load balancing technique leads to the ignorance of applications having a mixed nature of tasks. This is because load balancing strategies developed for one kind of job nature are not effective for the other kind of job nature. We have proposed a load balancing scheme in this paper, which is known as Mixed Task Load Balancing (MTLB) for Cluster of Workstation (CW) systems. In our proposed MTLB strategy, pre-tasks are assigned to each worker by the master to eliminate the worker's idle time. A main feature of MTLB strategy is to eradicate the inevitable selection of workers. Furthermore, the proposed MTLB strategy employs Three Resources Consideration (TRC) for load balancing (CPU, Memory, and I/O). The proposed MTLB strategy has removed the overheads of previously proposed strategies. The measured results show that MTLB strategy has a significant improvement in performance.

Hybrid grey wolf and improved particle swarm optimization with adaptive intertial weight-based multi-dimensional learning strategy for load balancing in cloud environments