• Home
  • Search
  • Adaptive Resource Relocation in Virtualized Heterogeneous Clusters
  • https://doi.org/10.25911/5d7a267ea3a59Copy DOI Icon

Adaptive Resource Relocation in Virtualized Heterogeneous Clusters

  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

Cluster computing has recently gone through an evolution from single processor systems to multicore/multi-socket systems. This has resulted in lowering the cost/performance ratio of the compute machines. Compute farms that host these machines tend to become heterogeneous over time due to incremental extensions, hardware upgrades and/or nodes being purchased for users with particular needs. This heterogeneity is not surprising given the wide range of processor, memory and network technologies that become available and the relatively small price difference between these various options. Different CPU architectures, memory capacities, communication and I/O interfaces of the participating compute nodes present many challenges to job scheduling and often result in under or over utilization of the compute resources. In general, it is not feasible for the application programmers to specifically optimize their programs for such a set of differing compute nodes, due to the difficulty and time-intensiveness of such a task. The trend of heterogeneous compute farms has coincided with resurgence in the virtualization technology. Virtualization technology is receiving widespread adoption, mainly due to the benefits of server consolidation and isolation, load balancing, security and fault tolerance. Virtualization has also generated considerable interest in the High Performance Computing (HPC) community, due to the resulting high availability, fault tolerance, cluster partitioning and accommodation of conflicting user requirements. However, the HPC community is still wary of the potential overheads associated with virtualization, as it results in slower network communications and disk I/O, which need to be addressed. The live migration feature, available to most virtualization technologies, can be leveraged to improve the throughput of a heterogeneous compute farm (HC) used for HPC applications. For this we mitigated the slow network communication in Xen; an open source virtual machine monitor. We present a detailed analysis of the communication framework of Xen and propose communication configurations that give 50% improvement over the conventional Xen network configuration. From a detailed study of the migration facility in Xen, we propose an improvement in the live migration facility specifically targeting HPC applications. This optimization gives around 50% improvement over the default migration facility of Xen. In this thesis, we also investigate resource scheduling in heterogeneous compute farm with the perspective of dynamic resource re-mapping. Our approach is to profile each job in the compute farm at runtime, and propose a better resource mapping compared to the initial allocation. We then migrate the job(s) to the best-suited homogeneous sub-cluster to improve overall throughput of the HC. For this, we develop a novel heterogeneity and virtualization-aware profiling framework, which is able to predict the CPU and communication characteristics of high performance scientific applications. Our experimental setup consists of a 32 node virtualized cluster with machines having CPU frequency between 2 GHz to 3 GHz. The framework was tested with a subset of NAS parallel benchmarks with the workload generated from Dror Feitelson’s workload archives. The prediction accuracy of our performance estimation model is over 80%. The framework implementation is lightweight, with an overhead of 3%. Our experiments show that we are able to improve the throughput of the compute farm by 25% and the time saved by the HC with our framework is over 30%. The framework can be readily extended to HCs supporting a cloud computing environment.

Similar Papers
  • Research Article
  • Citations2

Failure Analysis and Prediction in Compute Clouds

  • Jan 01, 2014
  • Open Collections
  • Xin Chen
  • Research Article

Recent Advances in Modularity Optimization and Their Application in Retailing

  • Jan 01, 2014
  • Repository KITopen (Karlsruhe Institute of Technology)
  • Andreas Geyer-Schulz +1
  • Research Article

Efficient and Secure Key Management Scheme for Mobile Ad Hoc Networks

  • Aug 22, 2015
  • Figshare
  • M El-Bashary +2
  • Research Article
  • Citations1

Analysis Study of Fuzzy C-Mean Algorithm Implemented on Abnormal MR Brain Images

  • Jun 18, 2015
  • Figshare
  • Journals Iosr +3
  • Research Article
  • Citations3

Crude Protein Profiling of Varieties of Capsicum annuum and Capsicum frutescens using SDS-PAGE.

  • May 12, 2015
  • Figshare
  • Journals Iosr +2
  • Single Report

2011 Computation Directorate Annual Report

  • Apr 11, 2012
  • D Crawford
  • Research Article

Dense Stellar Systems– Old and Young MODEST-5b

  • Jul 16, 2004
  • Astronomische Nachrichten
  • Spurzem +13
  • Research Article
  • Citations2

Dynamic Layered Clustering Routing Algorithm in Underwater Sensor Networks

  • Jun 19, 2015
  • 电子与信息学报
  • Hong Chang-Jian +2
  • Conference Article

Energy-efficient Clustering with Fast Data Compression in Sensor Networks

  • Nov 09, 2006
  • Xiaoling Wu +4
  • Supplementary Content

Massively parallel stream processing with latency guarantees

  • Jan 01, 2016
  • DepositOnce
  • Björn Lohrmann
  • Research Article

Fairly Power Allocation Scheme in Downlink NOMA Systems

  • Jan 01, 2020
  • 电子科技大学学报
  • Xinji Tian +2
  • Research Article

The $^{12}C$ continuum in a microscopic coupled channel calculation

  • Jan 01, 2014
  • GSI Repository (GSI Helmholtzzentrum für Schwerionenforschung)
  • Thomas Neff +1

Clustering of the Potential of Customers' Internet Bandwidth Upgrade on FTTH Broadband

  • Apr 30, 2021
  • Sasa Ani Arnomo
  • Research Article
  • Citations10

Extending an open source enterprise service bus for multi-tenancy support focusing on administration and management

  • Jan 01, 2012
  • OPUS Publication Server of the University of Stuttgart (University of Stuttgart)
  • Dominik Muhler
  • PDF
  • Supplementary Content
  • Citations32

Capturing Consumer Preferences for Value Chain Improvements in the Mango Industry of Pakistan

  • Sep 01, 2015
  • AgEcon Search (University of Minnesota, USA)
  • Hammad Badar +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.