Research computing in a distributed cloud environment

In recent cloud computing research, the allocation of virtual resources has become a major concern. The Virtual Machine (VM) dynamic deployment technique is one of the key virtual resource management techniques. Virtual resource management predicts the resources on the virtual machine and manages accordingly. The study of the Virtual Machine resource dynamic deployment focuses on the fine-grained resource adjustment strategy. So, there is a need for predicting the usage of resources for the virtual machine to avoid delays, overprovisioning and under provisioning. Time series forecasting models are used because predictions of cloud resources are dependent on time. In this paper, different time series models and ensemble techniques have been fitted to predict resource usage for successive hours. Different time series analysis techniques were implemented to study the dataset and to choose the appropriate model. Various types of resources such as CPU usage, disk read times, disk write times and memory usage are predicted using the above models.

Symbiotic Organism Search optimization based task scheduling in cloud computing environment

The emergence of academic and commercial Infrastructure-as-a-Service (IaaS) clouds is opening access to new resources for the HEP community. In this paper we will describe a system we have developed for creating a single dynamic batch environment spanning multiple IaaS clouds of different types (e.g. Nimbus, OpenNebula, Amazon EC2). A HEP user interacting with the system submits a job description file with a pointer to their VM image. VM images can either be created by users directly or provided to the users. We have created a new software component called Cloud Scheduler that detects waiting jobs and boots the user VM required on any one of the available cloud resources. As the user VMs appear, they are attached to the job queues of a central Condor job scheduler, the job scheduler then submits the jobs to the VMs. The number of VMs available to the user is expanded and contracted dynamically depending on the number of user jobs. We present the motivation and design of the system with particular emphasis on Cloud Scheduler. We show that the system provides the ability to exploit academic and commercial cloud sites in a transparent fashion.

The introduction of network function virtualization (NFV) leads to a new business model in which the Telecommunication Service Provider needs to rent cloud resources to infrastructure provider (InP) at prices as low as possible. Lowest prices can be achieved if the cloud resources can be rented in advance by allocating long-term virtual machines (VM). This is in contrast with the short-term VMs that are rented on demand and have higher costs. For this reason, we propose a proactive solution in which the cloud resource rent is planned in advance based on peak traffic knowledge. We illustrate the problem of determining the cloud resources in cloud infrastructures managed by different InPs and so as to minimize the sum of cloud resource, bandwidth and deployment costs. We formulate an integer linear problem (ILP) and due to its complexity, we introduce an efficient heuristic approach allowing for a remarkable computational complexity reduction. We compare our solution to a reactive solution in which the cloud resources are rented on demand and dimensioned according to the current traffic. Though the proposed proactive solution needs more cloud and bandwidth resources due to its peak allocation, its total resources cost may be lower than the one achieved when a reactive solution is applied. That is a consequence of the higher cost of short-term VMs. For instance, when a reactive solution is applied with traffic variation times of ten minutes, our proactive solution allows for lower total costs when the long-term VM rent is lower than the short-term VM one by 33%.

With the recent rapid development of cloud computing technology, how to reduce the costs of a cloud data center effectively has become an important issue. The study on virtual machine deployment mainly aims at deploying virtual machine resources required by users on a physical server rationally and effectively. This paper proposes a multi-population ant colony algorithm to solve problems of virtual machine deployment. With resource wastage and energy consumption as optimization objectives, this algorithm uses multiple ant colonies for the solution and determines strategies for information exchange among ant colonies according to the information entropy of each population to guarantee the balance of its convergence and diversity. The simulation results show that this algorithm has better performance than the single-population ant colony algorithm and can reduce resource wastage and energy consumption effectively for high-demand virtual machine deployment.

Cloud is a boon to the generation which provides services that can reduce the overhead in maintenance and computational complexities. Scheduling the user's job in the cloud resources plays an important role for the better performance. Task scheduling is an NP-hard problem, since it may have more than one solution to fit in. In this paper a hybrid algorithm is proposed by the amalgamation of artificial bee colony Algorithm and particle swarm optimisation named as ABPS algorithm. The proposed ABPS algorithm optimises the task scheduling on the cloud environment by providing minimised makespan, cost, and maximised resource utilisation and to balance the load. The proposed ABPS algorithm compared with ABC and PSO algorithm have been simulated in the CloudSim simulation tool. The proposed ABPS algorithm based on makespan outperforms ABC and PSO algorithms by 22.07% and 28.12%, respectively, also when compared with cost outperforms ABC and PSO algorithms by 32.41% and 44.49% respectively. ABPS algorithm based on resource utilisation outperforms ABC and PSO algorithms by 49.37% and 48.88% respectively and based on degree of imbalance outperforms ABC and PSO algorithms by 16.21% and 20.51% respectively.

HEPGA: A new effective hybrid algorithm for scientific workflow scheduling in cloud computing environment

Enhanced Particle Swarm Optimization for Task Scheduling in Cloud Computing Environments

BigTrustScheduling: Trust-aware big data task scheduling approach in cloud computing environments

Managing the allocation of cloud virtual machines at physical resources is a key requirement for the success of clouds. Current implementations of cloud schedulers do not consider the entire cloud infrastructure neither do they consider the overall user and infrastructure properties. This results in major security, privacy, and resilience concerns. In this paper, we propose a novel cloud scheduler which considers both user requirements and infrastructure properties. We focus on assuring users that their virtual resources are hosted using physical resources that match their requirements without getting users involved with understanding the details of the cloud infrastructure. As a proof-of-concept, we present our prototype which is built on OpenStack. The provided prototype implements the proposed cloud scheduler. It also provides an implementation of our previous work on cloud trust management which provides the scheduler with input about the trust status of the cloud infrastructure.

In order to solve the problems of uneven spatial distribution of data nodes and unclear weight relationship of virtual scheduling features in cloud computing platform, a virtual resource scheduling method based on improved ant colony algorithm is studied and designed to improve the performance of virtual resource scheduling in cloud computing platform by this method. After analyzing the information resource sequence change of the cloud computing platform, according to the STR - Tree partition graph, a simulated annealing-based algorithm is employed to classify the resource types after optimal scheduling into IO types, middle types and CPU types, and the time span and load balance are set as the measurement indexes. The simulation results show that after applying this method, the occupied resources of the main platform are 535 MB, which are much lower than the other two comparison algorithms, and the method has improved the allocation rationality, resource balance, maximum queue length and energy consumption. This result indicates that applying this virtual resource scheduling method can effectively improve the intelligent scheduling of virtual resources in the cloud computing platform.

This paper presents the design, implementation, and evaluation of a new system for provisioning of Hadoop clusters across multiple cloud domains. The Hadoop clusters are created on-demand and are composed of virtual machines from multiple cloud sites linked with bandwidth-provisioned network pipes. The prototype uses an existing federated cloud control framework called Open Resource Control Architecture (ORCA), which orchestrates the leasing and configuration of virtual infrastructure from multiple autonomous cloud sites and network providers. ORCA enables computational and network resources from multiple clouds and network substrates to be aggregated into a single virtual slice of resources, built to order for the needs of the application. The experiments examine various provisioning alternatives by evaluating the performance of representative Hadoop benchmarks and applications on resource topologies with varying bandwidths. The evaluations examine conditions in which multi-cloud Hadoop deployments pose significant advantages or disadvantages during Map/Reduce/Shuffle operations. Further, the experiments compare multi-cloud Hadoop deployments with single-cloud deployments and investigate Hadoop Distributed File System (HDFS) performance under varying network configurations. The results show that networked clouds make cross-cloud Hadoop deployment feasible with high bandwidth network links between clouds. As expected, performance for some benchmarks degrades rapidly with constrained inter-cloud bandwidth. MapReduce shuffle patterns and certain Hadoop Distributed File System (HDFS) operations that span the constrained links are particularly sensitive to network performance. Hadoop's topology-awareness feature can mitigate these penalties to a modest degree in these hybrid bandwidth scenarios. Additional observations show that contention among co-located virtual machines is a source of irregular performance for Hadoop applications on virtual cloud infrastructure.

This paper focuses on an integrated workload scheduling and pre-fetching model in a multimedia mobile cloud computing environment to enhance the performance of response time and reduce the cost to process multimedia data. The response time and cost optimization problems are presented along with the computation resources such as virtual machines (VMs) allocation, workload conservation, queueing stability constraints, and to overcome the total response time and cost, a heuristic approach of workload scheduling method is proposed. The integrated workload scheduling at pre-fetcher and cloud are considered to study the effects of various parameters such as VM’s processing speed, pre-fetcher’s utilization, the user requests arrival rate. The performance analysis results reveal that the cost and transmission speed are directly relevant factors, meaning that, once the rate of data transmission is increasing, the cost is also increasing and vice versa. Hence, the time and cost efficient workload scheduling is essential to satisfy both delay and cost in pre-fetcher enabled multimedia cloud systems.

In the current scenario, Cloud computing carved itself as an emerging technology which enables the organization to utilize hardware, software and applications without any upfront cost over the internet. The challenge before the cloud service provider is, how efficiently and effectively the underlying computing resources like virtual machines, network, storage units, and bandwidth etc. should be managed so that no computing device is in under-utilization or over-utilization state in a dynamic environment. A good task scheduling technique is always required for the dynamic allocation of the task to avoid such a situation. Through this paper we are going to present the Genetic Algorithm based task scheduling technique, which will distribute the load effectively among the virtual machine so that the overall response time (QoS) should be minimal. A comparison of this Genetic Algorithm based task scheduling technique is performed on CloudSim simulator which shows that, this will outperform the existing techniques like Greedy based, First - Come first - Serve (FCFS) techniques.

Load balancing is the core of virtual resource management and scheduling in cloud computing. For network applications, the cost of user would be greatly saved if load balancer could dynamically adjust cluster resources in accordance with the current applied load. The current load balancing products of cloud, such as Amazon's ELB, can be used to manage virtual machines in the cloud. The main drawbacks are still only supporting template-based deployment of new virtual machines, not supporting the trend prediction, failing to gain resources dynamically, and not sufficiently providing the elastic management of resources. Since the virtual machine for load balancing management in cloud computing can be dynamically applied and released, an algorithm of prediction-based elastic load balancing resource management (TeraScaler ELB) is presented to overcome the drawbacks. Experiments have shown that the required number of virtual machines change in compliance with the change of network load, thus TeraScaler ELB is able to dynamically adjust the processing capacity of back-end server cluster with the applied load. Besides it could make full use of the 'use on demand' feature of cloud computing, TeraScaler ELB leads to a better application of prediction based load balancing in cloud computing. It concludes that compared with the traditional elastic resource management algorithm, TeraScaler ELB is more reasonable for providing scalability and high availability.