- Research Article
7
- 10.1002/cpe.4517
Cloud computing and big data: Technologies and applications
- May 20, 2018
- Concurrency and Computation: Practice and Experience
- Mostapha Zbakh + 3 more +3
Cloud computing and big data: Technologies and applications
With the widespread application of cloud computing technology, the energy consumption problem of data centers is becoming increasingly severe. This research is dedicated to presenting a highly efficient and smart solution for optimizing energy usage in cloud computing systems, especially since effectively managing energy consumption through resource allocation and task scheduling has now become a critical challenge in the field. In view of this, the study first constructs a resource allocation model based on redundant resource recovery to dynamically identify and recover redundant resources. Second, a task scheduling optimization method based on dual deep reinforcement learning is established, which utilizes directed acyclic graphs to model task dependencies and optimizes task execution order and resource allocation strategies through reinforcement learning agents. In the testing of the resource allocation optimization model, when the number of tasks is 500, the redundant resource recovery rate is 87.0%, and the computational complexity is 16.7 s. The response time in the regional resource environment is 2.28 s, and the load adaptability is 95.8%. In the task scheduling method test, the proposed method has a memory occupancy of 85 MB and a scheduling delay of 0.13 s in a high-bandwidth environment of 300 Mbps. The results show that the proposed method outperforms traditional scheduling strategies in terms of resource utilization and energy consumption, and contributes to achieving the goal of green cloud computing. The research aims to provide an efficient and intelligent solution for energy optimization in cloud computing systems.
Cloud computing and big data: Technologies and applications
Cloud computing and big data: Technologies and applications
An energy-saving task scheduling strategy based on vacation queuing theory in cloud computing
High energy consumption is one of the key issues of cloud computing systems. Incoming jobs in cloud computing environments have the nature of randomness, and compute nodes have to be powered on all the time to await incoming tasks. This results in a great waste of energy. An energy-saving task scheduling algorithm based on the vacation queuing model for cloud computing systems is proposed in this paper. First, we use the vacation queuing model with exhaustive service to model the task schedule of a heterogeneous cloud computing system. Next, based on the busy period and busy cycle under steady state, we analyze the expectations of task sojourn time and energy consumption of compute nodes in the heterogeneous cloud computing system. Subsequently, we propose a task scheduling algorithm based on similar tasks to reduce the energy consumption. Simulation results show that the proposed algorithm can reduce the energy consumption of the cloud computing system effectively while meeting the task performance.
Read moreA Novel, Self-Adaptive, Multiclass Priority Algorithm with VM Clustering for Efficient Cloud Resource Allocation
Priority in task scheduling and resource allocation for cloud computing has attracted significant attention from the research community. However, traditional scheduling algorithms often lack the ability to differentiate between tasks with varying levels of importance. This limitation presents a challenge when cloud servers must handle diverse tasks with distinct priority classes and strict quality of service requirements. To address these challenges in cloud computing environments, particularly within the infrastructure of service models, we propose a novel, self-adaptive, multiclass priority algorithm with VM clustering for resource allocation. This algorithm implements a four-tiered prioritization system to optimize key objectives, including makespan and energy consumption, while simultaneously optimizing resource utilization, degree of imbalance, and waiting time. Additionally, we propose a resource prioritization and load-balancing model based on the clustering technique. The proposed work was validated through multiple simulations using the CloudSim simulator, comparing its performance against well-known task scheduling algorithms. The simulation results and analysis demonstrate that the proposed algorithm effectively optimizes makespan and energy consumption. Specifically, our work achieved percentage improvements ranging from +0.97% to +26.80% in makespan and +3.68% to +49.49% in energy consumption while also improving other performance metrics, including throughput, resource utilization, and load balancing. This novel model demonstrably enhances task scheduling and resource allocation efficiency, particularly in complex scenarios with tight deadlines and multiclass priorities.
Read moreJoint Optimization of Resource Allocation and Tasks Scheduling in Network Slicing Enabled Internet of Vehicles
With the increase of vehicular broadband services, network slicing is regarded as a promising technology to meet the various requirements. In the Internet of Vehicles (IoV) enabled by network slicing, both the resource allocation and task scheduling algorithms have received extensive attentions. However, most of the existing works on the joint optimization of inter-slice resource allocation and intra-slice task scheduling ignore the power consumption. In this regard, this paper further considers the queuing latency and the power consumption in the uplink transmission. Three sub Markov decision processes (MDP) are modeled and a layered deep reinforcement learning (DRL) based algorithm is designed for the joint optimization. The simulation consists of two parts, firstly the cumulative reward of different task scheduling algorithms are compared and analyzed, then the performance of all solutions are evaluated under different task densities. The results show that the proposed algorithm can significantly reduce the queuing latency while the power consumption is comparable to that of second best algorithm.
Read moreHybrid Heuristic Algorithm for Better Energy Optimization and Resource Utilization in Cloud Computing
Energy-efficient execution of the scientific workflow is a challenging task in cloud computing that demands high-performance computing to process growing datasets. Due to the interdependency of tasks in the scientific workflow applications, energy-efficient resource allocation is vital for large-scale applications running on heterogeneous physical machines. Thus, this paper proposes a hybrid heuristic algorithm based energy-efficient cloud computing service (HH-ECO) that offers a significant solution for resource allocation, task scheduling, and optimization of scientific workflows. To ensure the energy-efficient execution, the HH-ECO focuses on executing non-dominant workflow tasks through adaptive mutation and energy-aware migration strategy. HH-ECO adopts the chaotic based particle swarm optimization (C-PSO) principle to optimize the resource allocation, task scheduling, and resource migration by generating the global best plans without local convergence. C-PSO with adaptive mutation avoids the deterioration of global optima while finding the best host to place the virtual machine and ensures an appropriate resource allocation plan. By considering the workflow task precedence relationships during C-PSO based task scheduling, the novel hybrid heuristic method efficiently solves the multi-objective combinatorial optimization problem without dominance among the workflow tasks. The Cloudsim based simulation study delivers superior results compared to the existing methods such as the hybrid heuristic workflow scheduling algorithm (HHWS) and distributed dynamic VM management (DDVM). The proposed approach significantly improves the optimal makespan to 38.27% and energy conservation to 38.06% compared to the existing methods.
Read moreIncentives to Apply Green Cloud Computing
In recent years, there have been two major trends in the ICT industry: green computing and cloud computing. Green computing implies that the ICT industry has become a significant energy consumer and consequently, a major source of CO2 emissions. Cloud computing makes it possible to purchase IT resources as a service without upfront costs. In this paper, the combination of these two trends, green cloud computing, will first be evaluated based on existing research findings, which indicate that private clouds are the most green option to offer services. Hosting of private clouds can be outsourced, which allows companies to focus on their core competences. Furthermore, three case studies of state-of-the-art companies offering green hosting services are presented and incentives affecting their energy-efficiency development are analyzed. The results reveal that currently there is no demand in the market for green hosting services, because the only incentive for companies is low costs. Service providers should illustrate their greenness with transparent efficiency metrics, draw up green service level agreements and compete with greenness. Then it is up to end users to require more green web services and create derived demand for green cloud services and green hosting services.
Read moreTask Scheduling and Resource Allocation of Cloud Computing Based on QoS
With the enlargement of the scope of cloud computing application, the number of users and types also increases accordingly, the special demand for cloud computing resources has also improved. Cloud computing task scheduling and resource allocation are key technologies, mainly responsible for assigning user jobs to the appropriate resources to perform. But the existing scheduling algorithm is not fully consider the user demand for resources is different, and not well provided for different users to meet the requirements of its resources. As the demand for quality of service based on cloud computing and cloud computing original scheduling algorithm, the computing power scheduling algorithm is proposed based on the QoS constraints to research the cloud computing task scheduling and resource allocation problems, improving the overall efficiency of cloud computing system.
Read moreReview of Energy Reduction Techniques for Green Cloud Computing
The growth of cloud computing has led to uneconomical energy consumption in data processing, storage, and communications. This is unfriendly to the environment, because of the carbon emissions. Therefore, green IT is required to save the environment. The green cloud computing (GCC) approach is part of green IT; it aims to reduce the carbon footprint of datacenters by reducing their energy consumption. The GCC is a broad and exciting field for research. A plethora of research has emerged aiming to support the GCC vision by improving the utilization of computing resources from different aspects, such as: software optimization, hardware optimization, and network optimization techniques. This paper overviews the approaches to GCC and classifies them. Such a classification assists in comparisons between GCC approaches by identifying the key implementation approaches and the issues related to each.
Read moreHybrid Electro search beetle optimization based task scheduling and game theory SOA based resource allocation in multi cloud computing
The most complicated process in multi‐cloud computing is resource allocation, as it needs to cope with a number of configurations and constraints of cloud providers and customers. At the time of resource allocation, the centralized cloud broker monitors the virtual machines (VM) status, scheduling process, and fitness. However, VM scheduling is found tedious and has received huge attention in business, academia, and research. This enhances the demand for both task scheduling and resource allocation in a multi‐cloud environment. To bridge the gap between the consumer requirement and server infrastructure, a joint optimization‐based resource allocation and task scheduling concept is analyzed in the proposed framework. The first phase introduces the task scheduling mechanism, which uses Hybrid Electro Search and Beetle Swarm Optimization to determine the optimal task for specific VMs. The optimal selection procedure is done by analyzing a multi‐cloud environment's makespan, energy, cost, and throughput parameters. In the second step, an Adaptive Game Theory‐based Seagull optimization approach performs several rounds of reassignment iteratively to minimize the variation in the expected completion time, consequently decreasing high energy consumption and load balancing. The experimental analysis for the proposed model is implemented using Python. The proposed methodology is shown to achieve cheaper costs, shorter waiting times, improved resource allocation, and efficient load balancing. Finally, a comparative analysis is performed with some hybrid optimization models, which illustrate the efficiency of the proposed hybrid optimization model.
Read moreA PERFORMANCE OPTIMIZATION MODEL OF TASK SCHEDULING TOWARDS GREEN CLOUD COMPUTING
Cloud computing becomes a powerful trend in the development of ICT services. It allows dynamic resource scaling from infinite resource pool for supporting Cloud users. Such scenario leads to necessity of larger size of computing infrastructure and increases processing power. Demand on the cloud computing is continually growth that makes it changes to scope of green cloud computing. It aims to reduce energy consumption in Cloud computing while maintaining a better performance. However, there is lack of performance metric that analyzing trade-off between energy consumption and performance. Considering high volume of mixed users’ requirements and diversity of services offered; an appropriate performance model for achieving better balance between Cloud performance and energy consumption is needed. In this work, we focus on green Cloud Computing through scheduling optimization model. Specifically, we investigate a relationship between performance metrics that chosen in scheduling approaches with energy consumption for energy efficiency. Through such relationship, we develop an energy-based performance model that provides a clear picture on parameter selection in scheduling for effective energy management. We believed that better understanding on how to model the scheduling performance will lead to green Cloud computing.
Read moreA Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning
Automatic decision-making approaches, such as reinforcement learning (RL), have been applied to (partially) solve the resource allocation problem adaptively in the cloud computing system. However, a complete cloud resource allocation framework exhibits high dimensions in state and action spaces, which prohibit the usefulness of traditional RL techniques. In addition, high power consumption has become one of the critical concerns in design and control of cloud computing systems, which degrades system reliability and increases cooling cost. An effective dynamic power management (DPM) policy should minimize power consumption while maintaining performance degradation within an acceptable level. Thus, a joint virtual machine (VM) resource allocation and power management framework is critical to the overall cloud computing system. Moreover, novel solution framework is necessary to address the even higher dimensions in state and action spaces. In this paper, we propose a novel hierarchical framework for solving the overall resource allocation and power management problem in cloud computing systems. The proposed hierarchical framework comprises a global tier for VM resource allocation to the servers and a local tier for distributed power management of local servers. The emerging deep reinforcement learning (DRL) technique, which can deal with complicated control problems with large state space, is adopted to solve the global tier problem. Furthermore, an autoencoder and a novel weight sharing structure are adopted to handle the high-dimensional state space and accelerate the convergence speed. On the other hand, the local tier of distributed server power managements comprises an LSTM based workload predictor and a model-free RL based power manager, operating in a distributed manner.
Read moreHybrid resource allocation and task scheduling scheme in cloud computing using optimal clustering techniques
In diverse parallel and distributed computing systems, resource allocation is the progression of distributing consumer tasks for processing elements during execution in which some performance intentions are optimised. This research explains about the innovative resource allocation algorithm for the computing grid environment. In the scheduling problem of independent task in cloud computing, summarise other scheduling algorithms introduce a modified fuzzy C-means (MFCM) clustering algorithm. Our algorithm abstract resource into a model to analyse these characteristics of resources with the MFCM algorithm. From that our proposed technique could decrease a execution time and memory space allocation of the system. For the optimal selection of virtual machines hybrid whale genetic (HWGA) optimisation algorithm is used. Since the virtual machines are optimally selected on the basis of feature values, our proposed method provides reduced load balancing as well as improved parallel execution of tasks.
Read moreEnergy Efficient Task Scheduling to Implement Green Cloud
The research on green cloud computing is indispensable to reduce the emission of carbon as well as to rescue the environment from the hands of global warming, as the climatic change due to the emission of carbon is a critical issue now a days. The green cloud computing mainly concentrates on reducing the amount of cloud resources to condense the emission of carbon. So an effective and efficient scheduling algorithm, Green Task Scheduling (GTS) Algorithm, is proposed in this paper to bring down the amount of cloud resources in use. The proposed algorithm also reduces the hardware cost significantly. As green cloud computing also concentrates to save the energy consumption by providing only the required amount of voltage based on the frequency of work, a technique called as Dynamic Voltage Frequency Scaling (DVFS) is also used in this paper to control the voltage as well as the frequency of the processor without affecting the working performance. The numerical results prove that the makes pan, energy and cost are optimally minimized by implementing GTS along with DVFS in cloud computing environment.
Read moreChaotic symbiotic organisms search for task scheduling optimization on cloud computing environment
Recently, cloud computing have been witnessing high deployment rate of large scale scientific and business applications, this is due to the on-demand provisioning of shared pool of computational resources like networks, storage, and servers, it offers. Each of these applications is made up of various tasks whose execution determine the overall performance of the application. Task scheduling problem on cloud is an NP-hard problem, and thus task scheduling constitute one of the crucial aspects of resources management system in cloud computing, which ensures the attainment of the general user Quality of Service (QoS) performance in terms of response time, total execution time(makespan), throughput among others. In addition, appropriate task scheduling is effective in reducing the operational cost of cloud service providers in terms of energy consumption and resource utilization. This paper focuses on task scheduling problem using a novel Chaotic Symbiotic Organisms Search (CSOS) algorithm to minimize makespan and cost. The main idea is to prevent the premature convergence of SOS at early stages of optimization process by implementing a chaotic map, which enlarges the search space and provides diversity. The performance of the proposed CSOS algorithm is evaluated by extensive simulation using CloudSim toolkit simulation framework and compared with SOS and PSO. Simulation results reveal significant improvement in performance by the proposed CSOS in reducing cost and makespan, in task scheduling.
Read moreA Survey on Task Scheduling and Resource Allocation Methods in Fog Based IoT Applications
With the phenomenal growth of Internet, the technologies associated with the Internet like cloud computing and fog computing also grown massively. At the same time, the issues in this technology also need to be addressed while allocating and deploying resources. Regardless of rapid growth of cloud computing, plenty of issues are there which are necessary to be solved due to ingrained property of cloud computing such as location awareness, unreliable delay and lack of mobility support. Fog computing means bringing the centralized computing cloud resources to the edge of the network. Fog computing copes with the problems in cloud computing by giving flexible resources and services to end users, even cloud computing is more approximately imparting resource dispersed in the core network. The differences between compute-intensive applications and resource-limited gadgets result in restricting the quality of the system. In fog technology, the logical inconsistency should be tackled by task scheduling. Because of the mushrooming of Internet of things, effective means of allocating resource for the clients whenever it is requested is very challenging task in fog technology. This paper gives a survey of different mechanisms proposed for scheduling the tasks and allocating the resources in fog technology. The various techniques proposed by different authors are reviewed in detail. Finally, an analysis is made by comparing the disadvantages of each technique and suggestions are given for betterment in task scheduling and resource allocation in fog effectively.
Read more