- Research Article
7
- 10.1002/cpe.4517
Cloud computing and big data: Technologies and applications
- May 20, 2018
- Concurrency and Computation: Practice and Experience
- Mostapha Zbakh + 3 more +3
Cloud computing and big data: Technologies and applications
Cloud computing is a revolutionary cloud or Internet-based computing that provides services to customers with their on-demand requirements. The service providers who use cloud computing can provide their services to the users simultaneously. Also, cloud computing helps the users to use it like pay-as-you-go service over the internet providing them with uncertainty, dynamism, and elasticity-based services. It is being widely used by many clients in the fields of education, government services, telecommunication, and financial services. Due to the rapid growth of demand for cloud computing services and the heterogeneous nature of the resources for cloud computing, resource scheduling has become a major issue. In recent times, it can be observed that the increase of cloud services is to increase the workload in the cloud environment. The algorithms used for scheduling also need to optimize the main performance indicating parameters such as makespan time, response time, cost, reliability, energy consumption, availability, and resource utilization. Many state-of-the-art scheduling algorithms are proposed in the past to fulfill these requirements. These algorithms are based on heuristic, metaheuristic, and hybrid techniques. In this paper, systematic reviews of these scheduling techniques including the proposed scheduling techniques are provided with their classification.
Cloud computing and big data: Technologies and applications
Cloud computing and big data: Technologies and applications
Classification and Performance Study of Task Scheduling Algorithms in Cloud Computing Environment
Cloud computing is becoming very common in recent years and is growing rapidly due to its attractive benefits and features such as resource pooling, accessibility, availability, scalability, reliability, cost saving, security, flexibility, on-demand services, pay-per-use services, use from anywhere, quality of service, resilience, etc. With this rapid growth of cloud computing, there may exist too many users that require services or need to execute their tasks simultaneously by resources provided by service providers. To get these services with the best performance, and minimum cost, response time, makespan, effective use of resources, etc. an intelligent and efficient task scheduling technique is required and considered as one of the main and essential issues in the cloud computing environment. It is necessary for allocating tasks to the proper cloud resources and optimizing the overall system performance. To this end, researchers put huge efforts to develop several classes of scheduling algorithms to be suitable for the various computing environments and to satisfy the needs of the various types of individuals and organizations. This research article provides a classification of proposed scheduling strategies and developed algorithms in cloud computing environment along with the evaluation of their performance. A comparison of the performance of these algorithms with existing ones is also given. Additionally, the future research work in the reviewed articles (if available) is also pointed out. This research work includes a review of 88 task scheduling algorithms in cloud computing environment distributed over the seven scheduling classes suggested in this study. Each article deals with a novel scheduling technique and the performance improvement it introduces compared with previously existing task scheduling algorithms. Keywords: Cloud computing, Task scheduling, Load balancing, Makespan, Energy-aware, Turnaround time, Response time, Cost of task, QoS, Multi-objective. DOI: 10.7176/IKM/12-5-03 Publication date: September 30 th 2022
Read moreA New Approach to Cloud Resource Scheduling Using Genetic Reinforcement Kernel Optimization and Machine Learning Model
ABSTRACTTwo of the most essential elements of cloud resource management systems are resource management as well as scheduling. Due to the heterogeneity of resources, their interdependencies, and the unpredictable nature of load in a cloud environment, cloud Resource Scheduling (RS) is the most important issue to handle effectively. One of the most challenging tasks in cloud computing is RS, where resources must be assigned to the necessary tasks or jobs in accordance with the necessary Quality of Service (QoS) of the cloud applications. This study suggests a unique method for scheduling cloud resources based on a virtual data center and machine learning model. The genetic reinforcement kernel swarm optimization methodology and cloud data centers are deployed here. The suggested network analysis aims to balance energy usage and SLA. The suggested framework is assessed by doing experiments on the Google Cluster dataset, Planet Lab, and Bitbrains VM traces, as well as three real‐world workload datasets. Execution cost, Execution time, Makespan, Energy consumption, Resource utilization, and Scalability are factors that were examined here. The proposed framework is superior to several performance indicators compared to state‐of‐the‐art techniques. The project's methodology addresses the issues of load imbalance and excessive migration expenses.
Read moreVarious job scheduling algorithms in cloud computing: A survey
Cloud Computing provides a Computing environment where different resources, infrastructures, development platforms and software are delivered as a service to customers virtually on pay per time basis. Low cost, scalability, reliability, utility-based computing are important aspects of cloud computing. Job scheduling is an essential and most important part in any cloud environment. With increasing number of users, Job scheduling becomes a strenuous task. Ordering the jobs by scheduler while maintaining the balance between quality of services (QoS), efficiency and fairness of jobs is quite challenging. Scheduling algorithms are implemented considering parameters such as throughput, resource utilization, latency, cost, priority, computational time, physical distances, performance, bandwidth, resource availability. Though there are different scheduling algorithms available in cloud computing, a very less comparative study has been done on performance of various scheduling algorithms with respect to above mentioned parameters. This paper aims at a comparative study of various types of job scheduling algorithms that provide efficient cloud services.
Read moreA Crowd Sourcing Service Model for Optimizing User-Desired Storage Resource Scheduling
Cloud service, which has changed the business computing pattern, redefines the calculation scope of distributed systems by cloud computing approaches. The existing cloud service model provides different users an advanced on-demand computing model, and then achieves resource interaction between cloud service providers and users by cloud computing. However, this model, working as a centralized service strategy of resource allocation, also generates much scattered space debris resulting in a low efficiency of resource scheduling and utilization. To address this issue, we devise an improved model of crowd sourcing service and propose a technique of storage resource allocation and scheduling, where cloud service providers, as a resource agent, only provide leasing storage space for individual applicants instead of providing storage resources directly. This technique achieves allocation and scheduling of individual storage resources dynamically, according to user's expectations and individual storage providers constraints. The experimental results based on a large data set show that the improved model reduces the storage space of cloud service providers by 88.2% and improves storage space utilization rate of individuals by 65.2%, The improved model reduces the user fees by 64.8% and the cloud services development cost by 78.6% than that of existing models, In addition, the improved model and methods provide higher profits and efficiency for cloud service providers.
Read moreEnergy efficient task scheduling based on deep reinforcement learning in cloud environment: A specialized review
Energy efficient task scheduling based on deep reinforcement learning in cloud environment: A specialized review
An intelligent scheduling algorithm for energy efficiency in cloud environment based on artificial bee colony
The development of Cloud computing applications has initiated a substantial influence on the data center expansion around the world. This expansion has led to an increase in energy consumption within the different data centers. Aware of the environmental and financial impact of the energy efficiency, several research studies have tackled the problem of the energy efficiency at the level of the hardware as well as scheduling algorithms. However, the majority of the proposed solutions such as Dynamic Voltage and Frequency Scaling (DVFS) have demonstrated a negative effect on response time where most Cloud applications required a high responsiveness. This paper suggests an energy efficiency solution for Cloud computing by applying a fast and smart algorithm. Firstly, the paper provides an evaluation of the energy efficiency within the Cloud environment through the Taguchi experience plan. Secondly, the article introduces a bee colony scheduling algorithm which aims to optimize the energy efficiency while guaranteeing an optimum response time. The validation results acquired from the GreenCloud simulator emphasize the effectiveness of the suggested scheduling analysis methodology.
Read moreModified Sunflower Optimization Algorithm for Task Scheduling in Cloud Computing
Objective: To develop a task scheduling algorithm that efficiently approximates solutions for the multi-objective task scheduling problem in a cloud environment, optimizing resource utilization, execution time, cost, and overall system performance. Method: A Modified Sunflower Optimization Algorithm for Task Scheduling (MSOTS) is proposed to improve efficiency in cloud environments. The traditional sunflower optimization algorithm is enhanced with Levy flight, crossover, and mutation operations to achieve a better balance between exploration and exploitation while preventing entrapment in local minima. These enhancements help improve convergence speed and solution quality. CloudSim is utilized for comprehensive performance evaluation, comparing MSOTS with existing algorithms in terms of execution time, cost, and resource utilization. The randomly generated dataset (500 tasks and 50 VMs) is used for analyze the performance of the MSOTS. Findings: The results demonstrate that MSOA significantly reduces makespan and cost while enhancing resource utilization. Specifically, the proposed method reduces makespan by 22.43% compared to PSO and 14.95% compared to SOA. Additionally, cost is reduced by 14.47% and 10.38% compared to PSO and SOA, respectively, while resource utilization increases by 3.89% and 2.51% over these methods. These findings indicate that MSOA effectively improves task scheduling performance in terms of makespan, cost efficiency, and resource utilization, making it a promising approach for cloud computing environments. Novelty: The MSOA is integrated with single-point crossover, swap mutation, and Lévy flight to update the solution, which prevents entrapment in local minima and premature convergence and enhances task scheduling efficiency, optimizes resource allocation, minimizes execution time, and reduces overall costs in cloud computing environments. Keywords: Meta-heuristic, Sunflower optimization, Levy flight, Crossover, Mutation
Read moreDesign of Virtual Machine Scheduling Algorithm in Cloud Computing Environment
With the application of cloud computing services in more and more fields, it will undertake more computing tasks and storage tasks. The problem of high energy consumption in data centers will become more serious. Virtualization technology is very important in cloud computing, which can improve the utilization rate of resources. At the same time, it has flexibility in resource scheduling and can integrate multiple virtual machines to achieve power efficiency. Using online virtual machine migration technology for energy-saving planning in cloud environment is a hot research topic in academic circles. The scheduling strategy proposed in this paper can reduce the server downtime and the number of server hosts, so as to achieve the maximum use of resources. The main research work of this paper includes the following aspects: Firstly, the energy consumption in the process of virtual machine energy-saving design is modeled, and the relationship between energy consumption and resource usage under different load conditions is analyzed, and the problems are abstracted, e.g., packaging box problem. Secondly, this paper uses genetic algorithm to solve the problem of high energy consumption. Finally, based on the target allocation scheme of virtual machines obtained by the above method, the migration problem of virtual machines is abstracted as the problem of finding the maximum weighted independent set of graphs. And a greedy algorithm is designed to solve this problem. In this paper, CloudSim simulation platform is used to verify the effectiveness of the proposed algorithm. Experiments show that the proposed algorithm can reduce data energy consumption and avoid frequent migration of virtual machines.
Read moreExecution Analysis of Load Balancing Algorithms in Cloud Computing Environment
The concept oft Cloud computing has significantly changed the field of parallel and distributed computing systems today. Cloud computing enables a wide range of users to access distributed, scalable, virtualized hardware and/or software infrastructure over the Internet. Load balancing is a methodology to distribute workload across multiple computers, or other resources over the network links to achieve optimal resource utilization, maximize throughput, minimum response time, and avoid overload. With recent advent of technology, resource control or load balancing in cloud computing is main challenging issue. A few existing scheduling algorithms can maintain load balancing and provide better strategies through efficient job scheduling and resource allocation techniques as well. In order to gain maximum profits with optimized load balancing algorithms, it is necessary to utilize resources efficiently. This paper presents a review of a few load balancing algorithms or technique in cloud computing. The objective of this paper is to identify qualitative components for simulation in cloud environment and then based on these components, execution analysis of load balancing algorithms are also presented.
Read moreAn Energy and Performance Aware Scheduler for Real-Time Tasks in Cloud Datacentres
Datacentres provide the foundations for cloud computing, but require large amounts of electricity for their operation. Approaches that promise to reduce power use by minimizing execution time, for example using different scheduling and resource management techniques, are discussed in the literature. This paper summarizes some of the most important scheduling techniques in clouds focusing on power consumption, covering VM-level, host-level and task-level scheduling where the most promising approach is task level scheduling, with energy savings by means of load filtering, consolidation, adapted CPU throughput, or host power control. We explore use of the rate monotonic (RM) and backfilling algorithms for real-time task scheduling in cloud environment because RM is the simplest fixed priority scheduling technique, and thus the choice for modern real-time systems, and prior uses of RM in task scheduling have demonstrated power efficiency with optimal results. We specifically consider deadline-based tasks scheduling for real-time clouds which, to the best of our knowledge, has not been employed previously. RM with backfilling is experimentally evaluated and results show that, compared to the classical algorithms, all tasks were scheduled with minimum power consumption (5.5% – 29.3%), on minimum resources (3.9% – 25.2% less) while majority were meeting their deadlines (93.21% – 94.7%). The approach can guarantee deadline oriented Software as a Service (SaaS) in cloud if arrival rate i.e. network transfer time can be estimated in advance. We subsequently provided an extension of the proposed approach to task-based load balancing for almost balanced resource utilization and approximately 1.0% to 1.6% energy efficiency.
Read moreHybrid Evolutionary Algorithm based Task Scheduling Mechanism for Resource Allocation in Cloud Environment
Distributed computing, grid computing, and virtualization have all resulted in the growth of cloud computing.. The existing research work, introduced a strategic theory (ST) based Improved Elephant Herd Optimization (IEHO) for feasible allotment procedure where the result function confess a straight payment system to meet the users requirements. However total users is huge in cloud computing, total tasks and total data are as well vast. Because the cost of every work on cloud services varies, client task scheduling in the cloud is not like conventional scheduling approaches. In a cloud computing system, it's critical to figure out how to effectively schedule work. As a results, the research proposes an unique Task Scheduling Mechanism (TSM) was proposed, which may suit users' needs while also improving resource usage and so improving the overall performance of the cloud cloud computing. The goal of this project is to plan task groups in a cloud computing platform with varying resource costs and computational performance. The suggested cloud scheduling solution uses an enhanced cost-based scheduling scheme to efficiently map workloads to existing cloud services. The scheduling technique, that is founded on Hybrid Evolutionary (HE)algorithms, increases the computation/communication proportion by combining user jobs as per a cloud resource's processing capacity and sending the aggregated activities to the resource. The simulation experiments show that the proposed Hybrid Evolutionary based Task Scheduling Mechanism (HE-TSM) is efficiently schedule the tasks along with both costs and performance in the cloud computing environment.
Read moreDEEP LEARNING BASED LOAD BALANCING USING MULTIDIMENSIONAL QUEUING LOAD OPTIMIZATION ALGORITHM FOR CLOUD ENVIRONMENT.
Cloud computing becoming one of the most advanced and promising technologies in these days for information technology era. It has also helped to reduce the cost of small and medium enterprises based on cloud provider services. Resource scheduling with load balancing is one of the primary and most important goals of the cloud computing scheduling process. Resource scheduling in cloud is a non-deterministic problem and is responsible for assigning tasks to virtual machines (VMs) by a servers or service providers in a way that increases the resource utilization and performance, reduces response time, and keeps the whole system balanced. So in this paper, we presented a model deep learning based resource scheduling and load balancing using multidimensional queuing load optimization (MQLO) algorithm with the concept of for cloud environment Multidimensional Resource Scheduling and Queuing Network (MRSQN) is used to detect the overloaded server and migrate them to VMs. Here, ANN is used as deep learning concept as a classifier that helps to identify the overloaded or under loaded servers or VMs and balanced them based on their basis parameters such as CPU, memory and bandwidth. In particular, the proposed ANN-based MQLO algorithm has improved the response time as well success rate. The simulation results show that the proposed ANN-based MQLO algorithm has improved the response time compared to the existing algorithms in terms of Average Success Rate, Resource Scheduling Efficiency, Energy Consumption and Response Time.
Read moreImproved Task Scheduling Strategy Using Reinforcement Learning in Cloud Environment
Scheduling has recently been incredibly helpful in cloud computing due to the shift in resource consumption. Cloud provides many services (based on dynamism, elasticity, and uncertainty) to end-users based on their requirements which the users can access at any time, irrespective of the location, by paying for that service. The load on the cloud environment increases as the demand for applications in cloud services rises. The degradation in services (overuse of services) or the wastage of resources (underutilization) results from improper scheduling. The resources can be distributed appropriately to the different natured tasks through scheduling. The problem of resource imbalance can be avoided, and the optimization of the main execution factors, like availability, makespan time, utilization of resources, energy consumption, reliability, response time, etc., can also be optimized. Various algorithms like Heuristic, Meta-heuristic, and hybrid are proposed to justify the scheduling mentioned above. The proposed VTO-QABC is implemented and compared on the parameter throughput with different strategies. A significant improvement is found as compared to Max-Min(84.51%), MOPSO (37.82%), HABC_LJF (19.85%), Q-Learning (7.72%), VTO-QABC_FCFS(3.89%), VTOABC_LJF (3.89%) less time than MOCS.
Read moreA Systematic Approach to Enhancing Cloud Performance Using GA-Driven Adaptive Resource Allocation Strategies
In the last ten years, cloud computing has become a prominent a study providing scalable computing power through the Internet. As a consequence of extensive adoption and simultaneous increase in demand, a cloud environment often experiences burst loads. Therefore algorithms must be devised for efficient load balancing that works in real time while dynamically adjusting itself to any fluctuations in workload. Of these techniques, Genetic Algorithms (GAs) inspired by the processes of natural evolution have become prominent. Such algorithms use the process of natural selection to generate, combine and improve solutions for performance optimization. Researchers are continuously investigating these different GA-based techniques in order to assess their effectiveness, pinpoint the drawbacks, and assess their influence on key performance indicators like response time, resource utilization, energy consumption, and overall system throughput. Within this scope, an enhanced and new hybrid model combining Adaptive Resource Allocation with Genetic Algorithm has been proposed for better load balancing
Read more