Comparative Analysis of Energy-Efficient Scheduling Algorithms for Big Data Applications

Cloud computing and big data: Technologies and applications

Task scheduling optimization in heterogeneous cloud computing environments: A hybrid GA-GWO approach

To manage the huge data, cloud computing necessitates a significant number of disc I/O, network bandwidth and CPU cycles. To handle the massive volumes of data, the programming paradigm known as data flow integrates the tasks to a graph structure. Task scheduling is one of the main areas of current study that primarily aims to allocate the tasks to resources. It is essential to organise the tasks in a way which saves time and makes the most use of available resources. In recent years, scholars have offered different job scheduling techniques. This work intends to introduce a novel cost minimisation for big data processing with respect to task allocation or scheduling. Initially, the big data is processed under Map Reduce framework. This allocation processes is taken as the optimisation issue, which is solved under the consideration of minimisation factors like execution cost, makespan, communication cost, and energy consumption and risk. For solving this issue, a new hybrid optimisation algorithm known as Hybrid bald eagle and Archimedes Optimisation algorithm (HBE-ArCA) is developed. The betterment of HBE-ArCA is proven regarding cost, makespan and so on.

Cloud computing provides computing resources like software and hardware as a service by the network for several users. Task scheduling is one of the main problems to attain cost-effective execution. The main purpose of task scheduling is to allocate tasks to resources so that it can optimize one or more criteria. Since the problem of task scheduling is one of the Nondeterministic Polynomial-time (NP)-hard problems, meta-heuristic algorithms have been widely employed for solving task scheduling problems. One of the new bio-inspired meta-algorithms is Seagull Optimization Algorithm (SOA). In this paper, we proposed an energy-aware and cost-efficient SOA-based Task Scheduling (SOATS) algorithm. The aims of proposed algorithm to make a trade-off between five objectives (i.e., energy consumption, makespan, cost, waiting time, and load balancing) using a fewer number of iterations. The experiment results by comparing with several meta-heuristic algorithms (i.e., Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Ant Colony Optimization (ACO), and Whale Optimization Algorithm (WOA)) prove that the proposed technique performs better in solving task scheduling problem. Moreover, we compared the proposed algorithm with well-known scheduling methods: Cost-based Job Scheduling (CJS), Moth Search Algorithm based Differential Evolution (MSDE), and Fuzzy-GA (FUGE). In the heavily loaded environment, the SOATS algorithm improved energy consumption and cost saving by 10 and 25%, respectively.

In a mobile crowd sensing system, a smartphone undertakes many different sensing tasks that demand data from various sensors. In this paper, we consider the problem of scheduling different sensing tasks assigned to a smartphone with the objective of minimizing sensing energy consumption while ensuring Quality of SenSing (QoSS). First, we consider a simple case in which each sensing task only requests data from a single sensor. We formally define the corresponding problem as the Minimum Energy Single-sensor task Scheduling (MESS) problem and present a polynomial-time optimal algorithm to solve it. Furthermore, we address a more general case in which some sensing tasks request multiple sensors to report their measurements simultaneously. We present an Integer Linear Programming (ILP) formulation as well as an effective polynomial-time heuristic algorithm, for the corresponding Minimum Energy Multi-sensor task Scheduling (MEMS) problem. Extensive simulation results show that the proposed algorithms achieve over 79% energy savings on average compared to a widely-used baseline approach, and moreover, the proposed heuristic algorithm produces close-to-optimal solutions.

In cloud computing, optimizing task scheduling is crucial for improving overall system performance and resource utilization. To minimize cloud service costs and prevent resource wastage, advanced techniques must be employed to efficiently allocate cloud resources for executing tasks. This research presents a novel multi-objective task scheduling method, BSSA, which combines the Backtracking Search Optimization Algorithm (BSA) and the Sparrow Search Algorithm (SSA). BSA enhances SSA’s convergence accuracy and global optimization ability in later iterations, improving task scheduling results. The proposed BSSA is evaluated and compared against traditional SSA and other algorithms using a set of 8 benchmark test functions. Moreover, BSSA is tested for task scheduling in cloud environments and compared with various metaheuristic scheduling algorithms. Experimental results demonstrate the superiority of the proposed BSSA, validating its effectiveness and efficiency in cloud task scheduling.

MapReduce is now a significant parallel processing model for large-scale data-intensive applications using clusters with commodity hardware. Scheduling of jobs and tasks, and identification of TaskTrackers which are slow in Hadoop clusters are the focus research in the recent years. MapReduce performance is currently limited by its default scheduler, which does not adapt well in heterogeneous environments. In this paper, we propose a scheduling method to identify the TaskTrackers which are running slowly in map and reduce phases of the MapReduce framework in a heterogeneous Hadoop cluster. The proposed method is integrated with the MapReduce default scheduling algorithm. The performance of this method is compared with the unmodified MapReduce default scheduler. We observe that the proposed approach shows improvements in performance to the default scheduler in the heterogeneous environments. Performance improvement was observed as the overall job execution times for different workloads from HiBench benchmark suite were reduced.

Objective: To develop a task scheduling algorithm that efficiently approximates solutions for the multi-objective task scheduling problem in a cloud environment, optimizing resource utilization, execution time, cost, and overall system performance. Method: A Modified Sunflower Optimization Algorithm for Task Scheduling (MSOTS) is proposed to improve efficiency in cloud environments. The traditional sunflower optimization algorithm is enhanced with Levy flight, crossover, and mutation operations to achieve a better balance between exploration and exploitation while preventing entrapment in local minima. These enhancements help improve convergence speed and solution quality. CloudSim is utilized for comprehensive performance evaluation, comparing MSOTS with existing algorithms in terms of execution time, cost, and resource utilization. The randomly generated dataset (500 tasks and 50 VMs) is used for analyze the performance of the MSOTS. Findings: The results demonstrate that MSOA significantly reduces makespan and cost while enhancing resource utilization. Specifically, the proposed method reduces makespan by 22.43% compared to PSO and 14.95% compared to SOA. Additionally, cost is reduced by 14.47% and 10.38% compared to PSO and SOA, respectively, while resource utilization increases by 3.89% and 2.51% over these methods. These findings indicate that MSOA effectively improves task scheduling performance in terms of makespan, cost efficiency, and resource utilization, making it a promising approach for cloud computing environments. Novelty: The MSOA is integrated with single-point crossover, swap mutation, and Lévy flight to update the solution, which prevents entrapment in local minima and premature convergence and enhances task scheduling efficiency, optimizes resource allocation, minimizes execution time, and reduces overall costs in cloud computing environments. Keywords: Meta-heuristic, Sunflower optimization, Levy flight, Crossover, Mutation

In the past few years, with the rapid development of heterogeneous computing systems (HCS), the issue of energy consumption has attracted a great deal of attention. How to reduce energy consumption is currently a critical issue in designing HCS. In response to this challenge, many energy-aware scheduling algorithms have been developed primarily using the dynamic voltage-frequency scaling (DVFS) capability which has been incorporated into recent commodity processors. However, these techniques are unsatisfactory in minimizing both schedule length and energy consumption. Furthermore, most algorithms schedule tasks according to their average-case execution times and do not consider task execution times with probability distributions in the real-world. In realizing this, we study the problem of scheduling a bag-of-tasks (BoT) application, made of a collection of independent stochastic tasks with normal distributions of task execution times, on a heterogeneous platform with deadline and energy consumption budget constraints. We build execution time and energy consumption models for stochastic tasks on a single processor. We derive the expected value and variance of schedule length on HCS by Clark's equations. We formulate our stochastic task scheduling problem as a linear programming problem, in which we maximize the weighted probability of combined schedule length and energy consumption metric under deadline and energy consumption budget constraints. We propose a heuristic energy-aware stochastic task scheduling algorithm called ESTS to solve this problem. Our algorithm can achieve high scheduling performance for BoT applications with low time complexity $O(n(M+\log n))$ , where $n$ is the number of tasks and $M$ is the total number of processor frequencies. Our extensive simulations for performance evaluation based on randomly generated stochastic applications and real-world applications clearly demonstrate that our proposed heuristic algorithm can improve the weighted probability that both the deadline and the energy consumption budget constraints can be met, and has the capability of balancing between schedule length and energy consumption.

Task scheduling is an essential component of any distributed system because it routes tasks to appropriate resources for execution, such as grids, clouds, and peer-to-peer networks. Common scheduling algorithms include downsides, such as high temporal complexity, non-simultaneous processing of input tasks, and longer program execution times. Exploration-based scheduling algorithms prioritize tasks using a variety of methods, resulting in long execution times on heterogeneous distributed computing systems. As a result, task prioritization becomes a bottleneck in such systems. It is appropriate to prioritize tasks with the shortest execution time using faster algorithms. The genetic algorithm (GA) is one of the evolutionary approaches used to solve complex problems quickly. This paper proposes a parallel GA with a MapReduce architecture for scheduling jobs on cloud computing with various priority queues. The fundamental aim of this study is to employ a MapReduce architecture to minimize the total execution time of the task scheduling process in the cloud computing environment. The proposed method accomplishes task scheduling in two stages: first, the GA was used in conjunction with heuristic techniques to assign tasks to processors, and then the GA was used in conjunction with the MapReduce framework to assign jobs to processors. In our experiments, we consider heterogeneous resources that differ in their ability to execute various tasks, as well as running a job on different resources with varying execution durations. The results show that the proposed method outperforms other algorithms such as particle swarm optimization, whale optimization algorithm, moth-flame optimization, and intelligent water drops.

Big data analytics in business intelligence do not provide effective data retrieval methods and job scheduling that will cause execution inefficiency and low system throughput. This paper aims to enhance the capability of data retrieval and job scheduling to speed up the operation of big data analytics to overcome inefficiency and low throughput problems. First, integrating stacked sparse autoencoder and Elasticsearch indexing explored fast data searching and distributed indexing, which reduces the search scope of the database and dramatically speeds up data searching. Next, exploiting a deep neural network to predict the approximate execution time of a job gives prioritized job scheduling based on the shortest job first, which reduces the average waiting time of job execution. As a result, the proposed data retrieval approach outperforms the previous method using a deep autoencoder and Solr indexing, significantly improving the speed of data retrieval up to 53% and increasing system throughput by 53%. On the other hand, the proposed job scheduling algorithm defeats both first-in-first-out and memory-sensitive heterogeneous early finish time scheduling algorithms, effectively shortening the average waiting time up to 5% and average weighted turnaround time by 19%, respectively.

Objectives: In this paper various task scheduling algorithm along with load distribution techniques investigated to ensure efficient mapping of tasks to resources and for coherent resource utilization in heterogeneous environment. Statistical Analysis: A detailed comparative approach is conferred for different job scheduling and load balancing algorithm available in the literature. Considered parameters for comparisons are scheduling approaches, techniques, findings, benefits, pros and cons. This study facilitates to identify the efficient scheduling and load balancing approach which optimizes resource use, minimizes completion time as well as task waiting time and balanced load among resources. Findings: Make span time and flow time are two important parameters to compute and minimize job execution time. This paper explores an intense survey of minimization of task execution time with minimum make span time and flow time. Apart from this a joint review on job scheduling, load balancing and minimizing job completion time using Expected Time to Compute (ETC) matrix is presented. At the end of this paper a research model is proposed which is a novel and hybrid approach with a combination of the efficiency of scheduling and load balancing. Application: The Load Balancing (LB) is an important feature of the grid environment. The efficient LB algorithm assures cost effective resource usage by the provisioning of resources to users on demand schedule in pay-as-you-say-manner.

Task scheduling is a NP-hard problem and also an integral part of parallel and distributed computing.It becomes more complicated especially in the grid computing environment.An optimal task scheduling model and an algorithm were brought forward,which combined the advantages of immune clonal algorithm and simulated annealing.The simulation results show that this algorithm achieves resource load balancing,and it overcomes the shortcomings of genetic algorithm,and can be applied to the optimization of task scheduling successfully.

Because of the rapid rise in the count of corporations using cloud-dependent infrastructure as the foundation for big data storing and analysis. The fundamental difficulty in scheduling big data services in cloud-dependent systems is ensuring the shortest possible makespan while simultaneously reducing the number of resources being used. We have created a new, secure map reduce scheduling method that functions as follows. Initially, the cloud architecture is designed and the tasks are generated. In the pre-processing phase, the huge set of tasks was processed by the map-reduce scheduling framework. Afterward, the optimal task scheduling task is conducted which utilizes a hybrid algorithm named Tunicate Combined Moth Flame Algorithm (TCMFA), which provides better task scheduling via providing optimal makespan, execution time, and security. This proposed TCMFA is the hybridization of both Moth Flame Optimization (MFO) and Tunicate Swarm Algorithm (TSA). The error rate of the TCMFA gets reduced to 320 approximately over other conventional methods such as RSA, ACO, GHO, BTS, OWPSO, BES, PRO, SOA, COOT, TSA & MFO which proves the accuracy of our TCMFA and makes it more efficient and secure for optimal map-reduce scheduling.

IoT cloud computing provides all functions of traditional computing as services through the Internet for the users. Big data processing is one of the most crucial advantages of IoT cloud computing. However, IoT cloud job scheduling is considered an NP-hard problem due to the hardness of allocating the clients’ jobs to suitable IoT cloud provider resources. Previous work on job scheduling tried to minimize the execution time of the job scheduling in the IoT cloud, but it still needs improvement. This paper proposes an enhanced job scheduling mechanism using cat swarm optimization (CSO) with modified distance to minimize the execution time. The proposed job scheduling mechanism first creates a set of jobs and resources to generate the population by randomly assigning the jobs to resources. Then, it evaluates the population using the fitness value, which represents the execution time of the jobs. In addition, we use iterations to regenerate populations based on the cat’s behaviour to produce the best job schedule that gives the minimum execution time for the jobs. We evaluated the proposed mechanism by implementing an initial simulation using Java Language and then conducted a complete simulation using the CloudSim simulator. We ran several experimentation scenarios using different numbers of jobs and resources to evaluate the proposed mechanism regarding the execution time. The proposed mechanism significantly reduces the execution time when we compare the proposed mechanism against the firefly algorithm and glowworm swarm optimization. The average execution time of the proposed cat swarm optimization was 131, while the average execution times for the firefly algorithm and glowworm optimization were 237 and 220, respectively. Hence, the experimental findings demonstrated that the proposed mechanism performs better than the firefly algorithm and glowworm swarm optimization in reducing the execution time of the jobs.