Uncertainty-Aware Online Scheduling for Real-Time Workflows in Cloud Service Environment

With the vigorous development of Internet technology, applications are increasingly migrating to the cloud. Cloud, a distributed network environment, has been widely extended to many fields such as digital finance, supply chain management, and biomedicine. In order to meet the needs of the rapid development of the modern biomedical industry, the biological cloud platform is an inevitable choice for the integration and analysis of medical information. It improves the work efficiency of the biological information system and also realizes reliable and credible intelligent processing of biological resources. Cloud services in bioinformatics are mainly for the processing of biological data, such as the analysis and processing of genes, the testing and detection of human tissues and organs, and the storage and transportation of vaccines. Biomedical companies form a data chain on the cloud, and they provide services and transfer data to each other to create composite services. Therefore, our motivation is to improve process efficiency of biological cloud services. Users’ business requirements have become complicated and diversified, which puts forward higher requirements for service scheduling strategies in cloud computing platforms. In addition, deep reinforcement learning shows strong perception and continuous decision-making capabilities in automatic control problems, which provides a new idea and method for solving the service scheduling and resource allocation problems in the cloud computing field. Therefore, this paper designs a composite service scheduling model under the containers instance mode which hybrids reservation and on-demand. The containers in the cluster are divided into two instance modes: reservation and on-demand. A composite service is described as a three-level structure: a composite service consists of multiple services, and a service consists of multiple service instances, where the service instance is the minimum scheduling unit. In addition, an improved Deep Q-Network (DQN) algorithm is proposed and applied to the scheduling algorithm of composite services. The experimental results show that applying our improved DQN algorithm to the composite services scheduling problem in the container cloud environment can effectively reduce the completion time of the composite services. Meanwhile, the method improves Quality of Service (QoS) and resource utilization in the container cloud environment.

List based scheduling algorithms have been proven an optimistic strategy with a shorter response time to generate feasible solutions for the workflow scheduling problem. Data-intensive and computation-intensive workflow applications have different characteristics in terms of the ratio between data transfer time and task execution time. Workflow scheduling algorithms in a cloud-based environment should adequately consider the characteristics of the underlying cloud platform such as the on-demand resource provisioning strategy, the practically unlimited compute capacities, the booting times of virtual machines, the homogeneous network and the pay-as-you-go price model to produce an optimal scheduling solution within the deadline constraint of a given workflow. In this paper, a path based scheduling algorithm, named LPOD, is proposed to find the best workflow schedule solution with minimum monetary cost in a cloud computing environment. A series of case studies have been carefully conducted using synthetic workflows based on DATAVIEW, which is a popular open-source big data workflow management system. The experimental results show that the proposed algorithm is efficient and can generate better workflow schedules than the state-of-the-art algorithms such as IC-PCP and SGX-E2C2D.

U-shaped Line (UL) utilizes many manufacturing industries. It applies the just-in-time principle, the lean manufacture, and the quality improvement. It reduces the waste in the manufacturing operation. This paper presents two UL models which consist of eight certain tasks and three uncertain tasks. Two UL models include UL No.1 and UL No.2. The uncertain tasks assume the time distribution that is discrete. The uncertain task time of UL No.1 classifies to three types. The uncertain task time of UL No.2 classifies to five types. The objective function of two UL models is the minimum workstations. Two UL models formulate on a zero-one integer linear programming. This academic work applies MATLAB R2014a which solves the solution of two models. The procedure is run by INTLINPROG toolbox. The solution of the uncertain tasks for UL No.1 is the optimistic time. The solution of the uncertain tasks for UL No.2 is the most optimistic time. The number of workstations of UL No.2 is less than the number of workstations of UL No.1. The most early finish tasks which assign along the UL line affect the minimum workstations.

Due to the high growth of SNS population, service scalability is one of the critical issues to be addressed. The cloud environment provides the flexible computing and storage resources for services deployment, which fits the characteristics of scalable SNS deployment. However, if the SNS related information is not properly placed, it will cause unbalance load and heavy transmission cost on the storage virtual machine (VM) and cloud data center (CDC) network. In this paper, we characterize the SNS into a graph model based on the users associations and interest correlations. The node weight represents the degree of associations, which can be indexed by the number of friends or data sources, and the link weight denotes the correlation between users/data sources. Then, based on the SNS graph, the two-step algorithm is proposed in this paper to determine the placement of SNS related data among VMs. Two k-means based clustering schemes are proposed to allocate social data in proper VM and physical servers for pre-configured VM and dynamic VM environment, respectively. The experimental example was conducted and to illustrate and compare the performance of the proposed schemes.

Workflow applications comprise a number of structured tasks and computations featuring application services to be executed and the dependencies between these services. This paper deals with the problem of scheduling workflow applications, represented by directed acyclic graphs, on a set of virtualized servers. Each server hosts multiple virtual machines. Virtual machines sharing a host can communicate with each other, and with virtual machines hosted in different servers. The aim is to partition the application services and distribute each partition among the virtual machines in such a way that the dependencies are respected, the response time is minimized improving the quality of service and the intra- and inter-virtual machine communications are minimized. We model this problem as a workflow scheduling problem with hierarchical communications. The main contribution is to provide an evolutionary-based scheduling algorithm that considers this model when scheduling the applications. Simulation results demonstrate the effectiveness of the provided algorithm when compared with a related approach on a set of real-world applications emphasizing the interest of the approach.

scientific workflows are suitable models in many applications for recognition and execution of tasks in parallel, especially in the Cloud. Different aspects of resource provisioning and the resource pricing model in the cloud environment cause the scheduling problem of workflow very complex. One of the main aims of the scheduling algorithms is to satisfy the users’ different quality of service requirements. Communication delay between tasks is an important affecting factor in optimal scheduling of workflows. It can also highly increase the cost of workflow scheduling if proper actions are not taken. To solve this problem, we propose a tasks duplication-based list scheduling algorithm called Communication-Critical Task Duplication (CCTD). We first define the concept of communication critical task (CCT) for a workflow. Then, by presenting a ranking-based approach, we identify communication critical tasks in a workflow as well as duplicating candidates. Task duplication in idle time slots of leased virtual machines which their children tasks are mapped to. This idea, while eliminating the cost and time of data transfer between parent-child tasks, reduces the time of execution of tasks and effectively uses leased time intervals of resources. According to the proposed scheduling algorithm, a new heuristic method has been proposed for the budget distribution. This method distribute overall budget to tasks in proportional to the workload and duplication rank of each task. The proposed algorithm was evaluated and verified using four well-known scientific workflows. The simulation results show that the CCTD algorithm, while respecting the user budget constraint, reduces the workflow overall completion time.

Workflow scheduling is a long-studied problem in parallel and distributed computing (PDC), aiming at efficiently utilizing compute resources to meet user's service requirements. Recently proposed scheduling methods leverage the low response times of edge computing platforms to optimize application Quality of Service (QoS). However, scheduling workflow applications in mobile edge-cloud systems is challenging due to computational heterogeneity, changing latencies of mobile devices and the volatile nature of workload resource requirements. To overcome these difficulties it is important, but at the same time challenging, to develop a long-sighted optimization scheme with efficient modelling of the QoS objectives. In this work, we propose MCDS: Monte Carlo Learning using Deep Surrogate Models to efficiently schedule workflow applications in mobile edge-cloud computing systems. MCDS is an Artificial Intelligence (AI) based scheduling approach that uses a tree-based search strategy and a deep neural network based surrogate model to estimate the long-term QoS impact of immediate actions for robust optimization of scheduling decisions. Experiments on physical and simulated edge-cloud testbeds show that MCDS can improve over the state-of-the-art methods in terms of energy consumption, response time, SLA violations and cost by at least 6.13, 4.56, 45.09 and 30.71 percent respectively.

Resources Allocation in Virtualized Systems Based on Try- Before-Buy Approach

Cloud computing platforms have been extensively using scientific workflows to execute large-scale applications. However, multiobjective workflow scheduling with scientific standards to optimize QoS parameters is a challenging task. Various metaheuristic scheduling techniques have been proposed to satisfy the QoS parameters like makespan, cost, and resource utilization. Still, traditional metaheuristic approaches are incompetent to maintain agreeable equilibrium between exploration and exploitation of the search space because of their limitations like getting trapped in local optimum value at later evolution stages and higher-dimensional nonlinear optimization problem. This paper proposes an improved Fruit Fly Optimization (IFFO) algorithm to minimize makespan and cost for scheduling multiple workflows in the cloud computing environment. The proposed algorithm is evaluated using CloudSim for scheduling multiple workflows. The comparative results depict that the proposed algorithm IFFO outperforms FFO, PSO, and GA.

Resource provisioning and scheduling are essential challenges in handling multiple workflow requests within cloud environments, particularly given the constraints imposed by limited resource availability. Although workflow scheduling has been extensively studied, few methods effectively integrate resource provisioning with scheduling, especially under cloud resource limitations and the complexities of multiple workflows. To address this challenge, we propose an innovative two-stage genetic-based optimization approach. In the first stage, candidate cloud resources are selected for the resource pool under the given resource constraints. In the second stage, these resources are provisioned and task scheduling is optimized on the selected resources. A key advantage of our approach is that it reduces the search space in the first stage through a novel encoding scheme that enables a caching strategy, in which intermediate results are stored and reused to enhance optimization efficiency in the second stage. The proposed solution is evaluated through extensive simulation experiments, assessing both resource selection and task scheduling across a diverse range of workflows. The results demonstrate that the proposed approach outperforms existing algorithms, particularly for highly parallel workflows, highlighting its effectiveness in managing complex workflow scheduling under resource-constrained cloud environments.

Scheduling large-scale and resource-intensive workflows in cloud infrastructure is one of the main challenges for cloud service providers (CSPs). Cloud infrastructure is more efficient when virtual machines and other resources work up to their full potential. The main factor that influences the quality of cloud services is the distribution of workflow on virtual machines (VMs). Scheduling tasks to VMs depends on the type of workflow and mechanism of resource allocation. Scientific workflows include large-scale data transfer and consume intensive resources of cloud infrastructures. Therefore, scheduling of tasks from scientific workflows on VMs requires efficient and optimized workflow scheduling techniques. This paper proposes an optimised workflow scheduling approach that aims to improve the utilization of cloud resources without increasing execution time and execution cost.

The prime motive of this work is to develop an efficacious system for workflow scheduling in the cloud using optimized deep learning. Initially, the incoming tasks are fed into a multi-objective task scheduling module where the workflow is scheduled using proposed HMLO by considering the fitness, such as cost, predicted energy, reliability and make span. Here, the energy is predicted using DRN. The HMLO is obtained by the consolidation of Mutated Leader Optimization (MLO), and Hybrid Leader Optimization (HLBO). On the other hand, the deep learning-based task scheduling process is performed by getting the incoming tasks and it is given to DLSTM for workflow scheduling by assuming the live parameters of Virtual Machine (VM) and task. Finally, both outputs are fused to schedule the tasks in the VM. The proposed HMLO[Formula: see text]DLSTM has delivered superior results in terms of minimum make span of 0.499, maximum energy of 0.947 and minimum resource utilization of 0.079.

A multi-hierarchy particle swarm optimization-based algorithm for cloud workflow scheduling

Load balancing in cloud computing assigning the all load over each node of a mutual or shared system to make resource usage effective and to perform the response time to the job. This has used Round Robin (RR) and shortest job first (SJF) algorithm. Scheduling is the main part of cloud computing and jobs or workflow scheduling under the platform as a service. The work under infrastructure as a service work to virtual machine scheduling which machine is decided by scheduler should go on which job or virtual machine. Simply RR is a time quantum-based preemptive algorithm, and SJF is a burst time-based non-preemptive and preemptive algorithm. But RR work in two-phase like expected finish time adapted for each process. Each process with generally minimum completion times scheduled. This algorithm result as no longer waiting for the smallest task.

Improving reliability is one of the major concerns of scientific workflow scheduling in clouds. The ever-growing computational complexity and data size of workflows present challenges to fault-tolerant workflow scheduling. Therefore, it is essential to design a cost-effective fault-tolerant scheduling approach for large-scale workflows. In this paper, we propose a dynamic fault-tolerant workflow scheduling (DFTWS) approach with hybrid spatial and temporal re-execution schemes. First, DFTWS calculates the time attributes of tasks and identifies the critical path of workflow in advance. Then, DFTWS assigns appropriate virtual machine (VM) for each task according to the task urgency and budget quota in the phase of initial resource allocation. Finally, DFTWS performs online scheduling, which makes real-time fault-tolerant decisions based on failure type and task criticality throughout workflow execution. The proposed algorithm is evaluated on real-world workflows. Furthermore, the factors that affect the performance of DFTWS are analyzed. The experimental results demonstrate that DFTWS achieves a trade-off between high reliability and low cost objectives in cloud computing environments.