GA-Based Iterative Optimization System to Supervise Adaptive Workflows in Cloud Environment

As more and more data can be generated at a faster-than-ever rate nowadays, it becomes a challenge to processing large volumes of data for complex data analysis. In order to address performance and cost issues of big data processing on clouds, we present a novel design of adaptive workflow management system which includes a data mining based prediction model, workflow scheduler, and iteration controls to optimize the data processing via iterative workflow tasks. We proposed a new heuristic algorithm, called Upgrade Fit, which dynamically and continuously reallocates multiple types of cloud resources to fulfill the performance and cost requirements. The iterative workflow tasks can be bursty bags of tasks to be executed repetitively for data processing. A real application of weather forecast workflow has been used to evaluate the capability of our system for large volume image data processing. Experimental system has been set up and the results indicate that the system can effectively handle multiple types of cloud resources and optimize the performance iteratively.

The Industrial Internet of Things (IIoT) integrated with cloud computing resources offers effective advancements in the field of industrial automation with open connectivity and emergent computing. With such advancements, the supplementary services evolves in order to augment the automation process in manufacturing and production industries. However, the integration and transformation of cloud and IIoT environment poses serious limitation on reliability and computational performance. It is very necessary to develop a framework that handles and provides a bridge between the rapid input acquisitions from an IIoT device and rapid data processing or storage by the cloud. In this paper, model based engineering (MBE) model is used to control the workflow in cloud environment, which is integrated with a deep learning algorithm named recurrent neural network (RNN) algorithm. The RNN is utilized to control the future workload prediction and supply necessary resources to balance the workflow in cloud computing environment. The entire predictions are carried out based on the input data acquisition rate from the IIoT devices. The storage or processing of the input data is carried out by the Virtual Machines presents in cloud environment. The resources allocation and scheduling of workload is carried out in the control plane using RNN that optimizes the workflow. The RNN is resumed to solve iteratively the task of optimization. The experiments are conducted to test the efficacy of the model and the results are verified in terms of total energy consumption, task response time, and VM Utilization. The experiments are conducted with same/different types of IIoT devices. The simulation results reveal the improved efficacy of the MBE–RNN model than existing deep learning and MBE schedulers. The results show that the MBE–RNN acquires improved processing ability with reduced task response time and energy utilization.

In order to improve the reserve adequacy of high proportion new energy power systems and realize the efficient consumption of large-scale new energy, the source of reserve resources is expanded to the source network and load storage, and the operating reserve level that can be provided by the source network and load storage flexible regulating resources is quantified in terms of the direction of regulation and the mode of participation. Then, according to the regulation idea of “synchronous units bear the responsibility during regular hours and multiple types of resources share the responsibility during special hours”, a coordinated and participatory operating reserve capacity allocation strategy of source, network, load, and storage is proposed, and a reserve distribution optimization model incorporating multiple types of heterogeneous resources is constructed to achieve the optimization of reserve distribution of each type of reserve resource in the whole network. The optimization model of standby distribution is constructed to incorporate multiple types of heterogeneous resources and realize the step-by-step refinement of standby resources distribution among the whole grid - resources of the same type - units/stations. A case study is carried out based on a provincial power grid, and the effectiveness of the proposed strategy is verified by comparing it with the existing operational reserve allocation guidelines.

In recent years we have witnessed the development of adaptive workflow management systems. These systems offer an extended set of features to support both ad-hoc and evolutionary changes, while ensuring correctness of process definition and their running instances. Ad-hoc and evolutionary changes impose new access control requirements, which have been neglected by adaptive workflow research and are not met by existing models for traditional workflow management systems (WfMSs). In this paper, we extend the role-based access control model for adaptive workflows. This extension is done by defining authorizations for adaptive WfMSs and adaptive authorizations.KeywordsAccess controladaptive workflowadaptive authorizations

In grid systems, users compete for different types of resources such that they may execute their applications. Traditional grid systems are formed of organisations that join together for the purpose of collaborative projects. Resources of each of the participating organisation are pooled such that members of individual organisations may access the shared infrastructure. In general, each participant is both a provider and a consumer of resources. Whilst such systems address large organisations, in this paper we address democratic grid systems to satisfy needs of small organisations and even individuals, where on-demand grids may be formed by drawing idling resources available on the Internet. Whilst traditional grid systems resort to allocations that satisfy system specific objectives such as maximization of the resource utilisation, market mechanisms try to obtain allocations that are efficient economically. Economic mechanisms permit to achieve equilibrium between supply and demand and furthermore provide incentives for providers. Combinatorial auction has been argued as an effective mechanism to address the problem of resource allocation within grid systems. Auctions within which multiple types of resources in varying quantities may be traded eliminate the exposure problem by addressing co-allocation. In this paper, we describe a combinatorial exchange where multiple providers and multiple consumers may participate. We describe the winner determination problem that incorporates the time dimension, i.e. resource bundles may be requested for different time ranges, and describe a set of heuristics that have been designed to be fast. We show that these achieve a high level of efficiency as compared to exact solutions. The second part focusses on the pricing problem. The objective is to compute prices that represent the state of the market and bring trustworthy feedback to participants. Drawing on the approach taken by Kwasnica et al. (Manage Sci 51(3):419–434, 2005), we propose a pricing model that computes per-item pricing. Per-item pricing allows users to deduce the price of bundles that they require by linear summation. Furthermore, we propose a model that computes prices as a function of time, thus permitting users, in particular consumers to adjust their demand trading off price and time of execution.

Containers are gaining popularity in life science research as they encompass all dependencies of provisioned tools and simplifies software installations for end users, as well as offering a form of isolation between processes. Scientific workflows are ideal to chain containers into data analysis pipelines to sustain reproducible science. In this manuscript we review the different approaches to use containers inside the workflow tools Nextflow, Galaxy, Pachyderm, Luigi, and SciPipe when deployed in cloud environments. A particular focus is placed on the workflow tool’s interaction with the Kubernetes container orchestration framework.

Cloud computing makes utility computing possible with pay as you go model. It virtualizes the systems by polling and sharing the resources, thus we need to handle more than one workflow at the same time. Workflow is the standard to represent compute intensive applications in scientific and engineering domain. Hence, in this article, the authors presented the scheduling heuristic for multiple workflows running parallel in the cloud environment with the aim to reduce the energy consumption as it is one of the major concerns of cloud data centers along with the execution performance. In the proposed approach, first clustering is performed to minimize the energy consumption and execution time during communication corresponding to precedence constraint tasks. Then cluster are scheduled is on the best available energy efficient resources. Finally, DVFS is applied in order to reduce energy consumption further when the nodes are in the idle and communication stage. The simulation has been performed on CloudSim and the results show the reduction in energy consumption by up to 42%.

This study assessed secondborn adolescents' perceptions of changes in the allocation of family resources following their firstborn siblings' departure from home after high school, and whether perceived changes were related to changes over 1year in secondborns' academic functioning. Participants were secondborn siblings (mean age=16.58, SD=0.91) from 115 families in which the older sibling had left the family home in the previous year. Allocation of resources was measured via coded qualitative interviews. Most (77%) secondborns reported increases in at least one type of family resource (i.e., parental companionship, attention, material goods), and many reported an increase in multiple types of resources in the year following their older sibling's departure. Consistent with resource dilution theory, perceptions of increases in fathers' companionship, fathers' attention, and mothers' companionship were related to improvements over time in secondborns' academic functioning.

With the rapid development of processing and storage technologies and the success of the Internet, computing resources have become cheaper, more powerful and more ubiquitously available than ever before. This technological trend has enabled the realization of a new computing model, called cloud computing. In cloud, scheduling is an important application. In cloud environments, load balancing task scheduling is an important problem that directly affects resource utilization. Undoubtedly, load balancing scheduling is a serious aspect that should be considered because of its significant impact on both the back end and the front end of the cloud research industry. Good resource utilization is achieved whenever an effective load balance is achieved in the cloud. But, load balancing in cloud computing is an NP-hard optimization problem. In order to accomplish this problem, a novel load balancing task scheduling algorithm in cloud using Adaptive Dragonfly algorithm (ADA) is proposed. The ADA is a combination of dragonfly algorithm and firefly algorithm. Moreover, to attain the better performance, multi-objective function is developed based on three parameters namely, completion time, processing costs and load. Finally, the performance of proposed methodology is evaluated in terms of different metrics namely, execution cost and execution time. The experimental results demonstrate that a proposed approach accomplishes better load balancing result compared to other approaches.

This paper looks into the incorporation of Artificial Intelligence into cloud-native DevOps practices dealing in automation, workflow optimization, and proactive monitoring. The paper also outlines AI-based tools that support some of the main points in DevOps practice areas like CI/CD, infrastructure management, security, and observability. Additionally, it elaborates on new developments that shorten workflows in cloud environments. This study on AI-driven automation shows how these systems can help mitigate repetitive works, speed CI/CD Processes, and increase deployment efficiency. It further explains the possible AI contributions in the areas of real-time monitoring, anomaly detection, and rapid root-cause analysis. The paper also examines the existing challenges under which impeding aspects like scalability constraints, data issues, and overlook, as well as interpretability by human experts must be highlighted. Future research directions would also be mentioned at the end, intended to pave the way for practitioners and researchers to adopt artificial intelligence to develop.

Scheduling scientific workflows in cloud environments is challenging due to the need to balance multiple objectives while satisfying various constraints. In this paper, we propose a Hybrid cloud Economic and Secure workflow scheduling system (HyEcoSec) designed to minimize makespan and cost while ensuring security and meeting budget and deadline constraints. HyEcoSec comprises two modules: a Security Compliance Module to manage user-defined security requirements, and a Scheduling Module integrating a static scheduler based on the Multi-Portions Slime Mould Algorithm (MPSMA) and a dynamic re-scheduler to handle runtime failures. MPSMA, an enhanced version of the Slime Mould Algorithm (SMA), balances makespan and cost using a modified Pareto approach under budget and deadline constraints. Performance evaluation using the WorkowSim demonstrates the effectiveness of HyEcoSec. Compared to baseline methods, MPSMA achieves a competitive execution time, reduces makespan by 26% and cost by 43%, and improves resource utilization by 22%. It ensures compliance with budget and deadline constraints in almost all cases, outperforming the compared algorithms. The integration of security services shows minimal cost and time impact, confirming HyEcoSec's suitability for secure environments. Furthermore, the dynamic re-scheduler enhances execution efficiency, reducing makespan by 61% and cost by 49% in failure scenarios.

Cloud computing recent emanation has drastically diversified everyone's perception of software delivery, infrastructure architectures, application development and deployment models. It is a new touchstone preceding the enterprise application development that can effectively expedite and facilitate the execution of various workflows in business process management system. Workflow is an automation of a business entity that can manage the business processes efficiently satisfying the requirements of prevailing enterprises. Dynamic workflows in a cloud computing environment empower the utilization of various cloud services to facilitate workflow execution. Scheduling multiple workflows in cloud environment is always a challenge. Workflow scheduling is a critical issue in the management of workflow execution. In this paper, we have compared and analyzed the performance of different heuristic workflow scheduling algorithms with various QoS parameters and scheduling factors.

Decentralized orchestration of data-centric workflows in Cloud environments

Workflows are a commonly used model to describe applications consisting of computational tasks with data or control flow dependencies. They are used in domains of bioinformatics, astronomy, physics, etc., for data-driven scientific applications. Execution of data-intensive workflow applications in a reasonable amount of time demands a high-performance computing environment. Cloud computing is a way of purchasing computing resources on demand through virtualization technologies. It provides the infrastructure to build and run workflow applications, which is called ‘Infrastructure as a Service.' However, it is necessary to schedule workflows on cloud in a way that reduces the cost of leasing resources. Scheduling tasks on resources is a NP hard problem and using meta-heuristic algorithms is an obvious choice for the same. This chapter presents application of nature-inspired algorithms: particle swarm optimization, shuffled frog leaping algorithm and grey wolf optimization algorithm to the workflow scheduling problem on the cloud. Simulation results prove the efficacy of the suggested algorithms.

Petri nets are a kind of formal language that are widely applied in concurrent systems associated with resource allocation due to their abilities of the natural description on resource allocation and the precise characterization on deadlock. Weighted System of Simple Sequential Processes with Resources (WS3PR) is an important subclass of Petri nets that can model many resource allocation systems in which 1) multiple processes may run in parallel and 2) each execution step of each process may use multiple units from a single resource type but cannot use multiple resource types. We first prove that the liveness problem of WS3PR is co-NP-hard on the basis of the partition problem. Furthermore, we present a necessary and sufficient condition for the liveness of WS3PR based on two new concepts called Structurally Circular Wait (SCW) and Blocking Marking (BM), i.e., a WS3PR is live iff each SCW has no BM. A sufficient condition is also proposed to guarantee that an SCW has no BM. Additionally, we show some advantages of using SCW to analyze the deadlock problem compared to other siphon-based ones, and discuss the relation between SCW and siphon. These results are valuable to the further research on the deadlock prevention or avoidance for WS3PR. key words: concurrent systems, Petri nets, liveness, deadlock, resource allocation