An agent based network resource planner for workflow applications

The quality of the network services has so far rarely been considered in composing and executing scientific workflows. Currently, scientific applications tune the execution quality of workflows neglecting network resources, and by selecting only optimal software services and computing resources. One reason is that IP-based networks provide few possibilities for workflow systems to manage the service quality, and limit or prevent bandwidth reservation or network paths selection. We see nonetheless a strong need from scientific applications, and network operators, to include the network quality management in the workflow systems. In this position paper, we discuss our vision on this issue and propose an agent based solution to include network resources in the loop of workflow composition, scheduling and execution when advanced network services are available. Our approach is conducted in the context of the CineGrid project.

Currently, scientific application developers often configure available application components into a workflow of tasks that they can submit for execution on distributed systems. In this chapter, we present the Abstract Workflow Description Language (AWDL) for describing scientific workflow applications at a high level of abstraction. AWDL has been designed such that the user can concentrate on specifying scientific applications without dealing with either the complexity of distributed systems or any specific implementation technology (e.g., Web Services). AWDL is an XML-based language which allows a programmer to define a graph of activities that refer mostly to computational tasks. Activities are connected by control and data flow links. A rich set of control flow constructs (compound activities) is provided to simplify the specification of scientific workflow applications which includes dag, for, while, and alt. In addition, AWDL programmers can specify high-level constraints and properties for activities and data flow links which may be useful for underlying runtime systems to steer and optimize workflow executions. AWDL is the main interface to the ASKALON development and runtime environment for workflows on distributed systems and has been applied to numerous real-world scientific workflow applications. We describe a material science workflow that has been successfully ported to a distributed infrastructure based on the AWDL specification.

This paper presents a novel service-oriented network architecture to bridge the informational gap between user applications and optical networks providing technology-agnostic multigranular optical network services for clouds. A mediation layer (service plane) between user applications and network control is proposed to facilitate a mapping process between user application requests and the network services. At the network level, a multigranular optical network (MGON) is proposed and implemented to support dynamic wavelength and subwavelength granularities with different transport formats [optical burst switched (OBS), optical burst transport (OBT)], reservation protocols (one-way, two-way), and different quality-of-service (QoS) levels per service type. The service-oriented multigranular optical network has been designed, implemented, and demonstrated on an experimental testbed. The testbed consists of service and network resource provisioning, service abstraction, and network resource virtualization. The service-to-network interoperation is provided by means of a gateway that maps service requests to technology-specific parameters and a common signaling channel for both service and network resource provisioning.

Scientific Workflows (SWFs) are widely used to model applications in e-Science. In this programming model, scientific applications are described as a set of tasks that have dependencies among them. During the last decades, the execution of scientific workflows has been successfully performed in the available computing infrastructures (supercomputers, clusters and grids) using software programs called Workflow Management Systems (WMSs), which orchestrate the workload on top of these computing infrastructures. However, because each computing infrastructure has its own architecture and each scientific applications exploits efficiently one of these infrastructures, it is necessary to organize the way in which they are executed. \nWMSs need to get the most out of all the available computing and storage resources. Traditionally, scientific workflow applications have been extensively deployed in high-performance computing infrastructures (such as supercomputers and clusters) and grids. But, in the last years, the advent of cloud computing infrastructures has opened the door of using on-demand infrastructures to complement or even replace local infrastructures. However, new issues have arisen, such as the integration of hybrid resources or the compromise between infrastructure reutilization and elasticity, everything on the basis of cost-efficiency.\nThe main contribution of this thesis is an ad-hoc solution for managing workflows exploiting the capabilities of cloud computing orchestrators to deploy resources on demand according to the workload and to combine heterogeneous cloud providers (such as on-premise clouds and public clouds) and traditional infrastructures (supercomputers and clusters) to minimize costs and response time. The thesis does not propose yet another WMS, but demonstrates the benefits of the integration of cloud orchestration when running complex workflows. The thesis shows several configuration experiments and multiple heterogeneous backends from a realistic comparative genomics workflow called Orthosearch, to migrate memory-intensive workload to public infrastructures while keeping other blocks of the experiment running locally. The running time and cost of the experiments is computed and best practices are suggested.

This paper introduces an architecture for managing a wide range of different administrative domains, that already have its own management systems. The approach and requirements exposed enhance the different network resource provisioning system (NRPS) under scope, since the set of interfaces specified and implemented allow their interoperability through the network service provider (NSP). The northbound interface defined within the NSP provides single-step on-demand services across multi-domain networks, with advance reservation functionalities involving network resources. Another aim of the specifications defined, is the future interoperability with the JRA3 developments done within the GEANT2 project. The hierarchical client-server schema between the NSP and NRPSs could be replicated horizontally in order to allow different NSP communicating each other. The overall management system becomes more fault-tolerant and scales better than the single, centralised system. Moreover, hierarchical structures of NSPs could be implemented in order to add control over the management system, pursuing a balance between the scalability of the system and the control over the different domains.

Cloud computing, a novel and promising model of Service-oriented computing, provides a pay-per-use framework to solve large-scale scientific and business workflow applications. Workflow scheduling in cloud is challenging due to dynamic nature of the cloud, particularly, on demand provisioning, elasticity, heterogeneous resource types, static & dynamic pricing models and virtualization. An example of workflow scheduling is mapping workflow tasks to cloud computing resources. Additionally, these workflow applications have a runtime constraint—the most typical being the cost of the computation and the time that computation requires to complete. Therefore, the focus is on two criteria: makespan and cost. This paper presents an algorithm called NBWS (Normalization based Budget constraint Workflow Scheduling) which generates a workflow schedule which minimizes the schedule length while satisfying the budget constraint. The algorithm undergoes a process of min–max normalization tailed by computing expect reasonable budget $$ (erb) $$ for dispatching the workflow tasks into one of the virtual machines. To minimize the execution time, NBWS algorithm maps the workflow tasks to resources which are having the earliest finish time within the allocated budget. The experimental results demonstrate that NBWS outperforms current state-of-the-art heuristics with respect to budget constraint and minimizing the makespan.

This demo will present both real and mocked-up usecases of a software application that enables accurate probabilistic modeling of new IP based services and their impact on network resources. The demonstration should be of general interest to Network Service Providers (NSP), device manufacturers, and application developers. Network planning and service analysis is a very important issue for today’s NSP’s; new communications devices and services emerge quickly (e.g., VoIP, Peer-to-peer filesharing, etc.) and end-users demand network resources that are up to the task. For the analyst, our tool leverages familiar UML style views, used to capture prototypical service “sessions”. It then allows annotation of the service model with a rich grammar, model conversion into semi-Markov form, and algorithm execution for network resource usage estimation. Ultimately, analysts gain insight into the impact of new consumer devices and services on their networks.

Cloud computing, a novel and promising methodology in the distributed computing domain, provides a pay-per-use framework to solve large-scale scientific and business workflow applications. These workflow applications have a constraint that each of them must completed within the limited time (deadline constraint). Therefore, scheduling a workflow with deadline constraints is increasingly becoming a crucial research issue. However, many analytical reviews on scheduling problems reveal that existing solutions fail to provide cost-effective solutions and they do not consider the parameters like CPU performance variation, delay in acquisition and termination of Virtual Machines (VMs). This paper presents a Cost-Effective Firefly based Algorithm (CEFA) to solve workflow scheduling problems that can occur in an Infrastructure as a Service (IaaS) platform. The proposed CEFA uses a novel method for problem encoding, population initialization and fitness evaluation with an objective to provide cost-effective and optimized workflow execution within the time limit. The performance of the proposed CEFA is compared with the state-of-the-art algorithms such as IaaS Cloud-Partial Critical Path (IC-PCP), Particle Swarm Optimization (PSO), Robustness-Cost-Time (RCT), Robustness-Time-Cost (RTC), and Regressive Whale Optimization (RWO). Our experimental results demonstrate that the proposed CEFA outperforms current state-of-the-art heuristics with the criteria of achieving the deadline constraint and minimizing the cost of execution.

Workflow applications (WFA) support users working in business processes. Knowledge and comprehension of the underlying processes beyond one's own task is necessary but often not readily available. We have developed a generic taxonomy of help information for all types of workflow or process applications that considers the process-orientation of these applications. Based on this classification we have designed a user support system for one WFA. With this support system the user can not only retrieve information about field entries into the interfaces, but it also provides process-related information, including information about the work done on previous tasks as well as what will transpire with the entries, their consequences and results. Our goal is to demonstrate that the user support system enhances the users' process comprehension, ensures trustworthy decisions and hence improve human performance and accelerates processing times.

Energy-efficient execution of the scientific workflow is a challenging task in cloud computing that demands high-performance computing to process growing datasets. Due to the interdependency of tasks in the scientific workflow applications, energy-efficient resource allocation is vital for large-scale applications running on heterogeneous physical machines. Thus, this paper proposes a hybrid heuristic algorithm based energy-efficient cloud computing service (HH-ECO) that offers a significant solution for resource allocation, task scheduling, and optimization of scientific workflows. To ensure the energy-efficient execution, the HH-ECO focuses on executing non-dominant workflow tasks through adaptive mutation and energy-aware migration strategy. HH-ECO adopts the chaotic based particle swarm optimization (C-PSO) principle to optimize the resource allocation, task scheduling, and resource migration by generating the global best plans without local convergence. C-PSO with adaptive mutation avoids the deterioration of global optima while finding the best host to place the virtual machine and ensures an appropriate resource allocation plan. By considering the workflow task precedence relationships during C-PSO based task scheduling, the novel hybrid heuristic method efficiently solves the multi-objective combinatorial optimization problem without dominance among the workflow tasks. The Cloudsim based simulation study delivers superior results compared to the existing methods such as the hybrid heuristic workflow scheduling algorithm (HHWS) and distributed dynamic VM management (DDVM). The proposed approach significantly improves the optimal makespan to 38.27% and energy conservation to 38.06% compared to the existing methods.

The question of quality of service in networks of the latest generation is studied. The impact of heterogeneous traffic on the main quality of service indicators reflecting the probability of packet loss and delivery delay over time is evaluated. A model of quality of service in information communication networks has been developed to ensure low traffic delay. Taking into account the concept of quality in information communication networks, the issue of quality of service in networks of the latest generation becomes important from the standpoint of ensuring high reliability and reliability of information transmission, which significantly affects the quality of services provided to the user. This, of course, determines the need to control various parameters of equipment and communication lines, which are determined both by the features of the transmission system and by the parameters of leased channels and tracts. The basis for determining the quality of service, as mentioned above, is the provision of the international concept of quality of service (QoS). According to the requirements of the compliance control model proposed in the concept, the approach to quality indicators is structurally related to the OSI reference model. The main advantages of modern technologies of information communication networks are undoubtedly 5G technologies, which already support such resource-intensive services as massive machine-to-machine communication, ultra-reliable communication and low-latency communication. All of them require high data transfer rates, low millisecond latency, and connectivity of millions of devices. At the same time, it is necessary to ensure a high level of quality of service (QoS), which plays an important role in 5G technologies. The proposed model makes it possible to evaluate not only the quality of service of the combined traffic flow of Internet of Things, tactile Internet, and augmented reality services, but also the delivery delay and the probability of packet loss. The simulation results showed that the parameters of the quality of service of the traffic of the Internet of Things and other modern applications differ significantly during the service of the aggregate flow. When studying the service of aggregated traffic, it was established that the probability of packet loss is higher for it than for Internet of Things traffic. In addition, this difference becomes more significant when the intensity of the incoming load increases.

Network function virtualization (NFV) decouples network functions (NFs) from dedicated hardware, leading to significant cost reduction in network service provisioning. With NFV, a network service is represented by a series of inter-connected virtual network functions (VNFs), forming a service function chain (SFC). The problem of placing the VNFs on the NFV infrastructure (NFVI) and establishing the routing paths between them, according to the service chain template, is termed as SFC embedding. The objectives and constraints for the optimization problem formulation of SFC embedding may vary depending on the corresponding network service. We introduce the notion of trustworthiness as a measure of security in SFC embedding and thus network service deployment. We formulate the resulting trust-aware SFC embedding problem as a Mixed Integer Linear Program (MILP). We relax the integer constraints to reduce the time complexity of the MILP formulation, and obtain a Linear Program (LP). We investigate the trade-offs among the two formulations, seeking to strike a balance between results accuracy and time complexity.

Sponsored content policy enables a content provider to pay a network operator, and thereby their users access contents from the content provider through network services from the network operator with lower charge. In this paper, we study the interaction among three entities under the sponsored content policy, namely, the network operator or service provider, the content provider and the end-users. We consider a hierarchical three-stage setting to formulate the game theoretic model to analyze the interaction. Using the game model, we derive the user content demand, optimal sponsoring of content provider, and pricing of service provider based on backward induction. The model incorporates the network effects in social domain and congestion in network domain which enables us to obtain insights from the sponsored content policy. We derive the closed-form solution, i.e., equilibrium, and prove its existence and uniqueness in each stage of the game. Additionally, we develop an iterative algorithm to obtain the Stackelberg equilibrium of the entire three-stage game. The simulation results indicate that the revenue, profit, and utility of the service provider, content provider, and end-users have been improved to a large extent under the sponsored content policy because of the network effects.

The design, management, and operation of network infrastructure have evolved during the last few years, leveraging on innovative technologies and architectures. With such a huge trend, due to the flexibility and significant economic potential of these technologies, software-defined networking (SDN) and network functions virtualization (NFV) are emerging as the most indispensable key catalysers. SDN/NFV enhancing the infrastructure agility, thus network operators and service providers are able to program their own network functions (e.g., gateways, routers, load balancers) on vendor-independent hardware substrate. One of the most important considerations in NFV deployment is how to allocate the virtual resources that are needed to provide flexible virtual network services in an NFV-based network infrastructure. Thus, the most important prerequisite for NFV deployment is achieved fast, scalable and dynamic composition and allocation of networks functions (NFs) to implement network services (NSs). We have proposed a revised RA algorithm that integrates embedding and scheduling of virtual network functions (VNFs) simultaneously. In this paper, performance evaluation of the revised RA algorithm is performed and the proposed algorithm is applied more effectively in NFV environment where the demand for resources is flexible and concentrated.

Constrained routing in multi-partite graph to solve VNF placement and chaining problem