Service and Utility Oriented Distributed Computing Systems: Challenges and Opportunities for Modeling and Simulation Communities

Grids and peer-to-peer (P2P) networks have emerged as popular platforms for the next generation parallel and distributed computing. In these environments, resources are geographically distributed, managed and owned by various organizations with different policies, and interconnected by wide-area networks or the Internet. This introduces a number of resource management and application scheduling challenges in the domain of security, resource and policy heterogeneity, fault tolerance, and dynamic resource conditions. In these dynamic distributed computing environments, it is hard and challenging to carry out resource management design studies in a repeatable and controlled manner as resources and users are autonomous and distributed across multiple organizations with their own policies. Therefore, simulations have emerged as the most feasible technique for analyzing policies for resource allocation. This paper presents emerging trends in distributed computing and their promises for revolutionizing the computing field, and identifies distinct characteristics and challenges in building them. We motivate opportunities for modeling and simulation communities and present our discrete-event grid simulation toolkit, called GridSim, used by researchers world-wide for investigating the design of utility-oriented computing systems such as Data Centers and Grids. We present various case studies on the use of GridSim in modeling and simulation of Business Grids, parallel applications scheduling, workflow scheduling, and service pricing and revenue management.

Clusters, Grids, and peer‐to‐peer (P2P) networks have emerged as popular paradigms for next generation parallel and distributed computing. They enable aggregation of distributed resources for solving large‐scale problems in science, engineering, and commerce. In Grid and P2P computing environments, the resources are usually geographically distributed in multiple administrative domains, managed and owned by different organizations with different policies, and interconnected by wide‐area networks or the Internet. This introduces a number of resource management and application scheduling challenges in the domain of security, resource and policy heterogeneity, fault tolerance, continuously changing resource conditions, and politics. The resource management and scheduling systems for Grid computing need to manage resources and application execution depending on either resource consumers' or owners' requirements, and continuously adapt to changes in resource availability.The management of resources and scheduling of applications in such large‐scale distributed systems is a complex undertaking. In order to prove the effectiveness of resource brokers and associated scheduling algorithms, their performance needs to be evaluated under different scenarios such as varying number of resources and users with different requirements. In a Grid environment, it is hard and even impossible to perform scheduler performance evaluation in a repeatable and controllable manner as resources and users are distributed across multiple organizations with their own policies. To overcome this limitation, we have developed a Java‐based discrete‐event Grid simulation toolkit called GridSim. The toolkit supports modeling and simulation of heterogeneous Grid resources (both time‐ and space‐shared), users and application models. It provides primitives for creation of application tasks, mapping of tasks to resources, and their management. To demonstrate suitability of the GridSim toolkit, we have simulated a Nimrod‐G like Grid resource broker and evaluated the performance of deadline and budget constrained cost‐ and time‐minimization scheduling algorithms. Copyright © 2002 John Wiley & Sons, Ltd.

With the rapid development of national economy, our demand for power resources has also increased rapidly. The traditional power grid can not be achieved operational status of equipment for effective monitoring and control, we will not be able to ensure the quality and quality of electricity service. Cloudbased build smart grid information platform enables gathering the parameter of grid operation state, greatly increasing the economic efficiency of power enterprises. Firstly, cloud computing and smart grid briefly described, and then pointed out that cloud computing applications in the smart grid information platform, and specialized of cloud-based smart grid information platform, in order for the power grid enterprises to enhance their level of service and provide effective reference. KEYWORD: cloud computing; smart power grids; information platform; Application 4th International Conference on Mechanical Materials and Manufacturing Engineering (MMME 2016) © 2016. The authors Published by Atlantis Press 202 3 THE MAIN PROBLEM IS STILL THE SMART GRID INFORMATION PLATFORM FACING The transition to a cloud computing data center. Our data center because of the different regions and the degree of information presents different characteristics, some data centers already have a basic cloud computing capabilities, but most data centers still remain in the initial level, to ensure the normal operation of the existing data center power under the premise, gradual transition to cloud computing data center in order to meet future requirements of the development of smart grid. Restricting data transmission problem. State Grid Corporation between provincial branches and there are a lot of data needs to be transmitted, will bring massive data transmission efficiency decreases, taking into account the load balancing device between the cloud node, thus building a wide area network channels to resolve data transmission problem. Security of data transmission. Data Center for Cloud Computing power companies to build internal private cloud, capable of self-maintenance and management, system wide area network data transmission services, according to the third security zone defined security partition, and install proprietary equipment production area and external public network isolation . 4 CLOUD COMPUTING APPLICATIONS IN THE SMART GRID INFORMATION POLICY PLATFORM Integrate and optimize the management of heterogeneous resources. Among the smart grid systems, data resources, information systems between various platforms and mutually dispersed, severe heterogeneity, thus leading to the integration of the quality of information sharing to increase difficulty, unable to effectively improve network management efficiency for the power distribution system electrical work quality management and marketing operations system maintenance upgrade will not be able to effectively meet demand (Jie Yang, 2014). Cloud computing technology enables the power system to effectively integrate the various business data and computing resources on the basis of information for business collaboration platform built on effective, able to satisfy the huge demand for smart grid information resource sharing. Cloud computing network virtualization and application virtualization technology, the server virtualization technology and other resources to achieve each service abstraction, and then realize the difference between the system and the software effectively eliminated by the appropriate method to achieve the effective integration of information resources, and then to optimize the management, greatly improving the efficiency of heterogeneous resources management applications. Distributed storage and management of large volume of data. After application of smart grid information platform construction, customer service requests varied, huge data volume information brought it to the storage and management of data provides a great challenge. Cloud computing technology in general face amount of data each smart grid system and unit collected using forms distributed storage of data to achieve the effective storage, greatly to ensure the security and reliability of data storage. GFS effective use of technology with each node, the master server is responsible for the effective management of the whole file system, the file system and complete the work of the metadata stored in the data block is the main server for data storage block sliced, and by saving three or more redundant backup storage. At the same time, client access and complete access to the main block server information. Fast Power System Parallel Computation and Treatment. Smart grids and information platform for the large volume of data to provide stable operation of the power grid data base, but more critical is the analysis and design of each system operating data, planning and decision-making, and even failures and transient stability, state estimation and intelligent a variety of computing and content analysis and decision-making process, its huge computing needs, general computing system can not reach its goal (Wang, Mengxue, 2015). Cloud computing technology through parallel computing technology has greatly enhanced parallel processing capability of the power system, which provides parallel programming model makes more simple and efficient parallel algorithms. MapReduce as Google made an important parallel programming model, and its role in large data sets in parallel computing plays a very prominent. Parallel processing job results by the client to achieve effective user submitted, via decomposition automated master role, the corresponding task scheduling to working nodes and nodes work completed by the specific calculations and perform the appropriate task. 5 SMART GRID INFORMATION PLATFORM CLOUD COMPUTING ANALYSIS Cloud Computing Smart Grid Information Platform Architecture. Smart grid technology information platform architecture cloud computing include the following several levels and aspects (Yan Cui, 2015): First, that the platform infrastructure layer is the integration of the various hardware resources and related management capabilities of virtualized using virtual technology to achieve the corresponding hardware devices such as computers and network storage devices abstraction, and then complete the automation and optimization of resource data and load management; secondly, the main platform for the integration of the various layers of software

Fault tolerance, performance, and throughput have been major areas of research and development since the evolution of large-scale networks. Internet-based applications are rapidly growing, including large-scale computations, search engines, high-definition video streaming, e-commerce, and video on demand. In recent years, energy efficiency and fault tolerance have gained significant importance in data center networks and various studies directed the attention towards green computing. Data centers consume a huge amount of energy and various architectures and techniques have been proposed to improve the energy efficiency of data centers. However, there is a tradeoff between energy efficiency and fault tolerance. The objective of this study is to highlight a better tradeoff between the two extremes: (a) high energy efficiency and (b) ensuring high availability through fault tolerance and redundancy. The main objective of the proposed Energy-Aware Fault-Tolerant (EAFT) approach is to keep one level of redundancy for fault tolerance while scheduling resources for energy efficiency. The resultant energy-efficient data center network provides availability as well as fault tolerance at reduced operating cost. The main contributions of this article are: (a) we propose an Energy-Aware Fault-Tolerant (EAFT) data center network scheduler; (b) we compare EAFT with energy efficient resource scheduling techniques to provide analysis of parameters such as, workload distribution, average task per servers, and energy consumption; and (c) we highlight effects of energy efficiency techniques on the network performance of the data center.

New and emerging use cases, such as the interconnection of geographically distributed data centers (DCs), are drawing attention to the requirement for dynamic end-to-end service provisioning, spanning multiple and heterogeneous optical network domains. This heterogeneity is, not only due to the diverse data transmission and switching technologies, but also due to the different options of control plane techniques. In light of this, the problem of heterogeneous control plane interworking needs to be solved, and in particular, the solution must address the specific issues of multi-domain networks, such as limited domain topology visibility, given the scalability and confidentiality constraints. In this article, some of the recent activities regarding the Software-Defined Networking (SDN) orchestration are reviewed to address such a multi-domain control plane interworking problem. Specifically, three different models, including the single SDN controller model, multiple SDN controllers in mesh, and multiple SDN controllers in a hierarchical setting, are presented for the DC interconnection network with multiple SDN/ OpenFlow domains or multiple OpenFlow/ Generalized Multi-Protocol Label Switching (GMPLS) heterogeneous domains. In addition, two concrete implementations of the orchestration architectures are detailed, showing the overall feasibility and procedures of SDN orchestration for the end-to-end service provisioning in multi-domain data center optical networks.

Forest Service Research and Development is working to enhance the value of its network of Experimental Forests and Ranges (EFRs) by equipping them with modern data collection infrastructure to encourage cross-site and continental-scale monitoring, research, synthesis, and assessment. We envision a network of EFRs that shares resources across research sites, with managers and with scientists in other organizations. We anticipate focusing research across this network on the most pressing natural resource policy and management challenges of our time: natural disturbances and their interaction with human communities, sustainable production of forest and range goods and services, understanding and managing the effects of invasive species, understanding and coping with a changing environment, monitoring the pulse of our forest and range conditions, anticipating and addressing current and future land and resource management challenges, fully integrating urban natural resource research into our network, and optimizing the hierarchy of scales at which current and future natural resource challenges are addressed. As this vision is realized, the legacy of EFRs will be fully honored by sustained contributions to the well-being of the American people.

In this paper, we propose an FPGA-accelerated data optimization system for high-speed data transfer via Wide Area Network (WAN). To maintain high quality in cloud services, high-speed transfer among data centers via WAN is important. To accelerate transfers via WAN, data optimization techniques are used to reduce the size of the data transmitted to the WAN. Compression and deduplication are widely-used data optimization techniques; data sent is shrunk before transmission and restored upon receipt. However, the computational time of these techniques is so great that it is difficult to achieve high performance using only a CPU. To solve this problem, we developed an efficient data optimization system using both CPU and FPGA. We designed a dedicated accelerator whose architecture is suitable for each processing characteristic of data optimization. We also proposed an efficient data flow inside the FPGA and communication method between the CPU and the FPGA. Thanks to the appropriate pipelining and parallelization, and the data flow, the measured throughput of our developed accelerator achieves over 41 Gbps. With this accelerator, we can realize a data transfer system whose throughput is 40 Gbps end-to-end: as far as we know the fastest performance for a WAN optimization system.

Cloud data centres (CDCs) are the backbone infrastructures of modern digital society, but they also consume huge amounts of energy and generate heat.To manage CDC resources efficiently, we must consider the complex interactions between diverse workloads and data centre components.However, most existing resource management systems rely on simple and static rules that fail to capture these complex interactions.Therefore, we require new data-driven Machine learning-based resource management approaches that can efficiently capture the interdependencies between parameters and guide resource management systems.This review describes the in-depth analysis of the existing resource management approaches in CDCs for energy and thermal efficiency.It mainly focuses on learning-based resource management systems in data centres and also identifies the need for integrated computing and cooling systems management.A taxonomy on energy and thermal efficient resource management in data centres is proposed.Furthermore, based on this taxonomy, existing resource management approaches from server level, data centre level, and cooling system level are discussed.Finally, key future research directions for sustainable Cloud computing services are proposed.

SummaryThe cloud computing paradigm is an important service in the Internet for sharing and providing resources in a cost‐efficient way. Modeling of a cloud system is not an easy task because of the complexity and large scale of such systems. Cloud reliability could be improved by modeling the various aspects of cloud systems, including scheduling, service time, wait time, and hardware and software failures. The aim of this study is to survey research studies done on the modeling of cloud computing using the queuing system in order to identify where more emphasis should be placed in both current and future research directions. This paper follows the goal by investigating the articles published between 2008 and January 2017 in journals and conferences. A systematic mapping study combined with a systematic literature review was performed to find the related literature, and 71 articles were selected as primary studies that were classified in relation to the focus, research type, and contribution type. We classified the modeling techniques of cloud computing using the queuing theory in seven categories based on their focus area: (1) performance, (2) quality of service, (3) workflow scheduling, (4) energy savings, (5) resource management, (6) priority‐based servicing, and (7) reliability. A majority of the primary articles focus on performance (37%), 15% of them focus on resource management, 14% of them focus on quality of service, 13% of them focus on workflow scheduling, 13% of them focus on energy savings, 4% of them focus on priority‐based servicing for requests, and 4% of them focus on reliability. This work summarizes and classifies the research efforts conducted on applying queue theory for modeling of cloud computing (AQTMCC), providing a good starting point for further research in this area.

A distributed computing network (DCN), such as a content delivery network, not only improves the response time to clients but also reduce the traffic from/to the data center over a wide area network (WAN), thereby decreasing the power consumed in the WAN. In this paper, we focus on the energy-saving aspect of the DCN and evaluate its effectiveness, especially considering traffic locality, i.e., the amount of traffic related to the geographical vicinity. Numerical evaluations show that the DCN has a small advantage when the traffic has no locality. Meanwhile, when there is strong traffic locality, the system's power consumption is reduced to 64 % of the power consumption in the case where a DCN is not used.

Workflow generation, resource management, and scheduling of workflow are three main components of grid based computing architecture. The components in traditional grid architecture require human intervention due to lack of intelligence and use of knowledge from historical data. We have identified the need of intelligence based automatic workflow generation, and the prediction on run-time of tasks for optimal allocation and reservation of resources. We propose soft computing based architecture of intelligent grid. We also propose the neural network based algorithmic flow for the resource matchmaking, the allocation and reservation, and the scheduling of workflow. Our proposed architecture and algorithmic flow does not require human intervention in workflow generation and in finding and allocating optimal resource for task execution. As our proposed work provides flexibility and reduces complexity in workflow generation, it saves the valuable time and cost of user.

Contributors

Introduction. The rational use of material resources plays a crucial role in reducing production costs and enhancing the competitiveness of enterprises, particularly in the industrial sector. Rising prices for raw materials, energy, and logistics services necessitate improvements in resource management mechanisms. Optimization of planning and analysis of material resource utilization contributes to increased efficiency of production processes, cost reduction, and improved financial performance of enterprises. Objective of the study. The objective of this study is to examine approaches to planning and analysing the efficiency of material resource utilization at enterprises, identify key factors for optimization, and develop recommendations for improving information and analytical support for managerial decision-making. Methodology. The study employs methods of economic analysis, technical and economic justification, a systemic approach, and modelling. The analysis of regulatory documents, statistical data, and operational results of cement industry enterprises allowed for the identification of key factors influencing the efficiency of material resource utilization. Results. The study identifies key challenges in material resource management at enterprises and outlines directions for their resolution. It has been proven that effective planning and analysis of material costs contribute to reducing production costs, improving financial stability, and ensuring sustainable enterprise development. The study proposes the integration of modern information systems into the material resource management process, which will allow for automation of control processes, minimization of losses, and enhancement of forecasting accuracy in resource needs.

Improving the energy efficiency while guaranteeing quality of services (QoS) is one of the main challenges of efficient resource management of large-scale data centers. Dynamic virtual machine (VM) consolidation is a promising approach that aims to reduce the energy consumption by reallocating VMs to hosts dynamically. Previous works mostly have considered only the current utilization of resources in the dynamic VM consolidation procedure, which imposes unnecessary migrations and host power mode transitions. Moreover, they select the destinations of VM migrations with conservative approaches to keep the service-level agreements , which is not in line with packing VMs on fewer physical hosts. In this paper, we propose a regression-based approach that predicts the resource utilization of the VMs and hosts based on their historical data and uses the predictions in different problems of the whole process. Predicting future utilization provides the opportunity of selecting the host with higher utilization for the destination of a VM migration, which leads to a better VMs placement from the viewpoint of VM consolidation. Results show that our proposed approach reduces the energy consumption of the modeled data center by up to 38% compared to other works in the area, guaranteeing the same QoS. Moreover, the results show a better scalability than all other approaches. Our proposed approach improves the energy efficiency even for the largest simulated benchmarks and takes less than 5% time overhead to execute for a data center with 7600 physical hosts.

An extended Intelligent Water Drops algorithm for workflow scheduling in cloud computing environment