Using data mining techniques to improve replica management in cloud environment

In recent years data grids have been deployed and grown in many scientific experiments and data centers. The deployment of such environments has allowed grid users to gain access to a large number of distributed data. Data replication is a key issue in a data grid and should be applied intelligently because it reduces data access time and bandwidth consumption for each grid site. Therefore this area will be very challenging as well as providing much scope for improvement. In this paper, we introduce a new dynamic data replication algorithm named Popular File Group Replication, PFGR which is based on three assumptions: first, users in a grid site (Virtual Organization) have similar interests in files and second, they have the temporal locality of file accesses and third, all files are read-only. Based on file access history and first assumption, PFGR builds a connectivity graph for a group of dependent files in each grid site and replicates the most popular group files to the requester grid site. After that, when a user of that grid site needs some files, they are available locally. The simulation results show that our algorithm increases performance by minimizing the mean job execution time and bandwidth consumption and avoids unnecessary replication.

A data replication algorithm for groups of files in data grids

A Critical Survey of Data Grid Replication Strategies Based on Data Mining Techniques

The data grid is one of the infrastructures for data and storage resource management. Data replication can improve data access performance, fault tolerance, and reduce network transmission bandwidth. As the site storage space is limited in the grid, how to effectively use grid resources become an important challenge. This paper introduces a dynamic grid replication algorithm based on popularity support and confidence (BPSC). Through the algorithm, data and its associated copy can be placed on a suitable site, with reduced access latency. Through Optorsim, simulation results show that the algorithm can provide better performance compared with other algorithms Grid computing is an important branch of distributed computing. Grid computing constructs a virtual computer system through using a different location of the large number of heterogeneous resources (CPU and storage), provides the model to solve the problem of large scale computing and storage. The data grid is mainly the grid for data storage management. The data grid is defined as an effective combination of data and computing resources of distributed systems by Data Grid Group. The large-scale data- intensive applications, such as bioinformatics, high energy physics, space information generated vast amounts of data, these data may be TB scale, even up to the scale of PB. It is not practical to maintain large scale data at each site, and provide computing resources for visitors. At the same time, it will lead to increased access latency through a centralized way to storing and accessing these data from each site. Data replication technology is an effective method to reduce access latency in the data grid system. Data replication is one of the basic methods of distributed systems to improve the efficiency of system access, maintenance, system high availability. Data replication is to store data in the appropriate site for users to effective access to massive data. Data replication can reduce data access latency, reduce network bandwidth consumption, avoid congestion, adjust the load balancing in server-side, improve system reliability. Many data replication algorithm can reduce data access latency, improve system reliability, but which data will be copied, when the data will be copied at, where the data will be copied are the basic means of determining data replication algorithm. In this paper, we introduced a real-time dynamic data replication algorithm. The algorithm is based on the popularity of file dynamic to replicate data, and consider the storage space limitations of the site. Through optorsim simulation verified the algorithm. This paper is organized as follows: Section 2 is about related works on replication in data grids. In Section 3, system model and component in system are presented. The file

The Dynamic Adaptive Streaming over HTTP (referred as MPEG DASH) standard is designed to provide high quality of media content over the Internet delivered from conventional HTTP web servers. The visual content, divided into a sequence of segments, is made available at a number of different bitrates so that an MPEG DASH client can automatically select the next segment to download and play back based on current network conditions. The task of transcoding media content to different qualities and bitrates is computationally expensive, especially in the context of large-scale video hosting systems. Therefore, it is preferably executed in a powerful cloud environment, rather than on the source computer (which may be a mobile device with limited memory, CPU speed and battery life). In order to support the live distribution of media events and to provide a satisfactory user experience, the overall processing delay of videos should be kept to a minimum. In this paper, we propose a novel dynamic scheduling methodology on video transcoding for MPEG DASH in a cloud environment, which can be adapted to different applications. The designed scheduler monitors the workload on each processor in the cloud environment and selects the fastest processors to run high-priority jobs. It also adjusts the video transcoding mode (VTM) according to the system load. Experimental results show that the proposed scheduler performs well in terms of the video completion time, system load balance, and video playback smoothness.

Cloud Computing can be described as any where any time storage and it is able to provide on demand services like server, storage, software etc. to the users over internet. Cloud consists of multiple virtual machines which includes several facilities like computation and storage. The foremost aim of cloud computing is to provide systematic and well regulated access to distant and geographical resources. The efficient discharge of task scheduling computing is one of the primary factors. This paper deals with the subject of dynamic load balancing for task scheduling in cloud environment. Satisfactory results can be obtained by implementing the proposed algorithm by considering distribution of tasks on the basis of types (nature), processing efficiency and load efficiency factor. In this paper, algorithm has been implemented by taking into account cloud VM groups, configuration and nature of tasks which determines that resources have been utilized efficiently. The result has been compared and shows better results in terms of average turnaround time, average response time, average waiting time and cost efficiency ratio than existing load balancing algorithms [1, 2].

Distributed servers need good load balancing since changing client requests and resources can cause response times to vary and performance to degrade. Round-Robin methods have been used for a long time, however they don't account for server workload changes, and static approaches may overload servers and waste resources. This work creates and uses a load-balancing system that combines Round-Robin, Least Connections, and Weighted Response Time to solve these issues. The purpose is to reduce reaction time, allow workload changes, and improve server efficiency. The proposed method includes creating a hybrid load balancer that uses a fitness-based selection mechanism based on natural selection to keep an eye on backend servers and do regular health checks. This system checks servers on the fly based on current connections and average response times. This makes sure that requests are sent to the fastest server. We tested how clients and servers interact in a controlled setting with three servers and two clients. GUI-based monitoring was used to see the state at all times. The hybrid approach does better than traditional Round-Robin techniques by lowering error rates, increasing throughput, and shortening the average response time. The system can also grow as needed and handle errors, as broken servers are quickly taken out of the pool to keep service running smoothly. These results show that hybrid load balancing is a good way to improve distributed server systems. This makes it a strong and flexible choice for real-time applications that need reliable and consistent performance.

Abstract: The advantages of cloud computing can accordingly be seen in that; There is low price when the traditional computing systems are compared, there is a wide scope for expansion that is always needed by the business and fast availability of the resources. Since service users are a huge group controllable through a system and performance of the service is done according to traffic, user experience is as result incomparable and necessary service availability factors are catered for. It is that demand such as huge amounts of traffic and extensive utilization of huge cloud has become so common in the case of this before. Load balancing is the most important component of the cloud computing architecture, making it possible to provide the necessary uptime, capacity, reliability, and performance of web applications that are implemented by exploiting the load balancing technique. The specified train of thought addresses the implementation of as well as dealing with the procedures or types of cloud computing as well as working with the load balancing system.

A load balancing strategy for software defined networking (SDN) in the cloud seems to resolve the availability and latency difficulties in the HTTP server nowadays, thanks to the strong demand for cloud services. In this article, we show how to implement a load balancing system for software defined networking (SDN) in the cloud. The HTTP server handled 6 million requests, with 400 requests per second. Due to overload, the website's server will not reply to any more requests sent by the client. hence, after applying load balance, the same parameters may be measured using a load balancing system. The 6 million requests, each involving four connections, were all successfully accepted by the cloud environment's HA proxy, allowing the traffic from the HTTP server to be dispersed across a pool of servers rather than a single server. Having an HTTP load balancer helps to improve the availability and latency of HTTP requests, and the HA proxy eventually gives an adequate answer time with no requests lost, even after 6 million requests arriving at the same time.

Live virtual machine (VM) migration is a technique for achieving system load balancing in a cloud environment by transferring an active VM from one physical host to another. This technique has been proposed to reduce the downtime for migrating overloaded VMs, but it is still time- and cost-consuming, and a large amount of memory is involved in the migration process. To overcome these drawbacks, we propose a Task-based System Load Balancing method using Particle Swarm Optimization (TBSLB-PSO) that achieves system load balancing by only transferring extra tasks from an overloaded VM instead of migrating the entire overloaded VM. We also design an optimization model to migrate these extra tasks to the new host VMs by applying Particle Swarm Optimization (PSO). To evaluate the proposed method, we extend the cloud simulator (Cloudsim) package and use PSO as its task scheduling model. The simulation results show that the proposed TBSLB-PSO method significantly reduces the time taken for the load balancing process compared to traditional load balancing approaches. Furthermore, in our proposed approach the overloaded VMs will not be paused during the migration process, and there is no need to use the VM pre-copy process. Therefore, the TBSLB-PSO method will eliminate VM downtime and the risk of losing the last activity performed by a customer, and will increase the Quality of Service experienced by cloud customers.

The consumption of energy and carbon emission in cloud datacenters are the alarming issues in recent times, while optimizing the average response time and service level agreement (SLA) violations. Handful of researches have been conducted in these domains during virtual machine placement (VMP) in cloud milieu. Moreover it is hard to find researches on VMP considering the cloud regions and the availability zones along with the datacenters, although both of them play significant roles in VMP. Hence, we have worked on a novel approach to propose a hybrid metaheuristic technique combining the salp swarm optimization and emperor penguins colony algorithm, i.e. SSEPC to place the virtual machines in the most suitable regions, availability zones, datacenters, and servers in a cloud environment, while optimizing the mentioned quality of service parameters. Our suggested technique is compared with some of the contemporary hybrid algorithms in this direction like Sine Cosine Algorithm and Salp Swarm Algorithm (SCA-SSA), Genetic Algorithm and Tabu-search Algorithm (GATA), and Order Exchange & Migration algorithm and Ant Colony System algorithm (OEMACS) to test its efficacy. It is found that the proposed SSEPC is consuming 4.4%, 8.2%, and 16.6% less energy and emitting 28.8%, 32.83%, and 37.45% less carbon, whereas reducing the average response time by 11.43%, 18.57%, and 26% as compared to its counterparts GATA, OEMACS, and SCA-SSA respectively. In case of SLA violations, SSEPC has shown its effectiveness by lessening the value of this parameter by 0.4%, 1.2%, and 2.8% as compared to SCA-SSA, GATA, and OEMACS respectively.

Fog computing (also known as edge computing) is a decentralized computing architecture that seeks to minimize service latency and average response time in IoT applications by providing compute and network services physically close to end-users. Fog environment consists of a network of fog nodes and IoT applications are composed of containerized microservices communicating with each other. Due to limited resources of fog nodes, it is often not possible to deploy all the containers of an application on a single fog node. Therefore, communicating containers need to be distributed on multiple fog nodes. Distribution and management of containerized IoT applications is always a critical issue to the system performance in a fog environment. Kubernetes, an open-source system, has grown into a container orchestration standard by simplifying the deployment and management of containerized applications. Despite the progress made by the academia and industry with respect to container management and the wide-scale acceptance of Kubernetes in cloud environments, container management in fog environment is still in the early stage in terms of research and practical deployment. This article aims to fill this gap by analyzing the expediency of Kubernetes container orchestration tool in the fog computing model. The paper also highlights limitations with the current Kubernetes approach and provide ideas for further research to adapt to the needs of the fog environment. Lastly, we provide experiments that demonstrate the feasibility and industrial practicality of deploying and managing containerized IoT applications in the fog computing environment.

Creating many replicas in the processing of data-intensive jobs in data grid is an efficient strategy. Replica replacement is the crucial step to this strategy. Economic model, popularity model and hybrid model etc. have been proposed to solve this issue of replica replacement with analysis and prediction based on each data file, however, these models neglect association relationships among different data files. To find out these association relationships hidden in data-intensive jobs, Apriori algorithm in data mining field is adopted to analyze behaviors of each data-intensive job. An associated replica replacement algorithm based on Apriori approach in data grid is proposed in this paper. This algorithm has two major steps: 1) associated behavior analysis and classification of data files in each node; 2) generation and application of replica replacement rules. Our proposed algorithm is simulated in Optorsim to be compared with LFU algorithm. The experiment shows that there is a relative advantage compared with LFU in mean job times of all jobs, number of remote file access and effective network usage perspectives.

Recently, the development of cloud computing has received considerable attention. For cloud service providers, packing VMs onto a small number of servers is an effective way to reduce energy costs, so as to improve the efficiency of the data center. However allocating too many VMs on a physical machine may cause some hot spots which violate the SLA of applications. Load balancing of the entire system is hence needed to guarantee the SLA. In this paper, we present a simulated-annealing load balancing algorithm for solving the resource allocation and scheduling problem in a cloud computing environment. Experimental results show that this method is able to achieve load balancing, and performs better than the round robin and basic simulated-annealing algorithms.

In this paper, we propose an adaptive data replication algorithm, called the Popular File Replicate First algorithm (PFRF for short), which is developed on a star-topology data grid with limited storage space based on aggregated information on previous file accesses. The PFRF periodically calculates file access popularity to track the variation of usersi¦ access behaviour behaviors, and then replicates popular files to appropriate sites to adapt to the variation. We employ several types of file access behaviors, including Zipf-like, geometric, and uniform distributions, to evaluate PFRF. The simulation results show that PFRF can effectively improve average job turnaround time and data availability as compared with those of the tested algorithms.