Improving MapReduce Performance with Progress and Feedback Based Speculative Execution

Size of the data used by enterprises has been growing at exponential rates since last few years; handling such huge data from various sources is a challenge for Businesses. In addition, Big Data becomes one of the major areas of research for Cloud Service providers due to a large amount of data produced every day, and the inefficiency of traditional algorithms and technologies to handle these large amounts of data. In order to resolve the aforementioned problems and to meet the increasing demand for high-speed and data-intensive computing, several solutions have been developed by researches and developers. Among these solutions, there are Cloud Computing tools such as Hadoop MapReduce and Apache Spark, which work on the principles of parallel computing. This chapter focuses on how big data processing challenges can be handled by using Cloud Computing frameworks and the importance of using Cloud Computing by businesses

Speculative plan execution for information gathering

As a result of the continuing information explosion, many organizations are drowning in data and the resulting “data gap” or inability to process this information and use it effectively is increasing at an alarming rate. Data-intensive computing represents a new computing paradigm (Kouzes, Anderson, Elbert, Gorton, & Gracio, 2009) which can address the data gap using scalable parallel processing to allow government, commercial organizations, and research environments to process massive amounts of data and implement applications previously thought to be impractical or infeasible. Cloud computing provides the opportunity for organizations with limited internal resources to implement large-scale data-intensive computing applications in a cost-effective manner.

The data-intensive computing generates a huge number of data in wide area network. The DataGrid technology tries to manage such distributed data on the Internet to provide the quick and efficient data search/access mechanism. The difficulties of the data access on DataGrid systems is caused from the differences in the data management manner and policy among organizations which manage storage resources. In this paper, we propose the new distributed data management scheme and design for data-intensive computing. Especially, we focus on the data attributes. We define the pairs of data attribute and its values as its metadata. In our system, users can be find/access data with the metadata. We have been developing the prototype systems. We show the usage of our system with the applications.

With the advent of big data era, data intensive computing has been widely concerned by the industry, but the research on data intensive center is still in its infancy. With the development of science and technology, the emergence of cloud computing has brought great convenience to people, and has become a research hotspot for a time. This paper analyzes the application status of cloud computing in data intensive data center. In the research, we analyzed the application status of cloud computing in data intensive data center. Taking data decision-making as an example, we studied the application effect of cloud computing in data intensive data center by analyzing the influence of different number of data sets and data centers on the time cost of data transmission. The results of this paper show that by 2019, the corresponding rate of cloud computing in data decision-making, data security and data management has reached 88%, 92% and 95% respectively. Moreover, the research results show that the application of cloud computing in data intensive data center can effectively reduce the time cost of data transmission and achieve good application effect.

Many Big Data processing applications nowadays run on large-scale multi-tenant clusters. Due to hardware heterogeneity and resource contentions, straggler problem has become the norm rather than the exception in such clusters. To handle the straggler problem, speculative execution has emerged as one of the most widely used straggler mitigation techniques. Although a number of speculative execution mechanisms have been proposed, as we have observed from real-world traces, the questions of when and where to launch speculative copies have not been fully discussed and hence cause inefficiencies on the performance and energy of Big Data applications. In this paper, we propose a performance model and an energy consumption model to reveal the performance and energy variations with different speculative execution solutions. We further propose a window-based dynamic resource reservation and a heterogeneity-aware copy allocation technique to answer the when and where questions for speculative executions. Evaluations using real-world traces show that our proposed technique can improve the performance of Big Data applications by up to 30% and reduce the overall energy consumption by up to 34%.

In this era of the internet, a huge amount of news articles are added every minute of everyday. As a result of this evergrowing amount of news articles, news retrieval systems are required to process the news articles frequently and intensively. The news retrieval systems that are in use today barely cope up with these data-intensive computations. Cloudpress 2.0 presented here, is designed and implemented to be scalable, robust and fault tolerant. It is designed in such a way that all the processes involved in news retrieval, such as fetching, text processing, image processing, indexing, storing and summarising, exploit MapReduce paradigm and use the power of the cloud computing. It uses novel approaches for parallel processing, for storing the news articles in a distributed database and for visualising them as a 3D visual. It uses Lucene-based indexing for efficient and faster retrieval. It also includes a novel query expansion feature for searching the news articles. Cloudpress 2.0 also allows on-the-fly, extractive summarisation of news articles based on the input query.

Over the past decades, seismological data centers within the International Federation of Digital Seismograph Networks (FDSN; www.fdsn.org) have worked together to establish and implement standards for waveform data (miniSEED), metadata (StationXML), and web service APIs for data discovery, distribution and quality control. These have been broadly adopted, and today enable the seismological community to seamlessly access data from FDSN data centers across the world. Meanwhile, driven by applications related to environmental monitoring and Machine Learning, users increasingly demand access to massive data volumes and intensive computation, best addressed by using computational resources near to the data. Despite efforts to make an unprecedented amount of data openly and FAIRly available, the existing synchronous services, such as fdsnws-dataselect, are hampering the full use of data by the next generation of scientists. Faced with this challenge, some data centers are starting to design asynchronous data access mechanisms optimized for large volumes and efficient handling in object storage facilities. New use cases that are difficult to realize with the current infrastructure or to describe with current metadata formats include, for example, those involving large volumes generated by Distributed Acoustic Sensing (DAS) as well as dense, multidisciplinary experiments bridging land and sea observations. In spring 2024, supported by the Geo-INQUIRE project, European data centers representing the European Integrated Data Archive (EIDA; www.orfeus-eu.org/data/eida) within ORFEUS (www.orfeus-eu.org, a part of the EPOS Seismology TCS), and EarthScope (www.earthscope.org), met to coordinate joint developments to make the next-generation seismological data centers ready to address these challenges. This includes modern cloud-based storage systems, versatile formats for data and metadata, asynchronous data access, on-cloud processing, Quality Assurance (QA) and common Authentication and Authorization Infrastructure (AAI). This presentation will outline the envisioned framework, highlight progress in its various components, and present a roadmap for its realisation, to be coordinated within the FDSN over the coming years. It will also provide an opportunity for stakeholders - such as users, instrument manufacturers, data and service providers - to engage and shape the vision.

An emerging technology in data analytics, Cloud Computing (CC) is used to store, retrieve, and distribute data in a dispersed setting. Both individuals and businesses save their data on cloud servers. However, data privacy and security have become a rising problem that limits the organization from employing cloud services. In order to overcome that problem, the Paillier encryption method is proposed to offer safety for healthcare cloud data. Creating a pair of keys offers a comparatively easier method to ward against attacks and safeguard data confidentiality. Following data selection, the Paillier methodology is used to encrypt the input. The encoded health data were then kept in cloud environments, where they provided better read-write storage operations. The Paillier method is used in the data decryption procedure to get the data from the cloud environment. A Recurrent Neural Network (RNN) classifier is employed in healthcare diagnosis to categorize and forecast a patient’s ailment, after which the diagnosis is communicated to the patient and physician. The proposed work uses Python software, and a comparative analysis is conducted. An effective prediction using RNN showcases an accuracy of 96.4% with a better performance matrix. After classification, the outcomes are conveyed to patients and doctors for treatment.

Intercloud is a federated environment of private clusters and public clouds. The performance of Hadoop could be degraded significantly in intercloud environments. Because previous solutions for intercloud environments rely on speculative execution, they require additional cost in the cloud. In this paper, we propose a new task scheduler that improves performance without the help of speculative execution in intercloud environments.

Cloud computing has emerged as a transformative technology, offering diverse service models to meet varying organizational needs. This study employs the Complex Proportional Assessment (COPRAS) method to comprehensively evaluate and analyze different cloud environments, providing a nuanced approach to cloud service selection. The research investigates five distinct cloud environment types: general-purpose, service-centric, zone-centric, distance-centric, and cost-centric, examining their performance across critical parameters including quality of service, number of available services, availability zones, consumer distance, and hourly cost. The multi-criteria analysis reveals significant variations in cloud environment effectiveness. Zone-centric environments emerged as the top performer, achieving a remarkable quality index of 0.291 and a 100% usability degree. Service-centric environments followed closely, demonstrating a 92.03% usability degree and highlighting the importance of service availability. General-purpose environments showed moderate performance with a 63.97% usability degree, indicating their versatility. Conversely, distance-centric and cost-centric environments exhibited the lowest performance, suggesting limitations in meeting comprehensive organizational requirements. Key findings underscore that cloud computing is not a one-size-fits-all solution but a complex ecosystem requiring strategic selection. The research emphasizes that while cost is important, it should not be the sole determining factor in cloud environment selection. The study provides a robust framework for decision-makers, enabling them to align cloud infrastructure choices with specific organizational objectives. The methodology offers critical insights into the evolving cloud computing landscape, addressing the growing complexity of data management and the increasing demand for scalable, secure computing solutions. By presenting a comprehensive evaluation approach, the research contributes to a more sophisticated understanding of cloud environment selection, encouraging organizations to adopt a strategic, multi-dimensional approach to cloud infrastructure deployment.

Uncountable loops (such as while loops in C) and if-conditions are some of the most common constructs in programming. While-loops are widely used to determine the convergence in linear algebra algorithms or goal finding problems from graph algorithms, to name a few. In general while-loops are used whenever the loop iteration space, the number of iterations a loop executes is unknown. Usually in while-loops, the execution of the next iteration is decided inside the current loop iteration (i.e. the execution of iteration i depends on the values computed in iteration i-1). This precludes their parallel execution in today's ubiquitous multi-core architectures. In this paper a technique to speculatively create parallel tasks from the next iterations before the current one completes is proposed. If consecutive loop-iterations are only control dependent, then multiple iterations can be executed simultaneously; later in the execution path, the runtime system will decide to either commit the results of such speculatively executed iterations or undo the changes made by them. Data dependences within or between non-speculative and speculative work are honored to guarantee correctness. The proposed technique is implemented in SMPSs, a task-based dataflow programming model for shared-memory multiprocessor architectures. The approach is evaluated on a set of applications from graph algorithms and linear algebra. Results are promising with an average increase in the speedup of 1.2x with 16 threads when compared to non speculative execution of the applications. The increase in the speedup is significant, since the performance gain is achieved over an already parallelized version of the benchmarks.

Hybrid grey wolf and improved particle swarm optimization with adaptive intertial weight-based multi-dimensional learning strategy for load balancing in cloud environments

Now-a-days, cloud computing has become a cornerstone of modern technology, driving innovation, efficiency and accessibility across various industries and application. Distributed computing solves the difficulty of using distributed autonomous machines and communicating with each other over a network. Cloud computing provides clients with a range of services and capabilities that enhances productivity, accessibility and scalability while reducing the need for extensive hardware and infrastructure investments. When interest in distributed computing rises, it implies that more individuals, businesses, and organizations are exploring, adopting, and implementing distributed computing solutions. This surge in interest leads to increase in data traffic. There are two solutions for the issue of increase in data traffic, one is to server optimization (or) server performance enhancement (upgrade single server to a high performance server) but upgraded server may exceed its capacity (overload).Another one is multi servers. Ulti-server configurations are common in scenarios where the demands of an application or service exceed the capabilities of a single server, and the distribution of tasks across multiple servers is necessary for optimal performance and reliability. When there are multiple servers, there is an issue i.e. Load Adjusting. [1] Load balancing is one of the critical issues in cloud computing. In a cloud environment, where resources are often dynamically allocated and distributed, load balancing plays a central role in managing workloads efficiently. Load balancing in cloud computing is a technique used to distribute computing workloads and network traffic across multiple servers or resources within a cloud environment. The primary goal of load balancing is to optimize resource utilization, prevent individual servers from becoming overloaded, and ensure that the overall system can handle varying levels of demand efficiently. Here we also discussed about the hybrid of Cat and Mouse Optimization and Grey Wolf Optimization algorithms. This paper refers to Cloud Computing, Load balancing techniques, Load balancing algorithms.

Cloud computing not only provides convenience for users, but also brings certain information security threats to users. How to effectively protect the user’s information security has become one of the core issues of network security. This paper analyzes the problems and reasons of network security in cloud computing environment, and points out the main ways to improve network security. This paper analyzes the network security problems in the cloud computing environment, and discusses the information security technology in the network. This paper studies the definition, characteristics, classification and core technology of cloud computing security, analyzes the security risk of cloud computing network, and puts forward the cloud computing network security model and security level evaluation method from different angles. Level experiment results show that the network security model in cloud environment is feasible, computing can fully meet the development requirements of the current network information age, and effectively standardize the network security operation environment. The results show that: in 2016, the number of articles related to cloud computing is at least 2653, and in 2020, the number of articles related to cloud computing is at most 6438.