• Home
  • Search
  • MRPack: Multi-Algorithm Execution Using Compute-Intensive Approach in MapReduce
  • Cite Icon11
  • https://doi.org/10.1371/journal.pone.0136259Copy DOI Icon

MRPack: Multi-Algorithm Execution Using Compute-Intensive Approach in MapReduce

  • Aug 25, 2015
  • PLoS ONE
  • Muhammad Idris +5 more
Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Large quantities of data have been generated from multiple sources at exponential rates in the last few years. These data are generated at high velocity as real time and streaming data in variety of formats. These characteristics give rise to challenges in its modeling, computation, and processing. Hadoop MapReduce (MR) is a well known data-intensive distributed processing framework using the distributed file system (DFS) for Big Data. Current implementations of MR only support execution of a single algorithm in the entire Hadoop cluster. In this paper, we propose MapReducePack (MRPack), a variation of MR that supports execution of a set of related algorithms in a single MR job. We exploit the computational capability of a cluster by increasing the compute-intensiveness of MapReduce while maintaining its data-intensive approach. It uses the available computing resources by dynamically managing the task assignment and intermediate data. Intermediate data from multiple algorithms are managed using multi-key and skew mitigation strategies. The performance study of the proposed system shows that it is time, I/O, and memory efficient compared to the default MapReduce. The proposed approach reduces the execution time by 200% with an approximate 50% decrease in I/O cost. Complexity and qualitative results analysis shows significant performance improvement.

Loading PDF

Similar Papers
  • Research Article
  • Citations7

Cloud computing and big data: Technologies and applications

  • May 20, 2018
  • Concurrency and Computation: Practice and Experience
  • Mostapha Zbakh +3
  • PDF
  • Research Article
  • Citations11

A Maturity Analysis of Big Data Technologies

  • Mar 30, 2017
  • Informatica Economica
  • Radu Boncea +3
  • Conference Article

Survey on Scientific Data Processing Using Hadoop MapReduce in Cloud Environments

  • May 11, 2012
  • Xiangming Kong
  • Conference Article

IEEE IRI 2014 tutorial: Big data management and mining

  • Aug 01, 2014
  • Latifur R Khan +1
  • Book Chapter

Big Data Processing on Cloud Computing Using Hadoop Mapreduce and Apache Spark

  • Jan 01, 2018
  • Yassir Samadi +2
  • Conference Article
  • Citations2

A hybrid mapreduce model for prolog

  • Dec 01, 2014
  • Joana Corte-Real +2
  • Dissertation

A framework for clustering and adaptive topic tracking on evolving text and social media data streams.

  • Feb 07, 2018
  • Gopi Chand Nutakki
  • Book Chapter

Video Search Using Map Reduce Framework in an Education Cloud

  • Sep 24, 2013
  • E Iniya Nehru +3
  • Research Article
  • Citations2

Text Classification Using Genetic Programming with Implementation of Map Reduce and Scraping

  • May 08, 2023
  • JOIV : International Journal on Informatics Visualization
  • Wirarama Wedashwara +4
  • PDF
  • Research Article

A SURVEY ON PRIVACY PRESERVATIONTECHNIQUES FOR DATA CLUSTERING K-MEANS OVER LARGE-SCALE DATASET

  • Feb 20, 2018
  • International Journal of Advanced Research in Computer Science
  • C Srihitha
  • Conference Article
  • Citations5

FedLoop: Looping on Federated MapReduce

  • Sep 01, 2014
  • Chun-Yu Wang +5
  • PDF
  • Research Article
  • Citations16

A New Architecture for Real Time Data Stream Processing

  • Jan 01, 2017
  • International Journal of Advanced Computer Science and Applications
  • Soumaya Ounacer +4
  • Conference Article
  • Citations6

An efficient framework of data mining and its analytics on massive streams of big data repositories

  • Aug 01, 2016
  • D N Disha +3
  • Front Matter
  • Citations10

Taking a byte out of big data

  • Oct 26, 2015
  • The Journal of the American Dental Association
  • Michael Glick
  • Research Article

Implementation of Multi Node Hadoop Virtual Cluster on Open Stack Cloud Environments

  • Jan 01, 2019
  • International Journal of Business Intelligence and Data Mining
  • Karthikeyan Saminathan +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.