• Home
  • Search
  • Handling Data Skew in MapReduce Cluster by Using Partition Tuning
  • Cite Icon26
  • https://doi.org/10.1155/2017/1425102Copy DOI Icon

Handling Data Skew in MapReduce Cluster by Using Partition Tuning

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

The healthcare industry has generated large amounts of data, and analyzing these has emerged as an important problem in recent years. The MapReduce programming model has been successfully used for big data analytics. However, data skew invariably occurs in big data analytics and seriously affects efficiency. To overcome the data skew problem in MapReduce, we have in the past proposed a data processing algorithm called Partition Tuning-based Skew Handling (PTSH). In comparison with the one-stage partitioning strategy used in the traditional MapReduce model, PTSH uses a two-stage strategy and the partition tuning method to disperse key-value pairs in virtual partitions and recombines each partition in case of data skew. The robustness and efficiency of the proposed algorithm were tested on a wide variety of simulated datasets and real healthcare datasets. The results showed that PTSH algorithm can handle data skew in MapReduce efficiently and improve the performance of MapReduce jobs in comparison with the native Hadoop, Closer, and locality-aware and fairness-aware key partitioning (LEEN). We also found that the time needed for rule extraction can be reduced significantly by adopting the PTSH algorithm, since it is more suitable for association rule mining (ARM) on healthcare data.

Loading PDF

Similar Papers
  • PDF
  • Research Article
  • Citations138

Big Data in Healthcare Management: A Review of Literature

  • Jan 01, 2018
  • American Journal of Theoretical and Applied Business
  • Senthilkumar Sa
  • Conference Article
  • Citations21

Impact of big data in healthcare: A survey

  • Dec 01, 2016
  • D K Thara +3
  • Research Article

An Analysis on Future Prospects of Big Data in Healthcare

  • May 27, 2020
  • SSRN Electronic Journal
  • Abhishek Kumar +4
  • Front Matter
  • Citations10

Taking a byte out of big data

  • Oct 26, 2015
  • The Journal of the American Dental Association
  • Michael Glick
  • Book Chapter
  • Citations1

Utilizing Big Data for Health Care Automation: Obligations, Fitness and Challenges

  • Dec 31, 2018
  • Sherin Zafar
  • Abstract
  • Citations4

PHP136 - Big Data In Healthcare - Opportunities And Challenges

  • Oct 31, 2016
  • Value in Health
  • F Leppert +1
  • PDF
  • Research Article
  • Citations3

Legal Governance of Brain Data Derived from Artificial Intelligence

  • Jun 02, 2021
  • Voices in Bioethics
  • Mahika Ahluwalia
  • Research Article
  • Citations12

Big Data in Healthcare - Defining the Digital Persona through User Contexts from the Micro to the Macro. Contribution of the IMIA Organizational and Social Issues WG.

  • Aug 01, 2014
  • Yearbook of medical informatics
  • H Monkman +6
  • Conference Article
  • Citations1

Finding group structures in “Big Data” in healthcare research using mixture models

  • Dec 01, 2016
  • Shu-Kay Ng +1
  • Research Article
  • Citations34

Mining big building operational data for improving building energy efficiency: A case study

  • Apr 21, 2017
  • Building Services Engineering Research and Technology
  • Cheng Fan +1
  • Book Chapter

Big Data in Health Care: Opportunities, Challenges and Future Direction

  • May 11, 2025
  • Naheeda Zaib +2
  • Research Article
  • Citations3

Big data analytics in healthcare: opportunities, challenges and techniques

  • Jan 01, 2017
  • International Journal of Social Computing and Cyber-Physical Systems
  • Madhu Kumari +1
  • PDF
  • Research Article
  • Citations11

A Maturity Analysis of Big Data Technologies

  • Mar 30, 2017
  • Informatica Economica
  • Radu Boncea +3
  • Conference Article
  • Citations4

A big data framework for diabetes in Mauritius

  • Dec 01, 2017
  • Ravesh Joaheer +1
  • Discussion
  • Citations7

Invited Commentary on Stewart and Davis " 'Big data' in mental health research-current status and emerging possibilities".

  • Oct 25, 2016
  • Social Psychiatry and Psychiatric Epidemiology
  • Jonathan D Hafferty +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.