• Home
  • Search
  • RPig: A scalable framework for machine learning and advanced statistical functionalities
  • Cite Icon9
  • https://doi.org/10.1109/cloudcom.2012.6427480Copy DOI Icon

RPig: A scalable framework for machine learning and advanced statistical functionalities

  • Dec 1, 2012
  • Mingxue Wang +2 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

In many domains such as Telecom various scenarios necessitate the processing of large amounts of data using statistical and machine learning algorithms. A noticeable effort has been made to move the data management systems into MapReduce parallel processing environments such as Hadoop and Pig. Nevertheless these systems lack the features of advanced machine learning and statistical analysis. Frame-works such as Mahout on top of Hadoop support machine learning but their implementations are at the preliminary stage. For example Mahout does not provide Support Vector Machine (SVM) algorithms and it is difficult to use. On the other hand traditional statistical software tools such as R containing comprehensive statistical algorithms for advanced analysis are widely used. But such software can only run on a single computer and therefore it is not scalable. In this paper we propose an integrated solution RPig which takes the advantages of R (for machine learning and statistical analysis capabilities) and parallel data processing capabilities of Pig. The RPig framework offers a scalable advanced data analysis solution for machine learning and statistical analysis. Analysis jobs can be easily developed with RPig script in high level languages. We describe the design implementation and an eclipse-based RPigEditor for the RPig framework. Using application scenarios from the Telecom domain we show the usage of RPig and how the framework can significantly reduce the development effort. The results demonstrate the scalability of our framework and the simplicity of deployment for analysis jobs.

Similar Papers
  • Research Article
  • Citations132

Food flavor analysis 4.0: A cross-domain application of machine learning

  • Jun 10, 2023
  • Trends in Food Science & Technology
  • Xiangquan Zeng +7
  • Research Article
  • Citations1

Deciphering the complex links between inflammatory bowel diseases and NAFLD through advanced statistical and machine learning analysis

  • Jan 01, 2024
  • Computer Methods and Programs in Biomedicine Update
  • Deepak Kumar +3
  • Conference Article
  • Citations11

Winding Type Recognition through Supervised Machine Learning using Frequency Response Analysis (FRA) Data

  • Apr 01, 2019
  • Xiaozhou Mao +3
  • Book Chapter
  • Citations45

CloudSVM: Training an SVM Classifier in Cloud Computing Systems

  • Jan 01, 2013
  • F Ozgur Catak +1
  • Research Article
  • Citations122

Extracorporeal life support

  • Jan 01, 2010
  • BMJ
  • A M Gaffney +4
  • Preprint Article
  • Citations1

Google’s cloud computing platform performance assessment of spatio-temporal and estimation for global water quality analysis and surface water mapping using integrated satellite data of MODIS (TERRA), Landsat- (8, 9), Sentinel‑2A/B and Planets

  • Jan 20, 2025
  • Abhinav Galodha +3
  • Research Article

Predicting Adolescent Risk Behavior via Family Monitoring, Sensation Seeking, and Peer Deviance with Machine Learning Analysis

  • Jan 01, 2026
  • Journal of Psychosociological Research in Family and Culture
  • Andre Mckenzie +2
  • PDF
  • Research Article
  • Citations34

Synchronization patterns reveal neuronal coding of working memory content.

  • Aug 01, 2021
  • Cell Reports
  • Fahimeh Mamashli +7
  • Research Article

Machine Learning Analysis in the Field of Heart Disease.

  • Jul 05, 2024
  • Science and Technology of Engineering, Chemistry and Environmental Protection
  • Qianjun Zheng
  • Research Article

An efficient non-invasive model for predicting cognitive impairment based on comprehensive geriatric assessment: Machine learning and SHAP analysis

  • Jan 01, 2026
  • Journal of Alzheimer's Disease Reports
  • Jia Zhang +6
  • Supplementary Content
  • Citations231

Review of Machine Learning Algorithms for Diagnosing Mental Illness

  • Apr 01, 2019
  • Psychiatry Investigation
  • Gyeongcheol Cho +4
  • Research Article

Abstract A40: The SIAMCAT R package enables statistical and machine learning analyses for case-control microbiome datasets

  • Apr 15, 2020
  • Cancer Research
  • Jakob Wirbel +8
  • Research Article
  • Citations2

Machine learning for classifying chronic ankle instability based on ankle strength, range of motion, postural control and anatomical deformities in delivery service workers with a history of lateral ankle sprains.

  • Feb 01, 2025
  • Musculoskeletal science & practice
  • Ui-Jae Hwang +3
  • Research Article

Breast Cancer Classification based on Ultrasound Images using the Support Vector Machine (SVM) Algorithm

  • Jul 29, 2024
  • SISTEMASI
  • Nurazmi Aprilia +1
  • Research Article

Research on optimization strategy and practice of university course management based on machine learning and big data analysis

  • Jul 14, 2025
  • Journal of Computational Methods in Sciences and Engineering
  • Yanjie He
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.