• Home
  • Search
  • Real-Time Fraud ML on Spark Structured Streaming: Micro-Batch vs. Continuous Processing
  • https://doi.org/10.58425/ajt.v4i4.465Copy DOI Icon

Real-Time Fraud ML on Spark Structured Streaming: Micro-Batch vs. Continuous Processing

Show More
  • Abstract
  • Literature Map
  • Similar Papers
Abstract

Aim: The aim of this study is to evaluate the effectiveness of machine learning based real time fraud detection using Apache Spark Structured Streaming, with specific emphasis on comparing the Micro Batch and Continuous Processing modes. Methods: The study develops and evaluates a real time fraud detection pipeline trained on the IEEE CIS Fraud Detection dataset. The methodology includes feature engineering, supervised machine learning models, and stream processing to support near real time fraud classification. Apache Spark Structured Streaming is implemented using both Micro Batch and Continuous Processing modes. Experimental evaluations compare the two modes across multiple performance metrics, including latency, precision, recall, resource utilization, and system reliability. Results: The results show that the Micro Batch mode provides strong analytical capabilities, robust fault tolerance, and support for complex transformations, albeit with a slight increase in processing latency. In contrast, the Continuous Processing mode delivers significantly lower latency and higher throughput, making it suitable for environments requiring rapid fraud alerting. However, it demonstrates limitations in supported transformations and recovery mechanisms. The comparative analysis indicates that neither processing mode consistently outperforms the other across all evaluation criteria. Conclusion: The study concludes that the choice between Micro Batch and Continuous Processing modes in Apache Spark Structured Streaming should be driven by specific application requirements rather than a general preference for one mode. Each mode presents distinct tradeoffs between latency, analytical flexibility, and reliability in real time fraud detection systems. Recommendations: The study recommends adopting a use case driven approach when selecting Spark processing modes for fraud detection applications. Future research should explore hybrid architectures that combine the strengths of both processing modes, as well as advanced techniques such as active learning and graph-based machine learning to enhance adaptability and accuracy in real time fraud mitigation systems.

Similar Papers
  • PDF
  • Research Article
  • Citations1

An ESG-Compliant Framework for Fraud Detection in Online Payments Using Data Privacy and Machine Learning

  • Nov 17, 2024
  • International Journal of Computing and Engineering
  • Ashutosh Ahuja
  • Research Article
  • Citations90

A comprehensive investigation of external mass transfer and intraparticle diffusion for batch and continuous adsorption of heavy metals using pore volume and surface diffusion model

  • Apr 02, 2022
  • Separation and Purification Technology
  • Mohd Danish +6
  • Research Article
  • Citations25

Efficient methane production from petrochemical wastewater in a single membrane-less microbial electrolysis cell: the effect of the operational parameters in batch and continuous mode on bioenergy recovery.

  • Feb 06, 2019
  • Journal of Environmental Health Science and Engineering
  • Amin Arvin +4
  • Research Article

EVALUATING THE IMPACT OF MULTI FACTOR AUTHENTICATION ON CYBERSECURITY EFFECTIVENESS

  • Dec 08, 2024
  • International Scientific Journal of Engineering and Management
  • Ranga Premsai
  • Research Article
  • Citations2

Evaluation of the robustness of the enzymatic hydrolysis in batch and continuous mode by a central composite design

  • May 05, 2017
  • Journal of Food Processing and Preservation
  • N Abdi +6
  • Research Article
  • Citations33

Kinetic assessment of antibiotic resistant bacteria inactivation by solar photo-Fenton in batch and continuous flow mode for wastewater reuse

  • May 07, 2019
  • Water Research
  • I De La Obra Jiménez +4
  • Research Article
  • Citations99

Bioelectrochemical conversion of CO2 to chemicals: CO2 as a next generation feedstock for electricity-driven bioproduction in batch and continuous modes.

  • Jan 01, 2017
  • Faraday Discussions
  • Suman Bajracharya +4
  • Research Article
  • Citations31

High-capacity amino-functionalized walnut shell for efficient removal of toxic hexavalent chromium ions in batch and column mode

  • Feb 01, 2022
  • Journal of Environmental Chemical Engineering
  • Evans Dovi +6
  • Conference Article

Copper nanoparticle synthesis in continuous mode by the polyol method - Progress towards Nanomanufacturing

  • Aug 01, 2011
  • Sudhir Ramprasad +5
  • Research Article

Assessment of the Biological Treatability of Black Tea Processing Effluent

  • Aug 16, 2007
  • Journal of Civil Engineering Research and Practice
  • Z.N I Oonge +1
  • Research Article
  • Citations1

Technological Support of Real-Time Interaction in Web Clients of Analytical Fraud Detection Systems

  • Feb 15, 2018
  • KnE Social Sciences
  • Radygin V.Y +1
  • Research Article
  • Citations6

Construction and performance assessment of Recirculating packed bed biofilm reactor (RPBBR) for effective biodegradation of p-cresol from wastewater

  • Jun 19, 2023
  • Bioresource Technology
  • Vivek Kumar Jaiswal +2
  • Research Article
  • Citations12

Low-temperature evaporation of continuous pharmaceutical process streams in a bubble column

  • Sep 12, 2020
  • Chemical Engineering Research and Design
  • Phillip Roche +3
  • Research Article
  • Citations19

Alteration of Tecoma chip wood waste into microwave-irradiated activated carbon for amoxicillin removal: Optimization and batch studies

  • Jun 28, 2023
  • Arabian Journal of Chemistry
  • Mohamad Nasran Nasehir Khan +3
  • Research Article
  • Citations27

Performance of a solid-state immersion bioreactor for ligninolytic enzyme production: evaluation of different operational variables

  • Apr 24, 2002
  • Process Biochemistry
  • Susana Rodrı́Guez Couto +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.