• Home
  • Search
  • Balancing load in stream processing with the cloud
  • Cite Icon47
  • https://doi.org/10.1109/icdew.2011.5767653Copy DOI Icon

Balancing load in stream processing with the cloud

  • Apr 1, 2011
  • Wilhelm Kleiminger +2 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Stream processing systems must handle stream data coming from real-time, high-throughput applications, for example in financial trading. Timely processing of streams is important and requires sufficient available resources to achieve high throughput and deliver accurate results. However, static allocation of stream processing resources in terms of machines is inefficient when input streams have significant rate variations-machines remain under-utilised for long periods of average load. We present a combined stream processing system that, as the input stream rate varies, adaptively balances workload between a dedicated local stream processor and a cloud stream processor. This approach only utilises cloud machines when the local stream processor becomes overloaded. We evaluate a prototype system with financial trading data. Our results show that it can adapt effectively to workload variations, while only discarding a small percentage of input data.

Similar Papers
  • Research Article
  • Citations108

A Comprehensive Survey on Parallelization and Elasticity in Stream Processing

  • Apr 30, 2019
  • ACM Computing Surveys
  • Henriette Röger +1
  • Book Chapter

Streaming Data Processing forIoT

  • Mar 06, 2020
  • The Internet of Things
  • Carolina Fortuna +1
  • Conference Article
  • Citations59

Query processing for temporal databases

  • Feb 05, 1990
  • T.Y.C Leung +1
  • Book Chapter
  • Citations12

Chapter 11 - A Taxonomy and Survey of Stream Processing Systems

  • Jan 01, 2017
  • Software Architecture for Big Data and the Cloud
  • Xinwei Zhao +3
  • Conference Article
  • Citations7

High-Performance Stateful Stream Processing on Solid-State Drives

  • Aug 27, 2018
  • Gyewon Lee +4
  • Book Chapter
  • Citations10

StroMAX: Partitioning-Based Scheduler for Real-Time Stream Processing System

  • Jan 01, 2017
  • Jiawei Jiang +4
  • Book Chapter
  • Citations4

A Review of Dynamic Scalability and Dynamic Scheduling in Cloud-Native Distributed Stream Processing Systems

  • Jan 01, 2020
  • Ambalavanar Senthuran +1
  • Conference Article
  • Citations1

Hardware Support for Secure Stream Processing in Cloud Environments

  • May 15, 2017
  • Jeff Anderson +1
  • Conference Article
  • Citations13

Jarvis: Large-scale Server Monitoring with Adaptive Near-data Processing

  • May 01, 2022
  • Atul Sandur +4
  • Research Article
  • Citations18

Runtime‐aware adaptive scheduling in stream processing

  • Oct 31, 2015
  • Concurrency and Computation: Practice and Experience
  • Yuan Liu +2
  • Book Chapter
  • Citations8

Spatial Interpolation of Streaming Geosensor Network Data in the RISER System

  • Jan 01, 2015
  • Xu Zhong +3
  • PDF
  • Research Article
  • Citations4

S2p: Provenance Research for Stream Processing System

  • Jun 15, 2021
  • Applied Sciences
  • Qian Ye +1
  • Conference Article
  • Citations62

A Hybrid Approach to High Availability in Stream Processing Systems

  • Jan 01, 2010
  • Zhe Zhang +6
  • Conference Article
  • Citations55

Adaptive Provisioning of Stream Processing Systems in the Cloud

  • Apr 01, 2012
  • Javier Cervino +3
  • Book Chapter
  • Citations8

Dynamically Scheduling a Component-Based Framework in Clusters

  • Jan 01, 2015
  • Aleksandra Kuzmanovska +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.