Conference Proceedings

Predicting the stability of large-scale distributed stream processing systems on the cloud

TM Truong, A Harwood, RO Sinnott

Proceedings of the 7th International Conference on Cloud Computing and Services Science | SciTePress | Published : 2017

Abstract

©2017 by SCITEPRESS - Science and Technology Publications, Lda. All rights reserved. Large-scale topology-based stream processing systems are non-Trivial to build and deploy. They require understanding of the performance, cost of deployment and considerations of potential downtime. Our work considers stability as a primary characteristic of these systems. By stability, we mean that unstable systems exhibit large-spikes in latency and can drop throughput frequently or unpredictably. Such instabilities can be due to variations of workloads or underlying hardware platforms that are often difficult to predict. To understand and tackle this for large-scale stream processing systems, we apply queu..

View full abstract