Conference Proceedings

Performance Analysis of Large-scale Distributed Stream Processing Systems on the Cloud

Minh Truong Tri, Aaron Harwood, Richard O Sinnott, Shiping Chen

IEEE International Conference on Cloud Computing, CLOUD | IEEE | Published : 2018


Real-time data processing is often a necessity as it can provide insights that have less value if discovered off-line or after the fact. However, large-scale stream processing systems are non-trivial to build and deploy. While there are many frameworks that allow users to create large-scale distributed systems, there remains many challenges in understanding the performance, cost of deployment and considerations and impact of potential (partial) outages on real-time systems performance. Our work considers the performance of Cloud-based stream processing systems in terms of back-pressure and expected utilization. The performance of an exemplar stream application is explored using different Clo..

View full abstract