Benchmarking Message Brokers on Kubernetes : A Case Study in how to Evaluate Inter-Service Communication Technology

Sammanfattning

Establishing a system with a microservice architecture involves many decisions, one of which can be what message broker to choose. Camjo AB is about to start working on its system and is looking for a way to evaluate the performance of two message brokers: Apache Kafka and RabbitMQ. Despite that much has been written about these two messaging systems, there is little about how the performance can be tested in a distributed cloud environment. With Azure Kubernetes Services and an open-source project for benchmarking messaging systems called OpenMessaging Benchmark Framework, performance can be tested. This thesis presents how this can be done using Docker as a tool for containerization and Helm-charts for defining and installing the OpenMessaging Benchmark Framework in a Kubernetes cluster. Performance is evaluated regarding throughput and latency by sending messages to the brokers using the OpenMessaging Benchmark Framework. Five types of tests give data to evaluate different aspects of the brokers' performance: one test for when the message size differs, the second for how the latency is affected by a given throughput, and the third for how throughput and latency are affected when used by multiple producers and consumers. The final two tests focus on how Kafka's performance varies when the Kafka-specific properties of topics and partitions differ. The benchmark results confirm that Kafka is the overall better-performing messaging broker of the two. Previous studies also suggested this, but it is now confirmed for Kubernetes by running tests with the OpenMessaging Benchmark Framework.