Tinkering With Ktor and Kafka

Summary I wanted to practice setting up Kafka Producers/Consumers in Kotlin. Things have changed quite a bit since the last time I’ve touched it, so I thought I’d share my experience with it while setting up on Ktor. Docker setup Zookeeper is no longer needed and its support has been removed since Kafka 4.0, this makes our docker-compose file a bit more lightweight. We need to generate a CLUSTER_ID, a 22 character, base64 encoded string. You can craft anything fitting you want here, but the recommended way is to use Kafka’s script. You can run it locally using: docker run --rm apache/kafka:4.1.0 /opt/kafka/bin/kafka-storage.sh random-uuid. Armed with a generated CLUSTER_ID and a simplified configuration for single node Kafka setup, we have something like this: Read more