Project
Kafka broker discovery
A Java library and discovery server that let Kafka producers and consumers look up brokers and security settings by topic, then talk to Kafka directly.
Configuring every Kafka client with the right brokers and security settings for every environment is repetitive and error-prone. This project moves that knowledge into a discovery server: clients ask for a topic’s metadata and get back standard Kafka producers and consumers, already configured, that then talk to the brokers directly.
What it does
-
A discovery server with REST APIs for broker and topic metadata across environments (dev, staging, production), with health and admin endpoints.
-
A client library that looks up brokers and security settings (SSL, SASL) and builds ordinary Kafka producers and consumers — no proxy in the data path.
-
Producer and consumer interceptors for debugging and message augmentation.
How it is built
Plain Java: a server module (discovery service and metadata store), a client module (discovery client and producer/consumer wrappers over a REST metadata service), shared models and interceptor interfaces, and a full integration example. A Docker Compose file brings up Kafka and Kafka UI for local testing.