Project
Event Conduit (webhook_infra)
An outbound event-delivery platform: it consumes internal events from Kafka, matches them to subscribers, translates them to CloudEvents, and delivers them as signed webhook batches or WebSocket streams.
Event Conduit delivers internal events to external subscribers in near real time. Events come in from Kafka, are matched against each subscriber’s interests, converted to CloudEvents 1.0, and sent either as HMAC-signed webhook batches or over WebSocket streams.
What it does
-
At-least-once delivery with strict per-subscription ordering.
-
Durable per-subscriber queues (RocksDB/Pebble) that survive restarts, with configurable retention and background compaction.
-
Sub-500 ms end-to-end latency using a Redis ready queue instead of polling; webhooks batched into single HTTP POSTs with a 5 ms linger window; WebSocket streaming with no linger.
-
Round-robin scheduling so no subscriber starves the others; horizontal scaling of every stage.
-
An admin dashboard for pausing, resuming, replaying, and statistics, plus test producer and consumer UIs.
How it is built
A set of Go services: a subscriber registry (REST), a config distributor (gRPC streaming fed by PostgreSQL LISTEN/NOTIFY), an ingestion router, a queue writer, a subscriber-queue service with delivery leases, a webhook dispatcher, a stream gateway, and an admin API — with Valkey (Redis compatible) for mappings, the ready queue, and notifications.