Close and return to the grid
All projects
2024 · Backend engineer

Signal

Streaming ingestion that survives its upstreams.

Numbers

2.5K/s
Sustained events
99.98%
Delivery

Overview

A pipeline that swallows bursty webhook traffic from partners with no rate limits and no retry etiquette. Backpressure is explicit: the queue is bounded, overflow goes to cold storage, and replay is a first-class operation instead of an incident.

Most of the work was in the failure modes. Every partner is a separate circuit, so one misbehaving integration degrades exactly one tenant.

Stack

GoRedisClickHouseGrafana