Backpressure and flow control

Reference Card

• Origin/Prior Art: TCP Window Size (RFC 793), Reactive Extensions (Rx).
• Related Practices: Circuit breaking, Rate limiting, Message queues.
• Key Concepts: Flow control, Dropping packets, Bounded queues.
• Primary Failure Modes: Unbounded memory growth leading to Out-Of-Memory (OOM) crashes, Silent data drops.

Introduction

When a high-throughput firehose of data (e.g., a massive stream of incoming HTTP requests or a Kafka topic) feeds into a slower downstream consumer (e.g., a database that takes 100ms per write), physics dictates that a bottleneck will form.

If the system lacks backpressure, the downstream consumer will blindly attempt to process everything. It will queue the excess data in RAM. Eventually, the RAM fills up, the operating system triggers the OOM killer, the consumer crashes, and the data is lost forever. Backpressure is the operational pattern of explicitly communicating, “I am full; stop sending data,” all the way up the chain to the original source.

Mental Model

Think of a funnel pouring water into a narrow pipe. If you pour water faster than the pipe can drain, the funnel fills up and eventually overflows. Backpressure is adding a sensor to the funnel. When the water level reaches 90%, the sensor sends a signal to the pitcher to stop pouring. When the pipe drains, the sensor says, “You may resume.” The pitcher holds the excess capacity, preventing a mess on the floor.

Operational Pattern

Backpressure can be implemented dynamically (in real-time) or structurally.

1. Dynamic Flow Control (TCP/HTTP2): The most foundational example is the TCP Receive Window. The consumer explicitly tells the sender how many bytes of buffer it has left. As the buffer fills, the consumer advertises a smaller window. When the window reaches zero, the sender mathematically cannot send another byte until the consumer processes data and re-opens the window.
2. Structural Backpressure (Bounded Queues): In application code, never use unbound arrays for processing queues. Always specify a maximum queue depth (e.g., 10,000 items). When the queue is full, the application must explicitly reject new items.
3. The 429 Signal: When the bounded queue fills, the application immediately rejects incoming HTTP requests with a 429 Too Many Requests or 503 Service Unavailable, forcing the upstream client to buffer the data itself and try again later.

Failure Modes

Unbounded In-Memory Queues

The most common mistake developers make when consuming from a message broker (like RabbitMQ) is fetching messages faster than they can be written to the database, storing the pending messages in a generic List or Array. During a traffic spike, this array grows to millions of items, consuming gigabytes of RAM until the container is violently terminated by the orchestrator.

The Deadlock

If Service A calls Service B, and Service B calls Service C, backpressure must propagate perfectly. If Service C is slow, it applies backpressure to B. B must immediately apply backpressure to A. If B tries to buffer the requests instead of passing the backpressure up to A, B will crash. The bottleneck is merely shifted, not solved.

Security

Denial of Service (DoS) attacks actively attempt to defeat backpressure mechanisms by opening thousands of slow connections. Properly implemented backpressure at the edge (via load balancers and API gateways) protects internal databases by aggressively shedding malicious load before it can consume downstream connection pools.

Operational Guidance

Always combine Backpressure with Circuit Breakers and Load Shedding. Backpressure is the polite request to slow down; Load Shedding is the violent act of dropping traffic when the polite request is ignored. When monitoring backpressure, track the “Queue Depth” metric on your workers. If queue depth is consistently hitting the maximum bound, it indicates a structural capacity issue (you need more workers), not just a temporary spike.

Related topics

• Circuit breaker pattern
• Rate limiting strategies

Sources & further reading

• Reactive Streams: Backpressure - Reactive Streams Initiative