Resource Exhaustion

"What happens when capacity is exceeded?"

Applies To

APIs, connection pools, queues, file uploads

Why It Happens

A first-principles walkthrough of why unbounded work always finds a limit — connection pools, memory, queue depth, file size. How an unbounded `await file.read()` OOMs the process, and how bounded pools, token-bucket rate limiting, chunked streaming, and backpressure shed load gracefully instead of crashing.

How It Works Underneath

Every resource has a ceiling: file descriptors, DB connections, memory, queue depth. An unbounded await file.read() loads a 500 MB upload into RAM, the process OOMs, the orchestrator restarts it, the client retries the same upload, and the loop repeats. The fix is to bound: pool(max_size=20), TokenBucket(capacity=100), read(64*1024) chunked, and backpressure — when full, return 429, do not queue forever.

Cataloged Failure Modes

• pool_exhaustion
• oom_on_large_file
• burst_overload

Code Comparison

# NAIVE: Reading full file into RAM
@app.post("/upload")
async def upload(file: UploadFile):
    return {"size": len(await file.read())}

# IMPROVED: Chunked streaming with size caps
@app.post("/upload")
async def upload(file: UploadFile):
    while chunk := await file.read(64 * 1024):
        await stream_save(chunk)

// NAIVE: Reading full file into RAM
app.post('/upload', async (req, res) => {
  const buf = await req.file.arrayBuffer();
  res.json({ size: buf.byteLength });
});

// IMPROVED: Chunked streaming with size caps
import { pipeline } from 'node:stream/promises';
app.post('/upload', async (req, res) => {
  for await (const chunk of req.file.stream()) {
    await streamSave(chunk); // 64KB chunks, backpressure handled
  }
  res.json({ ok: true });
});

Mitigation Patterns

• rate-limiting
• bulkhead