Written from production.

One production failure per post, dissected in 10–15 minutes.

RACK 01
retries

Idempotency keys that survive concurrent retries

A client times out, retries — and the customer is charged twice

caching

Cache stampede: one expired key, one dead database

A hot key expires — a thousand requests rebuild it against the database at once

postgres

The five-second ALTER TABLE that took the site down

"Instant" DDL queued behind one long transaction — and every query queued behind it