Scaling
Vertical vs horizontal, replicas, sharding
The problem
A system that works for 100 users breaks differently at 1M and differently again at 100M. Scaling is knowing what breaks next and fixing it before it does — and, just as importantly, knowing when not to scale.
Two directions
Vertical (up): bigger machine. No code changes, doable this afternoon — but there’s a ceiling, the price curve goes exponential, and it’s still one machine that can die. Horizontal (out): more machines. No ceiling — but it demands a load balancer and stateless services. "Stateless servers are easy to load balance; state is what makes scaling hard" is the price of admission.
Scaling the database: copy or split
Read replicas copy the data: writes go to one primary, reads spread across copies. Perfect for read-heavy workloads — but replicas trail the primary by a beat (replication lag), and they don’t absorb writes, they multiply them: every write replays on every copy. Sharding splits the data instead: each shard owns a subset by shard key. Writes finally divide — at the cost of painful cross-shard queries and total dependence on choosing the key well.
The playbook — cheapest and most reversible first
1) Optimize the code (indexes, dumb queries). 2) Add caching. 3) Horizontally scale the stateless app tier — boring and reversible. 4) Read replicas for read-heavy load. 5) Async queues for deferrable work (floats earlier when obvious). 6) Shard — last, because it’s surgery on the data layer and brutally hard to undo.
Back-of-envelope numbers
A day ≈ 100K seconds. One server ≈ 1–10K requests/sec. 1M DAU × 10 actions ≈ 10M/day ≈ 100 QPS average; peak ≈ 5× average. An image ≈ 200KB–1MB. Run this math before designing: "1M DAU → ~500 QPS peak → one Postgres with an index and a cache is fine" is the strongest scaling sentence an interview can contain.
When things break
Hot shard: the celebrity lands on shard 3 and melts it — the cache block’s hot key at database scale (fixes: composite keys, shard by content ID, special-case the celebrities). Cascading failure: one slow service makes callers pile up, then their callers, until everything falls over — mitigated by timeouts, retries with backoff and jitter, and circuit breakers. Scaling too early is also a failure mode: complexity you paid for without evidence you needed it.