AdvancedSystem Design · Lesson 15 of 19 · Updated ·
System Design Case Study — Design a News Feed Like Instagram or LinkedIn
"Design the home feed of Instagram (or LinkedIn, or Twitter/X)" is one of the most asked system design questions. The core challenge fits in one sentence: when a user opens the app, show the latest posts from everyone they follow — fast — even though some accounts have millions of followers. Let's solve it with the 6-step method from the first lesson.
Step 1 — Requirements
Functional
- A user can create a post (text, photo, video link).
- A user can follow and unfollow others.
- The home feed shows recent posts from followed accounts, newest first (ranking comes later).
- Scrolling loads older posts.
Non-functional
- Feed loads fast: under about 200 ms for the first page.
- Highly available: a feed that is a few seconds out of date is fine (eventual consistency).
- Read-heavy: people scroll far more often than they post.
Step 2 — Estimates
Assume 100 million daily active users, each opening the feed 10 times a day, and 20 million new posts a day. Average 200 followers per user.
| Quantity | Calculation | Result |
|---|---|---|
| Feed reads / second | 100 M × 10 ÷ 86,400 | ≈ 11,574 |
| New posts / second | 20 M ÷ 86,400 | ≈ 231 |
| Feed inserts if every post is pushed to every follower | 231 × 200 | ≈ 46,296 per second |
Reads outnumber posts about 50 to 1. The interesting number is the last one: how much work does it take to put each post into every follower's feed? That depends on the fan-out strategy.
Step 3 — API
POST /v1/posts { "text": "...", "media_ids": [...] } -> 201 { "id": "..." }
POST /v1/users/{id}/follow -> 204
GET /v1/feed?cursor=...&limit=20 -> 200 { "items": [...], "next_cursor": "..." }
The feed uses cursor pagination (the API design lesson explains why): new posts arrive constantly, and offsets would show repeats.
Step 4 — Fan-out: push or pull?
Fan-out means delivering one post to many followers' feeds. There are two opposite ways to do it.
Fan-out on write (push). When Asha posts, a background worker inserts the post id into the stored feed of each of her followers. Reading a feed is then a single fast lookup: "give me the first 20 ids in Ravi's feed list".
- ✅ Very fast reads — and reads are the common case.
- ❌ Expensive writes for popular accounts: a cricketer with 50 million followers creates 50 million feed inserts for one post. Delivery can take minutes.
- ❌ Wasted work for followers who never open the app.
Fan-out on read (pull). Nothing happens when Asha posts. When Ravi opens the app, the system fetches recent posts from everyone Ravi follows and merges them by time.
- ✅ Cheap writes; no wasted work; a celebrity post costs the same as any other.
- ❌ Slow reads: following 500 accounts means up to 500 lookups and a merge, every time the feed opens.
Step 5 — The hybrid answer (what interviewers want to hear)
Use push for normal users and pull for celebrities:
- When a normal user posts, push the post id to followers' feeds.
- When an account above a threshold (say, 100,000 followers) posts, do not push. Just store the post.
- When a user opens the feed, read their precomputed list and fetch recent posts from the few celebrities they follow, then merge.
Here is the whole idea in a small, runnable model:
from collections import defaultdict
followers = {"asha": ["ravi", "meena"], "priya": ["ravi"], "virat": ["ravi", "meena", "asha"]}
following = defaultdict(list)
for author, fans in followers.items():
for f in fans:
following[f].append(author)
CELEBRITIES = {"virat"} # too many followers to push to
posts = defaultdict(list) # author -> [(time, text)]
feed_cache = defaultdict(list) # user -> [(time, text)] pushed in at write time
writes = 0
def create_post(author, t, text):
global writes
posts[author].append((t, text))
if author not in CELEBRITIES: # fan-out on write
for f in followers[author]:
feed_cache[f].append((t, f"{author}: {text}"))
writes += 1
def read_feed(user, limit=3):
items = list(feed_cache[user])
for c in following[user]: # fan-out on read, celebrities only
if c in CELEBRITIES:
items += [(t, f"{c}: {text}") for t, text in posts[c]]
return [text for t, text in sorted(items, reverse=True)[:limit]]
create_post("asha", 1, "DSA notes uploaded")
create_post("virat", 2, "Match day!")
create_post("priya", 3, "Got the internship")
create_post("virat", 4, "Thank you fans")
print("ravi :", read_feed("ravi"))
print("meena:", read_feed("meena"))
print("feed writes:", writes, "(virat's posts cost 0 writes)")
ravi : ['virat: Thank you fans', 'priya: Got the internship', 'virat: Match day!']
meena: ['virat: Thank you fans', 'virat: Match day!', 'asha: DSA notes uploaded']
feed writes: 3 (virat's posts cost 0 writes)
Ravi's feed mixes a pushed post (Priya's) with pulled celebrity posts (Virat's), merged newest first; Asha's older post is on his next page. Meena's feed mixes Asha's pushed post with Virat's pulled ones. Virat's two posts would have cost 6 feed writes with pure push; here they cost none.
Step 6 — Storage and the full picture
| Data | Store | Why |
|---|---|---|
| Posts (id, author, text, time) | sharded database, by post id or author | huge volume, simple lookups |
| Photos and videos | object storage + CDN | big files served close to users |
| Follow graph (who follows whom) | database with indexes both ways, cached | "followers of X" for push, "following of Y" for pull |
| Precomputed feeds | Redis lists of the latest ~500 post ids per active user | fast reads; older pages fall back to pull |
| Post details | cache in front of the posts database | the same popular post appears in millions of feeds |
Details worth mentioning:
- Store ids, not full posts, in feed lists. An edited post then only changes in one place, and memory stays small.
- Fan-out runs on a queue (the queues lesson), so posting returns instantly and workers deliver in the background.
- Skip inactive users during fan-out; build their feed by pulling if they come back.
- Ranking: real feeds are not purely by time. A ranking service scores candidate posts (closeness to the author, likes, comments, recency) and reorders them. In an interview, describe it as a step after candidate collection; do not invent the exact formula.
- Deleting or unfollowing: filter at read time (skip posts that are deleted or from unfollowed accounts) instead of rewriting millions of lists.
Trade-offs to say out loud
- Push = fast reads, costly writes; pull = cheap writes, slow reads; hybrid by follower count.
- Feeds are eventually consistent — a post may take a few seconds to appear for all followers. That is acceptable here, unlike for payments.
- Precomputed feeds cost memory; keep them only for active users and only the newest few hundred ids.
Common mistakes
- Choosing pure push and forgetting accounts with millions of followers.
- Building every feed on read with a giant SQL join across all followed users.
- Storing full post content in every follower's feed.
- Using offset pagination on the feed.
Quick recap
- About 11,574 feed reads per second vs 231 posts per second: optimise reads.
- Fan-out on write precomputes feeds (fast reads); fan-out on read builds them on demand (cheap writes).
- Hybrid: push for normal users, pull for celebrities, merge at read time.
- Redis lists of post ids for active users, posts in a sharded database, media on a CDN, fan-out through a queue.
- Cursor pagination, eventual consistency, ranking as a separate step.