Skills101

AdvancedSystem Design · Lesson 15 of 19 · Updated ·

System Design Case Study — Design a News Feed Like Instagram or LinkedIn

"Design the home feed of Instagram (or LinkedIn, or Twitter/X)" is one of the most asked system design questions. The core challenge fits in one sentence: when a user opens the app, show the latest posts from everyone they follow — fast — even though some accounts have millions of followers. Let's solve it with the 6-step method from the first lesson.

Step 1 — Requirements

Functional

  • A user can create a post (text, photo, video link).
  • A user can follow and unfollow others.
  • The home feed shows recent posts from followed accounts, newest first (ranking comes later).
  • Scrolling loads older posts.

Non-functional

  • Feed loads fast: under about 200 ms for the first page.
  • Highly available: a feed that is a few seconds out of date is fine (eventual consistency).
  • Read-heavy: people scroll far more often than they post.

Step 2 — Estimates

Assume 100 million daily active users, each opening the feed 10 times a day, and 20 million new posts a day. Average 200 followers per user.

QuantityCalculationResult
Feed reads / second100 M × 10 ÷ 86,400≈ 11,574
New posts / second20 M ÷ 86,400≈ 231
Feed inserts if every post is pushed to every follower231 × 200≈ 46,296 per second

Reads outnumber posts about 50 to 1. The interesting number is the last one: how much work does it take to put each post into every follower's feed? That depends on the fan-out strategy.

Step 3 — API

POST /v1/posts                 { "text": "...", "media_ids": [...] }   -> 201 { "id": "..." }
POST /v1/users/{id}/follow                                             -> 204
GET  /v1/feed?cursor=...&limit=20                                      -> 200 { "items": [...], "next_cursor": "..." }

The feed uses cursor pagination (the API design lesson explains why): new posts arrive constantly, and offsets would show repeats.

Step 4 — Fan-out: push or pull?

Fan-out means delivering one post to many followers' feeds. There are two opposite ways to do it.

Fan-out on write: when Asha posts, the post id is pushed into each follower's precomputed feed so reads are one lookup; fan-out on read: nothing happens at post time and Ravi's feed is built at read time by querying everyone he follows and merging

Fan-out on write (push). When Asha posts, a background worker inserts the post id into the stored feed of each of her followers. Reading a feed is then a single fast lookup: "give me the first 20 ids in Ravi's feed list".

  • ✅ Very fast reads — and reads are the common case.
  • ❌ Expensive writes for popular accounts: a cricketer with 50 million followers creates 50 million feed inserts for one post. Delivery can take minutes.
  • ❌ Wasted work for followers who never open the app.

Fan-out on read (pull). Nothing happens when Asha posts. When Ravi opens the app, the system fetches recent posts from everyone Ravi follows and merges them by time.

  • ✅ Cheap writes; no wasted work; a celebrity post costs the same as any other.
  • ❌ Slow reads: following 500 accounts means up to 500 lookups and a merge, every time the feed opens.

Step 5 — The hybrid answer (what interviewers want to hear)

Use push for normal users and pull for celebrities:

  • When a normal user posts, push the post id to followers' feeds.
  • When an account above a threshold (say, 100,000 followers) posts, do not push. Just store the post.
  • When a user opens the feed, read their precomputed list and fetch recent posts from the few celebrities they follow, then merge.

Here is the whole idea in a small, runnable model:

from collections import defaultdict

followers = {"asha": ["ravi", "meena"], "priya": ["ravi"], "virat": ["ravi", "meena", "asha"]}
following = defaultdict(list)
for author, fans in followers.items():
    for f in fans:
        following[f].append(author)

CELEBRITIES = {"virat"}              # too many followers to push to
posts = defaultdict(list)            # author -> [(time, text)]
feed_cache = defaultdict(list)       # user -> [(time, text)] pushed in at write time
writes = 0

def create_post(author, t, text):
    global writes
    posts[author].append((t, text))
    if author not in CELEBRITIES:               # fan-out on write
        for f in followers[author]:
            feed_cache[f].append((t, f"{author}: {text}"))
            writes += 1

def read_feed(user, limit=3):
    items = list(feed_cache[user])
    for c in following[user]:                   # fan-out on read, celebrities only
        if c in CELEBRITIES:
            items += [(t, f"{c}: {text}") for t, text in posts[c]]
    return [text for t, text in sorted(items, reverse=True)[:limit]]

create_post("asha", 1, "DSA notes uploaded")
create_post("virat", 2, "Match day!")
create_post("priya", 3, "Got the internship")
create_post("virat", 4, "Thank you fans")

print("ravi :", read_feed("ravi"))
print("meena:", read_feed("meena"))
print("feed writes:", writes, "(virat's posts cost 0 writes)")
ravi : ['virat: Thank you fans', 'priya: Got the internship', 'virat: Match day!']
meena: ['virat: Thank you fans', 'virat: Match day!', 'asha: DSA notes uploaded']
feed writes: 3 (virat's posts cost 0 writes)

Ravi's feed mixes a pushed post (Priya's) with pulled celebrity posts (Virat's), merged newest first; Asha's older post is on his next page. Meena's feed mixes Asha's pushed post with Virat's pulled ones. Virat's two posts would have cost 6 feed writes with pure push; here they cost none.

Step 6 — Storage and the full picture

News feed architecture: the post service saves posts and media; a fan-out queue and workers push post ids into Redis feed lists of followers; the feed service reads the Redis list, adds recent celebrity posts, fetches post details from a cache and returns the page

DataStoreWhy
Posts (id, author, text, time)sharded database, by post id or authorhuge volume, simple lookups
Photos and videosobject storage + CDNbig files served close to users
Follow graph (who follows whom)database with indexes both ways, cached"followers of X" for push, "following of Y" for pull
Precomputed feedsRedis lists of the latest ~500 post ids per active userfast reads; older pages fall back to pull
Post detailscache in front of the posts databasethe same popular post appears in millions of feeds

Details worth mentioning:

  • Store ids, not full posts, in feed lists. An edited post then only changes in one place, and memory stays small.
  • Fan-out runs on a queue (the queues lesson), so posting returns instantly and workers deliver in the background.
  • Skip inactive users during fan-out; build their feed by pulling if they come back.
  • Ranking: real feeds are not purely by time. A ranking service scores candidate posts (closeness to the author, likes, comments, recency) and reorders them. In an interview, describe it as a step after candidate collection; do not invent the exact formula.
  • Deleting or unfollowing: filter at read time (skip posts that are deleted or from unfollowed accounts) instead of rewriting millions of lists.

Trade-offs to say out loud

  • Push = fast reads, costly writes; pull = cheap writes, slow reads; hybrid by follower count.
  • Feeds are eventually consistent — a post may take a few seconds to appear for all followers. That is acceptable here, unlike for payments.
  • Precomputed feeds cost memory; keep them only for active users and only the newest few hundred ids.

Common mistakes

  • Choosing pure push and forgetting accounts with millions of followers.
  • Building every feed on read with a giant SQL join across all followed users.
  • Storing full post content in every follower's feed.
  • Using offset pagination on the feed.

Quick recap

  • About 11,574 feed reads per second vs 231 posts per second: optimise reads.
  • Fan-out on write precomputes feeds (fast reads); fan-out on read builds them on demand (cheap writes).
  • Hybrid: push for normal users, pull for celebrities, merge at read time.
  • Redis lists of post ids for active users, posts in a sharded database, media on a CDN, fan-out through a queue.
  • Cursor pagination, eventual consistency, ranking as a separate step.

Practice what you learned

📝 Quiz: Design a News Feed5 questions · instant answers & explanations · +10 XP · counts towards your free certificateTake the quiz →