System Design · Lesson 5 of 8 · Updated ·
Message Queues, Microservices, API Gateways and Rate Limiting
Two ideas help big systems stay fast and manageable: do slow work later (queues) and split big systems into smaller services (microservices) — when it's worth it.
1. Message queues
When a student pays for a course, you must: record the payment, send an email, send a WhatsApp message, generate an invoice, update analytics. If the API did all of that before responding, the user would wait seconds — and any failure would break the purchase.
Instead:
- The API saves the order and puts a message on a queue ("order 42 paid").
- It responds immediately.
- Workers pick up messages and do the slow jobs, retrying if something fails.
Benefits: fast responses, absorbs traffic spikes, failures are isolated and retried, services are decoupled.
| Pattern | Meaning | Tools |
|---|---|---|
| Queue (point-to-point) | each message processed by one worker | RabbitMQ, SQS, BullMQ |
| Pub/Sub | each message delivered to all subscribers | Kafka, Google Pub/Sub, Redis streams |
Idempotency
Messages can be delivered more than once. Workers must be idempotent — doing the same job twice has the same effect as once (e.g. "set paid = true", not "add ₹500 again"). Payment APIs like Razorpay send webhooks that can repeat for the same reason.
2. Monolith vs microservices
| Monolith | Microservices | |
|---|---|---|
| Codebase | one | many small services |
| Deploy | all together | each separately |
| Scale | the whole app | only the busy service |
| Failure | a bug can take down everything | isolated (if designed well) |
| Complexity | low | high: network calls, monitoring, data consistency |
| Best for | startups, small teams | large teams and systems (Amazon, Netflix, Swiggy) |
Start with a well-structured monolith. Split out a service when a part has very different scaling needs or a separate team owns it. Many startups regret going microservices too early.
3. API gateway
A single entry point in front of services that handles routing (/payments → payment service), authentication, rate limiting, logging and sometimes response caching. Clients talk to one URL; services stay internal.
4. Rate limiting
Protects your system from abuse (bots, brute-force logins) and from one client using everything.
| Algorithm | Idea |
|---|---|
| Fixed window | max 100 requests per user per minute (resets each minute) |
| Sliding window | counts the last 60 seconds at any moment (smoother) |
| Token bucket | tokens refill at a steady rate; each request spends one; allows short bursts |
Return HTTP 429 Too Many Requests with a Retry-After header. (Skills101's own signup is rate-limited this way.)
5. Service communication
- Synchronous (REST, gRPC): request and wait — simple, but failures cascade. Use timeouts and retries with backoff.
- Asynchronous (queues/events): fire and continue — resilient, but harder to trace.
Common mistakes
- Doing slow work (emails, PDFs) inside the request.
- Non-idempotent workers that double-charge or double-send on retries.
- Splitting into microservices before you have the problems they solve.
Quick recap
- Queues: respond fast, process in the background, retry; make workers idempotent.
- Queue (one consumer) vs pub/sub (all subscribers).
- Monolith first; microservices for big teams and different scaling needs.
- API gateway for routing/auth/rate limits; rate limiting with token bucket or windows → 429.