# Scaling and Load Balancing — Vertical vs Horizontal, Stateless Servers

Source: https://skills101.in/learn/system-design/scaling-and-load-balancing — Skills101 free tutorial (System Design). Updated 2026-11-09.

Your app runs fine for 100 users. Then a college fest promotes it and 10,000 students sign up in an hour. **Scaling** is how you survive that.

![Vertical scaling replaces a small server with a big one; horizontal scaling adds more servers behind a load balancer and needs stateless app servers](https://skills101.in/learn-img/system-design/scale-up-out.svg)

## 1. Vertical vs horizontal

| | Vertical (scale **up**) | Horizontal (scale **out**) |
|---|---|---|
| How | bigger machine (more CPU, RAM) | more machines |
| Limit | the biggest machine available | practically none |
| Failure | one machine = single point of failure | one dies, others continue |
| Complexity | simple | needs load balancing, stateless design |
| Cost | gets expensive fast | grows gradually |

Most systems start vertical (simple) and go horizontal when needed.

## 2. Load balancers

A **load balancer (LB)** sits in front of servers and spreads requests among them. Examples: **nginx**, HAProxy, AWS ALB.

### Algorithms

| Algorithm | How it picks a server |
|---|---|
| **Round robin** | 1, 2, 3, 1, 2, 3… |
| **Weighted round robin** | bigger servers get more requests |
| **Least connections** | the server with the fewest active requests |
| **IP hash** | the same user's IP always goes to the same server |

### Health checks

The LB pings each server (e.g. `GET /health` every few seconds). A server that fails is **removed** from rotation automatically until it recovers. This is how one crashed server doesn't take the site down.

**L4 vs L7:** a layer-4 LB routes by IP and port (fast); a layer-7 LB understands HTTP and can route by URL path (`/api` to one group, `/images` to another).

## 3. Stateless servers

For horizontal scaling, **any server must be able to handle any request**. So servers shouldn't keep user state in their own memory.

- ❌ Session stored in server 1's RAM → the next request goes to server 2 → "please log in again".
- ✅ Store sessions in **Redis** or the database, or use signed tokens (JWT).
- ✅ Store uploaded files in **object storage** (S3), not the server's disk.

(Sticky sessions — always sending a user to the same server — work, but make scaling and failures messier.)

## 4. Auto-scaling

Cloud platforms can add servers when CPU or traffic is high and remove them when it drops — you pay only for what you use. Example rule: *if average CPU > 70% for 5 minutes, add 2 servers.*

## 5. Remove single points of failure

- Run **at least two** of everything important (servers, load balancers).
- Use multiple **availability zones** (separate data centres).
- Database: a **replica** ready to take over (next lessons).

## Common mistakes

- Storing sessions or files on one app server's disk.
- One load balancer with no backup.
- Scaling app servers when the real bottleneck is the database.

## Quick recap

- Vertical = bigger machine; horizontal = more machines + load balancer.
- LB algorithms: round robin, weighted, least connections, IP hash; health checks remove dead servers.
- Stateless app servers: sessions in Redis/DB/JWT, files in object storage.
- Auto-scale with demand; remove single points of failure.


Practice quiz and free certificate: https://skills101.in/learn/system-design/scaling-and-load-balancing