Skills101

AdvancedSystem Design · Lesson 12 of 12 · Updated ·

Design a Unique ID Generator — Auto-Increment, UUID and Snowflake IDs

Every order, message, payment and short URL needs an ID. On one database, AUTO_INCREMENT gives 1, 2, 3… and the problem looks solved. But a chat system writing 70,000 messages a second across hundreds of servers cannot send every insert to one database just to get the next number. This lesson compares the standard answers and builds the one most interviewers expect: the Snowflake ID, first described by Twitter.

1. What do we need from an ID?

Clarify these first — they decide the design:

RequirementWhy it matters
Unique across all serverstwo orders with the same ID is a serious bug
Generated without coordinationasking a central server for every ID adds latency and a single point of failure
Roughly sorted by time"latest 50 messages" becomes a simple range scan; database indexes stay compact
Fits in 64 bitsa BIGINT column is smaller and faster to index than a 128-bit value
Not easy to guess (sometimes)sequential public IDs let anyone count your orders or scrape them

2. The options

Auto-increment in one database. Simple and perfectly ordered. But every insert depends on that one database, and it does not work once data is sharded across many databases.

Auto-increment with steps. With N databases, database k hands out k, k+N, k+2N… (with 3 databases: 1, 4, 7… / 2, 5, 8… / 3, 6, 9…). No clashes — but adding a fourth database changes the step for everyone, and IDs are no longer in time order across databases.

ID ranges (ticket server). A small central service hands each app server a block of IDs, say 1,000,000 at a time. The server then hands them out from memory. Only one network call per million IDs, so the central service is not a bottleneck. If a server crashes, the unused part of its block is lost — usually acceptable. The URL shortener lesson used this idea.

UUID v4. A 128-bit random value like 9f1c2e7a-4b3d-4f8e-9a61-2c7d5e0b8a43. Any server can create one with no coordination, and the chance of two clashing is so small that it is ignored in practice. But it is twice the size of a 64-bit ID, and because it is random, new rows land all over the database index instead of at the end, which makes inserts slower on large tables.

UUID v7. A newer UUID version (standardised in RFC 9562, 2024) that puts a millisecond timestamp at the front, so UUIDs are sorted by time while staying globally unique. A good choice when 128 bits is acceptable.

Snowflake. 64 bits, time-sorted, created locally on each server with no network call. Let's build it.

3. The Snowflake layout

A Snowflake ID packs three numbers into one 64-bit integer:

The 64 bits of a Snowflake ID: 1 unused sign bit, 41 bits of milliseconds since a custom epoch (about 69.7 years), 10 bits of machine id (1,024 machines) and 12 bits of sequence (4,096 IDs per millisecond per machine)

PartBitsWhat it gives
sign1always 0, so the ID is a positive signed 64-bit number
timestamp41milliseconds since a custom epoch you choose; 2⁴¹ ms ≈ 69.7 years
machine id102¹⁰ = 1,024 generators (often split as 5 bits data centre + 5 bits worker)
sequence122¹² = 4,096 IDs per millisecond on each machine

Because the timestamp sits in the highest bits, a larger ID always means a later millisecond. Sorting by ID is sorting by creation time.

Uniqueness comes from the combination: two machines never share a machine id, and within one machine and one millisecond the sequence number is different for every ID.

Maximum rate: 4,096 × 1,000 = about 4 million IDs per second per machine — far more than one server usually needs.

4. Building it in Python

We pass the current time in as an argument so the output is the same every time you run it. A real generator would read the clock itself.

EPOCH = 1735689600000            # 1 Jan 2025 00:00 UTC, in milliseconds

class Snowflake:
    def __init__(self, machine_id):
        assert 0 <= machine_id < 1024
        self.machine_id, self.seq, self.last_ms = machine_id, 0, -1

    def next_id(self, now_ms):
        if now_ms < self.last_ms:
            raise RuntimeError("clock moved backwards")
        if now_ms == self.last_ms:
            self.seq = (self.seq + 1) & 4095          # 12 bits: 0..4095
            if self.seq == 0:
                raise RuntimeError("4,096 IDs used this ms - wait for the next ms")
        else:
            self.seq = 0
        self.last_ms = now_ms
        return ((now_ms - EPOCH) << 22) | (self.machine_id << 12) | self.seq

def decode(snowflake_id):
    return {"ms_since_epoch": snowflake_id >> 22,
            "machine": (snowflake_id >> 12) & 1023,
            "seq": snowflake_id & 4095}

gen = Snowflake(machine_id=7)
t = EPOCH + 86_400_000            # exactly one day after the epoch
for now in [t, t, t, t + 1]:
    i = gen.next_id(now)
    print(i, decode(i))
362387865628672 {'ms_since_epoch': 86400000, 'machine': 7, 'seq': 0}
362387865628673 {'ms_since_epoch': 86400000, 'machine': 7, 'seq': 1}
362387865628674 {'ms_since_epoch': 86400000, 'machine': 7, 'seq': 2}
362387869822976 {'ms_since_epoch': 86400001, 'machine': 7, 'seq': 0}

Read the output line by line:

  • The first three IDs come from the same millisecond, so only the sequence changes (0, 1, 2).
  • The fourth ID is one millisecond later: the timestamp part goes up by one, and the sequence resets to 0.
  • Every ID is larger than the one before it — time order is preserved.
  • 86,400,000 ms is one day, which is exactly what the decoder reads back.

The shifts do all the work: << 22 moves the timestamp above the 10 + 12 lower bits; << 12 moves the machine id above the 12 sequence bits; & 4095 and & 1023 keep only the lowest 12 or 10 bits when decoding.

5. The hard parts interviewers ask about

Assigning machine ids. Each generator needs a unique 0–1023 number. Options: a config value per server, or a coordination service such as ZooKeeper or etcd that leases ids to servers as they start. In Kubernetes, the ordinal of a StatefulSet pod (gen-0, gen-1…) is a common trick.

Clocks going backwards. Server clocks are corrected by NTP (Network Time Protocol) and can jump back a few milliseconds. If the generator kept going, it could repeat an old timestamp and sequence — a duplicate ID. Our code refuses instead. Real generators usually wait until the clock passes last_ms again, and alert if the jump is large.

Sequence overflow. If one machine needs more than 4,096 IDs in one millisecond, it must wait for the next millisecond. In practice this is rare.

The 69-year limit. Pick an epoch close to your launch date, not 1970, so you do not waste decades of the range.

Guessability. Snowflake IDs reveal creation time and roughly how busy you are. For public URLs where that matters, show a separate random token instead of the raw ID.

6. Choosing

A comparison of ID strategies: auto-increment is ordered but needs one database; UUID v4 needs no coordination but is 128-bit and random; UUID v7 is time-sorted but 128-bit; Snowflake is 64-bit, time-sorted and local but needs machine ids and sane clocks

  • Small app with one database: auto-increment is fine.
  • Many writers, 128 bits acceptable: UUID v7 (or v4 if order does not matter).
  • Many writers, compact sorted 64-bit keys at high volume (messages, events, orders): Snowflake.

Common mistakes

  • Using MAX(id) + 1 from application code — two requests read the same maximum and create the same ID.
  • Using random UUID v4 as the primary key of a huge, insert-heavy table without thinking about index fragmentation.
  • Giving two servers the same machine id (for example by copying a config file), which silently creates duplicates.
  • Ignoring clock changes on the generator servers.

Quick recap

  • IDs should be unique, created without coordination, ideally time-sorted and compact.
  • Auto-increment needs one database; ID ranges reduce coordination to one call per block.
  • UUID v4 is random 128-bit; UUID v7 adds a time prefix so it sorts by time.
  • Snowflake = 41-bit time + 10-bit machine + 12-bit sequence in 64 bits: about 69.7 years, 1,024 machines, 4,096 IDs per ms each.
  • Watch for duplicate machine ids and clocks moving backwards.

Practice what you learned

📝 Quiz: Unique ID Generator5 questions · instant answers & explanations · +10 XP · counts towards your free certificateTake the quiz →