# Distributed Transactions — Two-Phase Commit, the Saga Pattern and the Transactional Outbox

Source: https://skills101.in/learn/system-design/system-design-distributed-transactions — Skills101 free tutorial (System Design). Updated 2026-11-21.

In a single database, a transaction is all-or-nothing: either every change in it is saved, or none is. That is the **A** (atomicity) of ACID, and it makes "move ₹500 from Asha to Ravi" safe. Now split the app into microservices, each with **its own database**: a payment service, a restaurant service and a delivery service. Placing one food order touches all three. There is no single database transaction that covers them. If the rider service fails after the payment succeeded, the customer has paid for food that will never come. This lesson covers how real systems handle that.

## 1. The problem: an order that is half done

Placing an order needs three steps in three services:

1. **Payment service:** charge ₹450 to the customer.
2. **Restaurant service:** the restaurant accepts the order.
3. **Delivery service:** assign a rider.

Each step commits in its own database. If step 3 fails (no rider nearby), steps 1 and 2 are **already committed**. Something must undo them, or the system is left in a state nobody wants: money taken, kitchen cooking, no rider. Network calls make it worse: a request can time out after the other side **did** the work, so the caller cannot even be sure whether a step happened.

## 2. Two-phase commit (2PC)

**Two-phase commit** makes several databases act like one transaction, using a **coordinator**:

- **Phase 1, prepare:** the coordinator asks every participant "can you commit?" Each one does the work, locks the rows, writes it durably, and votes **yes** or **no**.
- **Phase 2, commit or abort:** if **all** voted yes, the coordinator tells everyone to commit. If **any** voted no (or did not answer), everyone aborts.

![Two-phase commit: the coordinator sends prepare to the payment, restaurant and delivery databases, all three vote yes, then the coordinator sends commit; any no vote would make it send abort to all](https://skills101.in/learn-img/system-design/two-phase-commit.svg)

2PC gives real atomicity, but it has serious costs:

| Problem | Why |
|---|---|
| **Blocking** | participants hold locks between prepare and commit; if the coordinator crashes in between, they may wait with rows locked |
| **Slow** | at least two network round trips to every participant, while holding locks |
| **Availability** | every participant must be up; one slow service slows every order |
| **Support** | many modern services (most NoSQL stores, message brokers, third-party payment APIs) do not take part in 2PC |

2PC is used inside some databases and in tightly controlled systems, but between microservices most teams avoid it.

## 3. The saga pattern

A **saga** splits the big transaction into a sequence of **local transactions**, one per service. Each step commits on its own. For every step there is a **compensating action** that semantically undoes it: refund a payment, cancel a restaurant order. If a step fails, the saga runs the compensations for the completed steps **in reverse order**.

```python
def run_saga(steps, fail_at=None):
    done = []
    for name, action, compensate in steps:
        if name == fail_at:
            print(f"FAILED: {name}")
            for prev_name, undo in reversed(done):
                print(f"  compensate: {undo}")
            return "order cancelled, customer refunded"
        print(f"ok: {action}")
        done.append((name, compensate))
    return "order confirmed"

steps = [
    ("payment", "charge Rs 450", "refund Rs 450"),
    ("restaurant", "restaurant accepts order", "cancel restaurant order"),
    ("delivery", "assign rider", "release rider"),
]

print(run_saga(steps))
print()
print(run_saga(steps, fail_at="delivery"))
```

```text
ok: charge Rs 450
ok: restaurant accepts order
ok: assign rider
order confirmed

ok: charge Rs 450
ok: restaurant accepts order
FAILED: delivery
  compensate: cancel restaurant order
  compensate: refund Rs 450
order cancelled, customer refunded
```

![Saga with compensation, traced from the Python run: payment and restaurant steps succeed, the delivery step fails, so the saga cancels the restaurant order and then refunds the payment, in reverse order](https://skills101.in/learn-img/system-design/saga-compensation.svg)

Important properties of sagas:

- **No global locks.** Other users can see intermediate states ("payment done, waiting for restaurant"). This is called the lack of **isolation**. Design for it, for example with an order status like `PENDING` that the app shows honestly.
- **Compensation is not a rollback.** A refund is a new transaction; the charge still appears in the history. Some actions cannot be undone at all (an SMS already sent), so put them **last**, after the steps that may fail.
- **Every step and compensation must be idempotent** (safe to run twice), because messages get retried (see the API design lesson).

## 4. Orchestration vs choreography

There are two ways to drive a saga:

| | Orchestration | Choreography |
|---|---|---|
| Who decides the next step | a central **orchestrator** (an "order saga" service or a workflow engine) | each service reacts to **events** from the others |
| Flow | orchestrator: "payment, charge" → "restaurant, accept" → … | `PaymentDone` event → restaurant listens and acts → `OrderAccepted` event → … |
| Good for | complex flows, many steps, clear visibility | simple flows with few services |
| Risk | the orchestrator is extra code to run | the flow is spread over many services and hard to follow |

Workflow engines such as Temporal, AWS Step Functions and Camunda are often used as orchestrators because they persist the saga's state and retry steps for you.

## 5. The transactional outbox

Sagas and event-driven services depend on one small but tricky step: **save to my database AND publish an event**. If the service saves the order and then crashes before publishing `OrderCreated`, the other services never hear about it. If it publishes first and then the save fails, they react to an order that does not exist. This is the **dual-write problem**.

The **outbox pattern** fixes it: write the event into an `outbox` table **in the same local transaction** as the business data. A separate **relay** process reads unsent outbox rows, publishes them to the message broker (such as Kafka or RabbitMQ), and marks them sent.

```python
import sqlite3

db = sqlite3.connect(":memory:")
db.execute("CREATE TABLE orders (id INTEGER PRIMARY KEY, item TEXT, status TEXT)")
db.execute("CREATE TABLE outbox (id INTEGER PRIMARY KEY, event TEXT, sent INTEGER DEFAULT 0)")

def place_order(item, crash=False):
    try:
        with db:                                  # one local transaction
            cur = db.execute("INSERT INTO orders (item, status) VALUES (?, 'PENDING')", (item,))
            if crash:
                raise RuntimeError("server crashed")
            db.execute("INSERT INTO outbox (event) VALUES (?)", (f"OrderCreated:{cur.lastrowid}",))
    except RuntimeError as e:
        print("place_order failed:", e)

def relay():                                      # runs separately, every second or so
    for row_id, event in db.execute("SELECT id, event FROM outbox WHERE sent = 0").fetchall():
        print("publish to broker:", event)
        db.execute("UPDATE outbox SET sent = 1 WHERE id = ?", (row_id,))
    db.commit()

place_order("paneer biryani")
place_order("masala dosa", crash=True)
relay()
print("orders:", db.execute("SELECT id, item FROM orders").fetchall())
print("unsent events:", db.execute("SELECT COUNT(*) FROM outbox WHERE sent = 0").fetchone()[0])
```

```text
place_order failed: server crashed
publish to broker: OrderCreated:1
orders: [(1, 'paneer biryani')]
unsent events: 0
```

The crashed order left **no order row and no event**: both were rolled back together. The successful order and its event were saved together, and the relay published it. If the relay crashes after publishing but before marking the row sent, it publishes the event **again** on restart. That is why consumers must be **idempotent**: delivery is **at least once**, not exactly once.

## 6. What to say in an interview

- "Each service owns its database, so I cannot use one ACID transaction across them."
- "I'll avoid 2PC between services because it blocks and needs every participant up."
- "I'll use an orchestrated saga: charge, accept, assign rider; on failure, compensate in reverse: release, cancel, refund."
- "Each service writes its state change and its event in one transaction using an outbox; consumers are idempotent because delivery is at-least-once."
- "The user sees `PENDING` until the saga finishes, and irreversible steps like the SMS come last."

## Common mistakes

- Saving to the database and publishing to the queue as two separate steps (the dual-write problem).
- Forgetting to design compensating actions, or assuming every action can be undone.
- Non-idempotent steps that charge twice when a message is retried.
- Using 2PC across microservices and third-party APIs that cannot take part in it.
- Hiding intermediate states from users instead of showing a clear `PENDING` status.

## Quick recap

- Microservices with separate databases lose cross-service ACID transactions.
- 2PC: prepare, then commit or abort; atomic but blocking, slow and fragile across services.
- Saga: local transactions plus compensating actions in reverse order; no isolation, so design for visible intermediate states.
- Orchestration (central coordinator) vs choreography (services react to events).
- Outbox: business row + event row in one local transaction; a relay publishes; consumers are idempotent.


Practice quiz and free certificate: https://skills101.in/learn/system-design/system-design-distributed-transactions