# LLM Workflow Patterns — Chaining, Routing, Parallel Calls, Voting and Evaluator Loops

Source: https://skills101.in/learn/ai-engineering/ai-engineering-llm-workflow-patterns — Skills101 free tutorial (AI Engineering — Building Apps with Large Language Models). Updated 2026-11-21.

The tools lesson gave one rule: **start with a workflow, not an agent**. A workflow is a fixed path written in your code, where the LLM does one clear job at each step. This lesson covers the standard workflow patterns. Anthropic's engineering guide *Building effective agents* describes the same set, and you will see them in almost every serious LLM product.

**About the code.** Each example uses a small stand-in function called `call_llm` that returns fixed answers, so the **control flow** runs on your laptop with no API key and gives the same output every time. In a real app, `call_llm` would call `client.messages.create(...)` as in lesson 2. The patterns are about the code *around* the model, and that code is real.

![Five workflow patterns side by side: a chain of steps with a gate, a router sending inputs to different handlers, parallel calls whose results are combined, an orchestrator splitting work among workers, and a generator and evaluator in a loop](https://skills101.in/learn-img/ai-engineering/workflow-patterns.svg)

## 1. Prompt chaining with a gate

Split a task into fixed steps, where each step's output is the next step's input. Between steps, add a **gate**: a plain-code check that stops bad output early, before you pay for the next call.

Task: write a placement-drive notice. Step 1 extracts the facts; the gate checks that every required fact is present; step 2 writes the notice.

```python
import json

def call_llm(prompt):                       # stand-in for a real API call
    if prompt.startswith("EXTRACT"):
        if "Infosys" in prompt:
            return '{"company": "Infosys", "date": "28 Nov", "eligibility": "CGPA 6.5+"}'
        return '{"company": "TCS", "date": "", "eligibility": "all branches"}'
    facts = json.loads(prompt.split("\n", 1)[1])
    return f"Placement drive: {facts['company']} on {facts['date']}. Eligibility: {facts['eligibility']}."

REQUIRED = ["company", "date", "eligibility"]

def make_notice(email_text):
    facts = json.loads(call_llm("EXTRACT company, date, eligibility as JSON:\n" + email_text))
    missing = [key for key in REQUIRED if not facts.get(key)]
    if missing:                                          # the gate: plain code, no LLM
        return f"STOPPED: missing {missing} - ask the placement cell"
    return call_llm("WRITE a short notice from these facts:\n" + json.dumps(facts))

print(make_notice("Infosys is visiting on 28 Nov. Students with CGPA 6.5+ may apply."))
print(make_notice("TCS will visit soon. Open to all branches."))
```

```text
Placement drive: Infosys on 28 Nov. Eligibility: CGPA 6.5+.
STOPPED: missing ['date'] - ask the placement cell
```

Each step is easy to test on its own, and the gate catches the second email (no date) without spending a second call or publishing a wrong notice.

## 2. Routing

A **router** classifies the input first, then sends it to a specialised handler: a different prompt, a different tool, or a **cheaper model** for easy cases. The classification can be done by a small, fast model or, when possible, by plain rules.

```python
def classify(question):                    # stand-in for a small, cheap classifier call
    q = question.lower()
    if any(word in q for word in ("fee", "refund", "payment")):
        return "fees"
    if any(word in q for word in ("exam", "hall ticket", "result")):
        return "exams"
    return "other"

ROUTES = {
    "fees":  ("small model", "answer from the fee policy; never promise refunds"),
    "exams": ("small model", "answer from the exam rules; give dates exactly"),
    "other": ("large model", "general help-desk prompt; hand over to staff if unsure"),
}

for question in ["When will the hall ticket come?",
                 "I paid the fee twice, how do I get a refund?",
                 "Can I change my elective after week 3?"]:
    route = classify(question)
    model, instructions = ROUTES[route]
    print(f"{route:5s} -> {model}: {instructions}")
```

```text
exams -> small model: answer from the exam rules; give dates exactly
fees  -> small model: answer from the fee policy; never promise refunds
other -> large model: general help-desk prompt; hand over to staff if unsure
```

Routing keeps each prompt short and focused, which improves quality, and sends simple, frequent questions to a cheaper model, which cuts cost. Always include a safe fallback route (`other`) for inputs the router does not recognise.

## 3. Parallel calls: sectioning and voting

When parts of a task do not depend on each other, run them **at the same time**.

- **Sectioning:** split the work into independent parts, for example reviewing a resume for (a) grammar, (b) project descriptions and (c) formatting, with three focused calls run in parallel. API calls mostly wait on the network, so threads work well.
- **Voting:** ask the **same** question several times (or with differently worded prompts) and combine the answers. This is useful for high-stakes yes/no checks.

```python
from collections import Counter
from concurrent.futures import ThreadPoolExecutor

def call_llm(aspect):                        # stand-in: one focused review per aspect
    return {"grammar": "2 typos in the summary",
            "projects": "add numbers: users, speed-up, accuracy",
            "format": "one page, consistent dates"}[aspect]

with ThreadPoolExecutor(max_workers=3) as pool:      # sectioning: three calls at once
    reviews = list(pool.map(call_llm, ["grammar", "projects", "format"]))
print("sectioning:", reviews)

def decide(votes, min_agreement=0.8):                 # voting
    answer, count = Counter(votes).most_common(1)[0]
    share = count / len(votes)
    return answer if share >= min_agreement else f"UNSURE ({share:.0%} agree) - send to a human"

print("voting:", decide(["spam", "spam", "spam", "spam", "not spam"]))
print("voting:", decide(["spam", "not spam", "spam", "not spam", "spam"]))
```

```text
sectioning: ['2 typos in the summary', 'add numbers: users, speed-up, accuracy', 'one page, consistent dates']
voting: spam
voting: UNSURE (60% agree) - send to a human
```

`pool.map` returns results **in input order**, even if the calls finish in a different order. Voting turns disagreement into a useful signal: when the answers split 3–2, the case is genuinely unclear, so a person should look at it.

## 4. Orchestrator and workers

Sometimes you cannot know the sub-tasks in advance. Example: "update the syllabus pages for every subject affected by the new regulation". An **orchestrator** call reads the task and *decides* the sub-tasks (which subjects); **worker** calls handle each one; a final step combines the results.

This sits between a workflow and an agent: the model chooses **what** to do, but your code still controls **how** it runs: the maximum number of workers, the allowed tools, and a final review. Use it only when the fixed patterns above do not fit.

## 5. Evaluator–optimizer loop

One call **generates** a draft; another **evaluates** it against clear criteria and gives feedback; the generator revises. Repeat until the draft passes or a round limit is reached.

```python
def generate(feedback):                       # stand-in for the writer model
    drafts = {
        None: "Dear Sir, I am very very interested in the role and I have done many projects in many technologies.",
        "too vague": "Dear Hiring Team, I built a Django attendance app used by 300 students in my college.",
    }
    return drafts.get(feedback, drafts["too vague"])

def evaluate(draft):                          # evaluator: here plain rules; often a second LLM call
    if "Sir" in draft:
        return "too vague"                    # generic greeting means the draft is not tailored
    if not any(ch.isdigit() for ch in draft):
        return "too vague"
    return "PASS"

feedback = None
for round_no in range(1, 4):                  # always cap the rounds
    draft = generate(feedback)
    feedback = evaluate(draft)
    print(f"round {round_no}: {feedback:9s} | {draft[:50]}...")
    if feedback == "PASS":
        break
```

```text
round 1: too vague | Dear Sir, I am very very interested in the role an...
round 2: PASS      | Dear Hiring Team, I built a Django attendance app ...
```

This works when you can state **clear evaluation criteria** (word limit, required facts, tone rules, tests that must pass). Prefer code checks where possible because they are cheap and exact; use an LLM judge for qualities code cannot check, as shown in the evaluation lesson. The round cap stops endless loops and runaway costs.

## 6. Choosing a pattern

| Your situation | Pattern |
|---|---|
| Fixed steps, each easy to check | prompt chaining with gates |
| Different kinds of input need different handling | routing |
| Independent parts, or a high-stakes yes/no decision | parallel sectioning / voting |
| Sub-tasks depend on the input and can't be listed in advance | orchestrator and workers |
| Quality can be judged against clear criteria | evaluator–optimizer loop |
| Open-ended, many unpredictable steps | an agent (lesson 8), with limits |

The patterns combine: a router can send one route into a chain, and one step of the chain can use voting.

## Common mistakes

- Building an agent when a two-step chain would do.
- No gates between steps, so one bad extraction flows into everything that follows.
- A router with no fallback route for unexpected inputs.
- Voting on open-ended text, where answers never match exactly. Vote on short labels or yes/no decisions.
- An evaluator loop with no round limit, or with criteria so vague that nothing ever passes.
- Running dependent steps in parallel (step 2 needs step 1's output).

## Quick recap

- Workflows = your code fixes the path; the LLM does one clear job per step.
- Chaining splits a task into steps; gates are plain-code checks between them.
- Routing classifies first and sends each input to the right prompt or model.
- Parallel calls save time (sectioning) or add confidence (voting).
- Orchestrator–workers handles sub-tasks you cannot list in advance; evaluator loops refine drafts with a round cap.


Practice quiz and free certificate: https://skills101.in/learn/ai-engineering/ai-engineering-llm-workflow-patterns