AdvancedAI Engineering — Building Apps with Large Language Models · Lesson 14 of 15 · Updated ·
LLM Workflow Patterns — Chaining, Routing, Parallel Calls, Voting and Evaluator Loops
The tools lesson gave one rule: start with a workflow, not an agent. A workflow is a fixed path written in your code, where the LLM does one clear job at each step. This lesson covers the standard workflow patterns. Anthropic's engineering guide Building effective agents describes the same set, and you will see them in almost every serious LLM product.
About the code. Each example uses a small stand-in function called call_llm that returns fixed answers, so the control flow runs on your laptop with no API key and gives the same output every time. In a real app, call_llm would call client.messages.create(...) as in lesson 2. The patterns are about the code around the model, and that code is real.
1. Prompt chaining with a gate
Split a task into fixed steps, where each step's output is the next step's input. Between steps, add a gate: a plain-code check that stops bad output early, before you pay for the next call.
Task: write a placement-drive notice. Step 1 extracts the facts; the gate checks that every required fact is present; step 2 writes the notice.
import json
def call_llm(prompt): # stand-in for a real API call
if prompt.startswith("EXTRACT"):
if "Infosys" in prompt:
return '{"company": "Infosys", "date": "28 Nov", "eligibility": "CGPA 6.5+"}'
return '{"company": "TCS", "date": "", "eligibility": "all branches"}'
facts = json.loads(prompt.split("\n", 1)[1])
return f"Placement drive: {facts['company']} on {facts['date']}. Eligibility: {facts['eligibility']}."
REQUIRED = ["company", "date", "eligibility"]
def make_notice(email_text):
facts = json.loads(call_llm("EXTRACT company, date, eligibility as JSON:\n" + email_text))
missing = [key for key in REQUIRED if not facts.get(key)]
if missing: # the gate: plain code, no LLM
return f"STOPPED: missing {missing} - ask the placement cell"
return call_llm("WRITE a short notice from these facts:\n" + json.dumps(facts))
print(make_notice("Infosys is visiting on 28 Nov. Students with CGPA 6.5+ may apply."))
print(make_notice("TCS will visit soon. Open to all branches."))
Placement drive: Infosys on 28 Nov. Eligibility: CGPA 6.5+.
STOPPED: missing ['date'] - ask the placement cell
Each step is easy to test on its own, and the gate catches the second email (no date) without spending a second call or publishing a wrong notice.
2. Routing
A router classifies the input first, then sends it to a specialised handler: a different prompt, a different tool, or a cheaper model for easy cases. The classification can be done by a small, fast model or, when possible, by plain rules.
def classify(question): # stand-in for a small, cheap classifier call
q = question.lower()
if any(word in q for word in ("fee", "refund", "payment")):
return "fees"
if any(word in q for word in ("exam", "hall ticket", "result")):
return "exams"
return "other"
ROUTES = {
"fees": ("small model", "answer from the fee policy; never promise refunds"),
"exams": ("small model", "answer from the exam rules; give dates exactly"),
"other": ("large model", "general help-desk prompt; hand over to staff if unsure"),
}
for question in ["When will the hall ticket come?",
"I paid the fee twice, how do I get a refund?",
"Can I change my elective after week 3?"]:
route = classify(question)
model, instructions = ROUTES[route]
print(f"{route:5s} -> {model}: {instructions}")
exams -> small model: answer from the exam rules; give dates exactly
fees -> small model: answer from the fee policy; never promise refunds
other -> large model: general help-desk prompt; hand over to staff if unsure
Routing keeps each prompt short and focused, which improves quality, and sends simple, frequent questions to a cheaper model, which cuts cost. Always include a safe fallback route (other) for inputs the router does not recognise.
3. Parallel calls: sectioning and voting
When parts of a task do not depend on each other, run them at the same time.
- Sectioning: split the work into independent parts, for example reviewing a resume for (a) grammar, (b) project descriptions and (c) formatting, with three focused calls run in parallel. API calls mostly wait on the network, so threads work well.
- Voting: ask the same question several times (or with differently worded prompts) and combine the answers. This is useful for high-stakes yes/no checks.
from collections import Counter
from concurrent.futures import ThreadPoolExecutor
def call_llm(aspect): # stand-in: one focused review per aspect
return {"grammar": "2 typos in the summary",
"projects": "add numbers: users, speed-up, accuracy",
"format": "one page, consistent dates"}[aspect]
with ThreadPoolExecutor(max_workers=3) as pool: # sectioning: three calls at once
reviews = list(pool.map(call_llm, ["grammar", "projects", "format"]))
print("sectioning:", reviews)
def decide(votes, min_agreement=0.8): # voting
answer, count = Counter(votes).most_common(1)[0]
share = count / len(votes)
return answer if share >= min_agreement else f"UNSURE ({share:.0%} agree) - send to a human"
print("voting:", decide(["spam", "spam", "spam", "spam", "not spam"]))
print("voting:", decide(["spam", "not spam", "spam", "not spam", "spam"]))
sectioning: ['2 typos in the summary', 'add numbers: users, speed-up, accuracy', 'one page, consistent dates']
voting: spam
voting: UNSURE (60% agree) - send to a human
pool.map returns results in input order, even if the calls finish in a different order. Voting turns disagreement into a useful signal: when the answers split 3–2, the case is genuinely unclear, so a person should look at it.
4. Orchestrator and workers
Sometimes you cannot know the sub-tasks in advance. Example: "update the syllabus pages for every subject affected by the new regulation". An orchestrator call reads the task and decides the sub-tasks (which subjects); worker calls handle each one; a final step combines the results.
This sits between a workflow and an agent: the model chooses what to do, but your code still controls how it runs: the maximum number of workers, the allowed tools, and a final review. Use it only when the fixed patterns above do not fit.
5. Evaluator–optimizer loop
One call generates a draft; another evaluates it against clear criteria and gives feedback; the generator revises. Repeat until the draft passes or a round limit is reached.
def generate(feedback): # stand-in for the writer model
drafts = {
None: "Dear Sir, I am very very interested in the role and I have done many projects in many technologies.",
"too vague": "Dear Hiring Team, I built a Django attendance app used by 300 students in my college.",
}
return drafts.get(feedback, drafts["too vague"])
def evaluate(draft): # evaluator: here plain rules; often a second LLM call
if "Sir" in draft:
return "too vague" # generic greeting means the draft is not tailored
if not any(ch.isdigit() for ch in draft):
return "too vague"
return "PASS"
feedback = None
for round_no in range(1, 4): # always cap the rounds
draft = generate(feedback)
feedback = evaluate(draft)
print(f"round {round_no}: {feedback:9s} | {draft[:50]}...")
if feedback == "PASS":
break
round 1: too vague | Dear Sir, I am very very interested in the role an...
round 2: PASS | Dear Hiring Team, I built a Django attendance app ...
This works when you can state clear evaluation criteria (word limit, required facts, tone rules, tests that must pass). Prefer code checks where possible because they are cheap and exact; use an LLM judge for qualities code cannot check, as shown in the evaluation lesson. The round cap stops endless loops and runaway costs.
6. Choosing a pattern
| Your situation | Pattern |
|---|---|
| Fixed steps, each easy to check | prompt chaining with gates |
| Different kinds of input need different handling | routing |
| Independent parts, or a high-stakes yes/no decision | parallel sectioning / voting |
| Sub-tasks depend on the input and can't be listed in advance | orchestrator and workers |
| Quality can be judged against clear criteria | evaluator–optimizer loop |
| Open-ended, many unpredictable steps | an agent (lesson 8), with limits |
The patterns combine: a router can send one route into a chain, and one step of the chain can use voting.
Common mistakes
- Building an agent when a two-step chain would do.
- No gates between steps, so one bad extraction flows into everything that follows.
- A router with no fallback route for unexpected inputs.
- Voting on open-ended text, where answers never match exactly. Vote on short labels or yes/no decisions.
- An evaluator loop with no round limit, or with criteria so vague that nothing ever passes.
- Running dependent steps in parallel (step 2 needs step 1's output).
Quick recap
- Workflows = your code fixes the path; the LLM does one clear job per step.
- Chaining splits a task into steps; gates are plain-code checks between them.
- Routing classifies first and sends each input to the right prompt or model.
- Parallel calls save time (sectioning) or add confidence (voting).
- Orchestrator–workers handles sub-tasks you cannot list in advance; evaluator loops refine drafts with a round cap.