One model call answers one question. Real work needs more than that. A document gets read, checked, summarised and filed. A support ticket gets sorted, answered and sometimes escalated. Orchestration is how you wire several model calls, tools and people into one system.
There are only a handful of shapes, and we use the same seven over and over. Below is how each one works, when we pick it, and the way it usually fails in production.
One rule first: pick the simplest shape that does the job. Every step you add is another place to be wrong, another line on the bill and another thing to test. Most production systems end up as a pipeline with one or two of the others bolted on.
Pipeline
A pipeline is a fixed list of steps. The output of one step is the input of the next. Code decides the order, not the model.
Example: an invoice arrives as a PDF. Step one extracts the fields. Step two checks them against the purchase order. Step three writes the booking line. Each step has a small prompt with one job.
How it works: every call gets a narrow instruction and a defined output, usually JSON with a schema. Between the steps sits plain code that validates the JSON, checks that the totals add up, and stops the run if something is missing. Those checks are the point. They catch a mistake at step one before step three turns it into a confident wrong answer.
Use it when you can write the steps down in advance. That is more often than people expect.
Where it breaks: when the input does not fit the steps. A credit note in an invoice pipeline gets booked as an invoice. The fix is usually a router in front, not a cleverer pipeline.
Router
A router reads the request and sends it to one of several handlers. The routing is a single small call. The handlers can be anything: another pipeline, a template, a person.
Example: a support inbox. Billing questions go to a flow that can read invoices. Bug reports go to a flow that asks for the version and the steps to reproduce. Everything else goes to a person.
How it works: the router returns one label from a closed list, including an “unsure” option. Each handler gets a prompt written for its one kind of request, which works better than one giant prompt that tries to cover everything. You can route on cost as well: easy questions to a small, fast model, hard ones to a large one.
Where it breaks: at the edges between categories, and on requests that are two things at once. Always give it a fallback route and log what lands there. That log tells you which category you forgot.
Parallel
Parallel runs several calls at the same time and merges the results. It comes in two kinds.
Splitting: different parts of the job run side by side. A contract review checks liability, termination and data clauses as three separate calls, each with its own focused prompt.
Voting: the same task runs several times, or on several models, and you compare. Three independent reviews of a code change, and the change gets flagged if two of them find a problem.
How it works: fan out, wait for all of them, merge. The merge is the hard part. For splitting it is usually code that stitches the sections together. For voting, decide up front what counts as agreement.
Where it breaks: when the parts are not really independent. If the termination clause refers back to the liability clause, two separate calls each see half the story. And voting with the same model and the same prompt often gives you the same mistake three times. Change the prompt or the model if you want independent answers.
Orchestrator and workers
Here a model decides how to split the work. A lead agent reads the task, writes a plan, hands sub-tasks to workers and puts their results together.
Example: “find every place in this codebase that calls the old payments API and update it.” Nobody knows in advance how many files that is. The lead searches, makes the list and sends one worker per file.
How it works: the lead gets the goal and a way to hand out tasks. Each worker gets a short, self-contained brief and only the context for its own part. Workers report back in a fixed format. The lead reads the reports and decides whether the job is done or needs another round.
Use it when the shape of the work depends on the input. If you can write the sub-tasks down beforehand, use parallel instead. It is cheaper and much easier to test.
Where it breaks: in the briefs. A worker only knows what the lead tells it, so a vague brief comes back as a vague result. Leads also like to over-split, sending ten workers where two would do. Cap the number of workers and rounds, and log every brief so you can see where a run went wrong.
Evaluator loop
One call writes a draft. A second call checks it against criteria you wrote down. If it fails, the feedback goes back to the first call and it tries again.
Example: translating product copy into Dutch. The checker has a list: keep the brand terms, stay under the character limit, use the informal “je”, never the formal “u”. Anything that fails goes back with the reason.
How it works: the criteria are the product. “Is this good?” is useless as a check. “Does every number in the source appear in the translation?” is one a model can answer reliably. Where you can, replace the model checker with code: a length check, a schema, a test suite. A passing test is stronger evidence than a model saying it looks fine.
Where it breaks: loops that never end, and drafts that learn to please the checker instead of the reader. Set a maximum number of rounds. When a draft hits the limit, hand it to a person instead of shipping the last attempt.
Agent with skills
An agent is a model in a loop with tools. It looks at the goal, picks a tool, reads the result and decides the next step, until it thinks it is done. Skills are packaged instructions and tools it loads when a task needs them: how to query your database, how to read your docs, how your deploy works.
Example: an internal assistant that can search the wiki, read tickets and run read-only queries. Someone asks why last week’s numbers dropped, and the agent works out which of those it needs, and in what order.
How it works: the tools decide almost everything. Clear names, clear descriptions, small tools that each do one thing, and error messages that say what to try next. Skills keep the base prompt short, because the agent only loads a skill’s instructions when it is about to use them.
Use it for open-ended work where you cannot predict the steps. It is the most flexible shape and the least predictable one, which is why it is near the end of this list and not at the top.
Where it breaks: it wanders, repeats itself, or stops early and reports success. Give it a step budget, keep tools read-only unless writing is the point, and test it on real tasks you already know the answer to.
Human in the loop
This is less a system of its own than a gate you put inside any of the others. The flow stops at a point you choose, shows a person what it is about to do, and waits.
Example: a refund flow drafts the refund and the email to the customer, and a person clicks approve before any money moves.
How it works: put the gate in front of anything that cannot be undone, such as payments, customer emails, deletes and deploys. Show the decision and the evidence for it, not the whole transcript. Make approve and reject one click each, and record who decided.
Where it breaks: too many gates, and people start approving without reading. Gate the actions that matter and let the rest run.
How we pick
We start from the work, not from the pattern:
- You can list the steps: pipeline.
- There are a few different kinds of input: put a router in front.
- The parts are independent: run them in parallel.
- The number of parts depends on the input: orchestrator and workers.
- There is a clear pass or fail: add an evaluator loop.
- The path is unknown until you start: an agent, with a budget.
- A step cannot be undone: put a person in front of it.
Real systems mix these. The support inbox from earlier is a router whose billing route is a pipeline, with an evaluator on the reply and a human gate on refunds. Each piece is simple on its own, and that is what lets you test it, debug it and trust it.