Someone has to be the clock.

The LLM proposes. Memory stores. Planning revises. Tools execute. RAG fetches. Orchestration is the runtime that sequences those moves: who is called, in what order, with what budget, and when the loop is over. Call it a framework if you want. Do not pretend the model is scheduling itself.

What the loop actually does

A minimal orchestrator:

  1. Accept a goal.
  2. Assemble context (instructions, memory, optional RAG).
  3. Call the LLM.
  4. If the output is prose for a human, maybe stop.
  5. If the output is a function call, run the tool, append the observation, go to 2.
  6. Stop on success, refusal, max steps, timeout, or a human gate.

In pseudo code:

steps = 0
while steps < MAX:
    out = llm(context)
    if out is prose for a human: break
    if out is a tool request:
        obs = run(out)          # or refuse
        context.append(obs)
    steps += 1
# stop: success, MAX, timeout, or human gate

OpenAI’s agents guide is one example of that scheduler, including handoffs to another specialist. You can do the same with the loop above. The architecture is MAX and the break a.k.a. the loop.

That is the same observe → repeat shape From Models to Agents Part 4 drew. This part owns the scheduler: retries, branching, handoff, the kill switch.

Stop is a feature

Unlimited loops are not autonomy. They are a runaway bill and a runaway side-effect. Max steps, max tool calls, max dollars, max wall-clock — pick numbers. “Ask a human” is a transition, not a fallback, when the next tool effect is irreversible. Part 6 of AI Right Now is why that gate exists. Orchestration is where you implement it: do not call refund without an approval record.

Routers belong here too. Multi-agent is usually orchestration with extra name tags: researcher → writer → reviewer. Part 10 of From Models to Agents named the pattern. One example of a failure mode is a bad handoff — the reviewer never sees the 403 the researcher swallowed, so write into memory the whole chain.

Example

Goal: “Draft tickets for failed payments; do not refund.”

Orchestrator allows get_payments, draft_ticket. Forbids refund. After eight steps with no draft, it stops and returns the trace. After a draft, it stops even if the model wants to “just send it.” The model can beg. The clock says no.

Conclusion

Orchestration is not a smarter model. It is the clock: assemble context, call the LLM, run tools, append observations, and stop — on success, on max steps, on timeout, or when a human must approve. Budgets and gates live here, so do handoffs. Part 8 is how you prove that clock does what you think it does.

Takeaway: Orchestration is the loop and the stop — who calls the model, when tools run, when a human or a budget cuts in.

Sources

Part 6: RAG
Part 8: Evaluation