If you only grade the last outcome, you did not grade the agent.

Orchestration produces a path: model calls, tool calls, retrievals, stops. Evaluation is the layer that scores that path. It sits on the diagram because a stack you cannot measure will drift the first week a tool schema changes.

From Models to Agents Part 8 covers the enterprise process: golden tasks, a sandbox, injection cases, and repeated testing until you may ship. This part is narrower. It covers what must exist in the architecture so that process has something to test.

Score the trajectory

A trajectory is the ordered record: prompts, tool names, arguments, observations, final state. OpenAI’s trace grading (external link) puts it in one line: score the path, not the end result. Agent workflow evals (external link) start from those traces. You do not need their product. You need the log.

The unit of score is that record, not the closing paragraph:

[
  { "tool": "get_order", "args": { "id": "9" }, "obs": "paid" },
  { "tool": "draft_ticket", "args": { "order": "9" }, "obs": "ok" },
  { "stop": "success" }
]

Grade the list. A polite last sentence after refund(9) is still a fail.

Grade, at least:

  • Did it finish the goal?
  • Did it touch a tool it must not? (refund on a draft-only job)
  • Did a wrong observation poison step 5?
  • Did it stop? Max steps that never fire are decoration.

Last-sentence BLEU (Bilingual Evaluation Understudy) scores are for chatbot. Agents failures are the processes.

Evaluation is part of the system, not a formality

Wire it like any other layer. Staging credentials. Fixtures for get_order. A dataset of goals with a definition of done. A job that runs the orchestrator against that dataset on every schema change. Fail the build when the payment API is called in a “do not refund” case.

That is architecture. A demo with just a happy path is not a product.

When a retrieved doc tries to hijack the next tool, you are in OWASP LLM01 (external link) territory. Treat it as a test category. The Part 8 post lists the rest of the routine tests. Come back here when someone asks “where does evaluation sit in the diagram?” Answer: beside orchestration, reading the same traces.

Example

Trace: get_order(9) → paid → draft_ticket → stop.
Pass.
Trace: get_order(9) → paid → refund(9).
Fail, even if the closing sentence is polite.

The inspector does not care that the LLM is eloquent. The inspector cares whether the right tools were used correctly.

Conclusion

You now have the eight boxes from the hub. The model proposes. Memory persists. Planning revises. Tools exist. Function calling is the form. RAG grounds. Orchestration clocks. Evaluation watches the trip. For how a team runs that watch until production, read Evaluation and Testing. For what the agent may touch, Part 6 of AI Right Now still applies.

Takeaway: Evaluation is a layer that scores trajectories — tool calls and stops — not a quick feel-check of the final paragraph.

Sources