Trajectory evals for agents
For agents you check two things: whether the final result is right, and whether the path was sound.
Words on this page (4)
- agent
- : A model run in a loop that chooses its own tools and steps until the task is done.
- eval
- : A repeatable test of AI output quality, run before and after every change.
- trajectory
- : The sequence of steps and tool calls an agent took, not just its final answer.
- tool
- : A function the model can ask your program to run, described by a name, description and argument schema.
What it is
Output asks if the final artifact is correct: do the tests pass, is the answer right. evaluation inspects the sequence of steps: did the choose the right , avoid unsafe actions, and skip needless work? An answer that looks right but skipped its verification steps is more dangerous than an obvious failure, because it passes casual review.
How to run them
Define tasks with a checkable end state, such as a database row, a file or a passing test suite. Run each task several times, since agents vary from run to run, and report pass rate. Record the full transcript so you can read failures. Track cost and number of steps as well as success.
A worked example: scoring a run
Task: "Upgrade the library and keep tests passing." Outcome check: tests pass (yes). Path checks: did it modify test files (yes, 3, suspicious), did it run the tests after the change (yes), did it touch files outside the project (no), how many steps (14), what did it cost ($0.42). Outcome passed, path flagged: someone should read that transcript.
Because agents vary
Run each task five times. A task passing 5 of 5 is reliable. A task passing 2 of 5 is not, even though a single lucky run looks fine. Report pass rates, not single runs.
Why 5 of 5 is not proof
Suppose an agent really succeeds 80 percent of the time on a task. The chance it passes all 5 runs is 0.8 multiplied by itself five times, about 33 percent. So a flaky agent shows 5 of 5 in one test out of three. Five clean runs are encouraging, not a guarantee. More runs give a better estimate.
Errors also add up over steps. If each of 10 steps works 95 percent of the time, the whole chain works about 60 percent of the time (0.95 to the power 10). Long tasks need much better steps than short ones.
What goes wrong: grading the exact path
You write the expected steps and fail any run that differs. An agent solves the task in 9 steps instead of your 12 and is marked wrong. Symptom: failures whose transcripts look fine when you read them.
Check the rules that matter instead: no forbidden action, a required step happened (tests ran after the last edit), and the step count stayed under a budget.
How to tell the path check works
Test the checker itself. Write or record a run that deletes a failing test, and confirm the check flags it. Then feed it a clean run and confirm it passes. A check you have never seen fail may be checking nothing.
Common mistake
Judging only the final message. Agents can sound confident after doing the wrong thing.
Two agents both fix a bug and the tests pass. One did it by understanding the problem. The other deleted the failing test. The final result looks the same. Only by reading the steps do you see the difference. That is why agent evals look at the path as well as the outcome.
Check yourself
Why run each agent task several times?
Not quite. One run can pass or fail by luck.
What does trajectory evaluation catch that output evaluation can miss?
Not quite. The path matters when the result cannot be fully verified.
An agent makes the tests pass by deleting the failing test. What kind of eval spots this?
Not quite. The outcome passes, but a trajectory check shows what the agent did.
You got 0 of 3 right.
Go deeper
Spotted a mistake or something unclear? Report it on GitHub.