Measure agent quality beyond pass/fail — task completion, trajectory efficiency, tool call accuracy, and regression testing for agent systems.
Five passes over the same idea, each from a different angle. Do them in order, or jump to whichever you need.
Evaluating agents is hard because the output is a trajectory, not a single token. A correct final answer that wasted 20 tool calls is worse than a slightly less accurate answer reached efficiently. Agent eval frameworks like AgentBench, WebArena, and custom harnesses measure task success rate, step efficiency, hallucination rate in tool arguments, and regression across versions.
