How to evaluate the performance of AI agents?

Traditional software testing is straightforward: you give input X and expect output Y. If the function returns the wrong value, the test fails. LLM-based agents don't work that way. They're non-deterministic which means the same prompt can produce different outputs across runs. They operate over multiple steps, making decisions about which tools to call, what parameters to pass, and how to interpret results. An agent can complete an execution without errors and still hallucinate facts, miss the user's intent, or take unnecessary steps. Classical testing may not catch problematic outputs produced by an AI Agent. When building AI Agents, you face three main evaluation challenges: - You're evaluating trajectories, instead of just outputs. An agent might give the correct final answer but call the wrong tools, use the wrong parameters, or take five steps when one would do. If you…