Skip to main content
AI Agents and ADK

AI Agent Evaluation on Google Cloud: Simulation, Autoraters, Online Monitors, and Observability

AI agent evaluation on Google Cloud with Gemini Enterprise Agent Platform: eval cases, user and tool simulation, autoraters, online monitors, and observability.

Checked against Google's official documentation.

AI Agent Evaluation on Google Cloud: Simulation, Autoraters, Online Monitors, and Observability

Traditional software is tested by checking that the same input gives the same output. AI agents don’t work that way: they reason, choose tools, ask follow-up questions, and can give different (but equally good) answers to the same request. They can also fail in new ways: picking the wrong tool, looping, hallucinating a policy, or quietly getting worse after a model update. AI agent evaluation has to catch all of that before users do.

On Google Cloud, Gemini Enterprise Agent Platform has a full agent evaluation toolkit for this: evaluation with prebuilt and custom raters, user and environment simulation, multi-turn autoraters, online monitors for production traffic, observability dashboards and traces, and prompt optimization. This guide explains each piece and how to combine them into a practical evaluation program.

It applies whether your agents serve customers or employees working in Google Workspace and Gemini Enterprise.

Key Takeaways

  • Evaluate outcomes, trajectories, and behavior over time, not just final answers.
  • Simulation generates multi-turn scenarios and can mock tool errors and latency.
  • Online monitors watch production traffic for quality, safety, and drift.
  • Observability needs OpenTelemetry instrumentation; Model Armor emits it automatically.
  • Calibrate LLM judges against human labels before trusting them.

Table of contents

Open Table of contents

Why AI Agent Evaluation on Google Cloud Is Different

Traditional appAI agent
OutputDeterministicVaries between runs
CorrectnessExact matchOften a judgment (helpful? safe? grounded?)
PathFixed code pathsModel chooses steps and tools
ConversationSingle requestMulti-turn with clarifications
DriftChanges only when code changesCan change with model updates, data, and prompts

So you need to evaluate outcomes (did it solve the task?), trajectories (did it use the right tools in a sensible order?), and behavior over time (is it getting worse?).


The Evaluation Workflow

The agent evaluation loop on Gemini Enterprise Agent Platform

Google describes four phases:

  1. Design: define evaluation cases
  2. Execution: run inferences (the agent attempts each case)
  3. Scoring: compute metrics per trace with prebuilt or custom raters
  4. Refinement: improve the agent, then repeat

It works in two settings: local iteration on ADK agents during development, and assessment of deployed agents using historical traces or synthetic benchmarks.

Metric types

TypeNeeds a reference answer?Examples
Reference-basedYesExact match, expected tool call
Reference-freeNoHelpfulness, safety, groundedness, tool-use quality
CustomYour choiceCustom Python metrics, LLM-as-a-judge prompts
Adaptive rubricsNoRubrics generated per case (co-developed with Google DeepMind)

Multi-turn autoraters analyze whole conversations: they extract the user’s intent, generate rubrics dynamically, and issue validation verdicts, essential for agents that ask clarifying questions.


Simulation: Testing Conversations You Haven’t Seen Yet

Writing hundreds of multi-turn test conversations by hand isn’t realistic. Simulation does it for you.

User simulation

  • An LLM-driven User Simulator plays the user and drives multi-turn conversations
  • Agent Platform generates scenarios from your agent’s instructions and tool definitions
  • Each scenario has a starting prompt and a hidden conversation plan: the simulated user’s goals and how it reacts to questions

Environment simulation

Intercepts tool calls to return mocked data, a forced error, or added latency, without touching production systems. This tests the behavior that’s hardest to test in real systems: what does the agent do when the CRM is down?

Running a simulation

  1. Create at least one agent version in Agent Registry: simulation needs an immutable snapshot
  2. Generate scenarios in the console (Evaluation → New evaluation → Simulate sessions) or with generate_conversation_scenarios in the SDK; review and edit them
  3. Run with run_inference, setting the simulator’s max turns (default 5, to stop endless loops), model, and model configuration
  4. Score the resulting traces with your metrics

Online Monitors: Evaluation in Production

Tests before launch aren’t enough: real users ask things your test set didn’t include, and models and data change. Online monitors asynchronously score live traces with predefined and custom metrics, such as:

  • Response quality
  • Safety
  • Hallucination rates
  • Tool-use quality

They detect quality drift (a drop in agent performance over time, for example from changes in user behavior or external data), and you can configure quality alerts so the team hears about problems before users complain. Monitor results also appear in the agent’s Observability tab, next to traces and metrics.


Observability: Seeing Inside Every Run

Each agent’s Observability tab has six views:

ViewShows
OverviewSessions, turns, invocations, token usage (input vs. output), latency p50/p95/p99
EvaluationOnline monitor results
ModelsPer-model p95 latency, call counts, error rates, quota failures, token usage
ToolsPer-tool p95 latency, call counts, error rates, and how often no tool was called
UsageRuntime CPU and memory allocation, token usage
LogsFilterable agent logs with severity and execution summaries

Plus:

  • Traces: step-by-step execution as a graph of spans, with inputs and outputs
  • Topology: the agent’s inbound and outbound dependencies
  • MCP servers: request counts and p95 duration

Instrumentation

Agents and MCP servers must emit OpenTelemetry data following the OpenTelemetry generative AI semantic conventions. Google provides instrumentation guides for ADK and for other generative AI applications; Model Armor emits telemetry automatically. ADK telemetry metrics on Agent Runtime require ADK 2.6.0 or later.


Optimization: Turning Failures Into Fixes

Prompt optimization refines an agent’s system instructions and tool descriptions by analyzing failure patterns from evaluation and proposing targeted updates. Evaluation can also group failures into clusters, which makes the patterns easier to see. Re-run the evaluation set to confirm each improvement.

Treat suggestions like code changes: review, test against the full set, and deploy through a canary.


Building Your Evaluation Set

The quality of your evaluation depends on your cases. A good starting set:

CategoryShareExamples
Common tasks50%The ten most frequent real requests, with variations
Edge cases20%Ambiguous requests, missing information, unusual formats
Must-refuse10%Out-of-scope or unsafe requests the agent should decline
Tool failures10%Simulated errors, empty results, slow responses
Adversarial10%Prompt injection attempts, social engineering

Sources for cases: support tickets, chat logs (with privacy review), subject-matter experts, and (once live) failed production traces.


Worked Example: Evaluating an HR Policy Agent

An HR team builds an agent that answers employee questions from the handbook and benefits documents, and opens a ticket when it can’t help.

1. Cases. 80 questions from past HR tickets (common tasks), 15 ambiguous ones (“can I take time off for my move?”), 10 the agent must refuse or route (“what’s my manager’s salary?”), 10 with the ticketing tool returning errors, and 5 prompt-injection attempts hidden in a test document.

2. Metrics.

MetricTypeTarget
Correct policy answerReference-based (expected answer)90%
Cites the right documentReference-based95%
GroundednessReference-free95%
Refuses or routes correctlyCustom rubric100%
Handles tool errors gracefullyCustom rubric90%
Safety (no injection followed)Reference-free100%

3. Simulation. Scenarios generated from the agent’s instructions add 50 multi-turn conversations, including users who give incomplete information and change their minds. Environment simulation forces an error from the ticketing tool in a set of runs.

4. First results. Policy accuracy 84%, mostly due to an outdated leave policy still in the knowledge base; the agent also answered two salary questions instead of refusing.

5. Fixes. Remove the old policy document, add an explicit refusal rule to the instructions, and re-run: accuracy 93%, refusals 100%.

6. Production. An online monitor tracks groundedness and safety, with an alert if groundedness drops below 90% over a day. Failed traces are reviewed every Friday and added to the evaluation set.


Calibrating Autoraters and LLM Judges

Model-based raters scale evaluation, but they need calibration:

  • Start with human labels. Have two subject-matter experts score 50 traces. Compare the autorater’s verdicts with theirs.
  • Measure agreement. If the rater agrees with experts less than about 80–85% of the time, refine its rubric or criteria before trusting it.
  • Write specific rubrics. “Is the answer helpful?” is vague; “Does the answer state the correct leave entitlement and cite the policy section?” is testable.
  • Watch for bias. LLM judges can prefer longer answers or answers that sound confident. Include short correct answers and confident wrong ones in your calibration set.
  • Re-calibrate after changes. When you change the judge model or the rubric, repeat the check.
  • Keep humans in the loop for high-stakes domains. Use autoraters to triage, and people to decide.

An Evaluation Program That Works

StageWhat to doGate
DevelopmentRun the evaluation set locally on every changeScore doesn’t drop
Pre-releaseSimulation with user and environment scenarios; safety checksMeets targets for task success and safety
CanaryDeploy a new revision to ~10% of traffic (Agent Runtime guide)Online monitors and error rates stable
ProductionOnline monitors with alerts; weekly review of failed tracesAlerts acted on within a day
Model updateRe-run the full set before switching model versions (Model Garden guide)No regression

Metrics worth tracking

  • Task success rate
  • Tool-call accuracy (right tool, right arguments)
  • Groundedness / hallucination rate
  • Safety violations (and Model Armor detections)
  • Turns to resolution
  • P95 latency
  • Cost per successful task
  • Escalation rate to humans

Tooling: Console, SDK, and Coding Assistants

You can run evaluations three ways:

  • Console: create evaluations, simulate sessions, and view results under the agent’s Evaluation area
  • SDK: scenario generation (generate_conversation_scenarios), inference (run_inference), and scoring (client.evals.evaluate()), ideal for CI pipelines
  • AI coding assistant skills: Google publishes an Agents CLI eval skill (npx skills add https://github.com/google/agents-cli --skill google-agents-cli-eval) and an evaluation “flywheel” skill (npx skills add https://github.com/google/skills --skill agent-platform-eval-flywheel) for assistants that support skills

Google also provides a Colab notebook (Multi-Turn Agent Evaluation with User Simulation, Metric Registration, and Auto Loss Analysis) as a hands-on starting point.


Common Mistakes

  • Testing only happy paths. Most production failures are edge cases and tool errors.
  • Tiny evaluation sets. Ten cases can’t detect a 5% regression.
  • Only checking final answers. An agent can reach the right answer through an unsafe path. Check trajectories.
  • No baseline. Record scores before every change, or you can’t tell if things improved.
  • Ignoring “no tool called.” The Tools view shows when the agent answered without using a tool it should have, often a sign of hallucination.
  • Skipping evaluation for model upgrades. New model versions change behavior, sometimes with breaking parameter changes.
  • Not instrumenting. Without OpenTelemetry, you’re debugging blind.

Conclusion

Evaluation is what separates agent demos from agent products. Gemini Enterprise Agent Platform gives you the full loop: design cases, simulate users and failing tools, score with autoraters and custom metrics, watch production with online monitors and observability, and turn failures into better instructions with prompt optimization.

Start small (50 well-chosen cases, OpenTelemetry instrumentation, and one online monitor) and grow the program as your agents take on more responsibility. Agents that serve employees in Google Workspace and Gemini Enterprise deserve the same rigor as any production system people depend on.


Official Google documentation:

Related reading:

This guide contains referral or affiliate links. If you sign up through one, Workspace Blog may earn a commission at no extra cost to you. Read our referral and advertising disclosure.

Frequently Asked Questions

How do you evaluate an AI agent?
Define evaluation cases from real tasks, run the agent on them, and score each trace with metrics. Reference-based metrics such as exact match compare against an expected answer; reference-free metrics such as helpfulness, safety, and tool-use quality score the trace on its own. On Gemini Enterprise Agent Platform the workflow is Design, Execution, Scoring, and Refinement, and it works locally for ADK agents and for deployed agents using historical traces or synthetic benchmarks.
What is agent simulation?
Simulation uses an LLM-driven user simulator to play the user in multi-turn conversations. Agent Platform generates scenarios from your agent's instructions and tool definitions (each with a starting prompt and a hidden plan for how the simulated user behaves) and environment simulation can intercept tool calls to return mocked data, a forced error, or added latency.
What are online monitors?
Online monitors continuously score live production traces with predefined and custom metrics, such as response quality, safety, hallucination rates, and tool-use quality, so you can detect quality drift after deployment. You can configure quality alerts on them.
What does Agent Platform observability show?
Each agent's Observability tab has Overview, Evaluation, Models, Tools, Usage, and Logs views, covering sessions, turns, token usage, latency percentiles, per-model and per-tool error rates and latency, online monitor results, and runtime resource use. The Traces tab shows step-by-step execution as spans, and Topology shows the agent's inbound and outbound dependencies.
Do agents need special instrumentation for observability?
Yes. Agents and MCP servers must emit telemetry in OpenTelemetry format, following the OpenTelemetry generative AI semantic conventions. Google provides guides for instrumenting ADK agents and other generative AI applications; built-in components such as Model Armor emit telemetry automatically.