GenAI Evaluation: One Model, Three Goals - Deepshikha, Anamika Mukhopadhyay
Categories: Podcasts , Software Testing Unleashed
Evaluating generative AI demands moving beyond binary correctness to use-case-specific metrics that link technical performance to business value. Teams must address indirect prompt injection risks and frequent model changes by combining offline and online evaluations.
Software Testing Unleashed
Software Testing Unleashed - hosted by Richard Seidl. Different guest per episode. The official Show notes contain a comprehensive overview of the episode. Released as audio and video.
- https://www.richard-seidl.com/en/testing-unleashed
- https://www.youtube.com/playlist?list=PL48Mbm-L0hjB1OdwYi9h7jrq9t352-Zk_
Episode Details
- Show Notes: https://www.richard-seidl.com/en/podcast/genai-evaluation-metrics
- Published: 2026-11-19T05:00:00Z
- Duration: 00:27:59
- Author: Richard Seidl | Software Development & Testing Expert
Overview
The discussion covers how Gen AI testing has shifted from binary right/wrong evaluation to a quality-based framework. As the speakers put it, “all the answers are correct” in the sense that multiple responses to a single prompt can all be valid, so the framework has moved to “a best versus worst framework.” This fundamentally changes how metrics are calculated and what they measure compared to traditional AI classification tasks.
The speakers emphasize that metrics must be defined per use case rather than just for the model. The same model powering a customer support bot, a clinical summarization agent, and a coding agent requires entirely different definitions of “good”: satisfaction and issue resolution for the bot; faithfulness and adoption rate for the clinical tool; executability and security for the coding agent. They describe a continuous loop of offline evaluation (user research, golden datasets, adversarial testing) and online evaluation (observability, tracing every LLM call), with production data feeding back into offline testing.
A key example involves an indirect prompt injection discovered through a crafted Reddit post that exploited an agent’s web tool-calling capability, bypassing direct-injection guardrails. The speakers also highlight the challenge of frequent model changes and deprecations, and stress that technical signals like faithfulness scores must be translated into business-impact language for decision-makers.
What If
-
What if you redefined your metrics around your specific use case instead of your model?
- Move: Identify the three or four failure points in your Gen AI system’s architecture and define one metric per failure point tied to a business outcome (e.g., issue resolution rate rather than a raw faithfulness score).
- Why Now?: The source notes that teams commonly “measure what our tools give us by default” without asking whether those metrics actually reflect what their users need, and that “we don’t put our heads into thinking, do we really need this or do we really need that?”
- Expected Upside: You could potentially align your evaluation with the actual business value your system delivers, making it easier to justify continued investment or identify when to pivot. This is a proposed approach, not a guaranteed outcome.
-
What if you tested your agent’s tool-calling paths for indirect prompt injection?
- Move: Create a test document (a web page, forum post, or file) containing adversarial instructions such as “forget about the previous instruction” and verify that your agent’s web-search or file-reading tool calls do not execute those instructions.
- Why Now?: The source describes a case where direct prompt injection guardrails were in place but an indirect injection via a Reddit post still succeeded because the agent’s web tool call retrieved the malicious content, and the team had not considered that “agent is going to do a lot of tool calling.”
- Expected Upside: You might catch a vulnerability class that standard adversarial testing misses, reducing the risk of data exfiltration or policy violations in production. This is a possible benefit, not a source-confirmed result.
-
What if you built a model-change contingency into your evaluation pipeline?
- Move: Set up automated re-evaluation triggers so that when your model provider ships a new version or deprecates one, your offline golden dataset and online observability traces automatically re-run against the new model.
- Why Now?: The source describes a team that spent “almost two, three months” tuning a Gemini Flash model only to learn it would be deprecated in the next two months, and notes models change every “two weeks, three weeks.”
- Expected Upside: You could potentially avoid months of rework by detecting regressions such as increased sycophancy immediately after a model update rather than discovering them in production. This is a proposed mitigation, not a source-confirmed outcome.
Takeaway
-
Define metrics per use case, not per model. For each distinct use case (support bot, summarization, coding), write down what “good” means in business terms - satisfaction, faithfulness plus adoption rate, executability plus security - before selecting any technical metric.
-
Run offline and online evaluation as a continuous loop. Build a golden dataset from user research for offline testing, and deploy observability traces for every LLM call in production so that production data feeds back into your offline evaluation when models change.
-
Test indirect prompt injection through your agent’s tool calls. Craft adversarial content (a web page, forum post, or document) that your agent’s search or file-reading tools will retrieve, and verify guardrails hold when the injection arrives indirectly rather than in the user prompt.
-
Translate every technical metric into a business-impact statement before presenting it. Reframe scores like faithfulness values in terms of issue resolution rate, support ticket volume, or cost impact so that stakeholders can act on the number.
-
Plan for model deprecation and version changes. Maintain a mitigation plan with re-evaluation triggers so that when your provider deprecates or updates a model, you can detect regressions such as sycophancy shifts without repeating months of manual tuning.
For a PDF of longer Software Testing Podcast Episode Summaries with Briefing Notes and more detailed summary notes, visit EvilTester Patreon Podcast Summaries.