Agentic AI technologies are being utilized in software development to enhance efficiency and quality, with agents capable of observing, collecting data, reasoning, and autonomously executing tasks. A guest on the podcast has developed an agentic QE tool, blending software development and AI to improve test planning and test execution.

The EvilTester Show

The Evil Tester Show - hosted by Alan Richardson. Occasional guest episodes. Show notes have comprehensive descriptions and resource links. Released as audio and video.

• https://eviltester.com/show
• https://eviltester.com

Episode Details

• Show Notes: https://www.eviltester.com/show/030-agentic-ai-quality-engineering/
• Published: 2026-02-12T12:29:16Z
• Duration: 00:57:35
• Author: Alan Richardson

Overview

The podcast explores the concept of agentic AI in software development, highlighting its distinct approach compared to traditional large language models (LLMs). Agentic AI integrates autonomous agents that can observe, gather data, reason, and perform actions to achieve specific objectives, enhancing automation and decision-making in complex workflows. The discussion traces the evolution of agentic AI from simple prompt engineering to more sophisticated context engineering and full agentic development, which has led to significantly higher task success rates.

A key focus of the conversation is the agentic QE system, a quality-oriented orchestration tool designed to support the entire software development lifecycle. This system includes functionalities such as test planning, test case generation, and evaluating test effectiveness, and it operates as a fleet of specialized agents that handle different aspects of testing. These agents provide deeper insights and broader expertise than individual testers could achieve alone. The system is built using orchestration frameworks like Claude Flow and is designed to be vendor-independent, enabling integration with various LLMs and supporting multiple environments.

The podcast also emphasizes the role of human oversight and expert evaluation in maintaining the reliability of agentic AI systems. Continuous performance assessment ensures that the agents remain effective and aligned with quality goals. As agentic AI advances, it is expected to shift responsibilities in software development, focusing more on output validation and quality assurance rather than manual code inspection, potentially transforming the roles of developers and testers.

What If

• What if you set up a QA agent fleet using agentic QE and Playwright for automated UI testing?
  Concrete move: Install the agentic QE tool via npm install -g agentic-qe, configure it with Playwright as a fallback testing framework, and define agents.md files for UI testing scenarios.
  Why now: Modern development demands faster feedback loops, and agentic QE can automate repetitive testing tasks, freeing you to focus on edge cases or feature innovation.
  Expected upside: Reduce manual test effort by 40% in your projects, catch flaky tests earlier, and improve test coverage consistency via context-aware agents.

• What if you train an agentic QE agent to generate test strategies using Dragan’s QCSD Ideation Form?
  Concrete move: Modify the agents.md file to include the QCSD Ideation Form as a prompt template and use Claudes code interface to generate test strategies for your next project phase.
  Why now: Shift-left testing is becoming critical for agile teams, and automating strategy creation ensures alignment withSMART criteria and reduces human bias in planning.
  Expected upside: Cut test strategy drafting time in half, prioritize high-impact test scenarios, and reduce rework caused by unclear requirements.

• What if you integrate a security-focused agent into your agentic QE fleet using existing tools like OWASP ZAP?
  Concrete move: Extend the agents.md file to include a security agent definition that leverages OWASP ZAP for vulnerability scanning, using the MCP server to route tasks to the agent.
  Why now: Security testing is often overlooked by solo developers, and agentic QEs agent fleet can expose risks (e.g., XSS, SQLi) that you might miss manually.
  Expected upside: Identify critical security flaws early in development, reduce exposure to third-party tools, and build a reputation for robust security practices.

Takeaway

• Install and test Gentic QE Fleet on a compatible OS first before customizing agents or domain definitions, as recommended for initial evaluation and to avoid issues with non-standard environments like Windows.
• Customize agent definitions in agents.md after understanding the base setup, focusing on role-based prompts and context-markup to improve agent performance and specificity for your workflows.
• Use different LLM models based on task complexity (e.g., Haiku for simple tasks, Sonnet for medium-complex tasks, Opus for strategic work) to optimize cost and efficiency when integrating with the agentic system.
• Create golden examples and documentation for agent behavior and skills to standardize practices, ensure consistency, and improve the accuracy of agent-generated outputs during testing and development.
• Engage with the Agentic Foundation community through weekly events, public experiments, and educational resources (e.g., Stanford CS 230 lectures, 1-on-1 courses) to deepen your understanding of agentic AI and stay updated on tool advancements.