AI in software testing enhances efficiency with tasks like test case generation but lacks true understanding, requiring human oversight. Agentic systems - interconnected AI agents - streamline workflows but still depend on human validation for accuracy and reliability.

Software Testing Unleashed

Software Testing Unleashed - hosted by Richard Seidl. Different guest per episode. The official Show notes contain a comprehensive overview of the episode. Released as audio and video.

• https://www.richard-seidl.com/en/testing-unleashed
• https://www.youtube.com/playlist?list=PL48Mbm-L0hjB1OdwYi9h7jrq9t352-Zk_

Episode Details

• Show Notes: https://www.richard-seidl.com/en/podcast/agentic-testing-pipeline
• Published: 2026-09-17T04:00:00Z
• Duration: 00:28:11
• Author: Richard Seidl | Software Development & Testing Expert

Overview

The podcast discusses the current limitations and practical applications of AI in software testing, emphasizing that AI lacks true understanding and operates by processing data from its training. While AI tools can assist with tasks like requirement review, test case generation, and performance testing, they do not replace human judgment. A key distinction is made between individual AI agents and full agentic systems - interconnected ecosystems of agents that work autonomously but still require human oversight for validation and correction.

An example of an agentic system in testing involves 8 to 12 specialized agents handling various stages of the testing workflow, from analyzing requirements to executing tests and investigating failures. These systems aim to reduce manual effort while maintaining quality, but humans remain essential in evaluating AI-generated outputs, refining test cases, and ensuring accurate automation. The discussion also covers practical considerations such as data privacy, cost efficiency, and system configuration, highlighting the importance of balancing automation with controlled human intervention to build effective, reliable testing processes.

What If

• What if you built a minimal agentic system for test case validation using local AI to maintain data control?

  • Move: Set up a lightweight agentic workflow using n8n or Make.com to connect a locally hosted AI (e.g., Mistral) that reviews incoming JIRA requirements and generates test cases, with a manual approval step for final validation.
  • Why Now?: With rising data privacy laws (e.g., Swiss regulations), relying on US-based AI like ChatGPT poses compliance risks - local AI ensures control and avoids legal friction while still automating early testing phases.
  • Expected Upside: Reduce time spent on initial test design by 40 - 60% while maintaining full data ownership; create a reusable template for future agentic workflows in secure environments.

• What if you automated your regression testing pipeline by transforming functional test cases into performance checks using AI?

  • Move: Build a script-based agent that parses existing automated functional tests, identifies high-load user paths, and generates corresponding performance test configurations (e.g., for k6 or Locust) with human-defined thresholds.
  • Why Now?: Most solo developers overlook performance until late stages - by leveraging AI to repurpose existing functional logic now, you catch scalability issues early without writing tests from scratch.
  • Expected Upside: Cut performance test creation time by 70%, increase system reliability under load, and shift performance validation left in your development cycle.

• What if you delegated test execution and failure triage to an AI agent while retaining final judgment on bug reporting?

  • Move: Implement an agent that runs automated test suites on CI triggers, uses rule-based logic to detect UI changes (e.g., element shifts vs. true failures), and sends summarized results to you for decision-making via a dashboard or email digest.
  • Why Now?: As a solo operator, your time is constrained - automating execution and initial analysis frees you from repetitive debugging loops while avoiding blind trust in AI decisions.
  • Expected Upside: Save 5 - 10 hours per week on test monitoring, reduce false positives through contextual filtering, and focus your expertise on validating real issues instead of sifting through logs.

Takeaway

• Implement a modular agentic system using 8 - 12 specialized AI agents to automate repetitive testing tasks like requirement review, test case generation, and performance testing, tailored to your domain’s compliance needs.
• Configure AI agents with version-controlled content and iteratively refine system prompts to define clear skills, boundaries, and decision-making rules for reliable task execution.
• Use AI to convert functional test cases into performance test scenarios, but manually specify target systems and thresholds to ensure accurate bottleneck detection and compliance with scalability goals.
• Set up local AI models (e.g., Mistral, DeepSeek) on GPU-equipped servers when handling sensitive data, especially if operating under strict data residency laws like GDPR or Swiss regulations.
• Retain human-in-the-loop validation at critical decision points - such as test case approval, bug analysis, and automation logic - to verify AI outputs and prevent error propagation in production workflows.