Automation in quality is not just a new concept but also a rebranding of existing practices, raising questions about whether AI and generative AI enable new forms of quality automation beyond traditional testing.

The intersection of AI, automation, and quality assurance is blurring the lines between traditional testing and broader quality-focused efforts, with opportunities for leveraging tools to improve story quality, documentation, and system understanding.

The Vernon Richard Show

The Vernon Richard Show - hosted by Vernon Richards and Richard Bradshaw. Usually a Testing themed discussion between Vernon and Richard. Official show notes have summary description, timestamped chapter headings, resource links. Show notes on the website sometimes have a full transcript. Released as audio and video.

• https://thevernonrichardshow.com/
• https://www.youtube.com/@TheVernonRichardShow

Episode Details

• Show Notes: https://share.transistor.fm/s/a1b73d68
• Published: 2026-03-02T08:00:00Z
• Duration: 3072
• Author: Vernon Richards and Richard Bradshaw

Overview

The podcast examines the evolving role of automation in quality assurance, questioning whether it signifies a substantial advancement over traditional test automation or is merely a rebranded concept. It highlights potential applications of AI and large language models (LLMs) in quality assurance, including analyzing user stories, automating feedback loops, and enabling earlier-stage (shift-left) testing through static analysis. The discussion emphasizes the importance of distinguishing between quality and testing, exploring how AI could help enforce best practices while addressing challenges related to terminology and documentation.

The conversation also underscores the ongoing need for human judgment in quality assurance, noting that AI currently lacks the ability to replicate real-world expertise. It speculates on the future of AI-driven tools, such as automated agents and multi-agent systems, which could redefine quality assurance by enabling collaborative context-building and more dynamic testing processes. The dialogue balances optimism about AI’s potential with caution about its limitations, advocating for a complementary relationship between human insight and technological innovation.

What If

• What if you used an LLM to auto-analyze user stories for completeness and coherence before development?

  • Concrete move: Build a tool that ingests user stories and acceptance criteria, using an LLM to flag gaps (e.g., missing edge cases, unclear verbs) and suggest refinements.
  • Why now: Shift-left quality is critical, and AI can preemptively surface issues that would later become bugs or misaligned features.
  • Expected upside: Reduce rework by 30% by catching story flaws early, ensuring developers build whats actually needed.

• What if you automated Slack/Jira feedback loops using AI to evaluate PRs for scope misalignment?

  • Concrete move: Integrate an AI agent into your CI/CD pipeline that analyzes code changes in PRs, flags unrelated file edits, and triggers Slack alerts for PMs to verify alignment with tickets.
  • Why now: Manual PR reviews are error-prone, and AI can surface hidden risks (e.g., accidental refactoring) before merges.
  • Expected upside: Cut downstream regression issues by 40% and reduce QA time spent on unrequested features.

• What if you built a dynamic test selection system using LLMs to prioritize tests based on code changes?

  • Concrete move: Train a model to map file changes to system areas, then auto-select high-priority tests for immediate execution, deferring lower-impact tests to later stages.
  • Why now: Traditional “run all tests” pipelines waste time; AI can optimize feedback loops by focusing on critical paths.
  • Expected upside: Shrink test cycle times by 50% while maintaining coverage, enabling faster iteration for solo developers.

Takeaway

• Implement AI-driven analysis of user stories and acceptance criteria using LLMs to identify gaps, ensure coherence, and flag inconsistencies before development begins, reducing rework and improving story quality.
• Automate feedback loops with AI tools to trigger Slack notifications, Jira updates, or other actions based on AI evaluations of code changes, user stories, or test results, streamlining communication and issue tracking.
• Use static analysis tools integrated with AI to evaluate Jira tickets, analyze code changes, and predict system-wide impacts (e.g., via LLMs), enabling proactive refactoring and risk identification before development.
• Clarify roles and terminology (e.g., QA, QE, testing) within your team or documentation to avoid confusion in RFPs, sales materials, or internal processes, ensuring alignment with quality-focused automation goals.
• Adopt dynamic test selection strategies by leveraging AI to prioritize tests based on code changes, minimizing execution time while focusing on high-risk areas, and integrating this into CI/CD pipelines for faster feedback.