Software testing and QA roles are evolving with AI integration, shifting toward broader engineering responsibilities and quality-focused skills like risk mitigation and prompt engineering. Testers must adapt to new challenges such as non-deterministic AI behavior and model drift, leveraging traditional strengths while embracing monitoring, automation, and AI-driven quality pipelines.

The Vernon Richard Show

The Vernon Richard Show - hosted by Vernon Richards and Richard Bradshaw. Usually a Testing themed discussion between Vernon and Richard. Official show notes have summary description, timestamped chapter headings, resource links. Show notes on the website sometimes have a full transcript. Released as audio and video.

• https://thevernonrichardshow.com/
• https://www.youtube.com/@TheVernonRichardShow

Episode Details

• Show Notes: https://share.transistor.fm/s/6e9544bd
• Published: 2026-08-24T08:00:00Z
• Duration: 01:13:32
• Author: Vernon Richards and Richard Bradshaw

Overview

The podcast discusses the evolving role of software testing and quality assurance in the context of rapidly advancing AI technologies. As AI becomes integral to development workflows, traditional QA roles are shifting toward broader engineering responsibilities, with a growing emphasis on quality as a shared, cross-functional priority. While demand for conventional testing positions declines, the need for quality-focused skills remains strong, particularly in risk mitigation, prompt engineering, and evaluating AI-generated outputs. Testers are encouraged to leverage their existing strengths - such as attention to detail, structured thinking, and exploratory testing - to transition into AI engineering and related fields.

AI introduces new challenges, including non-deterministic behavior, model drift, and the difficulty of defining oracles for correctness. Testing AI systems requires adapting traditional techniques - like boundary analysis and regression testing - to probabilistic environments where outputs can vary significantly. The discussion highlights the importance of monitoring, observability, and automated quality pipelines that use AI to enforce standards in code, documentation, and ticketing at scale. Despite concerns about job displacement, the outlook remains positive: skilled quality professionals are well-positioned to lead in this new landscape by embracing change, experimenting with AI tools, and applying timeless testing principles to emerging technologies.

What If

• What if you became the AI quality gatekeeper for your solo development workflow?

  • Move: Integrate an automated AI-powered quality check into your CI/CD pipeline (e.g., using LLMs to review pull request descriptions, code comments, and test coverage reports before merge).
  • Why Now?: With AI generating large code blocks and rapid iterations becoming standard, manual oversight is no longer scalable - especially for a solo developer juggling multiple roles.
  • Expected Upside: Catches inconsistencies early (like outdated prompt-API mismatches), improves documentation quality, and reduces technical debt by enforcing standards without constant self-discipline.

• What if you treated every AI-generated feature as a black-box experiment requiring structured validation?

  • Move: Design end-to-end test cases before writing any prompt or integrating an AI model - treat it like a test-first approach, defining expected outputs, boundaries, and failure modes upfront.
  • Why Now?: AI systems drift over time and respond unpredictably to input variations; waiting until after deployment increases risk of undetected regressions in critical workflows.
  • Expected Upside: Builds a reusable regression suite that monitors model behavior changes (e.g., output distribution shifts from 70-30 to 55-45), giving you data-driven confidence to ship or roll back.

• What if you repurposed your test-case design skills into prompt engineering with built-in validation logic?

  • Move: Write prompts that include explicit output formatting rules, error-handling conditions, and self-validation steps (e.g., “If input contains X, respond Y; otherwise, flag Z”), then automate checks against edge cases.
  • Why Now?: Prompt fragility (e.g., misinterpreting separators or schema fields) is a common cause of AI system failures - structured prompts reduce ambiguity and increase reliability.
  • Expected Upside: Turns your QA expertise into a competitive advantage, enabling robust, maintainable AI features that require less debugging and earn stakeholder trust faster.

Takeaway

• Adopt AI tools in your development workflow to accelerate coding and testing, such as using AI to generate boilerplate code or set up infrastructure (e.g., Azure functions), then validate outputs with structured test cases.
• Reframe your QA skills for AI-related roles by treating prompt engineering as test case design - focus on input variation, boundary analysis, and expected outcomes to ensure reliable AI behavior.
• Build automated quality checks into your deployment pipeline using AI to enforce standards in documentation, ticket formatting, and code quality, reducing manual review overhead.
• Monitor AI system outputs over time for model drift by establishing baseline performance metrics (e.g., response distribution) and setting up alerts for significant deviations that could impact business outcomes.
• Position yourself as a quality specialist in small, fast-moving AI teams by leveraging your ability to review large code changes and assess risk - skills increasingly valuable as developers rely on AI-generated code that’s harder to verify line-by-line.