AI integration in software testing is framed as a collaborative tool, with Rex Blacks VREX system and risk-based strategies showcasing efficiency gains, while challenges like overhyping, disciplinary implementation, and testing AI-driven systems demand balanced, incremental adoption. Practical tools like Build Insights highlight cost savings, but QA professionals must balance AIs strengths in coaching and prioritization with irreplaceable human judgment and evolving skill requirements.

How To Test This?

interview episodes where Mamadou N’diaye talks with with software testing experts

• https://podcasters.spotify.com/pod/show/spidey1944
• https://www.youtube.com/@HowToTestThis
• https://www.linkedin.com/in/mamadou-ndiaye-consultant/

Episode Details

• Show Notes: https://podcasters.spotify.com/pod/show/spidey1944/episodes/How-to-test-with-an-AI-Partner---Rex-Black-e3l3rn6
• Published: 2026-06-22T03:53:32Z
• Duration: 00:40:23
• Author: Mamadou N’diaye

Overview

The podcast explores the integration of AI in software testing, emphasizing its potential to enhance QA processes when used as a collaborative partner rather than a standalone tool. Rex Black, a veteran in software quality with over four decades of experience, discusses his development of VREX, an AI system built on his extensive research and writings, aimed at improving testing efficiency. Key themes include historical context for AI in testing, with roots tracing back to 1980s research on neural networks and Lisp, and the challenges of modern AI adoption, such as overhyping tools and the need for disciplined implementation. Rex highlights practical applications, including risk-based testing strategies that prioritize high-impact areas, achieving 2040% efficiency gains without compromising quality. He also critiques unrealistic expectations around full automation, stressing that AI excels in supporting tasks like knowledge coaching, decision-making, and regression test prioritization rather than replacing human judgment.

The discussion also addresses workflows for implementing AI in testing, such as using tools like Build Insights to optimize regression testing and reduce manual effort, saving equivalent labor costs of three full-time employees annually. Challenges include managing the infinite scope of test cases in large systems and avoiding “overselling” AI capabilities, as many tools fail to deliver promised results due to insufficient integration with traditional testing principles. Rex emphasizes the evolving role of QA professionals, urging them to adopt AI literacy to remain competitive, while noting that human expertise in risk assessment and contextual decision-making remains irreplaceable. Additionally, he critiques gaps in AI training resources and the growing complexity of testing AI-driven systems themselves, which requires new methodologies and specialized skills. Finally, the podcast underscores the importance of incremental, practical AI adoption, focusing on efficiency gains and avoiding overambitious goals like full test automation.

What If

• What if you integrated AI into your regression testing workflow to prioritize test cases based on impact?

  • Move: Adopt a RAG-based system like VREX to analyze code changes and auto-prioritize regression test cases by risk.
  • Why Now?: Industry data shows 1530% of QA budgets detect only 15% of defects, creating a resource mismatch. VREXs growth phase (usage doubling every 3 months) indicates scalable adoption.
  • Expected Upside: Achieve 2040% efficiency gains in test case creation, reduce manual effort, and align testing with defect risks, potentially improving ROI by 150% as seen in case studies.

• What if you built a risk-based AI coach to streamline QA decision-making for your team?

  • Move: Develop a domain-specific AI coach (e.g., Quality Risk Coach) using your testing expertise and historical data to guide risk-based testing.
  • Why Now?: The Quality Risk Coach alone saved one FTE annually (20% rollout completion at time of discussion), and 40% of testing effort is wasted on non-critical risks.
  • Expected Upside: Reduce knowledge gaps in risk assessment, cut time spent on low-impact tasks, and improve team productivity by 40% through better prioritization.

• What if you launched an AI literacy training program tailored to QA professionals?

  • Move: Create a certification program combining AI basics (context windows, hallucinations) with risk-based testing, leveraging free AI resources and existing frameworks like PMP.
  • Why Now?: AI adoption risks overselling and under delivering, and testers need judgment to avoid over-reliance on tools. Manual QA still requires 80% of tester expertise per case study.
  • Expected Upside: Position yourself as a thought leader, attract clients needing AI-integrated testing, and reduce implementation failures by 60% through trained teams.

Takeaway

• Adopt risk-based testing to prioritize high-impact areas: Implement a structured risk analysis process to allocate testing resources effectively, reducing test case creation efforts by up to 40% while maintaining quality (e.g., prioritize testing based on likelihood and impact of defects, as demonstrated by the 150% ROI case study).
• Leverage AI for incremental, practical tasks: Use AI tools like VREX or AI assistants (e.g., Claude, ChatGPT) to streamline repetitive workflows (e.g., generating test strategies, summarizing documentation, or creating assessment questions) rather than pursuing full automation of test execution.
• Develop an AI-powered tool tailored to your expertise: Build a RAG-based system (like VREX) by curating your domain knowledge (books, articles, presentations) and integrating it with AI to act as a research assistant, improving productivity (e.g., saving equivalent labor of three FTEs annually).
• Avoid overhyping AI capabilities: Focus on realistic expectations for AI integration, avoiding claims of “full test execution automation” and instead emphasizing its role in decision support, knowledge coaching, and reducing manual effort (e.g., using AI to prioritize regression tests or analyze code changes).
• Invest in AI literacy and specialization: Acquire foundational AI knowledge (e.g., context windows, hallucinations, iterative prompting) and consider specializing in testing AI systems, as this is a growing field with opportunities for QA professionals (e.g., using AI coaches like the Quality Risk Coach to enhance risk-based testing).