The text examines how testing and quality engineering must evolve in the AI era, prioritizing adaptability, collaboration, and proactive system design over passive bug detection. It emphasizes balancing AI-driven automation with human judgment, ethical considerations, and the necessity of preserving creativity and intent in software development.

AB Testing

AB Testing - Each episode is a chat between Brent Jensen and Alan Page with an occasional special guest.

• https://www.moderntesting.org/

Episode Details

• Show Notes: https://podcasters.spotify.com/pod/show/abtesting/episodes/Episode-229-Testing-in-the-Age-of-AI-e3hhdh8
• Published: 2026-04-09T14:00:00Z
• Duration: 00:55:04
• Author: AB Testing

Overview

The podcast explores the evolving role of testing and quality engineering in the AI era, emphasizing adaptability, collaboration, and proactive influence over complacency. Smart Bear, a company rooted in the idea of making meaningful differences (inspired by a novel), has transitioned from a code review tool to a broader testing and quality engineering platform. Key discussions revolve around challenges posed by AI, such as whether testing is replaceable and how testers should adapt. Modern principles prioritize adaptability, curiosity, and collaboration, shifting testing priorities from merely finding bugs to influencing system design and integrating quality early in development. Testers are encouraged to advocate for their role, drive change, and prioritize business value over bug detection, while working closely with developers to embed quality from the start. The conversation also highlights the need for testers to embrace AI as a tool, leveraging it for automation, analysis, and iterative refinement, but cautioning against over-reliance without human judgment or context.

The episode delves into ethical considerations, systemic risks, and the balance between AI efficiency and human insight. While AI excels in rapid prototyping and identifying inefficiencies, it risks displacing hands-on experience and critical thinking, leading to “noise” and overproduction. Examples include AI-generated code or tools that bypass trial-and-error learning, potentially diluting the depth of understanding. The discussion underscores the importance of human creativity, intent, and the creative journey in software development, contrasting with AIs role as a catalyst for execution rather than idea generation. Challenges in modern testing include unclear success criteria, late-stage issues, and poor collaboration, with calls for upfront planning and traceability to ensure quality. Autonomous agents in testing, when guided by clear intent, can enhance coverage but require human oversight to avoid misalignment with goals. The future of testing is framed as a hybrid model, blending agentic tools with human expertise, while emphasizing the need for thoughtful, practice-driven approaches to maintain ethical responsibility and systemic integrity in software development.

What If

• What if you integrated AI-driven test automation with human oversight to refine exploratory testing strategies?

  • Concrete move: Use AI to generate test ideas (e.g., security checks, edge cases) from code repositories, then manually prioritize and execute the most critical ones.
  • Why now: Modern testing prioritizes influence over compliance, and AI can accelerate idea generation while retaining human judgment for context.
  • Expected upside: Faster identification of high-risk areas, reducing rework and improving system robustness with minimal manual effort.

• What if you shifted from “finding bugs” to co-designing quality with developers using early-stage collaboration techniques?

  • Concrete move: Implement a “definition of ready” process for tickets, requiring developers and testers to co-review requirements and design before coding.
  • Why now: Accelerated development cycles demand proactive quality integration, and late-stage testing is costly and inefficient.
  • Expected upside: Reduced late-stage defects, stronger alignment on objectives, and a culture of shared responsibility for quality.

• What if you leveraged AI to audit open-source dependencies and trace their impact on your projects quality and compliance?

  • Concrete move: Use AI tools to scan your projects open-source chain, flagging under-maintained libraries or security risks, then validate with manual review.
  • Why now: Systemic risks in open-source dependencies are critical in regulated industries, and AI can surface hidden vulnerabilities early.
  • Expected upside: Proactive risk mitigation, compliance readiness, and reduced exposure to cascading failures from flawed dependencies.

Takeaway

• Integrate testing early in the development cycle: Collaborate directly with developers to embed quality into system design rather than waiting until later stages. This includes participating in upfront planning, defining acceptance criteria, and ensuring testability is considered during feature design.

• Leverage AI for test automation but validate outputs manually: Use AI tools to optimize test suites, identify code redundancies, or generate test ideas, but critically review AI-generated outputs for context, alignment with business goals, and potential blind spots.

• Establish traceability between tests, code, and requirements: Link tests to specific code changes, user stories, and task management systems to ensure transparency, accountability, and informed decision-making during development and debugging.

• Prioritize systems thinking in testing: Focus on identifying root causes of issues (e.g., flawed requirements, design flaws) rather than surface-level bugs. Use examples like the “Malcolm in the Middle” light bulb analogy to challenge assumptions and seek holistic solutions.

• Advocate for structured requirement analysis using AI: Apply AI tools (e.g., risk storming, AI-driven documentation reviews) to clarify and flesh out requirements, ensuring gaps are identified early. Use clear definitions of “good” and structured practices to guide AI-assisted requirement validation.