Testing in software development is shifting toward collaborative, whole-team quality practices, with testers transitioning to coaching roles focused on critical thinking and shared responsibility. AI integration streamlines repetitive tasks but requires balancing automation with human judgment, emphasizing accuracy, ethical oversight, and strategic decision-making frameworks.

AB Testing

AB Testing - Each episode is a chat between Brent Jensen and Alan Page with an occasional special guest.

• https://www.moderntesting.org/

Episode Details

• Show Notes: https://podcasters.spotify.com/pod/show/abtesting/episodes/Episode-230-Kat-Obring-is-Back-e3iufbh
• Published: 2026-05-05T19:05:48Z
• Duration: 00:54:22
• Author: AB Testing

Overview

The podcast explores the evolving role of testing in software development, emphasizing a shift from traditional, siloed practices to collaborative, whole-team quality approaches. Testers are increasingly transitioning from managerial roles to coaching roles, focusing on decision-making, critical thinking, and fostering a culture of shared responsibility. AIs integration into testing is a central theme, with discussions on its potential to streamline repetitive taskssuch as automating invoice generation or filtering datasetswhile stressing the importance of balancing automation with human judgment to avoid over-reliance or underutilization. Key challenges include ensuring AI-generated outputs are evaluated for accuracy, defining clear outcomes and tolerances, and using AI as a thought partner to deepen exploratory testing rather than merely generating outputs. Decision-making frameworks, such as Daniel Kahnemans System 1 and 2 thinking, are highlighted, advocating for deliberate, context-aware choices using methods like the ASK, Reflect, Respond model to enhance strategic thinking and adaptability.

The discussion also addresses broader implications for testing and organizational practices, including the need for testers to understand code to effectively evaluate software, even if they arent developers. As AI tools democratize coding capabilities across teams, collaboration and domain knowledge become critical, with quality assurance evolving to address risks from non-specialized code contributions. Ethical considerations dominate, emphasizing the importance of human oversight in AI use, avoiding knee-jerk automation, and prioritizing security and privacy in internal tooling. The podcast underscores the growing relevance of testing in fast-moving environments, urging testers to identify and address incremental bottlenecks, adapt skills to infrastructure roles, and leverage data to improve efficiency. Finally, it touches on practical applications of AI in content organization, such as using transcription tools and AI-driven databases to manage podcast content, reflecting a broader trend of integrating AI into workflows while maintaining a focus on human-centric decision-making and problem-solving.

What If

• What if you used AI to accelerate exploratory testing by simulating test scenarios in parallel with code changes?
  Concrete move: Deploy an AI tool (e.g., LLM-based test idea generator) to auto-generate test cases based on code diffs or feature descriptions, then prioritize them using risk-based criteria.
  Why now: Modern AI tools can rapidly adapt to new code contexts, reducing manual effort in test design. Teams are moving faster, and this helps identify edge cases early without slowing development.
  Expected upside: Faster feedback loops, reduced rework from untested code, and more time to focus on high-risk areas.

• What if you integrated a decision-making framework like ASK, Reflect, Respond into your daily workflow to evaluate AI-generated outputs?
  Concrete move: Use a lightweight tool (e.g., Notion or Obsidian) to log AI-assisted decisions, including questions asked, analysis of results, and action taken. Review these logs weekly to refine your critical thinking.
  Why now: Automation and AI are increasing, but human judgment remains critical. Structured frameworks help avoid over-reliance on tools while improving accountability.
  Expected upside: More deliberate, data-backed decisions and a proven record of AI-human collaboration for future audits or scaling.

• What if you used AI to bridge the code literacy gap by analyzing code changes and suggesting test strategies without needing to write code?
  Concrete move: Train an AI model (or use an existing one like GitHub Copilot) to interpret code commits, identify potential risks (e.g., untested edge cases), and recommend exploratory testing scenarios.
  Why now: As more non-developers contribute code (e.g., product teams), testers must understand codes impact on quality without writing it. This aligns with the texts emphasis on code readability over writing skills.
  Expected upside: Faster identification of critical flaws in AI-generated or shared code, and reduced friction in cross-functional collaboration.

Takeaway

• Leverage AI for Repetitive Administrative Tasks: Automate time-consuming workflows like invoice generation or report creation using AI tools, freeing up time for strategic decision-making and creative problem-solving in your software business.
• Define Clear Tolerances for AI Outputs: When using AI in testing or content generation, explicitly outline acceptable outcomes (e.g., error margins, hallucination thresholds) and validate results with human judgment to avoid over-reliance on AI.
• Enhance Code Understanding with AI Tools: Use AI to analyze code changes, interpret complex logic, or generate test strategies, even if youre not a full-time developer, to improve your ability to evaluate software quality effectively.
• Build Custom AI Tools for Workflow Bottlenecks: Identify repetitive or inefficient tasks in your workflow (e.g., filtering datasets, test script generation) and create tailored AI solutions using frameworks like GitHub repositories or LLMs to accelerate your work.
• Adopt Decision-Making Frameworks for Testing: Apply structured approaches like ASK, Reflect, Respond to evaluate test scenarios or tool choices, ensuring decisions are data-driven, balanced, and aligned with long-term project goals.