The podcast critiques outdated QA terminology and testing principles, advocating for postmodern engineering to adapt to AI-driven development, human-AI collaboration, and dynamic socio-technical systems. It emphasizes real-time feedback, human oversight in AI tooling, and the shift toward trustworthy outcomes over rigid control in evolving software landscapes.

AB Testing

AB Testing - Each episode is a chat between Brent Jensen and Alan Page with an occasional special guest.

• https://www.moderntesting.org/

Episode Details

• Show Notes: https://podcasters.spotify.com/pod/show/abtesting/episodes/Episode-231-The-B-is-Back-e3jlkrl
• Published: 2026-05-20T19:13:49Z
• Duration: 00:51:38
• Author: AB Testing

Overview

The podcast delves into the evolution of quality assurance (QA) roles and challenges traditional terminology like “confidence engineer” proposed by Jason Arbon, arguing these concepts fall short of addressing current needs. It critiques modern testing principles, which once focused on accelerating quality delivery and improving business processes, as now inadequate in the context of rapid AI advancements and new delivery complexities. The discussion highlights persistent human bottlenecks in software pipelines despite AI-driven automation, referencing Brian Finsters ideas on continuous delivery. A proposed framework, postmodern engineering principles, is introduced to adapt to modern challenges where software development involves dynamic interactions between humans, AI agents, and platforms, rejecting rigid role boundaries and emphasizing adaptive socio-technical systems for trustworthy outcomes. The need for continuous feedback loopsreal-time rather than delayedemerges as critical, alongside redefining AI not as a “silver bullet” but as a simulated human tool prone to comparable errors as humans.

Key themes include the redefinition of AI as “LLM chatbot agents” (LOMs) and the exploration of systems thinking to manage complex, AI-driven environments. The podcast emphasizes the necessity of systematic oversight, transparency, and trust in AI systems, stressing that while AI can automate tasks like code review, human judgment remains vital for nuanced decisions. Examples include AI-driven tools for risk assessment and security checks, balancing machine capabilities with human oversight. The conversation also touches on the shift in software development, where coding becomes a commoditized skill accessible to non-traditional roles through tools like “vibe coding,” freeing developers for higher-level work. Broader implications include evolving roles for non-developers managing AI agents and the increasing importance of delivering trustworthy outcomes over rigid control. The discussion critiques deterministic thinking over overreliance on AI for simple decisions, emphasizing prompt engineering and critical thinking as essential skills.

The podcast reflects on agile methodologies evolution, advocating for faster adaptability and rethinking traditional testing practices in real-time environments. It addresses challenges in aligning intrinsic motivations with organizational goals, a recurring theme in prior episodes, while underscoring the need for homeostasis in systems to maintain balance. The job market is described as intensely competitive, with hiring increasingly reliant on personal networks rather than traditional recruitment. Professional narratives interweave with these themes, referencing shifts in roles, such as transitioning to Microsoft after high-stress misaligned positions, and the broader context of navigating career changes through persistence, networking, and market adaptability. The dialogue emphasizes the importance of closed-loop systems and real-time problem-solving to meet customer expectations, framing trust as a foundational element for long-term retention in an AI-driven world.

What If

• What if you integrated AI agents into your workflow as full-time team members, modeled after human roles like project owners or developers?

  • Concrete move: Set up a hierarchical AI agent system where junior agents (using free/cheap tools) handle initial tasks, and senior agents (using paid systems) review and refine outputs.
  • Why now: AI is becoming a commodity tool, enabling non-developers to manage workflows indirectly, and the text emphasizes AIs role in simulating real-world team structures.
  • Expected upside: Automate repetitive tasks (e.g., code review, security checks), freeing you to focus on high-level strategy while maintaining oversight and quality.

• What if you redefined your QA process as “confidence engineering” by leveraging AI for code review and risk assessment?

  • Concrete move: Use AI tools to generate algorithms that evaluate code risk, prioritize reviews, and flag areas needing human oversight.
  • Why now: Traditional QA is insufficient in AI-driven environments, and the text highlights AIs potential to mitigate human-level errors in code.
  • Expected upside: Reduce manual QA bottlenecks, accelerate delivery, and maintain trust in outcomes through AI-augmented confidence checks.

• What if you implemented real-time feedback loops to address bottlenecks at the millisecond level?

  • Concrete move: Build systems with continuous feedback mechanisms (e.g., real-time analytics, automated alerts) to adapt to changing contexts and constraints.
  • Why now: The text stresses that modern delivery requires millisecond-level feedback, and bottlenecks now occur at faster timescales than traditional methods.
  • Expected upside: Optimize processes dynamically, improve system resilience, and align with postmodern engineering principles that prioritize adaptive, socio-technical systems.

Takeaway

• Adopt postmodern engineering principles by integrating AI agents into your workflow, redefining traditional roles (e.g., using AI for code review or security checks), and prioritizing adaptive socio-technical systems over rigid boundaries.
• Leverage personal networks aggressively for job opportunities, as highlighted by the example of reconnecting with former colleagues (e.g., via LinkedIn outreach) to secure interviews and referrals in a competitive market.
• Implement real-time feedback loops to optimize processes, using millisecond-level data instead of slower survey-based methods, especially in AI-driven environments where bottlenecks occur at faster intervals.
• Experiment with AI tools for code generation and review (e.g., using LLM chatbots for initial coding or security checks), while maintaining human oversight to address errors and ensure trustworthy outcomes.
• Focus on outcome-driven delivery over rigid control, aligning with the shift from selling software to selling trust and adaptability, and embracing systems thinking to manage complex, AI-integrated workflows.