Getting a Seat at the (AI) Table
Categories: Podcasts , The Testing Peers
AI is transforming testing and quality engineering, requiring adaptation to non-deterministic outputs and AI-generated code while preserving human oversight and accountability. The future emphasizes strategic human guidance, ethical considerations, and maintaining core quality practices in AI-augmented development.
The Testing Peers
The Testing Peers - panel discussions about testing. Usually Chris Armstrong, Simon Prior, Russell Craxford and David Maynard, with occasional special guests. Show notes on the website have an episode description and resource links.
Episode Details
- Show Notes: N/A
- Published: 2026-09-28T11:00:00Z
- Duration: 00:43:49
- Author: Testing Peers
Overview
The podcast discusses the evolving role of testers and quality engineers in the context of rapidly advancing AI technologies. With the rise of AI-driven development, particularly since the emergence of large language models like ChatGPT, traditional testing practices must adapt. Key challenges include the non-deterministic nature of AI outputs, the complexity introduced by AI-generated code, and the risks of over-reliance on automation. The discussion emphasizes that while AI can accelerate development and assist with tasks like code reviews, it does not replace the need for critical thinking, risk assessment, or human oversight.
Participants highlight the importance of maintaining foundational software quality practices - such as small, modular pull requests, controlled experiments, and shift-left testing - even in AI-augmented environments. There is a strong focus on accountability, with concerns about who bears responsibility for AI-generated errors or hallucinations. The conversation also explores broader implications, including AI bias due to skewed training data, generational differences in trust toward AI, and the need for independent research. Ultimately, the future of quality engineering is framed as one where humans act as strategic advisors, guiding AI tools while upholding standards of quality, ethics, and system understanding.
What If
-
What if you treated AI-generated code like a junior developer’s pull request?
- Move: Implement a mandatory human-in-the-loop review process for all AI-generated code, enforcing small, modular PRs (under 50 lines), automated linting, and targeted test coverage. Use AI to draft code but require manual validation of logic, security, and alignment with architecture.
- Why Now?: AI coding tools are being adopted rapidly without guardrails - leading to bloated, risky, or redundant codebases. The window to establish healthy norms before technical debt accumulates is closing fast.
- Expected Upside: Reduce long-term maintenance costs by 30 - 50%, improve system reliability, and position yourself as a quality gatekeeper in a market where speed often overrides safety.
-
What if you built your own AI-augmented testing toolkit as a solo operator?
- Move: Develop a lightweight, open-source CLI tool that uses LLMs to auto-generate test scenarios from user stories, then flags high-risk areas (e.g., auth, payments) for manual deep dives. Integrate with GitHub Actions to analyze PR diffs and suggest exploratory testing paths.
- Why Now?: Off-the-shelf AI testing tools are overcomplicated or enterprise-focused. As a solo dev, you can move faster, solve real pain points (like flaky tests or missed edge cases), and monetize via sponsorships or pro features.
- Expected Upside: Establish authority in the AI+QA niche, attract early adopters, and create a productizable asset that scales beyond hourly work - turning insight into IP.
-
What if you ran weekly “AI Refactor Sprints” on legacy code as a value accelerator?
- Move: Pick one critical but tangled module per week. Use AI to visualize dependencies, propose refactors, and generate unit tests - then manually validate outputs, measure performance pre/post, and document trade-offs. Share results as case studies.
- Why Now?: Businesses are desperate to modernize aging systems but fear AI-driven rewrites. By demonstrating controlled, auditable improvements, you offer a safe path forward when trust in AI is low.
- Expected Upside: Differentiate your services with measurable outcomes (e.g., “Reduced function complexity by 60%”), build client trust, and create reusable patterns that reduce future project ramp-up time.
Takeaway
- Implement shift-left testing by actively pairing with developers and data engineers to catch issues early and reduce assumptions in AI-integrated workflows.
- Break down code changes into small, modular pull requests (under 300 lines) to maintain reviewability, even when using AI-generated code.
- Establish a controlled experimentation framework where only one variable is changed at a time to accurately assess the impact of AI tools in development and testing.
- Regularly validate AI-generated outputs (code, tests, or reviews) through manual verification to prevent propagation of hallucinations or low-quality artifacts.
- Focus on risk-based decision-making by acting as a quality advocate who questions AI-driven suggestions and evaluates business and technical risks before implementation.
For a PDF of longer Software Testing Podcast Episode Summaries with Briefing Notes and more detailed summary notes, visit EvilTester Patreon Podcast Summaries.