AI is transforming software development by generating and refining code, though challenges like review fatigue and creative limitations persist. Testing shifts toward real-world data, automated audits, and modular architectures, with AI agents acting as collaborative team members needing oversight and training.

Richard Seidl Software Testing

This is the other podcast on Software Testing by Richard Seidl, the episodes are in spoken German but the show notes and site are written in English. Our summaries are generated from AI transcript translations.

• https://www.richard-seidl.com/en/blog/tag/podcast-software-testing
• https://www.richard-seidl.com/en/

Episode Details

• Show Notes: https://www.richard-seidl.com/de/podcast/feedbackschleifen-coding-agenten
• Published: 2026-09-22T04:00:00Z
• Duration: 00:41:10
• Author: Richard Seidl - Experte fur Software-Entwicklung und Testautomatisierung

Overview

The podcast discusses the evolving role of AI in software development, focusing on how AI-generated code impacts software quality, testing strategies, and developer workflows. It distinguishes between supervised (human-in-the-loop) and unsupervised (autonomous) AI code generation, emphasizing the importance of feedback loops, rules, and hooks to guide AI behavior and ensure adherence to best practices. The conversation explores how agents can mimic iterative human development processes - writing initial drafts, refining them, and applying architectural patterns - while also highlighting challenges such as review fatigue and the limitations of AI in creative problem-solving or cross-project knowledge transfer.

Testing in AI-assisted development is approached through agent-written tests across multiple levels - unit, integration, system, and contract tests - with a shift away from strict test-driven development toward using real-world usage data and customer feedback to refine requirements. Non-functional aspects like security, performance, accessibility, and licensing are addressed through automated pipelines and AI-driven audits, balancing supervised oversight for critical areas with unsupervised optimization where feasible. The discussion also covers architectural modularity, continuous improvement via frequent retrospectives, and the use of deterministic and non-deterministic feedback loops to enhance both code quality and system resilience. A key theme is treating AI agents as collaborative team members that require training, evaluation, and constraints to effectively contribute to complex software projects.

What If

• What if you implemented a non-deterministic testing layer to catch edge cases your AI agent misses?

  • Move: Introduce randomized input generators and chaos testing into your agent’s test suite (e.g., fuzzing API contracts or UI flows) to surface flaky behavior or hidden bugs.
  • Why Now?: As AI agents generate more code autonomously, deterministic tests alone may miss emergent failures - non-deterministic methods expose weaknesses in both code and test quality.
  • Expected Upside: Discover 15 - 30% more integration bugs pre-deployment, reduce production incidents, and improve agent training by feeding failure cases back into its feedback loop.

• What if you treated your AI agent as a junior developer requiring structured feedback loops to enforce architectural rules?

  • Move: Set up Git-based hooks that automatically flag and reject AI-generated commits violating modularity (e.g., CSS over 4k lines, missing unit tests, broken contracts) with actionable feedback.
  • Why Now?: Without proactive constraints, AI agents drift into technical debt; real-time enforcement mimics supervised learning and prevents compounding rework.
  • Expected Upside: Maintain clean architecture across solo projects, reduce refactoring time by 40%, and train your agent to self-correct within days instead of weeks.

• What if you combined two AI models - one for creation, one for critique - to automate code review and boost output quality?

  • Move: Use Claude to generate features and Codex (or another rule-bound model) to review the output, run checks against best practices, and suggest fixes before commit.
  • Why Now?: Relying on one model creates blind spots; leveraging complementary strengths (creativity vs. rigor) mimics peer review at zero marginal cost.
  • Expected Upside: Cut manual review fatigue by 60%, increase test coverage via AI-written tests, and ship higher-quality code faster while preserving your oversight bandwidth.

Takeaway

• Implement deterministic hooks (e.g., pre-commit or file-change triggers) to automatically enforce architectural rules and coding standards in AI-generated code.
• Set up a feedback loop using rule-based skills to iteratively correct recurring AI mistakes, such as enforcing test patterns or accessibility checks before code is committed.
• Use a dual-agent approach - assign one AI model (e.g., Claude) to write code and another (e.g., Codex) to review it - automating initial quality validation and reducing manual review load.
• Schedule automated, frequent retrospectives (e.g., every few hours) using agent-powered analysis of recent changes to detect process inefficiencies or recurring bugs early.
• For high-stakes components (e.g., security logic or core architecture), disable autonomous AI generation and manually implement or review code, treating AI output as a draft rather than final.