AI in software development faces challenges like increased errors, higher costs, and unreliable reviews, especially in critical systems where flaws can be catastrophic. The discussion calls for balanced, human-supervised approaches to prioritize quality, sustainability, and developer well-being over excessive automation.

Richard Seidl Software Testing

This is the other podcast on Software Testing by Richard Seidl, the episodes are in spoken German but the show notes and site are written in English. Our summaries are generated from AI transcript translations.

• https://www.richard-seidl.com/en/blog/tag/podcast-software-testing
• https://www.richard-seidl.com/en/

Episode Details

• Show Notes: https://www.richard-seidl.com/de/podcast/qualitaet-ki-generierter-code
• Published: 2026-09-15T04:00:00Z
• Duration: 00:35:12
• Author: Richard Seidl - Experte fur Software-Entwicklung und Testautomatisierung

Overview

The podcast discusses the growing challenges and limitations of using AI in software development, particularly focusing on quality assurance, cost, and long-term sustainability. While AI enables faster code generation, it often leads to increased errors, redundant code, and higher expenses - especially when using agent-based systems or advanced models that consume large amounts of computational resources. A major bottleneck is the mismatch between rapid AI-driven development and slow, resource-intensive review and testing processes. Traditional static analysis remains more cost-effective than AI reviews, which are found to provide useful feedback in only about 30% of cases, with the rest being irrelevant or misleading.

Serious concerns are raised about the reliability and safety of AI-generated code, especially in critical systems like automotive software, where undetected flaws could lead to catastrophic failures. The lack of comprehensive studies on long-term impacts, combined with anecdotal evidence of declining code quality, highlights the risks of unsupervised AI use. Human oversight remains essential, as both developers and AI struggle with complex, large-scale systems. Additionally, the strain on open-source communities, dependency management, flickering tests, and security analysis overload underscore broader systemic issues. The discussion advocates for a more deliberate, balanced approach - prioritizing meaningful software improvements, developer well-being, and sustainable practices over unchecked speed and automation.

What If

• What if you built a lightweight AI-assisted code review filter to reduce noise and cost?

  • Move: Implement a two-step review process: first run static analysis (e.g., SonarQube, ESLint), then use a smaller, cost-efficient AI model (e.g., Llama 3 70B via API) to interpret only the high-priority findings and suggest fixes - ignoring low-severity or duplicate issues. Automate this into your CI pipeline for pull requests.
  • Why Now?: AI review costs are skyrocketing, and most AI-generated feedback is irrelevant. With teams already overwhelmed by tool noise and rising token expenses, filtering before AI involvement cuts cost and improves signal-to-noise ratio immediately.
  • Expected Upside: Reduce AI review spending by 60 - 80% while increasing actionable output; avoid $10k+/year in wasted compute and developer time sifting through false positives.

• What if you used AI to auto-generate only regression test guards for critical modules instead of full test suites?

  • Move: Identify core business logic (e.g., billing calculation, export logic) in your app. Use AI to generate narrowly scoped, deterministic unit and integration tests only for these functions after changes - enforced via git hooks or PR checks. Pair with test selection to skip unrelated suites.
  • Why Now?: AI-generated tests often flicker and bloat maintenance. But skipping tests entirely risks failures in production, especially as code complexity grows. Focusing AI on regression protection for mission-critical paths balances speed and safety without test explosion.
  • Expected Upside: Prevent critical regressions (e.g., incorrect invoicing) with minimal test overhead; achieve 90%+ confidence in core logic while avoiding the trap of maintaining thousands of flaky AI-generated tests.

• What if you paused feature generation and used saved AI dev time to refactor legacy dependencies instead?

  • Move: For one month, disable AI-driven feature work. Redirect all AI coding capacity toward identifying and upgrading vulnerable or deprecated open-source dependencies. Use AI to analyze package.json/pom.xml, map exploit risks (via OSS Index or Snyk), and draft safe migration patches - then manually validate.
  • Why Now?: The open-source ecosystem is burning out, and dependency vulnerabilities are ticking time bombs. AI lets solo devs move fast - but technical debt and security risks compound silently. Now is the time to invest “speed dividends” into foundational stability before a breach or breakage occurs.
  • Expected Upside: Cut future incident response time by 70%, reduce emergency patching, and increase system resilience - all while contributing responsibly to the sustainability of tools you rely on.

Takeaway

• Implement static analysis early in your development workflow to catch issues quickly and cheaply, especially since AI-based reviews can be 10 - 100x more expensive and often less effective.
• Limit AI-generated code in safety-critical or production systems by enforcing manual review and regression testing, ensuring you retain control over functionality and avoid liability risks.
• Use test selection strategies - run only relevant tests based on code changes - to speed up feedback loops and avoid wasting time on unrelated test suites when iterating with AI assistance.
• Avoid blind reliance on AI for code generation; manually rewrite or refactor critical components when needed to prevent degradation in code quality and maintain long-term system integrity.
• Evaluate AI tools pragmatically by measuring their actual output quality (e.g., % useful feedback, false positives) and cost per task, favoring traditional scripts or tools where they are more efficient (e.g., duplicate detection, security scanning).