The software testing profession faces decline due to over-automation, commodification, and Agile-driven quality compromises, while AI risks displacing testers and overwhelming human oversight. To remain vital, testing must prioritize critical thinking, ethical accountability, and human judgment over blind reliance on tools.

Quality Remarks

Quality Remarks is the podcast from Keith Klain. Each episode has a different guest. Recorded as audio and video.

• https://qualityremarks.com/
• https://qualityremarks.com/qr-podcast/
• https://www.youtube.com/@KeithKlain

Episode Details

• Show Notes: https://www.spreaker.com/episode/qr-podcast-michael-bolton--73094441
• Published: 2026-07-21T22:08:57Z
• Duration: 02:05:43
• Author: Keith Klain

Overview

The podcast discusses the declining state of the software testing profession, marked by a post-pandemic deterioration despite brief improvements in the late 2000s. A key concern is the industry’s overreliance on rigid test cases and automation, often at the expense of exploration, critical thinking, and meaningful problem detection. Agile practices are criticized for accelerating delivery while compromising quality, and conferences are seen as having lost their educational value. The commodification of testing, along with the devaluation of manual testing in favor of automated checks, has contributed to a loss of depth and accountability in the field.

Emerging trends like AI are viewed with skepticism, as they threaten to displace testers who focus solely on automation, while generating vast amounts of output that exceed human review capacity. The concept of “human in the loop” is challenged as a fallacy when oversight is practically impossible at scale. Ethical concerns arise from how AI is trained and deployed, particularly regarding exploitation and lack of accountability. Despite these challenges, the role of testing remains vital - rooted in systematic doubt, risk identification, and advocating for what truly matters. The profession must evolve toward deeper critical thinking, adapt to technological changes, and resist blind confidence in tools, ensuring that human judgment remains central to software quality.

What If

• What if you stopped writing automated checks and focused on finding high-impact risks instead?

  • Move: Pause all new test automation work for 30 days. Spend that time conducting exploratory testing on your product’s most used or most fragile features, documenting risks in a lightweight format (e.g., mind maps or bullet notes).
  • Why Now?: AI is accelerating the production of automated checks, making manual script writing an increasingly low-value task; meanwhile, real bugs hide in complexity and edge cases that only human exploration can uncover.
  • Expected Upside: You’ll shift from commodity labor to high-leverage problem-finding, differentiating your work and increasing your value as a solo developer who prevents actual user harm.

• What if you made customer support your primary testing sensor?

  • Move: Set up a simple dashboard (e.g., Airtable or Notion) that logs recurring support issues, categorizing them by feature, frequency, and emotional impact (e.g., “angry email,” “repeat caller”). Use this to guide your next testing session.
  • Why Now?: Support channels are underutilized sources of real-world failure data; while AI generates synthetic test cases, real pain is already being reported by users - and ignored.
  • Expected Upside: You’ll align your testing with actual user pain, increasing business impact and justifying your role with evidence that stakeholders can’t dismiss.

• What if you treated AI-generated code as a liability until proven otherwise?

  • Move: For any AI-generated code (yours or third-party), add a mandatory “trust audit” step: test it under stress, edge inputs, and integration points before deployment. Document assumptions and failure modes.
  • Why Now?: AI tools produce outputs at scale, but without human scrutiny, they introduce undetected risks - mirroring the financial crises where complexity masked failure. The cost of ignoring this is rising.
  • Expected Upside: You position yourself as a risk mitigator, not just a builder, creating a defensible niche as systems grow more complex and failures more costly.

Takeaway

• Focus on identifying and exposing real risks instead of generating test artifacts or seeking false confidence - shift your value from confirmation to critical problem-finding.
• Invest time in deeply understanding AI systems you’re testing by studying foundational material (e.g., Wolfram’s ChatGPT essay, Brooks’ IEEE interview) to move beyond superficial automation.
• Prioritize hands-on experimentation and peer feedback to refine your testing skills, rather than relying on certifications or automated checklists that don’t reflect actual testing competence.
• Position yourself as a critical voice in development by asking who’s been overlooked and how features might fail stakeholders - emulate a skilled critic, not a compliance checker.
• Resist over-reliance on AI-generated tests or outputs by maintaining skeptical oversight; assume human review bandwidth is limited and focus on high-risk areas where automation alone fails.