The episode explores AI’s role in software testing through Munich’s public administration, highlighting tools like Sherlock for ISO-compliant test case generation and Muck-GPT’s integration across departments. Challenges include tool limitations, training needs, and the need for centralized data, with future plans for user customization and expanded AI capabilities.

Richard Seidl Software Testing

This is the other podcast on Software Testing by Richard Seidl, the episodes are in spoken German but the show notes and site are written in English. Our summaries are generated from AI transcript translations.

• https://www.richard-seidl.com/en/blog/tag/podcast-software-testing
• https://www.richard-seidl.com/en/

Episode Details

• Show Notes: https://www.richard-seidl.com/de/blog/ki-gestuetzte-testfallerstellung
• Published: 2026-01-27T05:00:00Z
• Duration: 1210
• Author: Richard Seidl - Experte fur Software-Entwicklung und Testautomatisierung

Overview

The podcast explores the implementation of AI in software testing within Munich’s public administration, where a custom GPT model known as Muck-GPT has been used since 2023 to support various AI-assisted tasks. A specific focus is placed on an AI tool named Sherlock, which is designed to aid non-IT departments in creating test cases that comply with the ISO 29119 standard, requiring little to no prior testing expertise. The tool provides guided prompts to simplify the process, and has been well-received during training sessions, although it currently has limitations, such as only exporting up to 10 test cases at a time to external tools like TestLink and X-Ray.

The discussion highlights that while the AI can assist in generating test cases, it still depends on clear input and requires human validation to ensure accuracy. There is also mention of a future tool called Watson, which aims to automate the generation of test reports, reducing manual efforts further. The potential for improving AI performance through the use of a centralized knowledge base is noted, as it could enhance the accuracy and customization of AI assistants. Additionally, ideas for integrating a chatbot to help with test case reviews are being explored. Overall, AI is viewed as a supportive tool rather than a replacement for human decision-making in testing, with continued efforts to improve its usability and compatibility with existing test management systems.

What If

• What if you prototype an AI test case generator that auto-generates ISO 29119-compliant test cases for your solo projects?

  • Concrete move: Use a GPT-based model like Sherlock or Watson to input business requirements, then refine the output using prompt engineering and manual validation to ensure compliance.
  • Why now: Businesses increasingly require standardized testing (e.g., ISO 29119), and solo developers can bypass the learning curve of training teams by leveraging AI tools directly.
  • Expected upside: Reduce testing time by 60% for non-IT teams by automating repetitive case creation, freeing up time for strategic work.

• What if you reverse-engineer the integration hurdles of AI tools with TestLink/X-Ray and build a lightweight export script?

  • Concrete move: Develop a Python or Node.js script that takes AI-generated test cases (via Sherlock) and batches them into XML/CSV files for TestLink or X-Ray, bypassing the 10-case token limit.
  • Why now: Existing tools like TestLink are widely used, but their integration with AI is limited. Solving this gap could position you as a go-to resource for SMEs.
  • Expected upside: Enable seamless tool adoption for solo developers, potentially monetizing the script via a SaaS model or GitHub repository.

• What if you prototype a chatbot that simulates a test review process involving a test architect, developer, and tester?

  • Concrete move: Use a GPT model to create a chatbot that accepts test cases, asks clarifying questions, and suggests outdated cases based on input from a simulated review.
  • Why now: Users actively requested this feature to automate error documentation in tools like Jira, and solo developers can test this concept with minimal infrastructure.
  • Expected upside: Reduce manual error logging by 50% in projects with high test case turnover, while showcasing your ability to build AI-driven QA workflows.

Takeaway

• Leverage AI tools like Sherlock to generate ISO-compliant test cases for non-IT departments, reducing the complexity of manual test case creation and ensuring alignment with industry standards.
• Implement structured prompt engineering when using AI (e.g., I help you create test cases that comply with standards) to guide the assistant effectively and avoid vague or inaccurate outputs.
• Address integration limitations by prioritizing compatibility improvements with test management tools like X-Ray and SAP Sub Solution Manager, especially when exporting test cases in bulk (currently limited to 10 cases).
• Create a centralized knowledge base (e.g., internal data repository) for project specifications, user stories, and system details to ensure AI tools like Sherlock/Watson can access accurate, relevant information instead of generating fabricated content.
• Focus on prompt-based training for business users, emphasizing how to interact with AI as a test analyst (not a technical expert), reducing dependency on formal testing certifications or lengthy training programs.