AI automation in testing and agentic engineering could redefine QA roles, emphasizing structured planning, domain expertise, and human oversight over outdated tools. The shift demands skills like curiosity and technical fluency to guide AI agents, prioritizing product impact and recursive automation over rigid script-based workflows.

Test Guild

Test Guild - hosted by Joe Colantonio has main topic focus on Testing or Automating. Each episode has a different guest. Show notes have comprehensive links and usually a full transcript. Released as audio and video.

• https://testguild.com/
• https://testguild.com/podcasts/automation/
• https://www.youtube.com/playlist?list=PL9AgRtJkydU1jqvx46esyr56BXtm1QEds
• https://www.youtube.com/@JoeColantonio

Episode Details

• Show Notes: https://testtalks.libsyn.com/agentic-engineering-for-testers-how-to-automate-your-way-to-the-top-with-amit-rawat
• Published: 2026-07-07T16:35:00Z
• Duration: 43:30
• Author: Unknown

Overview

The podcast explores advancements in AI-driven automation, particularly in test case development and agent engineering. It speculates that AI could replace humans in writing manual test cases within a year, while tools like PromptWritedeveloped by Demitra Watt to translate natural language prompts into Playwright scriptsare highlighted. PromptWrite, though popular with over 100 stars, requires updates to keep up with evolving AI models. The discussion emphasizes the importance of structured planning in AI projects, using brainstorming sessions and tools like STML (Structured Markdown Language) for visualizing workflows. Key challenges include balancing AIs probabilistic outputs with human oversight to avoid generic results and ensuring detailed planning for feature completeness. The transition to “agentic engineering” is framed as critical for future knowledge workers, requiring skills like curiosity, technical proficiency, and domain expertise to guide AI agents effectively rather than relying on predefined prompts.

The role of QA professionals in the agentic era centers on managing AI agents and optimizing workflows, leveraging their deep understanding of product systems over automation frameworks. The podcast critiques traditional testing approaches focused on technical tools, advocating instead for domain knowledge and impact-driven testing. Cost considerations for AI tokens, such as a $200 monthly investment, are analyzed for their value in automation projects. Technical implementations include using GitHub Copilot SDK for agent intelligence, headless browser modes, and logging systems for troubleshooting automation. The discussion also covers Gherkin scenario generation, which enhances test accuracy by incorporating real browser data, and record-playback tools for deriving test cases from manual workflows.

Agentic engineering is contrasted with “white coding,” emphasizing the need for technical understanding over outcome-based AI interactions. Examples like loop engineeringwhere agents scan GitHub repositories for issues and execute tasksillustrate continuous AI-human collaboration. Prompt engineering techniques, including verbosity and structured task breakdowns, are emphasized to avoid AI overstepping requirements. The podcast highlights the shift in productivity toward recursive automation loops, such as managing AI workflows via Telegram or remote servers, and the importance of privacy in localized data management. Future success in testing and automation is tied to curiosity, domain expertise, and the ability to influence product quality, rather than clinging to outdated methodologies or tools.

What If

• What if you leveraged PromptWrite to automate test case generation for legacy systems?

  • Move: Upgrade PromptWrite to integrate with newer AI models like Febbel or Clotsone, enhancing its ability to convert natural language prompts into accurate Playwright scripts.
  • Why Now?: Manual test case creation is costly and time-consuming, while AI-driven tools like PromptWrite can rapidly generate scripts with minimal QA intervention, especially for legacy systems with outdated documentation.
  • Expected Upside: Reduced manual QA overhead, faster test coverage for legacy apps, and opportunities to scale to new clients targeting automated regression testing.

• What if you implemented a loop engineering agent to monitor GitHub repositories for issue resolution?

  • Move: Build an agent using OpenClaw or Hermes to scan public GitHub repos every 10 minutes, flagging new issues (e.g., bug fixes, feature requests) and auto-generating code snippets or tickets for review.
  • Why Now?: GitHubs API is stable, and open-loop agents can handle low-priority tasks, freeing you to focus on strategic work while maintaining engagement with the community.
  • Expected Upside: Streamlined issue tracking, faster client onboarding, and a portfolio of demo projects showcasing your ability to solve real-world automation challenges.

• What if you redefined your QA workflow using agentic principles to optimize test orchestration?

  • Move: Create a “Chief QA Agent” using LangChain, trained on your domain-specific test scenarios and STML plans, to prioritize test cases, trigger execution, and log outcomes in structured markdown.
  • Why Now?: QA professionals are uniquely positioned to guide AI agents by combining domain knowledge with technical skills, ensuring tests align with product goals rather than just automation scripts.
  • Expected Upside: Higher test accuracy, reduced false positives/negatives, and a scalable system where your QA expertise becomes a bottleneck for competitors.

Takeaway

• Adopt PromptWrite for Test Automation: Convert natural language prompts into Playwright scripts using PromptWrite, but regularly update its integration with current AI models (e.g., Febbel, Clotsone) to maintain relevance and efficiency in automation.
• Implement Structured Planning with STML: Use Structured Markdown Language (STML) to detail project workflows as interactive web pages, ensuring clarity, traceability, and high feature completeness (e.g., 9099%) before executing AI-driven tasks.
• Leverage QA Expertise in Agent Management: QA professionals should focus on managing and optimizing AI agents, using domain knowledge to refine test strategies, prioritize critical scenarios, and ensure alignment with product quality goals over automation tools.
• Benchmark AI Token Costs Against ROI: Evaluate whether $200/month for AI tokens (or cheaper alternatives like Moonshot API) is justified by comparing token usage, task complexity, and measurable productivity gains from agentic workflows.
• Iterate with Logging and Playback Tools: Use automation logging (e.g., “key steps” mode), recorded video playback, and resource tracking (token consumption, API costs) to troubleshoot failures, refine agent behavior, and iteratively refine workflows in PromptWrite or similar tools.