The text discusses modern QA strategies leveraging AI and tools like Playwright, emphasizing automation, risk management, and AI agent compatibility for efficient, reliable testing. It highlights the evolving QA role in balancing AI-driven automation with human oversight, foundational skills, and collaboration across teams to ensure accuracy and security in increasingly automated workflows.

How To Test This?

interview episodes where Mamadou N’diaye talks with with software testing experts

• https://podcasters.spotify.com/pod/show/spidey1944
• https://www.youtube.com/@HowToTestThis
• https://www.linkedin.com/in/mamadou-ndiaye-consultant/

Episode Details

• Show Notes: https://podcasters.spotify.com/pod/show/spidey1944/episodes/How-to-Test-With-Playwright-CLI--AI---Lucas-Smit-e3i195t
• Published: 2026-04-17T00:40:00Z
• Duration: 00:43:13
• Author: Mamadou N’diaye

Overview

The podcast explores modern testing methodologies, emphasizing risk management, AI integration, and automation tools like Playwright. It highlights the migration from traditional frameworks (e.g., WebDriver.io) to Playwright, praised for its robust end-to-end testing capabilities, Microsoft-backed development, and AI agent compatibility (e.g., Planner, Generator, Healer). Playwright CLI is discussed as a resource-efficient alternative for test generation, network interception, and exploratory testing, with a focus on reducing flakiness through role-based selectors and customizable “skill” files to guide AI agents. Best practices include prioritizing verification of AI-generated test outputs, avoiding monolithic AI tooling, and fostering collaboration across QA, engineering, and security teams.

The discussion also addresses AIs growing role in QA, including AI red teaming for security testing, adversarial testing with tools like PromptFool, and the need for QA professionals to adapt by mastering both traditional testing principles and AI-driven workflows. Challenges include ensuring AI outputs are accurate, verifying non-deterministic behavior in AI systems, and addressing security risks in AI integration. The podcast underscores the evolving QA roleshifting from manual tasks to ensuring AI-generated software reliability, requiring dual expertise in testing AI systems and leveraging AI for automation. It stresses continuous learning, practical application of AI tools, and the importance of verification and human oversight in AI-assisted workflows.

Key takeaways include the necessity of context engineering over prompt engineering for AI agents, modular AI architectures to avoid complexity, and the critical role of QA in balancing AIs efficiency with accuracy and security. The content also highlights industry trends, urging testers to embrace AI as a tool to enhance, not replace, human expertise, while staying proactive in adopting emerging tools like Playwright and AI-driven red teaming frameworks. The podcast emphasizes that QA professionals must prioritize foundational testing skills, adapt to AIs transformative impact, and focus on verifying outputs to maintain quality in an increasingly automated landscape.

What If

• What if you integrated Playwright CLI with an AI agent to generate automated test cases for a new feature?

  • Move: Use Playwright CLIs trace viewer and skill files to train an AI agent (e.g., Cloud Code) to auto-generate test cases for your latest feature.
  • Why now: Playwright CLIs lightweight selectors and trace analysis reduce flakiness, while AI agents like Cloud Code have matured to handle this task efficiently.
  • Expected upside: Cut test writing time by 50% and ensure alignment with internal QA standards via the skill file.

• What if you ran an AI red teaming exercise to evaluate your QA processes for AI-specific risks?

  • Move: Use PromptFool or similar tools to simulate adversarial attacks on your AI-powered testing workflows (e.g., testing if AI-generated tests miss edge cases).
  • Why now: As AI systems become central to testing (e.g., self-healing agents), vulnerabilities like hallucinations or misaligned logic must be addressed proactively.
  • Expected upside: Identify critical security gaps in AI-driven test outputs and refine verification protocols to prevent undetected bugs.

• What if you migrated a legacy WebDriver.io test suite to Playwright with AI-assisted tools?

  • Move: Leverage Playwrights CLI and healer agent to automate migration, starting with a small module (e.g., login flow) to validate stability.
  • Why now: Playwrights AI integration (Planner, Generator) and robust community support make migration faster than traditional methods, with tools now reducing manual effort to weeks.
  • Expected upside: Achieve a 30% reduction in flaky tests and full adoption of Playwrights modern features like trace viewer for faster debugging.

Takeaway

• Adopt Playwright CLI with a custom skill file: Integrate Playwright CLI into your test automation workflow, and create a skill file to enforce best practices (e.g., prioritizing get by role over brittle selectors) to reduce flakiness and ensure AI agents follow your teams standards.
• Migrate test frameworks to Playwright using AI tools: Use AI-assisted migration tools to transition from WebDriver or Cypress to Playwright, cutting down manual effort from months to weeks while leveraging Playwrights stability and AI agent integrations.
• Manually verify AI-generated test cases: Always review AI-generated test assertions for accuracy (e.g., confirm pop-up content validation and error message checks) to avoid incorrect logic, even if the test runs successfully.
• Implement AI red-teaming for security testing: Use tools like PromptFool to adversarially test your AI systems, identifying vulnerabilities (e.g., unintended outputs or breaches of safety boundaries) as part of your QA process.
• Master manual testing fundamentals before AI automation: Build a strong foundation in manual testing, including critical thinking and edge case identification, before leveraging AI for automation to avoid gaps in testing coverage and quality.