Data from production environments can be used to improve software testing by applying data-driven approaches like Process Mining.

By analyzing real process execution data and visualizing actual process paths, teams can focus on the most critical test cases, prioritizing the 80% of process executions that occur through the top 10 most frequent paths.

Richard Seidl Software Testing

This is the other podcast on Software Testing by Richard Seidl, the episodes are in spoken German but the show notes and site are written in English. Our summaries are generated from AI transcript translations.

• https://www.richard-seidl.com/en/blog/tag/podcast-software-testing
• https://www.richard-seidl.com/en/

Episode Details

• Show Notes: https://www.richard-seidl.com/de/blog/testabdeckung-verbessern
• Published: 2026-02-17T05:00:00Z
• Duration: 1298
• Author: Richard Seidl - Experte fur Software-Entwicklung und Testautomatisierung

Overview

The podcast explores a novel method for enhancing software test coverage by comparing it to football analytics. Just as football teams use performance data to optimize training and strategies, the speakers explain how similar data-driven techniques can be applied to prioritize test cases based on real-world usage patterns. They describe a project that utilized actual process execution data from production systems to identify frequently used workflows, enabling testing teams to concentrate on the most critical paths. This approach, known as Process Mining, visualizes real workflows and helps uncover discrepancies between what is tested and what is actually used in production, resulting in more targeted and effective risk-based testing.

Despite the benefits of this method, the podcast highlights its challenges, particularly in maintaining sustainability over time. The reliance on manual processes limits the ability to consistently apply the technique, suggesting that automation is essential for scaling and long-term success. The discussion also emphasizes the broader significance of data analysis and workflow prioritization in testing, showing how real-world data can guide more informed and efficient testing strategies. Overall, the approach encourages a shift toward testing that aligns more closely with practical system behavior, potentially improving both the quality and relevance of test coverage.

What If

• What if you leveraged process mining to prioritize test cases based on real production workflows?
  Move: Analyze your applications production logs to identify the most frequently executed workflows (e.g., user sign-ups, payments). Use this data to build a risk-based test suite focused on these paths.
  Why now: Your team has access to tools like Ceylonis or open-source process mining platforms, and existing production data can be repurposed without additional resource investment.
  Expected upside: Test coverage becomes data-driven, reducing blind spots and aligning test efforts with actual user behavior, potentially cutting regression testing effort by 2030%.

• What if you automated the extraction of workflow insights to sustain your testing strategy?
  Move: Set up a script or tool (e.g., Python + ELK stack) to parse production logs, aggregate workflow frequencies, and generate prioritized test scenarios automatically.
  Why now: Manual analysis is unsustainable for long-term projects, and automation ensures consistency even if key contributors leave.
  Expected upside: Reduced manual overhead by 50%, enabling continuous alignment of test cases with production realities and faster adaptation to changing workflows.

• What if you applied the KVP (Key Value Points) approach to monitor test coverage anomalies in real time?
  Move: Implement periodic deep-dive analyses of workflow execution gaps (e.g., workflows active in production but untested) using a dashboard (e.g., Grafana + process mining data).
  Why now: The text highlights the risk of untested workflows; early detection via KVP can prevent critical failures and improve stakeholder trust.
  Expected upside: Proactive identification of coverage gaps, reducing post-deployment defects by 1525% and enabling data-backed conversations with stakeholders about test priorities.

Takeaway

• Implement process mining to identify high-impact workflows: Use real production data to map frequently used workflows (e.g., top 10 paths covering 80% of usage) and prioritize testing these for risk-based coverage, ensuring critical processes are validated.
• Automate data collection and analysis for sustainability: Set up automated tools to extract, aggregate, and analyze process execution data, reducing manual effort and ensuring the process can be maintained independently of individual team members.
• Quantify the financial impact of testing: Calculate the cost of errors (e.g., Vestas wind turbine fines) and compare it to testing investment, using this metric to justify resource allocation and prioritize high-value test scenarios.
• Audit test coverage against production workflows: Periodically compare executed test cases with production data to identify untested workflows, addressing gaps caused by outdated processes or overlooked paths.
• Adopt KVP (Key Value Points) for continuous monitoring: Establish regular intervals to analyze anomalies in process data, ensuring test coverage aligns with actual system usage and adjusting strategies when discrepancies arise.