You are here

Test Trend Analysis: Towards Robust, Reliable and Timely UI Tests

Writing good UI Automation is challenging. Slow, unreliable tests are typical problems that people can face. In this talk you will get ideas about how you can instrument your test result information to provide valuable insights, paving the way for more robust, reliable and timely test results.

By capturing this information over time, and when combined with visualization tools, we can answer different questions than with existing solutions (Allure / CI tool build history). Some examples of these are:

  • Which tests are consistently flaky
  • What are the common causes of failure across tests
  • Which tests consistently take a long time to run

Using this information we can move away from the ‘re-run’ culture and better support continuous integration goals of having quick, reliable, deterministic tests

Key takeaways: 

  • Why slow and non-deterministic tests are a problem
  • How visualization of test result information will help you have insights
  • Why capturing test result information over time is important