Part IV — Win Some. Lose Some.

Coverage Velocity

The coverage and the coverage velocity defines how many of your business flows are covered by the tests, and how fast you can achieve this coverage. As your requirements and hence your software functionality is likely to change often, it also means how fast you can adapt your tests to those changes.

Resilience

E2E tests are infamous for being flaky. Changes in the underlying software is one example, but also careless design that does not take into considerations the activity that occur in the system under the hoods, such as the time it takes to get the data from the server, can make the tests extremely brittle. Verifying each screen element prior to handling it and waiting until it is available reduces the chances of tests to break, but takes it costs elsewhere.

Performance

This simply says how long the tests will run. Releasing multiple times a day requires that the tests will run often and relatively fast. But with less frequent releases, extra minutes during the run may not be so critical, or may be compensated with additional hardware.

What Does It mean?

Writing a very small set of E2E tests, verifying basic scenarios such as login and a main flow, will probably run relatively fast (high performance) and will not break often (high resilience) but is only testing small part of functionality (low coverage). This is sometimes referred to as Smoke Tests (or Sanity tests).

export function safeClick(el, options?, timeout?) {
options = options || {};
browser.waitForExist(el, timeout);
if (!options.skipVisible) {
browser.waitForVisible(el, timeout);
}
let tagName = browser.getTagName(el);
if (['button', 'input', 'textarea', 'a'].indexOf(tagName) > -1 ) {
browser.waitForEnabled(el, timeout);
}
browser.click(el);
}
  • Finding that the element exists
  • Finding the element tag name
  • Checking the element enable state
  • Clicking on the element.

What is Right For me?

Needless to say, there is no one correct answer for that. It is common to focus on performance of the tests and ignore other aspects, but eventually the correct answer is the one that serves your business goals the best.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Tally Barak

Tally Barak

151 Followers

If you can’t explain it simply, you don’t understand it well enough.” — Einstein