UI/UX: Establishing a Regression Testing Firewall

Introduction

Finding a balance between the benefit the tests give you and the cost of spending time and effort writing tests can be challenging. Processes that work for unit tests can be cumbersome when it comes to UI/UX tests (for help writing good tests see Brandon Dickson’s blog post). Throughout the development lifecycle of an application designs can change, often requiring several UI/UX test suites to be thrown away or rewritten. The best balance of effort to benefit is to follow a practice I call a testing firewall.

Testing Firewall

A testing firewall is achieved by designing your front-end to be testable and then writing one happy path test for each major feature or interaction. Do not worry about testing every possible permutation. Just ensure that your UI is testable and straightforward. Then, as bugs come up in QA or through general usage, you create additional tests that recreate that bug. Throughout the development and maintenance of a project, you will accumulate a large test suite that covers your entire UI and UX.

Accessibility

To ensure your UI is testable means following the accessibility standards for your platform. Most UI testing automation libraries leverage accessibility to perform interactions when testing. For instance, on mobile, this means ensuring all your views have proper IDs and info tags. Making your app testable is also making your application accessible; getting two things for the price of one is always a win.

Separation of Concerns

This process works well for a front-end that has a good separation of concerns from its backend; business logic is kept to a minimum and the backend does all the heavy lifting. The front-end should just make network calls, transform domain models into view models, and then display them.

Testing Firewall as a Regression

Creating a test for each bug makes deciding what needs to be tested straightforward. It also ensures that you will only ever encounter any bug or issue once, since your growing test suite will catch it before it is released to production, creating a regression firewall. Thus, you should only ever see a specific bug reported a single time, with future tests preventing it from getting back into production.

Testing Efficiency

Another benefit of creating a test for each bug is that you know what you are testing. Most projects focus on the number of tests or the percentage of code coverage, but usually, half of those tests either do not test anything or are not testing what the engineer thinks they are testing. Writing a test for each bug or feature ensures that tests are actually testing what you think they are. To do this you simply write a test to recreate a known bug then make that test pass.

Conclusion

I find that projects that use a UI testing firewall proceed at a faster velocity, while also enforcing the creation of testable code that, by the end of development, will be accompanied by a robust test suite that is testing what matters.