Testing Frontend Code in 2021

Testing highly interactive web applications have evolved over the last decades. Some years ago, testing frontend code integrated with a browser environment was a big pain point. Nowadays, new utilities and libraries exist, best practices have been established and have evolved.

Introduction

The first StateOfJS study in 2016 concluded that many developers struggled with testing and therefore avoided writing automated tests in general.

Abstractions and frameworks like Selenium and Nightwatch were helpful but also in their early days and even though those frameworks existed, it was still hard for frontend developers to write reliable, performant as well as refactor-friendly automated tests.

The developers still had to write a lot of glue code and had to evolve their own best practices in their codebase. The overall experience for writing tests for frontend applications was not mature, but a lot has changed in the last four years.

Frameworks and automated headless browser environments like Cypress, Playwright, Puppeteer as well as utility frameworks like Testing-Library and MSW have been introduced and are trending in usage. These changes also introduced a new perspective for frontend testing.

Recent advances in end-to-end (e2e) and integration testing for frontend applications have shifted testing strategies from unit towards more integration and end-to-end testing.

I think the main reason for this trend is that modern web applications got more complex and more developers got into testing. Thereby a lot of really great content, abstractions, and frameworks got created.

As the latest StateOfJS study in 2020 shows that more people are using and are satisfied with frameworks that are focused on (integration and e2e testing) like Cypress, Playwright, and helper utilities like Testing-Library.

In this blog post, I want to cover some historical aspects of testing with a focus on frontend applications and discuss the current state as well as some of the best practices of the current industry.

TLDR;

• Testing pyramid switches (trend) towards integration and e2e tests for frontend applications.
• With Cypress (first released 2015), Playwright (first commit Nov 10, 2019), Puppeteer (first released in 2017), Codecept (released in 2016) Testing Library (released in 2018) a lot of great tooling has been introduced in the last few years.
• Integration and end-to-end testing got more reliable and faster to execute because of better abstractions and new best practices.
• Also, integration and e2e tests provide more confidence that your app works as expected.

Testing history

While several years ago many people would have suggested implementing a lot of unit tests, several integration tests, and only some e2e tests as mentioned and referenced often by the testing pyramid principles, the testing strategy for frontend applications has shifted a bit.

In general, this is still really good advice but the landscape has evolved and integration and e2e tests are faster to execute. For example by using parallel and split testing or the continuous updates to Jest and JSDOM abstraction. These types of tests are also easier to write with better DOM queries based on Testing-Library API and user event abstractions for handling user interactions.

When speaking about testing in general we have to mention and talk about the so often referenced testing pyramid paradigm. So let’s take a look at the testing pyramid.

The Testing pyramid

The testing pyramid was first introduced by Mike Cohn in the book „Succeeding with agile“. It describes three main layers of testing with a reference to how much testing you should do on each of these layers.

• Unit test
• Service test (also often described as integration-test)
• User interface test (also often described as end-to-end test)

Many people adopted their own explanation for the different surface areas and adapted some terminology but most people would conclude that tests should follow the following principles

While it is generally a good approach to have tests with different granularity, the main purpose of testing is to have a healthy, fast and maintainable test suite which gives you the maximum confidence that your application works as you have expected.

Martin Fowler later referenced the testing pyramid in his blog post from 2012 and shared his thoughts on why he prefers having more unit tests instead of slow-running UI (e2e) tests. But as a side note, he also mentioned that he prefers unit tests mainly because they are more reliable and fast to execute. I would argue that writing and mocking tests that run in a real browser environment got better in the execution time, the documentation, as well as the developer experience than it was back in 2012. As a comparison to 2021 shows, are Playwright and Puppeteer generally faster in test execution than the industry standard selenium. While Cypress is slower, it provides an awesome developer experience (as mentioned in several blog posts, webinars and podcasts) and can be highly parallelized.

Testing strategies: Return on investment (ROI)

One of the core decision drivers when writing tests is about the return on investment (ROI). There are several factors that we have to account for when evaluating ROI:

• Developer experience (DX)
• Confidence and preventing bugs that may get shipped
• Maintaining the codebase
• What kind of errors are you and your team mostly worried about and want to prevent to happen

As I have mentioned, the tooling to create an integration and e2e test got much better in many different ways. Unit tests are very well suited to test application logic but can lead to false confidence. When we can assume that running an integration and e2e test has gotten and will get significantly better each year, we could argue that writing and maintaining those tests prevent most bugs and provides the best developer experience (as for example writing Cypress tests is a highly interactive experience).

Modern tools like Cypress allow developers to write reliable and cheap to modify tests while contributing to more confidence and safety shipping your application to your users. Those tests give you much more confidence while with the new modern tools you do not lose a lot in developer experience and performance.

In 2016 Guillermo Rauch (CEO of Vercel) tweeted this and started a lot of discussions.

Later the creator of the Testing-Library Kent C. Dodds responded to this quote and referenced this tweet in several discussions about testing frontend applications.

Kent C Dodds a thought leader in testing afterward changed the testing pyramid construct and created the testing trophy. He shifted the testing principle towards writing more integration tests and added a new layer. The static testing layer describes the usage of Linters (ESLint), Types (TypeScript, Flow), and auto code formatting tools (Prettier).

The Testing Trophy and the focus on integration tests

There are several reasons why the testing trophy strategy got more attraction and popularity

• Attempting to refactor a codebase with a lot of unit tests is difficult, especially when implementation details get tested.
• Unit tests in frontend applications where you mock out a component can lose a lot of confidence.

In reality, a well-tested and healthy codebase has all types of tests and also the distribution depends highly on the current status of the project as well as on the things you want to have tested (not everything needs to be tested). In most cases, if you want to test a user-facing frontend application, I would argue that the relation the testing trophy provides is a good one and satisfies business as well as developer needs.

So let me sum up the points and the core reasons why the testing pyramid switches towards integration and e2e testing:

Why testing pyramid has shifted

• Writing integration and e2e test got easier (tooling got better) Cypress, Testing-Library.
• More confidence for e2e and integration tests than unit tests.
• Users will use the frontend and interact with the DOM, not the actual code.

Example with React

What does that actually mean in a codebase and what do people talk about when mentioning the problem of testing implementation details?

I want to show an oversimplified example of what I mean when we test implementation details instead of real code shipped to the user and show why integration tests can lead to more confidence while keeping to the principles mentioned by this quote.

While it is generally a good approach to have tests with different granularity, the main purpose of testing is to have a healthy, fast and maintainable test suite which gives you the maximum confidence that your application works as you have expected.

Let’s say we have the following util function which prefixes a price value with a currency symbol.

We want to use our prefixAmountWithCurrency function to format values in our frontend application. We want to make sure that it works properly. So let’s set up a basic Create-React-APP (CRA) project using that function and write a unit test for it.

You can check out the code in this sandbox or set up the files using CRA following these commands:

We then create a utils file as well as a test file.

In our App.tsx file, we import our utils function and use it to format a price.

); }; export default App;

Let’s write a short unit test for our utils function and run the tests with
npm run test

As we can see, our App renders the price correctly and our function works as expected.

The problem: False confidence

Let’s assume that another engineer steps in and wants to modify the price because he thinks that the value should have a character in between or adds a space accidentally. That is totally normal and, as we are humans, things like that happen. So an engineer takes the simplest route and adds a symbol after the dollar sign.

); }; export default App;

My question is, does the application still work as expected?

Our unit test still works well but is this really what we expect? Maybe but maybe not. At least as an engineer I want to get notified that I potentially broke some code.

I actually tested the implementation detail of the above util function but not the real code shipped to the user.

What makes it worse, my test did not warn me about my potential breaking change. This is a really small example to visualize the main idea but you hopefully get the idea and see how this can affect a larger codebase.

I do not want to neglect the necessity and potentials of unit tests but you have to be aware of such behavior.

Fix the problem: Let’s write an integration test

One possible solution to this problem is to write tests that interact more directly with what the end-user is seeing and give you therefore more confidence that your application works as expected. Let’s add a test that makes sure that the rendered output HTML is the same as the user sees it. You can write such tests with react-testing-library. This utility library gives you a jQuery-like abstraction layer for writing user-centric tests.

As you see, we now use the render function of the @testing-library/react package to evaluate what the end-user will see when the App gets rendered. You now should see that this test is failing since we are missing the space in our test. We can now either adjust our test or our code and do not have to worry about shipping unexpected features to our users.

Conclusion

Unit tests are still important and should not be neglected, but as I mentioned integration and e2e test got easier to write. Because of newer and better tools and abstraction, I would argue that writing more integration tests gives you more confidence while you can faster iterate and test out new ideas. The react testing library provides you with a solid abstraction you can think of jQuery but for testing. With a unified API using react testing library, you do not have to invent the wheel of testing and get implicitly a more accessible UI.

 

Resources and further notes

• Testing-Library
• Kent C Dodds common mistakes with react-testing library
• Martin Fowler: Practical test pyramid