Good old Selenium

Is tooling finally moving forward? Can or should we say goodbye to Selenium? If so, what are the successors?

Working in IT as a Test Automation Professional made us all come across Selenium. A wide spread tool which has some cool features and a number of frameworks built on it with even more features and flexibility. When properly set up they definitely provide us with reliable tests.

However, in an ever changing world we aren’t just utilizing all these cool features. Unfortunately, we are also facing the situation that we lack some features. Just think about the hassle that is called configuration..

To name a few:

  • Getting/installing drivers for browser support.
  • Creating capabilities and profiles.
  • Matching of driver and browser versions to avoid compatibility issues.
  • Grid configuration and separate launchers for running the tests in parallel.
  • Clear demarcation of parallel runnable tests so that they don’t run into each other’s states.

The new kids on the block

Now, let’s shift our focus towards the horizon, here we see two new players arriving at the scene.

Their names?
Cypress and TestCafe.

Will either one of them make our lives better? Should we forget about Selenium and embrace these new kids?

Looking at the both of them there are a few similarities between them.

  • Both are open source end-to-end testing frameworks written in JavaScript.
  • Both run partially in the browser and partially in Node.JS.
  • Both transparently retry assertions which eliminates a lot of the flakiness associated with Selenium based tests.

Oh yeah, and both of them are under heavy development with responsive developers and an active community. Are you going to be an early adopter and help shape the future of tooling 😉?

Two new challengers arrive

For me personally it’s not about the similarities though. I’m looking for that next generation tool that stands out. A tool that actually makes writing tests a fun activity! (Just think about the entry of a tool like BrowserStack that nowadays is widely accepted by the community). So which one to support? Or should we stick with Selenium for now?

In the red corner: TestCafe

TestCafe definitely has some valuable features I’m very much interested in. A top 5 you say?

  1. Browser handling: It can run tests simultaniously in any browser, and no, it doesn’t need any complex configuration (including mobile devices and cloud services).
  2. Headless testing: It can run tests in headless mode (Chrome, Firefox) without having to render DOM.
  3. Proxy server: TestCafe uses a proxy server. It injects scripts into the page which can inspect and control elements on the page. This amazing tool also allows interaction with native alerts, file upload inputs and iframes.
  4. DB fixtures: The ability to call out to parts of your Node server application directly from the test (setting up and clearing database fixtures, starting and stopping the test server).
  5. Waiting mechanism:Automatic Waiting Mechanisms and Smart Assertion Query Mechanism (faster and more stable tests).

Well, I guess you would want to read more about this right?

Faster tests, more stability, less hassle? It certainly puts a smile on my face. But what about Cypress then. Surely it can’t get any better than this, or can it?

In the blue corner: Cypress

Well, Cypress has some cool features as well that makes it stand out from the crowd. Want another top 5?

  1. DOM: It runs the actual testcode in the browser giving access to real DOM elements.
  2. Configuring: It doesn’t require installing or configuring servers, drivers or any other dependencies.
  3. Automatic waiting: Cypress waits for commands and assertions before moving on (no more async hell).
  4. Real time reloads: Making changes to your test triggers automatic reload of Cypress.
  5. JQuery selectors: Cypress uses JQuery selectors which have some usefull extra capabilities (for instance :parent and :first).

Need additional info again?

And the verdict is…

TestCafe

Cypress

Selenium

Configuration +++ +++ +
Browser support +++ + +++
Maintainability ++ ++ ++
Reliability ++ ++ +++

Selenium, of course, remains a great tool to use. When properly set up and configured, they provide us with reliable and maintainable test suites that do the trick. It gives flexibility with regards to being able to add stuff, however this requires a good amount of knowledge to do so. Especially when it comes to reliability, Selenium has the advantage of having proven themselves over a few years.

Cypress momentarily only supports chrome as a browser, but have Firefox, IE11 and edge on their roadmap. The cool thing about Cypress is that it’s actually a complete framework. It contains Mocha, Chai, Sinon and Chai-JQuery. Starting to write tests is something you can start on immediately after installing.

TestCafe actually recognizes any browser installed on the target device which makes it very easy to get proper browser support. It’s also quite easy to install, configure and get started.

Being able to do the same with less configuration and easier browser support is definitely something that gets me excited about writing tests. As we all know choosing the right tool is an essential part of the success and depends on the challenges we face (some of you might stick with Selenium as it remains a strong tool). For me personally, both Cypress and TestCafe bare the promise of an improvement compared to the traditional tools we all know. The fact that TestCafe has better browser support and less configuration puts them slightly ahead of the game for me.

Are you going to be an early adopter? Let me know in the comments!

4 COMMENTS

  1. Great article. We want to install in our company one of both tools (testCafe or Cypress). Our projects are mainly based on java ee, and react, etc. I guess we will go for testCafe. I will let you know how is everything going :).

    • Thank you Mr Angel Lopez.

      I am very curious as to find out what your experiences will be implementing one of these tools in an existing environment/project. What are your thoughts on where testing should move too in the coming years?

  2. Thanks for the nice writeup. I prefer to be an early adopter of testing tools on a green field project. What is your recommendation on using these new tools on a brown field project?

    • Hi Phani,

      One of the challenges of a brown field project you might be facing is remodeling of the infrastructure. With this in mind I wouldn’t immediately throw away the old.
      My personal belief (as an early adopter) is to implement new tooling in an isolated segment/project and to get a good look and feel about the possibilities of the tool. If succesfull, I would gradually phase out the old by migrating stuff like test sets, test requirements, test data etc

LEAVE A REPLY

Please enter your comment!
Please enter your name here

*