This blog post is the first one in the series of “UI tests with Selenium”. This post will be about Selenium, how it’s often used and the problems that could arise when using it an a CI setup.
Selenium is one of the most used frameworks to create tests that interact with the browser. There are two flavors: Selenium IDE and Selenium WebDriver. The former is is not well suited for creating and maintaining test suites due to it’s record-and-playback setup: recorded tests are difficult to maintain and become outdated quickly. Selenium Webdriver provides a programmer with an API to interact with the browser in a uniform way. In combination with a test framework like jUnit, TestNG or Cucumber, a programmer can create UI tests in similar way as creating unit tests. Additionally, Selenium has features to record screenshots during the tests, which facilitates analyzing failures of your test runs.
You can easily run Selenium from your own desktop. When you fire up your tests, you’ll see Selenium start the browser, do all the steps you specified, and close the browser when it’s done. But when you’re doing Continuous Integration or Continuous Delivery, you don’t want to just run the tests from your desktop, but you also want to run them using your CI server whenever a change is committed and a new build is started.
And this is the point where things get more interesting, since out of the box, the build server software doesn’t run in a graphical environment, it runs headless. It probably runs either as a daemon on *nix systems or as a Windows Service. That means that a browser can’t run in this environment. Of course, there are ways to overcome this, but you could end up with a lot of interference between the libraries and dependencies of the build server and of the browser.
There are however a couple of downsides to using HtmlUnit . First, because is it headless, you can’t automatically take a screenshot when something fails. The biggest downside, however, is that HtmlUnit is not a real browser, it just simulates them. None of your customers will ever be using HtmlUnit to make use of your products. You think you are thoroughly testing your application, but you will still get defects from live applications. Of course there are web standards, but the reality is that each browser has it’s own quirks and interpretations of certain rules. Therefore, HtmlUnit is not the best option from a user perspective.
So you want to test with the browsers your customers use, but you don’t want any interference with your build environment. One of the options you have is to run the browser or multiple browsers in a standalone environment. For this, you could use Selenium Grid. The next blog post will talk about Selenium Grid and it’s pros and cons.
This blog was written by Harm Pauw.