In my previous blog, I talked about using WebdriverCSS to help Agile teams verify the UI of their application automatically. But later the community for the same, had died down. Support was necessary as it started breaking down with the latest versions of dependent components. It was obvious to me that visual verification is a thing which will be of importance in the future, but I was surprised why there were less and less tools and need for it. Then I came across AppliTools. AppliTools has made the visual verification so easy that it hardly takes 5 minutes to start leveraging the benefits from it. So, let’s get to it then. Before we start, this is not an entirely free tool. You will get 1000 free tests per month with the free account.
AppliTools eyes helps us verify an application UI between changes, it uses an AI powered program to find only differences which makes sense to a human eye. This does not need any tweaking of tolerance and thresholds like in WebdriverCSS or other traditional tools, it’s all been taken care of. The diagram below gives you an idea of the workflow from AppliTools. First it takes a baseline which you must verify manually. Then it checks the differences in the UI against the baseline. When there is a new change, it will give let you know (#6 in the image). If you accept it, the new image will become part of the baseline (see #7 and #8). You are able to review the results in the very intuitive online dashboard.
How to Implement AppliTools
Assuming you already have a functional test suite for your application, It takes 4 steps to implement it:
Step 1: Registration
Register for a free AppliTools account at Create Account and retrieve the API KEY
Step 2: Incorporating AppliTools to your existing/new functional test solution
A) Add AppliTools Eyes to your project. Depending on your environment you can find most suitable way to include Eyes to project from here. I ‘ve used the NPM package as I was using Protractor.
npm install eyes.selenium --save-dev
If you’ve C# selenium, you can consume the NuGet package:
B) Create an object of the Eyes using the API key and call the Open method before your test steps run.
var Eyes = require("eyes.selenium").Eyes; var eyes = new Eyes(); eyes.setApiKey("YOUR_API_KEY"); describe …. It …. eyes.open(browser, "YOUR_APPLICATION_NAME", "YOUR_TEST_NAME");
var eyes = new Eyes(); eyes.ApiKey = "YOUR_API_KEY"; eyes.Open(driver, "YOUR_APPLICATION_NAME", "YOUR_TEST_NAME", "VIEW_PORT_SIZE");
C) Insert checkpoints during your test with the CheckWindow() function. This takes an argument which shall work as a tag to identify the application page between runs. You may call this function wherever you deem necessary in your workflow. Other functions like CheckRegion(), CheckFrame() and CheckElement() are also available, usages can be found here.
D) Close the connection by calling eyes.Close() in your tear down function before quitting the driver.
Step 3: Run your tests for the first time and creating the baseline
Running your test will capture the results in https://eyes.applitools.com. We must mark these tests as the baseline. This can be easily done from the dashboard. You can find a tutorial on how to do this in the following video: https://youtu.be/W0GdsOdg7Xw
Step 4: Subsequent runs
Subsequent runs of the test will check the application UI against the (earlier created) baseline. The test would be marked as FAIL “during run time” if there are changes observed. Which makes it convenient to add the test as a sanity test to your CI pipeline. You can check the failures in the dashboard. You can accept the changes and update the baseline with this new change. This will make the next test run PASS.
What makes AppliTools a great tool is the flexibility around checks that it provides via the dashboard. You can exclude an area from being checked by creating some layers on the baseline images. You can also make it so that the verification only is performed on the layout and not on content or vice versa. The reports and videos of the tests can be shared via dynamic links. These flexibilities make this a great tool to invest in. Here are some screenshots to give a feel of what you can expect:
Test Result View
Flexibility Around Verifications