Blog: Test Automation Starts at Specification

Several years ago I was involved in a team that needed to create a new product. We had a product owner and one team member that were working on the vision and roadmap for several months when we started. We agreed to adopt Scrum and had a customer that needed this product so we had a launching customer. This helped to fund the product development.

So we started with our first sprints and pretty soon came to the conclusion that the user stories weren’t as clear as we thought they were. The product owner and team member that were working on the vision for a few months already understood one another pretty well and fast but the rest of the team did not. We struggled a while with this and eventually we agreed to try out specification by example.

At first we did not use fancy tools to note down our specifications or even use a specific syntax. During refinement meetings we discuss the examples the product owner and one of the team members had written down. With that information we brainstorm on how to build and test the examples. It really helped us to quickly find the missing information we needed to build and test the new functionality.

After a while we started using SpecFlow (C# equivalent of Cucumber) to write down the specifications so we could also reference our tests to the specifications. We did this based on reading some sites and books. Looking back at this product development and the team this was the most productive team I can remember with a very satisfied client.

So why did this work? Why did we have a very satisfied product owner and customer? Why were we able to deploy to production whenever we liked? Eventually this all came down to the way the team worked. The team members had a good mix of experience, knowledge, enthusiasm, eager and motivation. Beside that working with Specification by Example really made the difference.

But what were the things that made the difference by using specification by example?

By using Specification by Example…

  • All team members new what to build and how to test it before a letter code was produced;
  • Things that were unclear got cleared up during refinement instead of during a sprint or worse;
  • We had living documentation of the functionality of the product backed up by automated tests;
  • Expanding or changing functionality got validated each build instead of during a manual test once a sprint or worse;
  • Setting up the automated test environment stimulated also the automation of deployment of the application and infrastructure;
  • We were able to deploy every two weeks to production, half a year later we were able to deploy to production whenever a story was ready and the product owner wanted to go to production.

I’m proud that I was part of this team and gained the experience on how a team can be successful.

Recently I read the book Specification by Example by Gojko Adzic because I wanted to know if there was more to learn from that approach. I also like to draw and as a practice I made a summary sketch note of the book on my whiteboard at home. See the result down here. Hope this inspires you to start with this great way of working.


Like to hear your experience with specification by example. Please leave a comment below?

Kind regards

Pieter Versteijnen

Previous articleModern programming languages – bloated or powerful?
Next articleEmerging Practices to Cheerlead Your Distributed Teams From Good to Great
Hi, my name is Pieter Versteijnen, married and father to two sons. At DevOn, I work as Technical Agile coach. This means that I, besides coaching teams in Agile/Scrum, also support teams on technical aspects such as setting up a Continuous Delivery pipeline. I also facilitate TFS related training courses, Test Automation courses and several Agile/Scrum foundation training courses. When I'm not working, I like to play tennis and working on, or of course riding my motorcycle. Training courses: Continuous Delivery training, Dealing with your legacy code & Continuous Delivery training, Scrum and Team Foundation Server, Automated builds with Team Foundation Server, Source control with Team Foundation Server, Work item management with Team Foundation Server, Test management withTeam Foundation Server, Scrum framework training.


Please enter your comment!
Please enter your name here