Continuous Delivery Best Practices

Speed by itself is not a success metric. Without quality, speed is useless. There is no value in having CI/CD pipelines shoot buggy features into production at speed. So, in the world of continuous delivery, velocity means responsible speed, and not suicidal speed. In this article we will take a look at some Continuous Delivery best practices that can help a team achieve quick time-to-market & responsible speed:

1. Automate almost everything

Automation is a prerequisite for the deployment pipeline because it is only through automation that we can guarantee that people will get what they want at the click of a button. However, we don’t need to automate everything at once. We should start by looking at that part of the build, test, deploy or release process that is currently the bottleneck. You can, and should, automate gradually over time.

In general, your build process should be automated to the point where it needs specific human direction or decision making. This is also true of your deployment process & perhaps your entire software release process.

2. Keep everything in version control:

Everything you need to build, deploy, test and release your application should be kept in some form of versioned storage.

This includes configuration properties, requirement documents, test scripts, database creation, upgrade, downgrade & initialization scripts, technical documentation etc.

It should be possible for a new team member to sit down at a new workstation, checkout the project’s revision control repository & run a single command to build & deploy the application to any accessible environment.

3. Implement (or refine) Continuous Integration: 

You can practice continuous integration without continuous delivery, but you can’t practice continuous delivery without continuous integration. Continuous integration (CI) involves merging software code into an automated build process and running automated tests when updated code is added to the source code depository.

Once CI is in place, continuous delivery (CD) simply extends the integration process to the actual release.

4. Build your binaries only once: 

The binaries that get deployed into production should be exactly the same as those that went through the acceptance test process. If we recreate binaries, we run the risk that some change will be introduced between the creation of binaries and their release & that binary we release will be different from the one we tested.

So, during the commit stage of the build, we should only build our binaries once & store them in package repositories like artifactory, nexus or Archiva.

5. Don’t check binaries into version control as part of build:

It can often seem like a good idea to check binaries into version control as part of build process. However, we should resist this as the binaries are derivatives of our code baseline, not part of its definition.

Instead, we should put binaries onto a shared filesystem. If we lose them or need to recreate them, we can get the source from version control & create them again.

The general thumb rule is not to check in anything created as part of our build, test & deploy cycle into version control. Instead, we should treat these artifacts as metadata to be associated with the identifier of the revision that triggered the build.

6. Provide fast, useful feedback:

Failures in the commit stage can usually be attributed to one of the following three causes:

  1. a) either a syntax error introduced into the code caught by compiler/interpreter
  2. b) semantic error has been introduced into the application causing one or more tests to fail
  3. c) there is a problem with application configuration or its environment.

Whatever the problem, in case of failures, it is the job of the commit stage to notify the developers as soon as the commit tests are complete and provide a concise summary of reasons of the failures.

7. Deploy the same way to every environment:

It is essential to use the same process to deploy to every environment – whether a developer or analyst’s workstation, a testing environment or production – in order to ensure that build & deployment process is tested effectively.

However, every environment is different in some way, like different DB credentials, service host name, IP address etc. This does not mean we should use a different deployment script for each environment.

Instead, we can use separate properties file to hold configuration environment for each environment & keep the generic configuration in same file across environments.

8. Smoke test your deployments :

When we deploy our application, we should have an automated script as smoke test to make sure that it is up & running. This could be as simple as launching the application & checking to make sure that the main screen comes up with expected content.

Smoke test should also check that any services (such as database, messaging bus or an external service) our application depends on are up & running.

This gives us a confidence that our application is actually in a usable state.

9. Don’t make changes directly on the production environment:

Most downtime in production environments is caused by uncontrolled changes. Production environments should be completely locked down so that only your deployment pipeline can make changes to it.

This includes everything from the configuration of the environment to the application deployed on it and their data.

10. Deploy into a copy of production:

One of the main problems that many team experiences going live is that their production environment is significantly different from their testing and development environments.

To get a good level of confidence that going live will actually work, we need to do our testing and continuous integration on environments that are as similar as possible to production environment.

To ensure this, we need to have infrastructure, system/application configuration & application’s data in sync with each other.

11. Include the Database in your pipeline:

In order to have a truly automated and repeatable application deployment, you should include all changes to the database as part of release.

However, many times this is neglected because the database is unlike other software components or compiled code that are easily copied from development to testing to production.

We can use database versioning tools like Flyway or Liquibase to apply controlled changes to DB along with maintaining the state of database for each commit.

Concluding Note: 

Adopting these continuous delivery best practices will allow us to create a process with increased productivity, faster time to market, reduced risk and increased quality.

Thanks for reading. If you have some feedback or want to share some experience or have some questions, please leave a comment.

References: Continuous Delivery book by Jez Humble & David Farley



Please enter your comment!
Please enter your name here