Continuous Delivery in an environment of microservices

Oct 20, 2014

There are many processes that encompass what we consider to be a deployment. And one of them is what we call build.  A build consist of compilation (if needed), testing, inspection and then deployment itself.  It’s just one process to get all the source code and put it together and verify if the software itself keeps working as expected. A good practice is to perform a build whenever you have a code pushed to the master branch in your repository.

To apply it in your project, you must configure a Continuous Integration Server. If you do a quick search in Google you’ll see that it comes in many flavors: CruiseControl, [Jenkins](http://jenkins-ci.org/ and so forth.

Continuous Integration Server

The CI server is responsible for the integration of changes committed to the repository by executing your scripts (for instance: ant, rake, make, etc). You can set it up to make a build on a regular frequency, such as every couple of hours, once a day. BUT, you can think that it’s a continuous integration, and it’s NOT. A good approach is to configure your CI Server to poll your repository looking for a new commit. If it finds some, it should make a new build.

Basically, these are the steps that a CI Server follow to make a new build:

  • Get the latest changes from the main repository.
  • Run the build script, that includes execution of all your tests (unit, acceptance, ..).
  • Deploy the changes.

What if something goes wrong with the build? It’s your responsibility to stop whatever you’re doing to fix the build. You must keep the build in the green, which means passing all the tests and integrating the code successfully. Otherwise, other developers will keep pushing code to an unstable environment.

Another tip is: don’t use your development workstation to run your integration server. You may be asking yourself “why?”. It avoids the typical scenario: “It Works on My Machine”: You will prefer to use a separate machine solely. In our development machine, we have everything configured and ready to run our own code. Committing code frequently to the version control repository is the only effective way to implement CI, and this means that all developers need to embrace this development practice by grabbing smaller chunks of code and breaking up their tasks into smaller work items.

Build Pipeline

At globo.com, we work in an environment of micro services, where each service has its own infrastructure and the communication between them occurs over the HTTP protocol. One advantage of this environment is the flexibility that each service has to evolve independently of the others. Ian Robinson wrote an excellent article where he describes a pattern of evolution for the services, so that the coupling between them becomes visible, called: Consumer-Driven Contracts: A Service Evolution Pattern. I strongly recommend you reading this article.

And, one of our practices to make this coupling visible is through our integration server. As you can see in the figure below, we add all the dependencies in our build pipeline. In other works, after a change/evolution in a service, if the unit/integration/acceptance tests are successfully executed, the acceptance tests of other services are triggered, so if there is any break in the contract during that change, the build will fail, preventing the commit reaches prod-rc (production release candidate).

image

This way we have full confidence to move the code to our production-release-candidate environment.

Cover photo by Visakhan Kattappuram