Recent entries
Recent Comments

Jez Humble

Deployment Pipelines: Revolutionizing Release Management

Continuous integration was one of the original twelve XP practices introduced in Kent Beck's book Extreme Programming Explained, back in 1999. It was designed to solve a specific and acute problem in the software development process: making sure that when an individual developer completes new functionality, or makes a change to the code, the software as a whole still works. Continuous integration requires that every time a developer writes some code, he or she builds the software and runs some tests on a non-development environment to make sure that his or her changes haven't broken anything. If done manually this process can be error-prone and time consuming, so it needs to be automated. The upshot is that to do continuous integration well, you need an automated build and a set of automated tests, and once the team is large enough, a continuous integration server to run them for you every time a developer submits code. Thus, CruiseControl and its many siblings were born.

However, the problem that continuous integration solves is only a part, albeit an important one, of a greater problem: how to get software that satisfies its functional requirements actually working in production. The part of the software development process between an application being functionally complete and releasing it into a production environment is known as the "last mile" (the opening chapter of the ThoughtWorks Anthology covers this issue in more detail). If sufficient effort has been put into preparing for a production release during the development process, the last mile can take seconds or minutes. However at ThoughtWorks we have been called in to projects where it takes weeks or even months.

This phenomenon has several causes. First of all, any kind of deployment of a complex application — whether or not to a production environment — is difficult. When an application is first deployed to a non-development environment, or when there are significant changes in the application or its environment, it is usual for all kinds of bugs to be exposed. When the application undergoes user testing, further issues are often discovered, such as violations of non-functional requirements. These problems have to be fixed, and the whole process repeated until the application is stable enough to be deployed into production. If deployment to testing environments is hard, it's a good bet that production release will be risky. The options open to you when a production deployment goes wrong are usually very limited — in most cases, backing out is the best option — and downtime costs money. This leads most organizations to be extremely conservative about production releases. This in turn means that releases don't happen very often, which translates to big changes in process between releases, which means the risk doesn't decrease over time. These various factors combine to make releasing applications expensive and slow. Your release process can be the difference between being a market maker and a late entrant.

There is a solution to the problem of the last mile. It is regular, automated deployment and testing of your software in a production-like environment — and ultimately, into production itself. In this paradigm, which we call the deployment pipeline, software is built and then passes through a series of tests — unit tests, functional tests, performance and other non-functional tests and user acceptance testing — before being deployed into staging and then production. The benefits of putting in place an automated deployment pipeline are huge, and ThoughtWorks has been helping companies to solve their last mile problems with this technique for several years. In one organization we turned a deployment process that took days into one that took less than 15 minutes and resulted in split-second downtime, with full rollback, through automating the release process. This had positive knock-on effects throughout the delivery phase of the software lifecycle, making it a push-button process to prepare testing environments. This in turn reduced defects and proved out the deployment process, significantly reducing the risk of performing releases. As a result releases became more frequent and required less effort, freeing up the teams to concentrate on what they wanted to be doing: developing new features.

That's why we at ThoughtWorks Studios have been working to bring you Cruise. We wanted to create a tool that embodies the best practices we have learned through years of solving the last mile problem for our clients. Cruise provides you with an engine that lets you implement deployment pipelines simply and intuitively. It uses a robust, reliable architecture that allows you to scale pipelines across your whole operation by creating fault-tolerant grids of hardware so you can parallelize your long-running builds and test your software on a variety of different platforms. It includes an artifacts repository to manage binaries, reports and other artifacts. And because we are committed to making continuous integration and automated deployment an ubiquitous industry-best practice, we will be offering a one-agent free edition of Cruise which gives you all the features of the commercial edition, as well as free licenses to open source projects, non-profits, and academic institutions.

The other major roadblock to a low-risk deployment is writing a sufficiently comprehensive automated test suite. That's why ThoughtWorks Studios has created Twist, a testing IDE that makes creating functional tests a snap for analysts and business users. This, in combination with Cruise's ability to run test suites in parallel to minimize cycle time, will provide a powerful and simple solution to the problem of the last mile.
Tags :

Comments > (HTML is allowed)

  1. Sargon Benjamin
    May 23rd, 2008 @ 11:36 PM

    Can't wait to here more about Cruise! From what I gather, most teams that use CruiseControl are hacking away at setting up a test environment and then executing functional tests via a combination of post build publishers, listeners, and ant targets that execute shell scripts. Results then have to reported back some how and this may involve custom modification of the dashboard to report on the FVT results. I've used STAF to set up a test farm and have thought about setting this up with CruiseControl to execute and report on FVT tests. Considering the effort, I may just wait on more press about Cruise before doing so. Keep it up!

Sorry, comments are closed for this article.


Products  |  Customers  |  Contact Us
Copyright 2008 ThoughtWorks, Inc.