Our journey

Here at EndGame we have been using Octopus Deploy since 2015 and we think that it’s an amazing tool. So much so that we have partnered with Octopus to become their New Zealand implementation and licensing partner! Given this exciting news I have been roused to write about something dear to my heart. The art of deploying, or why your build pipeline isn’t the place for deployments.

At EndGame we partner with our clients long term, and as such we need to ensure that the whole lifecycle of a product is considered, and we have a lot of them. Early on we encountered a problem that you may already have or are walking into. We were writing scripts and tools to do the deployment in our build pipelines. Each of which was slightly different, and the configuration a particular mixture of values in source control and as build variables.

When we came across Octopus it was like a light had gone off, in that we had found a long lost family member. Octopus was built on two principles, which I have learned the hard way are good; that the same build should be released to all your environments, and that the deploy have an easily configurable process that can have steps that have variables injected into them.

Long story short, the following is an examination of those principles.

Building vs Deploying

There is a difference between building and deploying. While these two terms are often used interchangeably, they are inherently different.

A build is working towards collecting a set of artefacts that will be (or maybe) released.  It should be red or green. Either you have successfully created the artefacts (green), or you haven’t (red). These artefacts are stateless, in the sense they don’t care what comes before them or goes after, they are not effected by any other build, just what has gone into them.

The opposite is true of a deploy, it needs to care about not just the package, but the services and infrastructure that will run the package. The world of deployment is full of nuances and it's not so easy to declare a deploy as simply "deployed" or "not deployed". We're dealing with a task that could potentially leave a system in a precarious state, and escaping from that bitten apple is often a risky endeavour. Anyone who has gone through a failed automated database deploy, will tell you it’s not quite the same as a unit test breaking a build.

These failed states lead us to another difference in building and deploying, if a build is run multiple times with the same files one can expect that you get the same artefacts, and you therefore expect to do it once. With a deploy you expect not just to run it once to release to stage, but to run it again on UAT and live (and that other live for client 2).  And each of these runs are expected to have different results, as the places they release to are different (if only by IP address).

With this clear separation of intent, it makes sense for one tool to build the artefacts (we use gitlab runners for this) and then one tool to deploy it - Octopus!

Scripts vs Process Engine

Ok the second principle of deployment, it should be easy, and variable based on the place you want to deploy it.

The way it starts is that software developers are familiar with coding scripts, a task often approached with an esprit of simplicity. Shelling out some scripting to deploy your latest app in a build pipeline might seem like a quick and easy solution to solve a problem. Then you have to add a cache that needs to be flushed, or that separate live environment (because of client 2). So the script gets more complicated with more twists and turns, and before long you are thinking about the tech debt held in the scripts, and adding cards to rewrite the scripts in something else.  Even then the problem doesn’t go away as deploying over multiple environments is a long process, not a simple pipeline (which should either fail or succeed). It may seem ok as pipelines have steps, so release to stage as one step, wait, release to UAT, wait, release to Live (and other live).  It’s the waiting that’s the killer, maybe you are the type to push your code straight to production, but the rest of us have QA sign offs, and change boards. So now we have to find the right pipeline that had the right release, as the world of the code has moved on since you last looked at it, and please please don’t ask me to test the rollback on stage.

The answer to this problem is not to use the pipeline, but use an engine that can run steps needed to release the artefacts to each environment, and hold the state of those versions for as long as it takes a change board to discuss, even if the stage environment has moved on since then. Our extreme example of this is a client that has 100s of different deployments each with different variables and databases, across three different environments (stage, UAT, live) and different locations in those environments. To do this with a script and pipeline would be an engineering feat, but with a simple repeatable process for each client against the same package it seems like child’s play.

Enter Octopus: Taming the Deployment Beast

Facing these challenges, developers require a robust, streamlined, and reliable deployment tool. Cue - Octopus. Octopus cuts through the chaos and exclusively focuses on deployment. It’s like having a trusted ally who’s adept at navigating uncharted depths.

Octopus works by managing packages that are built once by a pipeline and pushed to the server for safe keeping, but can be released multiple times.  Our example of this is having three “epic” stage environments each with different feature flags enabled, but the same code releases.  Once tested against all the different combinations, the code could then be pushed to multiple client live environments knowing that the feature flags would work as expected.

Octopus also enhances the deployment management process with its unique feature, tenants. Managing multi-tenant deployments become a breeze with them, as mentioned in the example above. In addition, Octopus allows for streamlined redeployment, should an environment need fixing. A roll-back example in Octopus would be a perfect testament to its redeployment capabilities.  So testing that roll back in stage becomes a click of the button, rather than a lesson in git mechanics.  And I haven’t even started on it handling all your variables and secrets securely, including certificates and cloud credentials, which will have to be a blog for another day.

In conclusion

While scripting in the build pipeline may seem like the easiest answer, builds vastly differ from deployments. With Octopus at your disposal, the complex world of software deployments becomes a lot less intimidating.

Remember, in the realm of software development, anyone can code, but deploying? That’s a whole other game!

If you would like to get in contact with us about how we can help streamline your deployment process then contact me at andrew.cox@end-game.com