Many teams at OVO are using CircleCI for their continuous integration and deployment. The new orbs feature (https://circleci.com/docs/2.0/orb-intro/) of CircleCI allows for packaging of configuration into reusable elements called ‘orbs’, which can be used from any CircleCI configuration file.
CircleCI uses a simple yaml configuration file at a fixed path inside your git repository (
.circleci/config.yml). The first key is
version which should be set to
2.1 to use orbs.
A basic configuration consists of jobs and workflows:
version: 2.1 jobs: test: docker: - image: python:3 steps: - checkout - run: ./test.sh deploy: docker: - image: ovotech/deploy-image:04-11-2018 steps: - checkout - run: ./deploy.sh workflows: commit: jobs: - test - deploy: requires: - test
This contains two jobs:
deploy. Each job runs in a container, and we specify the image to use with the
docker key. CircleCI will run each of the steps for a job inside the container for that job.
There are a number of step types available including
checkout will checkout your source code inside the container;
run executes a shell command inside the container.
workflows key defines how the jobs should be run. In this example the
test job is run and only if the tests pass is the
deploy job run. CircleCI will trigger this workflow every time someone pushes a change to the repository.
A CircleCI Orb is a way to package reusable definitions of jobs, executors, and commands which you can import into a CircleCI config. To import an orb you add a top level
orbs amp which contains the orbs you want to use (as the value) and the name you refer to the orb by in the rest of the configuration file (as the key).
orbs: terraform: firstname.lastname@example.org
In the rest of the config file we can now reference the contents of this orb using the name
terraform. The orb is identified by the namespace '
ovotech', the name '
terraform' and the version '
An orb is a yaml file which is very similar to a CircleCI config file. An orb can contain jobs, executors, and commands.
An executor is the environment that runs your steps. In many cases this will be the Docker image, environment variables and other Docker settings that are used to create a container.
terraform orb we created an executor named '
default' which is a docker image we built containing terraform and associated tools. To define the executor, we use the top level
executor key in the orb yaml file:
executors: default: description: "Terraform executor" docker: - image: ovotech/terraform:latest
To use an executor defined in a orb, set the executor key for a job in the CircleCI config file:
jobs: setup: executor: terraform/default steps: - checkout - run: <command>
This job runs the steps in the executor named
default from the
You can define commands inside an Orb that consist of multiple individual steps. Once imported into your CircleCI config you can then use the command as a single step. This is useful for packaging long commands (or series of commands) so they can be reused, simplifying your CircleCI config.
To enable reuse of commands, they support parameters. Define the parameters under the
parameters key of the command, including a
description. You can make a parameter optional by including a
Within the steps of a command you can substitute the value of a parameter by using
<< parameters.parameter_name >>. Here's an example:
commands: check: description: "Check that terraform state hasn't drifted" parameters: path: type: "string" description: "Path to the terraform module" default: "tf" steps: - run: name: "terraform check << parameters.path >>" command: | ./check.sh
We can then use this command from a CircleCI config:
version: 2.1 orbs: terraform: email@example.com jobs: check_all: executor: terraform/default steps: - checkout - terraform/check: path: uat - terraform/check: path: prod workflows: commit: jobs: - check_all
Just like in our config file, an orb can define jobs. Jobs can have parameters just like commands. This example defines the
check job which run the
check step we already looked at inside the
default executor. Inside the orb you can reference other commands and executors by their name.
jobs: check: executor: default description: "Check that terraform state hasn't drifted" parameters: path: type: "string" description: "Path to the terraform module" steps: - checkout - check: path: << parameters.path >>
We can use a job that's contained inside an orb by including it as part of a workflow:
version: 2.1 orbs: terraform: firstname.lastname@example.org workflows: commit: jobs: - terraform/plan: path: prod
This workflow will run the
plan job from the
Let's look at how we used these elements to create the
ovotech/terraform orb. This is an orb which packages tasks that were being duplicated many times across our repositories, in some cases implemented quite differently.
At OVO we aim to treat our infrastructure as code, with Terraform being a popular tool for achieving that aim. When making a change to an infrastructure git repository we follow this workflow:
- A short lived branch is created and edits made.
- The branch is pushed to GitHub and a Pull Request is created.
- CI generates a Terraform plan and attaches the plan to the PR.
- A developer reviews the plan and the PR approved or rejected.
- The PR is merged and the approved plan automatically applied to the infrastructure.
This is similar to the process that teams follow for making changes to their application code.
The CircleCI configuration to enable this workflow has been reimplemented multiple times by individual teams, bloating the size of the CircleCI config in each repository. By creating an orb, we can package this functionality and simply import it. This allows improvements to be made centrally.
Here is an example of how simple it is to implement this workflow using the ovotech/terraform orb:
version: 2.1 orbs: terraform: email@example.com workflows: test: jobs: - terraform/plan: path: tf filters: branches: ignore: master - terraform/apply: path: tf filters: branches: only: master
In this example a plan for the module in
tf/ is generated and attached to the open PR. If that PR is then merged, the plan is applied.
Full documentation and source code for the ovotech/terraform orb is in the ovotech/circleci-orbs GitHub repo. All orbs in CircleCI are publicly available - these are in the ovotech namespace.