DevOps: From code to production

With Octree, we carry out several website projects per month with a team of three developers. Thus, to ensure quality development in a short period of time, we have implemented a process that allows us to develop quickly and in a controlled manner using a DevOps approach.
In this article, we look at the main tools we use at Octree and how we put them into practice.

The tools

To implement this DevOps approach, we use tools that are already widely used and approved. Open-source solutions are often our default choice, but we also prioritize stable, proven services.

Source management with GitLab

GitLab is a Git repository manager whose operation is largely inspired by the GitHub site. Unlike GitHub, GitLab is a free and open-source tool that allows us to install it on a self-hosted server. Thus, we have a total control of the sources that we produce without limitation on the number of managed repositories.

More than just a source manager, GitLab integrates a large number of tools to facilitate team development, deployment and continuous integration of projects.

In our case, we mainly use the GitLab continuous integration tool(GitLab-CI). Through a simple script, we can automate all the tasks from the push (upload of source files on the GitLab server) of the code to the delivery of the project.

Development organization with PivotalTracker

PivotalTracker is an organization tool for Agile projects. We use it to organize, delegate and weight all the stories. It serves as a dashboard on the progress of the project.
It integrates very well with GitLab and allows us to easily and accurately find which commit corresponds to which story.

Simplicity of deployment with Docker

Docker is a tool that has been in the news for some time. Indeed, it proposes a new approach in the field of development and deployment: containerization.

Containerization is the hottest tool in the DevOps world because it reduces the friction between development and operations more than ever. Engineers produce an application inside a container with all the required dependencies and then move it to a production server once the application is ready (more on this in a future article). Thus, development and operations always use a common environment.

The delivery continues

Continuous delivery is a process that requires the automation of all the actions between the code of an application just written and its delivery. Not to be confused with continuous deployment, which also manages the automation of application deployment.

Differences between continuous delivery and continuous deployment

Continuous delivery brings us several advantages in the development of our web applications:

• Accelerated deployment: with each push, GitLab launches the tests on the new version of the application and builds a Docker image ready to be deployed in production. All we have to do then is place this image on the customer's server at his hosting company.

• Improved quality: Since we perform the tests at each push, we are alerted quickly in case of regression or other problem on the latest version of the project deployed. This allows developers to focus on other things than bug fixing, such as performance improvement or security testing.

• Easier development: Since we only have a limited amount of time to develop our projects, it's nice as a developer to only worry about the code to create and not the usual multiple deployment concerns.

Of course, not every solution has advantages. The transition to a DevOps way of doing things has been the source of some worries, but more related to the process changes than to the solution itself. Indeed, being little or not experienced in DevOps culture, it was difficult for us to set up the automated delivery pipeline. The adoption of Docker also generated some "grumbling" from the developers because of its paradigm shift. However, after a few months of getting used to it, we greatly appreciate the ease that this solution brings us for visible efficiency.

To master the DevOps methodology as quickly as possible, we relied heavily on the book "Discovering DevOps which is now a reference in the field.

The automated delivery pipeline

To get a little more practical now, we'll cover how we implement our automated delivery pipeline.

As we said, GitLab, through its GitLab-CI tool, allows us to manage a pipeline for a repository as easily as possible, like Jenkins.

The definition of the pipeline is done through a YAML file ".gitlab-ci.yml" at the root of the repository. We will not dwell on the configuration of this file because the documentation provided by GitLab does not need any addition but we will detail the few steps we have for a commit pipeline by taking the example of Thyme, our volunteer management service.

The pipeline is divided into 3 stages (or stages according to Gitlab nomenclature):

1. build: We build a Docker image of the application with the build tag and then we push it to our internally hosted Docker registry. This registry is used as a Docker image cache to facilitate the pipeline.

2. test: We retrieve the build image from the registry and perform all the tests on our application. Note that we use a database container for the occasion that we destroy at the end of the tests. If a test does not pass, the pipeline stops and the developers receive a notification through Slack messaging.

3. release: If the build image has passed all the tests successfully, then we assign it the latest tag and push it to the registry.

Thus, we are assured that the image with the latest tag on the Docker registry contains the most up-to-date valid version of our application. So we can easily deploy the latest version in production with a single command on the production server:

docker run -d -P docker.octree.ch/thyme:latest

docker.octree.ch is the address of the Docker registry

This pipeline is launched every time a developer makes a push to GitLab. It is possible to monitor the steps through the web interface.

Visualizing the pipeline on GitLab

As you can see in the screenshot, we add instructions in the commit messages to update PivotalTracker. For example, the commit containing [#13378337 delivers] will inform PivotalTracker that story #13378337 is to be set to Delivered status, ready for acceptance by the project manager.

For more information

In this article, we didn't have time to cover how we create a Docker image for our applications. We will cover that in a future, more detailed article.

Now that we have a pipeline for continuous delivery, we want to go all the way and achieve continuous deployment by automating the last step: release.

In the medium term, our goal is to have a real DevOps culture within the company and the implementation of these tools is our first step in this direction. After a few months, we have already observed a real increase in the efficiency and motivation of the developers because everyone can fully concentrate on what they like to do: develop.

References:

Discover DevOps by Goudeau & Metias: Reference book on DevOps culture

https://www.continuousdelivery.com/ : Continuous Delivery Article