Understanding DevOps Before Hiring a DevOps Engineer

Hiring a DevOps engineer is a challenge.
This role is relatively new in our world, and recruiting a DevOps engineer successfully requires a deep understanding of DevOps, and even being willing to make organizational and cultural changes to embrace it.

This series of articles is meant to provide you the tools to hire a DevOps engineer.
We will start by clarifying what “DevOps” means, go through how the role should look like in your company, and finish with a guide for interviewing a candidate.

By the end of this series of articles you should be able to have both an understanding of how DevOps should look like in your company, and a process for recruiting a DevOps engineer.

‍

So, what’s the challenge with recruiting a DevOps engineer?

Going about hiring a DevOps engineer is starting a process that needs to be thoroughly thought through, that is, if you want to make sure the person you hire is the best fit for your organization.

This is not new, and not unique to DevOps engineers. You could replace the words “DevOps engineer” with any other role name, and it would still be true.

The unique challenge here is that it’s not trivial to even define what the DevOps engineer should do when it starts working in your company.

In most organizations I’ve seen, the role of the DevOps engineer was not defined at all, and it was more often than not just “whatever we want to save the developer’s time from doing”.

This is a bad definition, as it’s not sustainable nor healthy for your development team.

It’s not sustainable nor healthy since over time it creates a thinking pattern in the developer’s mind that “whatever I don’t feel like doing should be done by this guy - I don’t want to be responsible for it, so let the DevOps team do it.”

Now, you might think, “Yes! That’s exactly what I want!”, and that’s ok, but that’s not the idea of doing DevOps, and my claim is that it won’t be as good as doing DevOps.

Let’s start tackling this challenge by explaining what we mean when we say “DevOps”.

‍

What do we mean when we say “DevOps”?

We will use  an analogy to explain the principles at the core of DevOps.

Let’s imagine 2 parents, a mother and a father, Joe and Joana, blessed with 2 identical twins.

They name the twins “Baby A”, and “Baby B”, of course (sorry about that, I’m an engineer not a storyteller).

Those twins are just 1 day old when their parents, Joe and Joana, come to an agreement.

Joe says: “I’ll be responsible for Baby A, and you’ll be responsible for Baby B, this way each of us can be focused on one child, and we’ll each specialize in raising that child.”

Joana of course says: “Yes, what a great idea.”

From now on each parent will provide solutions to all of the needs of the child to which it’s responsible.

What do you think is going to happen as a result of this separation of responsibility?

I’ll tell you what I think is going to happen, they won’t be able to collaborate, because they will have no shared responsibility.

The solutions Joe provides to Baby A won’t be applied to Baby B (despite probably having similar needs), and same goes with Joana and Baby B.

Not only that, Joe and Joana will each notice different needs with their child, and the knowledge that a certain need even exists might never get communicated, and so that need might never get answered.

We can go on, but it boils down to this: a team works well when they collaborate, and it’s only able to collaborate when the team’s members have shared responsibility.

We know 2 people can achieve more if they contribute to the same product or service, but we don’t always remember the basics - it’s only true when they have good collaboration.

The idea of DevOps is the same - it takes two different roles in R&D organizations, that historically didn’t manage to collaborate at all, and had no shared responsibility, Developers and Operators, and it then says “Let’s give these roles as much shared responsibility as possible in order to make them collaborate”.

Instead of a Developer writing code and passing it to a system administrator to deploy it, there’s now a developer and a DevOps engineer, both responsible for making it work.

This shared responsibility can now manifest itself in multiple ways:

1. The DevOps engineer implemented a tool to allow developers to deploy changes and rollback.
2. The DevOps engineer wrote a guide for how to deploy and rollback changes step by step, or automated that process.
3. The DevOps engineer pointed at the right tutorials for the developers to accumulate knowledge needed to deploy and rollback changes however they like.

But wait, if it’s shared responsibility, why is the DevOps engineer doing all of that?
That’s a good question, and we have to understand the role of the DevOps engineer to explain it.

‍

What’s the role of the DevOps engineer?

If the responsibility is shared between development and operations, what’s the role of the DevOps engineer? and why does it look like it is the one responsible for the operations?

The answer is that the DevOps engineer will be responsible for introducing the knowledge, processes and tools that will enable the developers to ship code and operate the system.

This means the DevOps engineer won’t deploy to changes developed by the developer, but it will introduce the deployment tools that will enable it, alongside processes and knowledge that will enable doing it.

The effect of the shared responsibility here will be the following: both the DevOps engineer and the developer are responsible for performing a successful deployment, but the developer is accountable for making the code work when deployed, and the DevOps engineer is accountable for enabling the deployment by providing the platform to do so.

Deployment is, of course, just one example.
There are other things within the DevOps engineer’s scope of responsibility to enable.

Except for deployment, what other fields could the DevOps engineer enable in the company?

‍

What is the DevOps engineer’s responsibility

There are many things the DevOps engineer can be responsible for, and so we will stick to the principles that could help you define the role in your company.

If we accept the role of the DevOps engineer as someone that shares responsibility over the system with the developers, and we accept it is accountable for enabling the developers to operate the system, then it makes things easier to define.

We can now say the role of the DevOps engineer is to build the platform for operating the system.
The word “platform” here is key.

Building the platform doesn’t mean “abstracting away everything from the developers”.
The platform is something meant to enable various abilities.

What abilities should the platform enable?
Here is a number of them:

1. Infrastructure - Enable provisioning, maintaining and scaling infrastructure
2. Monitoring - Enable monitoring logs, metrics and traces, and triggering alerts
3. Continuous Integration - Enable continuously collaborate on the same codebase
4. Secrets Management - Enable storing and retrieving sensitive configuration data
5. Etc.

But what does it mean to enable these things?
We have said it means building a platform, and we also mentioned that abstracting things from developers isn’t “building a platform”.

The 3 things a platform is comprised of are these:

#1 - Tools - Ready to use systems that already solve an existing problem

#2 - Processes - Well defined series of steps that should take place to solve a problem we don’t want the developers to invest time and thought into

1. Documented - The process could be a guide for committing that series of steps
2. Automated - The process could be fully automated to save the time required to perform its steps

#3 - Knowledge - The information and thinking patterns required to solve a problem

Let’s take one of the abilities it’s the DevOps engineer’s role to enable, and see how we build a platform using tools, processes (automated and non-automated), and knowledge.
We’ll focus on monitoring for the example.

Monitoring:

‍The developers in the company have expressed their desire to understand what’s going on with the apps they wrote that are running in production.
The DevOps team took note, and did the following:

#1 - Told all of the developers to start exposing metrics from the app

1. It gave them a community SDK to do so (Tools)
2. Pointed them at the tutorials (Knowledge)

#2 - Edited the company’s microservice template to utilize the SDK out-of-the-box (Automated Process)

#3 - Deployed an instance of Prometheus to start collecting the Metrics (Tools)

#4 - Deployed an instance of Grafana to start displaying the Metrics (Tools)

#5 - Arranged a workshop for using Prometheus and Grafana (Knowledge)

#6 - Wrote a guide for building a dashboard with graphs for an app (Documented Process)

By doing all of the above, we have enabled the developers to start monitoring their applications using metrics, and gave them the platform to do it.

This is just one example, and there are countless other examples.
Let me know if you would be interested in an article with more of those examples.

‍

Let’s summarize

Recruiting a DevOps engineer is challenging, mostly because one must first deeply understand DevOps in order to make it a success.
DevOps is a culture meant to improve collaboration by increasing shared responsibility.
The DevOps engineer increases shared responsibility with the developers over the operation by enabling the developers to operate the system they are building.
The DevOps engineer does that by building a platform for the developers, which is a collection of tools, knowledge and processes meant to enable various abilities.

Defining the role of the DevOps Engineer, and understanding what DevOps even means is the first step, and hopefully you have taken that step just now.