04-10-2022
DevOps: what is it and how to implement it? Everything you always wanted to know
by Daniel Silva
DevOps is a word we hear ever more often. But why? Is it just a spur of the moment thing? What is its usefulness? Is it a title? Is it a culture? Let’s get these concepts aligned! DevOps is the contraction of two words: Development and Operations. Many disagree on what DevOps actually is, and it ranges from a culture, a title/position, a methodology, a set of tools. There are also those who say it’s a buzzword just to describe something that was done before.
DevOps is more than a dedicated team; methodology; tool; culture; position or title. Personally, I consider it to be an aggregation of the first four items, and much more than the last one. Let’s break this into parts.
Title/position
There is a genuine demand for DevOps positions. Typically, this demand serves to integrate teams dedicated to helping different teams/areas in all processes from development, maintenance, delivery, or communication, using a variety of tools and methodologies. These teams are important because they create rigour and enforce good practices, thus raising the quality of the final product. In addition, they allow each team to focus on its own goal, without having to worry about how they are going to automate processes.
Methodology
As the name implies, methodology is the application of methods. DevOps is very much about applying methods. Methods to automate processes, assist teams, monitoring, infrastructure generation, among other things. Such a DevOps methodology also aims to integrate software development work with its operations, thus promoting a culture of collaboration and shared responsibility between teams.
From software development to implementation, DevOps aims to promote efficiency throughout the process. Thus, teams can improve the quality of their code, make the solution reach the market faster and, consequently, involve the entire team for better planning of the application in the future. At its genesis, the DevOps methodology is based on four principles:
- Automation of the software development life cycle
- Collaboration and communication
- Continuous improvement and minimization of wasted time
- Focus on user needs, with short feedback cycles
Set of tools
Over time more and more tools and libraries have emerged to assist in the application of these methodologies. A few examples include pipelines, containers, Kubernetes, Terraform among many others.
Why is it not just a new word to define something that was already being done?
Those who defend this idea usually say that “there have always been development and integration strategies“. Note that I have crossed out the part about strategies because this is usually the part they overlook. And even in cases where there are strategies, these are often not practical – they are manual, which makes them error-prone. This is where DevOps comes in!
What does a DevOps Engineer do anyway?
When someone in my field, whether developer, project manager, team lead, and others, ask me what I do as a DevOps Engineer, I always answer with this question: “How does your team handle shifts to production?“
They are often confused and do not understand why I have answered with a question that in no way answers theirs. But I ask this question for a simple reason: one of the big areas where DevOps has a lot of impact is in reducing the effort/work required in shifting to production. This makes it an excellent entry point to explain/prove why DevOps is “so trendy”.
A practical scenario that portrays reality
Time and again, the answer to my question goes something like this:
“We schedule the shift to a certain date. Whoever manages it on that day compiles the code, runs the tests, prepares the settings, publishes the code, applies the settings and runs some tests to make sure everything is operational”.
My reaction usually goes something like this:
And when I ask how long these shifts take, the answer is that it usually depends on the complexity of the application/component. But if all goes well and no one makes a mistake in any configuration, it’s only a short time … a couple of HOURS.
Again, my reaction:
This process is costly in several ways:
- The process of compiling the code and running the tests can take quite a long time, and usually runs on the publisher’s machine, which can make it unusable during this period
- That person (often it even involves more than one person) has to make a compilation of settings to change and point everything out so you can do it after publishing
- The likelihood of failing/forgetting configurations is directly proportional to the amount and complexity of the configurations, which will imply later troubleshooting in case of an error
- There will have to be someone (be it the surveyor or someone else) applying the necessary changes, which is even more time consuming for that person and the whole process
And we must not forget that we are dealing with a transition to the production environment. This means that (unless there is something very wrong with the team’s process, but that’s a whole other topic), for this to go into production, these configurations already had to be done in at least one other environment! If we consider that many teams use 3 environments: Development, Quality and Production, the time required is trebled.
In this example, considering the optimistic scenario (a couple of hours), this means that whoever goes from one environment to another will spend a total of 3 x 2 hours = 6 hours if everything goes well!
How is DevOps useful in this case?
There are various aspects that can be improved. And the best thing is that improvements can be gradual and adaptive. This means that you can start by taking the points that take the longest and improve them over time.
The ultimate goal is to try to automate as much as possible so that intervention is minimal, and the likelihood of error is reduced as much as possible. Therefore, our goal is to have a pipeline that will perform this whole process: compile the code, run the tests, generate the artefact, publish it in the different environments, as well as the necessary configurations.
Pipelines
In a very simplified way, a pipeline allows us to execute a flow defined by us. Let’s break this into sections:
- You could start by defining the compilation and execution of the tests and, if everything is compliant, generate the artefact at the end. Then you simply download it and publish it for each environment. This already made it possible to compile and execute the tests outside the developer’s machine, thus freeing the developer to continue working.
- Next, your flow would evolve to be able to publish to different environments, without having to download the artefact and do it manually.
- Regarding configurations, you could use automation tools such as Terraform, Ansible, Puppet, among others, as needed to define the configurations and integrate them into your pipeline.
Whilst all of this has a gradual learning curve and the initial setup is time consuming, the time gained from then on is invaluable.
My experience
When I started working as DevOps Engineer, everything was new, and it was a world with numerous possibilities and different tools that made it possible to achieve the same goal. The team I joined had a very similar pattern to the one I described: processes were mostly manual, configurations were not centralised, you had to bring different people/teams together to analyse what needed to be changed and much more. Initially I followed shifts for production environments that took 2 or 3 hours.
Currently we have more teams and more projects, however our shifts (for three environments take about 15 minutes), and usually there are only 2 people involved: the one responsible for the shift, and a second person who makes a review to confirm if everything is OK and approves the shift to the production environment.
Conclusion
There are several parts/areas where DevOps makes a big impact. The example I brought you today is one that I consider one of the most impactful and easy to explain to show the difference between companies/teams with and without DevOps practices.
Personally, it’s quite rewarding and satisfying automating the processes where I know the teams have the most difficulty or waste the most time, because I know I’m taking the load off them, reducing the likelihood of error, and it’s a win-win for everyone.
As I mentioned in the beginning, I don’t think it’s just a name to define something that was already being done. I believe that, increasingly, there is a need to follow these good practices and automate as much as possible, because the results are easily visible.
But it is also important to bear in mind that this is a constant and evolving process. The industry is evolving at lightning speed and there are more and more tools, technologies and techniques to solve problems.
