DevOps, What’s it all about? Part 3 — CI/CD Why do I need it?
After we had a long gaze on version control, what’s the next logical step? CI/CD i would say!
In my previous articles, I defined DevOps and checked out (pun is intended) version control. The next step would be to give an over of another vital aspect of the DevOps tooling world. How do we automate our development and release process, and why? In the world of software development, this process is called merely CI\CD pipeline.
When companies look for DevOps engineers, they typically need someone who has experience working with CI/CD pipelines, but what does it mean? Why do we need a CI/CD pipeline? In this article, I try to explain some of the ideas behind it, and why we should use it when we release software.
What is the CI/CD pipeline?
In short, a CI/CD pipeline is a method in which teams of developers and operators (who have a winning DevOps culture in their organization) can deliver apps automatically and efficiently. Specifically, CI/CD introduces ongoing automation and continuous monitoring throughout the lifecycle of apps, from integration and testing phases to delivery and deployment.
What are we trying to solve here? Well, grab your self a cold smoothie and let me tell you a story.
A long time ago, when I was still young and sassy, deployments were not an easy task. It could take us sometimes a week to deploy our software on all the servers. It would take a lot of effort; everyone had to integrate their code into a working application. Most of the time, it was manual, or we had some bash script to help us with some magic (not magic, it was horrible).
Yours truly used to wear the hat of Integration specialist in some companies, a job title for someone who can integrate all the pieces of software into a working app.
Those shenanigans are called “Integration Hell.” Those times were so horrible that I can write a full article only on this subject. Still, luckily in some organizations, those days are over.
How can CI/CD solve this problem?
When designed and implemented in an orderly manner, CI/CD should help the operators and developers with automatic workflow for their testing, integration, creating the artifact, and deploying the software.
When specific triggers — manual or automatic — generate a job that would result in a deployment, you have a CI/CD pipeline. In many cases, some parts can be manual, but the aim is to have an automatic and fast process, one that is easily reproducible. When you can achieve those merits, I think you did the first step into a better organization.
So what’s the CI, and who’s the CD?
The acronym CI/CD has a few different meanings. The “CI” in CI/CD always refers to continuous integration, which is an automation process for developers. Successful CI means that a new code is automatically built, tested, and is merged into a shared repository.
The “CD” in CI/CD refers to continuous delivery and continuous deployment, which are related concepts that sometimes get used interchangeably. Both are about automating further stages of the pipeline, but they’re sometimes used separately to illustrate just how much automation is happening.
Continuous delivery usually means a developer’s changes to an application are automatically bug tested and uploaded to somewhere where it is ready to be deployed. The deployment process itself can be done by a developer, operators, or both.
Continuous deployment can refer to automatically releasing a developer’s changes from the repository to production, where it is usable by customers. It removes a lot of the manual work that operation teams used to do back in the days.
How does it help?
With a proper CI/CD pipeline, you can find errors fast, and most importantly, it should help you have a faster Lead Time. As I explained before, if the goal is shipping code quickly, this is a key DevOps metric. I would define lead time as the amount of time that occurs between starting on a work item until it is deployed.
When correctly done, it enables you to deploy your software more often, and with better confidence, this removes a lot of frustration from your teams. When everything is automatic, the team would need to rely less on their memory and have more ability to do other tasks.
You will gain better visibility on your code and deployments with more measurable information. You can get an even better feedback loop, and as mentioned by one of my smart and elegant colleagues, your pipeline is a feedback loop itself.
A good pipeline will remove reliance on one or two team members who are the only ones who can build and deploy your application.
Using CI/CD tools offers a better option for knowledge transfer between teams. Instead of using some manual labor, most tools use some conventions for the configuration, for example, YAML files. This option makes onboarding and knowledge transfer much more straightforward.
How does it work?
A developer downloads some code from the shared repository to work on it. He might start a new branch for a new feature. Once the new feature is complete, the developer would push it to the shared repository.
The CI server checks the changes and begins to build and test the application, ensuring that the changes have not caused errors in the code.
The development team is notified of the test results. In the event of a failure, the team will know that the new code has caused the crash and can begin to isolate and fix the issue. If the changes are successful, the group moves on to the continuous delivery phase.
The CD side of things would create an artifact. It would automatically deploy the object on the relevant environment (Dev\QA\Staging\Prod) in a somewhat automatic way while sending status messages when each step is finished.
When it becomes a burden
Having a proper CI/CD can save you a lot of frustration and time. But sometimes maintaining the pipeline can create a lot of frustration and over-engineered solutions that are very hard to keep are not precisely what’s want to achieve.
Before you begin implementing, you should design and think about what do you want to achieve with your pipeline. Think about the tools that you are using, how well they work together? Would you need another tool? How easy is it to customize your pipeline?
The pipeline cant becomes more critical than its objective, which is to make releases and deployments faster and more accessible for everyone. If you are failing this need, you are doing something wrong.
One of the biggest pitfalls of any DevOps project is a horrible CI/CD pipeline. I can give many examples, but let’s stay on the positive side of things.
What about DevOps culture?
I think that having a well organized CI/CD pipeline is only part of what makes a high performing organization. The pipeline should be implemented with a strong understanding of DevOps culture; otherwise, you will have a high-speed pipeline with many frustrated developers and super angry ops.
It again comes down to a question of responsibility, and developers should take responsibility for their parts of the pipeline. If we do not think holistically, we will create just another silo, and your organization would have a big team of “DevOps engineers” that are endlessly working on a complicated pipeline that is never actually working.
Furthermore, a CI/CD can be an essential part of your development process, the question you need to ask yourself: “What kind of a pipeline do I need?”, “How could I scale my pipeline when the demands grow?” “What’s the budget that I can have for using and building a pipeline?”
Do you need some help? Feel free to contact me or anyone from our team at Polar Squad. Empathy is one of our values, and now might be a good time to start doing some first steps in the realm of DevOps transformation. Feel free to drop us an email or Linkedin message! Maybe we can help you or guide you in the right direction.
