The State of DevOps 2019 was published in August, right in the middle of summer break. If you are like us, you debated whether or not to read it immediately.
In between soccer practice and family weekends at the beach, we snuck in a few sections here and there. We are glad we did. The study findings exactly reflect the kind of results we have seen from customers at Agile Stacks.
If you’re reading this blog, you’ve likely already realized how essential software development is for the modern enterprise. Even if the products you ship out the door are cars, pizzas, or oil, you need software to create a competitive advantage in the modern world.
One of the things that jumped out at us, as a metrics-driven executives working at a metrics-driven company, was the huge difference between ‘low-performing’ companies and ‘high-performing’ companies. High performers:
- Deploy code over 200 times more frequently
- Get from commit to deploy more than 100 times faster
- Recover from incidents 2600 times faster
- Have a failure rate that is one seventh the failure rate of low performers
How does your company win the World Series of software development? This blog post will show you how to start and where to focus to make your software organization ready for the playoffs.
When at Bat, Swing Through for More Power
Here’s the big takeaway:
There is no perfect time to start your DevOps transition.
Either you do it and learn. Or you hold off... and fail by default. The 2019 results show that high-performing companies are now moving to a new category called Elite Performers in double-digit droves. There are many reasons these top performers are getting even better. But for now, let's focus on two that we think are particularly important: using automation tools, and using tools that are easy to use. The issue isn’t just about using tools, but rather about choosing tools strategically and focusing on tools that will save time by automating away drudgery without adding complexity just to learn how to use the tool. High-performance teams create a rhythm to tooling adoption just like a baseball player establishes a rhythm when batting. These rhythms create power through pace because consistency is built into muscle memory.
Let us take a step back and look at DORA’s definition of productivity p.57 of the report. “Productivity is the ability to get complex, time-consuming tasks completed with minimal distraction and interruptions.” Productivity is about producing widgets. In our case, it’s about producing software that works. Using the right tools is the best way to get more done in a shorter amount of time — and that is what DevOps is all about.
Useful, Easy-To-Use Tools
The Useful, Easy-To-Use Tools summary is provided on pages 58-59. According to the report, “When building complex systems and managing business-critical infrastructure, tools are even more important because the work is more difficult.” We couldn’t agree more. In addition, page 20 points out that most DevOps professionals have more than ten years of experience—again, because the work is more difficult and complex.
This is why tooling is so important for Kubernetes. First of all, it’s a new technology. Second, the abstraction of workloads, networking, state and more is still foreign to all of us who grew up building monoliths apps on bare metal. The metaphysical nature of containers is creating a cognitive dissonance.
It reminds us of our early days learning C (or in one of our backgrounds writing Pascal and Fortran). Having to think about where code ran in a physical memory register was a foreign concept to an institutionally trained engineer who lived in the world of capacitors and voltage. We can still remember the shocks from discharging a capacitor but we still forget to deallocate memory with malloc() or calloc(). The more concrete something is, the easier it is for humans to grasp. That’s one of the core challenges with Kubernetes.
So at Agile Stacks, we’ve taken that cognitive dissonance away by building a very simple and easy-to-use UI that never requires our customers to write a single command line.
Being a company built by engineers, we’ve lived through the pain of writing thousands of lines of code and just as many clicks to build a simple software stack. As a matter of fact, we recently wrote another blog about scripts being the new technical debt.
With this pain point in mind, we created a UI to abstract away the complexity of pulling software down, installing it, integrating it and then praying it all works together securely. In minutes, a customer can select software components and put a fully secure, integrated and scalable infrastructure in place. Importantly, our platform also inherently builds in repeatability since we subscribe to a GitOps approach. This fits the DORA report conclusion of two key attributes that drive productivity:
- how easy it is to use the toolchain, and
- how useful the toolchain is in accomplishing job-related goals.
At Agile Stacks, we agree that both are important. The tool has to actually be useful, but it also has to be easy to use. We’ve created a tool that is both.
Although we weren't surprised to learn that 33% of Elite performers use a mix of proprietary, open source and COTS tools - we did find shocking that 15% of Elites used open source with little customization.
The Automation and Integration By Performance Profile summary is on page 60.
“Automated provisioning and deployment to testing environments,” is cited as the third most important reason for adopting automation tools. Certainly, automated build (#1) and automated unit test (#2) are critical, but it is the deployment to test environments that is a high point of failure. It’s the first time that a developer’s code is hitting a full environment and interacting with other developers’ code. Things can and often do go wrong. A large portion of our own engineering work at Agile Stacks is debugging the heavy integrations of our core platform when we add new components, upgrade existing components and fix bugs. Yet what we have found is that by empowering teams to shift their stack build process to the left, we find problems earlier.
The key here is to ensure that all environments and templates are consistent and immutable. A baseline has to be set; otherwise your teams will lose trust because their target environments will never be repeatable. Then 85% of their effort will go to blaming other departments and teams for broken code. Culture in DevOps is another topic we will cover in a different blog.
One of our customers is an AI-powered advertising platform. The company runs it's artificial intelligence engine several times a day to maximize the pricing and placement of ad space for retailers. The Ops team found it difficult to maintain the test environments quality as the business introduced more variables into the algorithm. So, the team looked to Agile Stacks to help them speed up the deployment of Kubernetes nodes and the management of its Cloud instances. What the customer knew was that there was a proportional relationship between compute needs and compute cost. What the team didn’t know was how to lower the denominator. We helped the company by using AWS Spot Instances and Auto Scaling groups. The powerful combination of this solution is now saving the business 70%.
On page 60, the report points out that “We also see that elite performers automate and integrate tools more frequently into their toolchains on almost all dimensions.” The report correctly points out that, in order to achieve amazing returns in DevOps, the company must automate. This is why at Agile Stacks we focus not just on automation, but on automating the automation—in other words, making the integration points work seamlessly, too.
The challenge is that you can automate yourself into technical debt. The authors are reminded of the age-old cartoon of a salesperson selling modern artillery to the busy medieval king in a battle with swords.
One of the keys to using automation successfully as part of your DevOps practice is the ability to eliminate future technical debt. Too often, automation is done with one user or use case in mind. When circumstances evolve, automation tools can make it harder to shift tactics. When this happens, companies have the following options:
- Start the automation all over.
- Ignore the new requirements.
- Have a separate set of automation tools for the new requirements and leave the existing automation unchanged in place.
- Avoid having any automation at all, and just do everything manually as this 'automation thing' is just too hard.
None of those are good options. The solution is to use DevOps tools that “automate the automation,” like SuperHub.
On page 17, the report states, “Many professionals approach these metrics as representing a set of trade-offs, believing that increasing throughput will negatively impact the reliability of the software delivery process and the availability of services.” This is unfortunately a very common misconception that people have, based on the theory of constraints. In manufacturing, the theory was always that moving faster always meant reducing quality. When it comes to modern software development, however, using the right tools means you don’t have to trade speed for quality anymore. You can get both.
Software is like compound interest - both good and bad, depending on if you’re the investor or the debtor. When things go wrong in a complex system, then they really go wrong, and triaging the problem can be a nightmare. On the flip side, when you create a synergistic ecosystem, you end up getting code delivered at speeds like Amazon and Facebook. Their fabled tale of hundreds of releases a day is only achievable because they’ve discovered the King Solomon’s Mine of automating their easy-to-use tools to create an ecosystem of speed, reliability and repeatability. You don’t need to have Amazon or Facebook’s revenue numbers to move at the same software development speed. The key is to start the transformation now, to focus on choosing tools that are useful yet simple and to automate away low-value tasks. If you do that, you’ll be on your way to reaching Elite Performer status and perhaps will even grow as big as Amazon in the future.