Can DevOps problems actually cause projects to fail?
DevOps isn't perfect. There are times when DevOps problems can overwhelm the potential benefits. So, why do some DevOps projects succeed while others fail?
DevOps is broken for many companies, leading to more DevOps problems than the new process and methodology solves. Yet, at the same time, innovative companies are utilizing core DevOps principles to unseat established titans in virtually every industry, creating a duality in which DevOps adoption is not stifled by potential DevOps problems. The best way to explain this apparent paradox lies in reconsidering how the enterprise thinks about DevOps in the context of overarching enterprise goals.
Buried in the recent 2017 State of DevOps Report, sponsored by Puppet and DevOps Research and Assessment, was the observation that all enterprises are building software faster. Even companies ranked as low performers were rolling out software updates at an average pace of 32 deploys per year, with change lead times of one week to a month. This represents an improvement from 2016 when low performers were deploying two to 12 updates per year and required change lead times of one to six months. But over the same period, the change failure rate grew from the range of 16% to 30% to the range of 31-45%. So, basically, DevOps laggards are failing faster as DevOps problems reverse their progress.
Quality more important than speed
The core ideas of DevOps lie in improving the communications and interchange between developers and operations. The problem is that this focus can leave out the importance of other roles that go into building quality enterprise software, including user experience (UX) designers; QA; security; governance, risk management and compliance (GRC) leads; and business development. Over the last couple of years, many experts have suggested that enterprises should think about concepts like AIOps, ChatOps, DevTestOps, DevSecOps, DesignOps and BizDevOps.
High-performing DevOps enterprises reported a change failure rate of 0% to 15% in 2017. But this metric only addresses one -- albeit an important -- component of modern enterprise needs. Perhaps a more useful metaphor for achieving the kind of impressive results of Amazon and Netflix might be QualOps. In this case, the ops here might be more about pursuing quality opportunities for the enterprise on top of the operational infrastructure.
Gradient of DevOps problems
The traditional notion of a software defect came from the literal bug found to have broken the execution of an app running on a Harvard Mark II electromechanical computer in 1946. But today, software defects take on different connotations for the various roles that play a part in developing modern apps:
UX designers worry about defects that frustrate users.
Developers are concerned with logical flaws that break the execution of their code.
QA is concerned with exploring edge-case scenarios that developers hadn't considered.
Security teams are worried about defects that open new doors for malicious hackers.
GRC leads worry about getting sued or sanctioned by various government agencies.
Business leads worry about defects that cause financial losses.
Harmonizing quality as a priority
Defects are never going to completely go away. All apps are going to frustrate some users. Large software is just too complex not to break sometimes. New security vulnerabilities are going to be discovered in popular libraries that no one had ever considered. Consumers will creatively find some loopholes to save money at an enterprise's expense.
One of the big differentiators of DevOps leaders was the adoption of tools for test automation.
But these defects can be managed by shifting the focus of QA from finding logical errors to harmonizing the different perspectives of quality across the enterprise. The 2017 State of DevOps Report found that one of the big differentiators of DevOps leaders was the adoption of tools for test automation. This frees up QA teams from the manual efforts of testing apps to allow them to spend more creative time exploring the various dimensions of software defects across the enterprise.
DevOps problems and software quality
It can be useful to have a metric for enterprise software quality to gauge efforts to improve the software development process and address potential DevOps problems as they arise. However, how does one compare the relative impact of app quality in a way that considers performance, UX, security and business needs? To address this DevOps challenge, the State of DevOps researchers suggest measuring unplanned works and rework as proxies.
In 2016, DevOps leaders spent 22% less time on unplanned work and rework compared to DevOps laggards. As a result, they could spend 29% more time on new work, such as new features or code. This year, DevOps leaders spent 21% less time on unplanned work and rework and 44% more time on new work. DevOps leaders also spent 50% less time remediating security issues. This suggests a need to involve security and quality teams in the development process early and often.
It's not that the DevOps paradigm is fundamentally broken. But all this information suggests there should be a focus on technical quick fixes to speed up the software development lifecycle. The bigger and more significant problem lies in working through the internal politics and processes for improving quality. Once these are addressed, better tools and technologies will have a meaningful impact on enterprise goals.
