Creative ways to encourage the integration of DevOps processes
It can be a challenge to get developers to abandon their bad habits and adopt DevOps processes, but Verizon is finding innovative ways to make that happen.
Many organizations approach a DevOps transformation by starting with the tools and pushing them onto developers. Verizon has been experimenting with a different approach, though, that involves immersing developers, managers and business experts in a DevOps experience that they can bring back to their normal workflows. This helps shed some of the anxiety many developers might otherwise experience if they try to kick-start DevOps processes from their normal office environment.
Verizon has already created over 1,000 teams that use DevOps toolchains and pipelines as they undertake a large scale move to the public cloud. The company is finding that a big part of successfully adopting DevOps processes lies in tapping into developers' pride and passion, rather than pushing a pre-defined agenda and tech stack, insisted Ross Clanton, a fellow of technology transformation at Verizon.
One of the company's key strategies has been to launch a number of dojos in the US and India where developers can try out DevOps processes on real projects, without the stress involved in their traditional development practices. Verizon currently has three dojos in the US and two in India.
Enterprise DevOps faces people bottlenecks
It's much more challenging to adopt DevOps processes at a large company like Verizon than to do so at smaller organizations.
"We have a pretty familiar toolchain," Clanton said. The real challenge, then, was the people. The company wanted to find a way to bring DevOps to 20,000 developers. They had to find a way to break larger development teams into smaller ones that could be more agile and inspired to work together. Verizon went about creating a network of dojos to spread DevOps skills.
"The dojo is an immersive learning environment modeled on the Japanese concept of a place you go to build your skill," Clanton said. "We are bringing teams in for extended periods and working with coaches while learning about the products they are creating for the organization."
I have to figure out how to get people excited from the bottom up and from the top down.
Ross Clantonfellow of technology transformation, Verizon
Engineers practice skills around DevOps, Agile and cloud development. Part of that means learning to break work down so they can demo it with internal business customers to assess whether they are on the right path.
"Speed to market has been challenging, and the dojos and DevOps model has been a way to accomplish this," Clanton said. "The dojo puts them in an environment where they work more cross-functionally. The dojo is an immersion center where teams dedicate six weeks to accelerate learning modern engineering practices."
Teams come into the dojo with a backlog of real work they are trying to deliver and are paired with DevOps coaches for six weeks. Some managers expect the teams to deliver these projects faster over the course over this period. Sometimes it happens, but Clanton explained it is really about building the skills that will allow them to deliver faster software and with better quality when they return to the office.
The biggest adjustment has been in aligning the management expectations with the purpose of the dojo. Most teams see this as an opportunity to get work done faster. But rarely do they deliver software faster during the actual dojo.
"The purpose of the dojo is learning, and we prioritize that over everything else," Clanton said. "That means you have to slow down to speed up. Over the six weeks, they will not speed up. But they will get faster as an outcome of the process."
Spotlighting DevOps and continuous learning
What do DevOps and skateboarding have in common? To find out, read our profile of Gene Kim, of Phoenix Project fame, who also explains how both the software development methodology and a 40-something DevOps leader have room to grow and learn.
Spreading DevOps processes globally
Company size has been a problem for Verizon, with teams physically distributed around the world. It's not easy to get a whole team of engineers to leave their homes and families for six weeks. Verizon has done some experimentation with remote workers, but it hasn't worked as well as physically immersing developers with a team.
In part to address this, Verizon has developed a program called Dojo-in-a-box, where it sends guidance out to teams. Sometimes, but not always, it means sending out a live coach as well. These are the steps bringing a new kind of culture across Verizon.
"I have to figure out how to get people excited from the bottom up and from the top down," Clanton said.
Verizon has even created an internal DevOps conference just for employees. But it's important to get people excited rather than force them to come. That takes the right ingredients.
"Get the external speakers that will draw a crowd, and get people to share and talk internally. Market it like crazy, make it fun and keep doing it," Clanton said.
Build better DevOps processes with gamification
A key part of Verizon's DevOps strategy was the launch of an innovative program for creating friendly competition. Launched by Nanda Kumar, fellow at Verizon Global Technology Services, the program presents a team with an actual cup for showcasing mastery of DevOps processes. gamification gets teams from around the world to flex their DevOps skills and share their secret sauce. Tools are a particularly sensitive topic, since the point is to emphasize best practices and not alienate developers by forcing them to use a specific toolchain.
"With gamification, we wanted to define the high-level guiderails to achieve and let developers figure out if the current tools satisfy these needs," Kumar said. "If not, they can gravitate towards tools that work for them."
This approach has also lead to better feedback with tool providers. Verizon works with lots of commercial off-the-shelf products that do some, but not all, of what its developers need. The competition encouraged developers to open a dialogue with these vendors so that they can build the capabilities developers need in their products. Developers are also inspired to learn from the winning teams' tools and practices.
Verizon uses the Hygieia open source dashboard to share each team's progress. As part of the cup competition, Verizon brings teams together at a tech fair that is partly sponsored by tool vendors, which helps keep costs down. Kumar was surprised at the way this approach engaged senior management. One senior manager, who was scheduled to talk for 15 minutes, canceled the rest of his meetings for the day when he saw the excitement in the room.
"This can provide the value to get things going," Kumar said.
