The DevOps movement promises a lot of benefits to enterprises wanting to turbocharge their application development and delivery processes. Unfortunately, many DevOps implementation projects end up going off the rails as the team gets distracted with meaningless minutia and organizational dynamics. To keep your team from wandering into the DevOps wilderness, you need to have a map of the territory.
Value Stream Mapping
Value stream mapping is a lean management technique for understanding the end-to-end flow of materials and artifacts through some sort of production process. Value stream mapping originally came from the manufacturing world, where it was used to understand the flow of materials through a factory. In the DevOps world, we use value stream mapping to understand the flow of application artifacts through our software development and operations processes.
A value stream mapping is related to a process description, but differs in what information it tries to capture. When we create a value stream mapping we want to visually show the current state of the system as it stands today. We try to capture:
- Process steps — What are the steps in our current value stream?
- Personnel involved — Which people and departments are involved with each step? Who does what?
- Artifact flow — What artifacts (“material” in a factory setting) are created or enhanced at each step? What paths do artifacts take through the value stream? The paths are often non-linear, with both splits and joins along the way. Artifacts may include both final products (our application code, web pages, graphics, etc.) as well as intermediate artifacts (configuration files, test scripts, etc.).
- Information flows — What is the information flow between people and process steps? Information flows are often related to the artifact flows, but may not take the same path. Information flows may take place in formal, written forms (run-books, written approvals, ITSM change records, etc.) or information, often verbal forms (“Bob gives the release approval to Sally verbally at her desk.”)
- Ordering — Which steps and flows happen in serial and which in parallel? What is the order? We need to understand what happens first, second, third, and onwards.
- Delays — How long does each step take? What are the start and stop events that define the duration, so that we can get consistent measurements over time? What are the timings between steps (dead time), so that we can understand the end-to-end delay through the value stream? Sometimes this information is captured in a “lead time ladder” at the bottom of the diagram.
Show the value stream mapping in a graphical form, not just text. The goal here is to get the whole value stream down onto just a single (possibly large) page, or a couple of pages at most if your value stream is tremendously complicated. You want everybody involved in your DevOps project, as well as senior management, to be able to look at the value stream mapping and understand it without a lot of explanation. You want to be able to project the value stream onto the wall at team meetings. You want the value stream to fit on a slide or two for executive presentations.
In other words, your value stream mapping is not a checkbox artifact that you create and then stuff into a binder of project documentation, never to be viewed by a living being ever again. Instead, it’s the single most important document in your transformation effort. It’s a living document that should be updated whenever the value stream changes (see below).
So, draw a picture. Make it easy to understand and useful to everybody on the team. Use icons to represent parts of the value stream. Draw lines and arrows to connect the icons. Annotate the pictures with text describing the exact artifacts, departments, and delays involved.
There is absolutely no “right way” to do this.
Let me say that again: There is absolutely no “right way” to do this.
The goal here is not lean management purity, conforming to some official value stream mapping standard. Yes, you can Google for “value stream mapping symbols” and play with Visio all day long to get them set up. But if you do, you’re just procrastinating and avoiding the hard work of creating the diagram. You’ll also quickly find that most existing value stream symbol sets were designed for the manufacturing world and won’t have applicability to software development… unless your software gets produced in a factory and shipped on trucks.
So, ignore that and just start sketching on a piece of paper. You can clean it up with nice icons later.
The main questions you need to concern yourself with when creating your diagram are:
- Is it accurate? If your value stream mapping isn’t accurate, it’s a waste of time. You’re creating a false map of the territory, which is just as valuable as a false map in the real world — it won’t help you get where you want to go.
- Does it capture the necessary information to allow you to make decisions? You need to capture all the information that matters most on the mapping diagram itself. That doesn’t mean every single bit of information about your value stream needs to appear. That would overwhelm the diagram and make it cluttered. You just need enough to reason about the forthcoming transformations and optimizations of the value stream, but not so much that you can’t see the forest for the trees.
- And above all: Is it useful? Again, this should be a living document that gets a lot of daily use as you go through your DevOps transformation. If it isn’t useful, it won’t get used.
Go and See
When you start the process of creating your value stream mapping, you’ll quickly find out that people in the organization will disagree about what they think are the actual set of process steps and information flows in the value stream.
One person will say, “We do Step X first, followed by Step Y.”
Another will say, “No, that’s not true. We do Step Z first, then Step Y, then Step X.”
Which one is right? Who knows?
Rather than argue about it, use a technique from the Toyota Production System: go and see.
Walk around your organization and follow the flow of artifacts from point to point. Talk to the people who generate the artifacts and participate in the information flows. Gather information from them about the flows, ordering, and delays at each step.
Show your in-process value stream mapping to each of the people that you talk to and ask them if it’s accurate. If not, update it appropriately.
Over time, you’ll converge to a map of the true current state.
The key thing is to avoid ivory-tower discussions with people who think they understand how things work but who, in fact, hold outdated knowledge or are just plain wrong. Get out “into the field,” speak with the people really doing the work, and observe the process for yourself if at all possible.
Look for Waste and Constraints
Once you have an accurate value stream mapping, it’s time to start analyzing it.
You want to ask yourself a couple key questions:
- Where is the waste? “Waste” in this context means anything that doesn’t add value for the end user: waste of time, waste of human resources, waste of monetary resources, or waste of capital resources.
- Where are the constraints? What you’re looking for here is the “weakest link” in your process, the choke-point that slows down the entire system.
There is always some amount of waste in the system and there is always a constraint. Keep asking these questions until they are answered.
When you first start the process, you’re probably going to be overwhelmed with waste and you’ll find several points of constraint. The key here is to develop the list in priority order. It isn’t enough to identify a few candidates; you need to know which is the worst.
That’s particularly true with the list of constraints since the overall process can only go as fast as the worst-case constraint. Thus, if you spend time eliminating constraints that aren’t the worst in your system, it won’t have much of an effect on the end-to-end efficiency.
In another post, we’ll talk about a process to attack these problems. For now, just get a handle on your value stream mapping and develop your lists of constraints and waste.
Keep it Updated
As your DevOps implementation progresses, the value stream mapping should be the focal point of your strategic decision making. You’ll be coming back to it again and again over time and asking the same two questions as above:
- Where is the waste?
- Where are the constraints?
Once you’ve attacked the worst instances of waste and constraint, your value stream mapping will be out of date. It might have changed only a little bit (perhaps you reduced a single process step time), or it might have changed a lot (you reorganized the flow of application artifacts to make it more efficient).
So, update your value stream mapping to reflect the new current state. As you did originally, go and see the new value stream itself. Take measurements of the processing times at each step. Talk to people. Update your drawing and get their feedback.
Repeat Until Satisfied
Then repeat the process, attacking the (new) biggest source of waste and the biggest constraint. And again, and again. You can do this almost forever. Toyota is relentless about optimizing the value stream as part of the Toyota Production System, for instance, and they have been doing it for decades with no sign of reaching “the end.”
But, like most optimization problems, you’ll find that much of the gain comes quickly at the beginning, with diminishing returns over time. So, it’s your choice how far you take it. Even if you pause for a while to focus on other activities, it’s always good to keep the value stream map updated and review it periodically to see if anything has changed (technology, organization, priorities, etc.) and whether you want to optimize the value stream further.
No matter how far you take it, you’ll be doing DevOps and you’ll have a map of the territory. The map will help keep you focused and on-track.
If somebody wants to discuss a problem or the particulars of organizational dynamics, ask them whether it relates to the current biggest source of waste or the biggest constraint? If not, shelf the discussion for another day. It isn’t that these things aren’t issues worthy of discussion, but they aren’t the priority for today. At some point, those issues may be related to the biggest source of waste or biggest constraint and you can deal with them at that time. But not today.
The value stream mapping helps the whole team stay focused on the most important work items and it sidelines discussions that would otherwise take you off track. You’ll make faster progress, particularly at the start, and that will give you some “quick wins” that will help build organizational credibility for your DevOps project.