See The Whole - Avoiding Sub-Optimisation

Hi Folks

In my last post, I mentioned in passing the phenomenon of sub-optimization - where optimizing one part of a process negatively affects the throughput of the whole. This is covered by the seventh (and probably least understood) principle of Lean - See The Whole.

So how does sub-optimization arise? Let's look at an example - a fairly typical software development process where development passes work to testing for review. In many companies, development and testing are under different managers and in some cases completely different business units. In our example, the test manager is under budget pressure so he decides to reduce his headcount. This solves his budget problem so he looks good, but what happens to the development process? Developers are still producing just as fast as before but testing can't keep up. With less testers, work starts to pile up, feedback becomes slower, quality starts to drop and the development process starts to slow down. To make things worse, the increased bug fixing load prompts the development manager to increase head count to maintain output but strangely, this seems to make things even worse. What we have here is a classic (and unfortunately very common) case of sub-optimization. Let's look at an even more common scenario - our development and test teams are running close to capacity and there are new projects on the horizon, so the development manager gets approval for more headcount but testing doesn't (no budget). Suddenly, the testers can't keep up, work piles up, quality drops, etc, etc. Sub-optimization again.

Any process, whether it is manufacturing widgets or writing software consists of a number of individual steps. Each step takes inputs from the step before and send outputs to the step after. For work to flow efficiently through the process, lean tells us that the rate of output from one step should match as exactly as possible, the rate at which the step after requires its inputs and so on down the line - each step should match its outputs to the step after. If inputs and outputs are unmatched you get waste. I'll reveal my electrical engineering background here and make the comparison to matching the impedances of a transmitter and receiver. For maximum power transmission the impedances have to match. If they don't there will be losses.

If you take one process step in isolation and adjust its efficiency without considering the steps up and down stream, you will break the smooth flow and cause waste. That's pretty easy to understand. Why then do I say that See The Whole is the least understood lean principle? The difficulty many organizations have is that they never grasp what the whole is. In our example above, the development manager sees their "whole" process as the process from receiving requirements to handing over to testing. The test manager sees their "whole" process as the process from receiving code from development to handing over to production. In our examples each manager has optimized their "whole" process and disaster has arisen. "Ah-ha" we say. That's easy. The development process and test process are really two halves of a larger process. They should be combined. And indeed they should. Most organizations can make that leap and most do. The trouble is that this isn't really the whole process.

In Lean thinking, all processes start and end with the customer. From customer request (or value proposition) through to successful delivery and maintenance, software development is only a tiny part of that larger process. There's a whole bunch of other stuff that goes on right across the organization that is part of the larger "concept to cash" process. That whole process needs to be considered and made to flow smoothly, not just the bit labelled "development". That's what most organizations fail to understand.

Have you ever spoken to an organization that boasts of its efficient development process but had 15 years worth of customer requests in their backlog? Or a company with lots of releases but a 5 year backlog of customer bugs in their support organization? Or a company that has a backlog of product that is developed but not rolled out to market yet? I worked for a company once where the marketing organization could only handle 5 product releases a year. The development team had been optimized (after a senior management order to become more efficient) to provide 4 releases/year across 5 products. That's 20 releases a year. Into a process that could only handle five. That's 75% of their output that was never released to market. The extra efficiency in development cost the company a lot of money but delivery to the customer actually slowed down due to the overworked marketing people. If the increased development budget had been shared with marketing, the flow could have been evened out. There would have been fewer releases per year from development but all of them would have actually made it to the customer.

For a true, whole process view, you need to consider the whole process - starting and ending with the customer. From Concept to Cash. The process cuts across organizational barriers. It has sections under different managers and business units. It cuts across budgets and P&L's. In most organizations this make it very difficult to optimize it as a whole but it needs to be optimized as a whole. Not as independent pieces. Because those pieces aren't really independent. That's what most organizations fail to understand. See the whole isn't about optimizing your process. It's about optimizing your whole organization. From concept to cash.