Before You Start Changing, Measure Where You Are

What's the first thing you do when you look at a map? Find your destination? Maybe. Start planning a route? Sounds logical. But there is something missing. One fundamental step that renders the other two useless. That first step is locating where you are. Obvious really, but essential. Unless you can position yourself accurately on the map, no amount of accuracy in destination identification, or time spent in route planning, will get you where you want to go.

That's obvious when looking at a map. Very few of us (my mother excluded) will locate our destination then confidently set off without working out where we are now. My mother, on the other hand, will locate her destination, see that it is on the left hand side of the map and confidently set out towards the left. Consequently her excursions often end up in interesting places. Trouble is, the same principle applies to organisational change and in that context, very few of us perform the first step. We jump straight into desired state, plan a few actions and off we go. We don't spend much time, if any, on step one. We don't measure where we are first. The result is exactly the same as looking at a map without locating yourself on it. You will start off confidently in a random direction and end up... somewhere. If it's at your intended destination that will be by good luck (or the help of someone you asked for directions) rather than good map reading.

Anyone who has been involved in an organisational change program will know that one of the biggest problems you will face is showing benefits. Most of the time you are trying to drive the organisation in a way that its current systems were not set up to measure. Agile programs are a perfect example of this. How can you measure success of an agile transformation when the only organisational metrics available are the old waterfall metrics like schedule variance? How do you measure things like faster time to market when the organisation has no idea how to measure cycle time? What about behavioural changes? How do you measure them? The metrics almost certainly don't exist.

The reason change programs run into this problem is that they almost always forget step one. Find out where you are first. If you are going to show benefits, you need to measure your current state, so you have something to compare with. A lot of change programs don't do this. One of the big reasons for not doing it is that data just doesn't exist in any form that makes sense for the change program, so they give up and just dive in to making the change. They will say something like "we will create the measures we need as part of the change program" and dive straight in. This is a big mistake.

You have just made it impossible to show benefits. You can create all the measures you like but all you can measure is the new state. You have nothing to compare to. You can't show that new is better than old because they are measured in different ways. Even worse, you can't discover that old is better than new - that the change isn't working. We always assume that the change will be positive (if we are the ones that proposed it) and we base everything on that assumption but we have no way of proving (or disproving) it.

The very first thing you need to do when looking at a change is step one. Measure where you are. Work out what measures you want to use. Work out whether the data exists (probably not). Go out and create any data that is missing. Measure everything against those new measures. Now we know where we are. Now we can look at starting the change and measuring its effects.

Before anyone feels the need to point this out, yes, this will slow down the change. Measuring a baseline can take some time. We are all keen to jump towards the new state but the investment in time is well worth it. It doesn't even have to take much time. Not if we are clever about it.

We may not have the data we want, but we may have data we can use to derive the measurement we want. Take cycle time. We may not have a cycle time measurement, but rather than build one and wait for a bunch of projects to finish to build a baseline, can we approximate a cycle time metric by creative use of project start and end dates (which the organisation almost certainly has)? Of course if we do this, we need to be completely transparent about what we are doing and get agreement from the organisation that the derivation is valid. We want to avoid accusations of cooking the books to make our change look good.

We can also run change experiments. Build the measures we want and start measuring everything against them. Then change a small part of the organisation to the desired state. This way you are collecting baseline data at the same time that you are collecting new state data which speeds things up enormously. Again, complete transparency is required to avoid accusations of cherrypicking the bits you change to make the results look good. You could even run a randomised trial if you want to get really scientific - each project gets either old process or new process, based on the flip of a coin.

So, just like setting out on a trip, we need to work out where we are before we set out towards our destination. Otherwise we run the risk of ending up in an... interesting place, rather than where we were trying to go.