So far we have looked at two of the four key elements for real business agility - distributed decision making and execution efficiency. It's time now to look at the third element - measuring what matters. Organisations tend to collect a lot of data They measure a lot of stuff. The problem with many of those measurements is that they are often data that is easy to collect rather than data that is important. 

What's the problem with that? Data is data. If it's easy to measure, why not measure it? Having more data has to be better than less. Not necessarily. There is something important about making a measurement that makes it vitally important to measure the right things, rather than just measuring stuff just because you can. The important thing about making a measurement is that measuring drives behaviour. As soon as you measure something, people will naturally try to optimise that measurement and if you're measuring the wrong things, that can drive some very bad behaviour.

Ok. So I have a team and I'm looking at their board. On that board, if they are like 90% of the teams out there, they will have a burn down chart somewhere. Let's take a closer look at the burn down shall we? Does it, as so many do, show a flat line until the day they all update the tool at the end of the sprint? Does it show a nice progression of tasks done over the sprint but at the end of the sprint are there no stories completed because they worked on everything at once? 

The burn down is one of the classic tools for an agile team. Everyone does burn down charts. Mostly they do them badly. Over the years I have come to dislike the burn down chart for a number of reasons. Chief among those reasons is that it really doesn't do what it is supposed to - give an indication of whether the team is going to achieve its sprint goal. On the surface it looks like it should - there is a list of stuff to be done and a rate at which it needs to be completed to meet the date but in reality, the burn down doesn't give a very good indication of where things really are. The other big problem is that while they are supposed to be easy, most teams find keeping a burn down chart up to date a massive headache. So let's take a look at the burn down and some of the things that cause teams pain. We'll also look at a really good way to replace it completely.

As soon as you talk to anyone senior in an organisation about agility, one of the first questions that comes up is "How do we measure the benefit?" It's a perfectly valid question to ask. We are (usually) asking them for a bunch of money to implement change and quite often the benefits we describe are fairly poorly defined - better staff engagement, the ability to respond faster, faster time to market. That sort of thing. They all sound great but what really matters to an executive sponsor is the bottom line impact. Will spending this money generate additional profit? 

Unfortunately, the question of measuring agile benefit is a hard one. Other than time to market (which is actually a hard one to pull off for a bunch of reasons), all the traditional agile benefits are pretty soft. It's hard to come up with a set of hard, bottom line benefits we can easily measure and show the difference agile is making. This tends to make a lot of agile sales pitches a bit like the underpants gnomes in South Park - Step 1: Implement agile. Step 2:Ummmm. Step 3: Profit! While the ultimate metric is the profit, for the reasons I mentioned above, that's actually quite a hard metric to impact on the short term. What we need are a set of good metrics that can show progress towards increased profitability that we can measure and track in the short term. Don Reinerstein is an advocate of using an economic model to show benefits and while this is undoubtedly the right way to go, most organisations simply don't have the maturity to even consider things like cost of delay accounting until they are quite a long way into their agile/lean journey. What we need are things that any company can measure right from day one that we can use as leading indicators for increased profitability. Over the years I have come up with a set of metrics that seem to do the trick.

Last time we looked at the most common question we get from management -

What metrics will I get?

And saw that our usual answer - velocity - really isn't very good at answering that question. It's really only designed to be an internal measure for a single team. You can't use it to compare teams. I mentioned that Cycle Time is a much better metric to use for this sort of question, so let's take a look at Cycle Time. We will also look at its cousin - Lead Time.

When implementing agile in an organisation, one of the first questions management will ask is “what metrics will I get”. This is not an unreasonable question. They are financially responsible for their part of the business and are used to dealing with a particular set of metrics that let them assess the performance of their teams and take action if action is needed. RAG (Red Amber Green) status on projects, schedule variance, earned value, actuals vs estimates, risk profile, and so on.

When agile comes on the scene, those measures go away and management feels like they have lost control. So a question about agile metrics is perfectly reasonable. How do I, as a manager, see that my process is working well? How do I tell when things will be delivered? How do I tell how much stuff will cost? When answering this sort of question, the answer we most frequently turn to is velocity. Most agile systems have some concept of velocity – how much a team can produce in a given time. The problem is that velocity is a very poor way to measure those things at any scale beyond a single team. Velocity just wasn’t designed to answer this sort of thing. Why? Well to see why velocity isn’t a good measure for these questions, we need to look at what velocity is designed to measure.