As soon as someone starts looking at the topic of metrics, the measurement fallacy pricks up its ears (I always imagine it looking somewhat rodent-like with mangy fur, evil eyes and sharp teeth) and gets ready to emerge from its hole behind the database server. When people start discussing what should be measured in order to keep track of a process, it gets ready to strike. Most people have fallen prey to it at one point or other. Mostly without ever knowing they have been bitten. The bite is painless. The only symptom is that the bitten suddenly assumes that because we can measure something, it must be important. More serious cases assume that the easier something is to measure, the more important it must be. This dreadful scourge is responsible for making Lines Of Code the primary measure of developer output for years.

It's a typical case of a severe bite - we can measure lines of code. Therefore it must be an important measurement. It's really easy to measure so it must be a really important measurement. Therefore we must measure it and use it to drive developer behaviour. Once it sets in, it's hard to shift. Despite the fact that the behaviour it drove - writing masses of wordy code to inflate your LOC counts and never, ever remove code - was completely counterproductive, the LOC (or KLOC) still hangs around to this day.

When you work in a large company, one of the things you hear quite often is “we have to follow the process”. Large companies, for very good reasons, have a need to standardise their processes. If you have 50,000 staff, having one way to do things makes a lot of sense. No matter where someone goes in the organisation, the process for ordering a new pen, or whatever, will be the same. The problem with defined processes though, is that unless they are regularly reviewed and cleaned up, they tend to accumulate complexity. Each time something happens that is just outside the normal way the process works, someone will add some extra checks into the process to make sure that that situation is now covered. Over the years it will collect enough of these extra checks that your carefully considered and streamlined pen ordering process now requires a 10 page form, 15 signatures and about 4 hours (and in some companies a pint of cockerel’s blood). The end result is that everyone spends all day looking for pens.

In the last post, we looked at estimating by essentially not estimating. To do that we broke down stories into two categories - small and the rest. Small stories were ready to be accepted into the team's backlog, the rest were too large and need to be broken down further. By doing this, velocity becomes just a count of stories completed and all the hassles involved with story point estimation just go away.

To me, this is a real no-brainer. Why wouldn't you estimate this way? But whenever I mention this in polite company, I tend to get some uncomfortable silences, strange looks and the inevitable - "but....". These buts, tend to come in three types -

Last time we looked at T-shirt sizing and some of the benefits and problems that method has. We found that its greatest benefit was also its biggest disadvantage. The use of something completely abstract (T-shirt sizes) removes all our cognitive biases around numbers but by not using numbers we can’t really compare estimates against each other and make predictions except by converting back to numbers which of course brings our biases back.

We can use T-shirt sizes usefully if we make an adjustment to the scale we use. Rather than have Small, Medium, Large and Extra Large, let's just have Small and Extra Large. Now, this would obviously never work for clothing because people come in a range of sizes. Stories come in a range of sizes as well, so what gives? What makes this useful? The trick here is that unlike people where we can’t dictate what size someone should be (outside the modelling industry and certain trendy nightclubs), we can, and should, be pretty strict about what size a story can be before we accept it onto a sprint.

Last time we started to look at relative estimates and the most common method of relative estimation using story points. We looked at why they work well but also at some of their limitations. The biggest limitation is the fact that they are numbers and we have some built in cognitive biases when it comes to numbers. We mistake precision for accuracy and tend to agonise for ages over the story point numbers which turns story points from a fast, lightweight and accurate method of estimation into a slow, heavyweight and accurate method. It's still accurate but we waste a lot of time.

There is a way to keep the accuracy of story points but remove the cognitive biases we have around numbers. It’s a simple as not using numbers in our estimates. The usual way to do this is by using T-Shirt sizing – stories are small, medium, large or extra-large. Some teams go a bit further and add Extra Small and XXL but we’re getting into false precision there so I would recommend against that.

Last time we looked at the concepts of accuracy and precision and how getting the two mixed up can lead to all sorts of problems. We also looked a little at our cognitive bias, that has us assuming that precise numbers are also automatically accurate. The upshot of that is that we humans are absolutely terrible at estimation. We mistake precision for accuracy and our accuracy is really bad to begin with.

That last statement is only half true. We are really, really bad at things like guessing how many jelly beans are in a jar, or how tall that person is, or how much does that thing weigh. What we are bad at is absolute estimates. To make up for that, we are really, really good at relative estimates.

Last time we looked at why we estimate and why there is always pressure to make our estimates more accurate. We have come up with a vast number of methods for estimation all of which aim to improve accuracy. The problem is that most of them don't. What they improve is precision instead.

Most people think of accuracy and precision as being the same thing. But they aren't. My nerdy and pedantic engineering background tells me that accuracy is how close to the true value a measurement is, while precision is a measure of how reproducible the measurement is. A more formal definition (thanks to Wikipedia) is -

A couple of posts back I mentioned Estimation and my desire to poke a stick at the hornet's nest that estimation can be. Estimation is always a controversial topic. It's often at the heart of serious conflicts within organisations. There are a huge number of estimation methods and techniques but nothing seems to prevent these issues from coming up. Before I poke a stick into the hornet's next (well, not so much poke as take a full bodied swing with run up and follow through), I'll spend a little while looking at why we estimate in the first place.

Any time we have two parties involved in something there is estimation happening. Right back from prehistoric times -