Over the last few posts we have been looking at the key changes I feel are necessary for an organisation to be agile, rather than just do agile. We have looked at distributed decision making, execution efficiency and measuring what matters. It's time now to cover the fourth key change - inspect and adapt.

This is probably the hardest of all the four changes for an organisation to adopt in anything but the most superficial of ways. By adopting inspect and adapt, they are not just adopting the need to continuously improve. They are also adopting a view of the world that is fundamentally non-deterministic. Where uncertainty is not just normal, but accepted and even embraced. Where long term plans give way to rapid experimentation. This may be a step too far for many organisations.

Most organisations react to the world in a deterministic way. Something happens, we analyse or categorise what happened, we form a plan, we execute the plan. The world operates in predictable ways. We can make long term plans because cause and effect can be determined up front and because actions have predictable, repeatable outcomes.

There is some acceptance of unpredictability, but unpredictability is to be minimised. We need to plan better. Anticipate better. Cover all our bases. Have contingency plans. There will be events that occur that derail our plans but these are rare, large and catastrophic - natural disasters, market crashes and so on.

The problem is that unpredictability isn't the exception - it is the norm. Most organisations are operating in a complex environment and the distinguishing feature of complex environments is that you can't make predictions about the future. Cause and effect can't be determined in advance - you can analyse all you like, but what you predict will happen often isn't what actually happens. What happens is something completely unexpected.

In hindsight, what happened is obvious. You can trace the effect back to the cause easily. You can compare that to your prediction and see what you got wrong but that doesn't do you much good for next time because, no matter how much you analyse, a complex environment will always catch you by surprise.

So what can you do? If you can't predict the future, how can you plan? The answer is - inspect and adapt. Rather than try to predict the future in advance, try something, see how it works and adapt from there. Dave Snowdon, author of the Cynefyn framework, calls this method - Probe, Sense, Respond.

Probe - try something. Preferably something small and fast. A quick experiment.
Sense - see how the system responds. Did it behave as expected? Did something unexpected happen? 
Respond - adjust what you are doing based on the response of the system. Is what you are doing working? If so, then keep going. If not, try something else.

We tend to think of inspect and adapt in an agile environment as something to do with retrospectives - inspect what you are doing and adapt to improve. It's more than that though. Inspect and adapt sounds like a simple concept, but what it's really asking you to do is accept that the world is a fundamentally non-deterministic place. Inspect and adapt, or probe, sense, respond, becomes the primary way you run your enterprise.

Don't try to analyse and predict what the market will do, Run an experiment - make a prediction. Run a quick experiment and see how the market responds. Use that to run the next experiment. Don't plan a cultural transformation in advance, run a series of transformation experiments. See what works and what doesn't. Then use that knowledge to run another series of experiments. Rinse and repeat.

The key to responding in a complex environment is experimentation - the ability to run a series of small, fast experiments to see how the system responds. This is a real problem for most organisations. They are geared around large, complex, slow programs of work, not quick experiments. They are set up to build large, complex, fully-featured products that they release to market in a very controlled way. All their systems. All their processes. Their entire corporate culture is set up around large, predictable (or the illusion of predictability) programs of work.

In this sort of environment, running small experiments is extremely difficult. Any attempt to do something quickly, something simple, something not absolutely polished and complete, is blocked by the organisation's own systems and processes.

Inspect and adapt is probably the hardest change an organisation can make. It sounds simple, but what it is really asking the organisation to do is to switch from a deterministic view of the world to a fundamentally non-deterministic view of the world. That's a massive change in the way we see things. To make the switch means fundamentally re-engineering the entire organisation around small, fast experiments to probe the environment, sense the response and adapt to that response quickly.

To do that is an enormous effort. To make things even harder, the organisation itself is a complex environment, so you can't come up with a plan to change mindsets and execute on that in the way we run most change programs (or try to Ru most change programs). The change to an organisation built around inspect and adapt has to be done through inspect and adapt.

If the organisation can make the change, it will allow them not just to respond to complexity, not just to survive in a complex environment - it will allow them to harness that complexity and channel it into successful products and services that really meet the needs of their customers.

This is a change that most organisations will fail to make. But those that can manage the change will dominate those that can't. As environments become more and more complex, the organisations that survive will be those that navigate the change in mindset and start exploiting complexity instead of trying to fight it.