Moving Beyond Problem Solving

One of the most highly prized skills these days is problem solving. Being known as a problem solver is an almost surefire way to guarantee success in whatever organisation you are in. As the great Vanilla Ice once said - "If there was a problem, Yo, I'll solve it". Mind you, he did also say "Word to your mother" which is somewhat less profound. Problem solving is indeed a very valuable thing, but at the same time a very limited one. Problem solving implies finding the root cause of an issue and solving it - making sure that it doesn't occur again. But what happens when the root cause of a problem isn't something that can be fixed?

Many of the problems that confront organisations (and society as a whole) these days are not problems where you can look at the problem, see the root cause and fix the system so the problem goes away. Many of the key problems we face are caused not by an easily fixed flaw in the system, but by the system itself. They aren't really problems at all but behaviour that emerges from the operation of the system. What I'm talking about, of course, is complexity. When a system behaves in a complex way, it exhibits unexpected behaviour and we see that as a problem. But you can’t just go in and fix a complex system. Any fix you make will itself have unexpected consequences. Complex problems can't be fixed. At least not in the conventional sense of finding a root cause and applying a discreet solution. What do problem solvers do when confronted by complexity?

At the moment, many of them try to apply the same thinking they do for complicated or simple problems - find a root cause, apply a fix and wait for the problem to go away. But complex systems don’t work like that. Cause and effect are not so easily determined. As Dave Snowdon says in his Cynefin sense-making system - in a complex system cause and effect can only be determined in hindsight - you can trace what happened between cause and effect in a problem that has already happened but you cannot accurately predict the impact of a change on the system in the future. So when these well reasoned fixes are applied to complex systems, they don't work as expected. They have unintended consequences. The fix does not fix the problem and may even make things worse. So what's a problem solver to do?

Complex "problems" are the emergent behaviour of a complex system. You can't fix complexity, the problem isn't one piece of the system, it is the whole system. So you can't "solve" complex problems. We need to add some extra words to our problem vocabulary to deal with complexity, words like simplification, adaptation and experimentation.

Simplification is taking the system as a whole and trying to simplify it until it no longer exhibits complex behaviour. Cutting the number of communications paths, the number of people, breaking up the system into smaller chunks that are isolated from each other. All these things will, if done with sufficient vigour, reduce the system to a complicated or simple system rather than a complex one. Of course, done with insufficient vigour to reduce the system's complexity, they will just take one complex system with known problems and turn it into another complex system with problems you don't know about yet.

Once you are back to a non-complex system, traditional problem solving will work just as it always has. Sounds great doesn't it? But be careful what you wish for. By reducing the system and killing its complex behaviour, you will also kill off other emergent complex behaviours like creativity and innovation. You may end up with a simple, easy to control system, but one that is incapable of actually achieving anything. You are simplifying the system essentially by removing its brain. 

So if simplification isn't the way to go, what is then? Adaptation is a good one. If you know the system exhibits a particular behaviour then you can wait for that behaviour to occur and deal with it when it does. If we find that the system always causes a huge bottleneck between development and production, we can just adapt to it - schedule more production folks on deployment days and pad our estimates to compensate for the known source of delay. If we know that the system tends to underestimate a certain type of work, then have an independent estimate done of that work so you aren't caught out. If you can't solve the problems in the system, you can make the system resilient to those problems.

Adaptation is all very well but it doesn't actually get rid of any problems. It adds extra steps to compensate. It adds to the complexity of the system. It’s wasteful and inefficient. Getting rid of the problematic behaviour would be a much better way to go. But how?  We can't analyse our way through a complex problem. We need to tweak the system so the behaviour we want to get rid of goes away. We do that essentially through trial and error, or to give it its scientific name - experimentation. We try lots of things. But in a scientific way. We make a guess at an intervention in the system, we make a guess on the effect it will have on the system. If you want to be really fancy you can call this guess your hypothesis. We try the intervention out in a small, controlled way, measure the actual impact and use that extra knowledge to come up with a new guess (hypothesis) about an intervention. We go around that loop lots of times until the system is behaving in the way we want.

It’s not nearly as easy and straightforward as traditional problem solving - see a problem, analyse, fix. This is see a problematic behaviour, make an initial hypothesis, do the experiment, measure the result and repeat as necessary. It’s not as easy or straightforward but it is effective. For a really good primer on where to intervene in complex systems, Donella Meadows, one of the founders of complex systems analysis, published an article a few years before her death, describing a series of key intervention points and how effective they are. It’s well worth a read -http://donellameadows.org/archives/leverage-points-places-to-intervene-in-a-system/ 

If you have a reputation as a problem solver, how about adding a few extra words to your problem vocabulary? Become a problem adaptor, a problem simplifier and a problem experimenter as well.