Why leaders need competence and confidence in complexity
Complexity is a ‘thing’
Leaders need competence and confidence in complexity! Complexity is very much a regular feature of the business landscape. It has been for decades but never as obviously as during these pandemic years. Few would deny its presence and impact now, even though barely a couple of years ago it hardly featured on people’s radars, let alone on daily business agendas.
That’s all changed now, which is just as well because working and managing in complex contexts is profoundly different from getting the job done in more ‘ordered’ contexts. Let’s take a quick look at some examples of the difference.
A quick distinction between ‘ordered’ and ‘unordered’ contexts
What do we mean by ‘ordered’ or linear contexts? These are contexts generally characterised by stability, certainty and predictability. Ikea’s furniture concept provides a good example. Buy the item which comes as a flatpack for easy delivery. Then, unpack the component parts, follow the step-by-step instructions and – voila! – you’ve constructed your piece of furniture. Anyone who follows that series of steps exactly can be confident they’ll build an item identical to all the others. It’s the classic rational-linear approach to problem solving that featured large in your school years.
Let’s shift focus now to an example of an unordered system. Acquiring and building your flatpack item took place over a few days. During that time the weather outside was quite changeable. Some of the time there was bright sunshine but at other times the winds gusted, thunder clouds swirled and it rained. You noticed that while the weather forecast for your region seemed fairly accurate, in your own location there was quite a bit of variation and plenty of uncertainty about exactly what conditions you’d face even a few hours ahead. The weather is an example of an unordered system, with its characteristic volatility, uncertainty, complexity and ambiguity.
Now, think about work (and life) during the past couple of years. Overall, it’s surely been more like the weather example and less like the flat-pack construction project!
Complexity requires a different approach
Dealing with complex challenges is necessarily not the same as the approach taken to resolving ordered problems. The way you approached the flat-pack construction was likely quite different from how you resolved your weather challenges. Since your past experience of both types of challenge would have kicked-in quickly, you probably didn’t even give a thought to the fact that the basic assumptions and approaches you were using differed. But, essentially, they were different problem types.
The ordered problem had you follow the recipe. You assumed all the parts had been supplied. You identified them all after unpacking and then put them together, in the right order, until the whole was built and looked like the picture (no guarantees though if you didn’t read the instructions!)
The weather, on the other hand, required you to continually monitor the context. You considered a number of contingencies and kept options open as far as possible. For example, you timed different activities based on current weather conditions. You intentionally kept a number of options available, ready to apply when the time seemed right for each. In one case, you took your rain jacket on your longer walk, just in case. And you’re glad you did, because there was a short light shower that hadn’t been forecast. You got a little damp but avoided the soaking that would have occurred if you hadn’t taken the raincoat, and you also got in your exercise. Your assumption of no showers failed but the consequences weren’t life-threatening. You hadn’t bet the house on the forecast. Clearly though, that more experimental approach is not how you approached your flat-pack construction – and just as well for that project type!
A false confidence in competence with complexity
While people believe problem solving capability is generally high, there is an often-unrecognised problem in how most people tackle complex challenges! Taking a step sideways, there is usually much greater competence (and associated confidence) when problem solving and decision making relates to ordered contexts. The mechanistic, rational-linear approach (used by engineers, for example) is so embedded in our thinking approach and problem-solving tools that we hardly give it a second thought. Assess the problem, break it into its component parts, tackle each of the parts methodically and eventually the problem is resolved! Rinse and repeat the process for any problem.
Mostly, when you apply all this to problems in ordered contexts, you get the outcomes you anticipate. Unsurprisingly, these successes reinforce the worth of the linear approach, methods and tools. Regular and repeated use turns them into habits: that desired brain state where you expend minimal cognitive effort to execute. It seems as if you do it almost automatically, without thinking.
So, even when the context isn’t well served by a rational-linear approach (complex contexts), it’s likely used anyway, out of habit. When things don’t work out as anticipated, a rational-linear problem analysis process will probably be used and it’s unlikely to uncover that problem context mismatch is a/the root cause of failure. Instead, there would be attempts to tweak the rational-linear approaches and processes next time round.
Recent McKinsey research on change projects in organisations confirms that only 30% of change projects are regarded by affected organisations as being successful. This 70% ‘failure’ rate of organisational change projects is one that has been consistently reported, in many different studies, over the past 50 years or more. It’s UGM’s firm belief that inability to recognise and deal with complex problems is at the heart of this problem. For many managers, there isn’t even recognition that the characteristics of complex contexts differ profoundly and fundamentally from ordered contexts.
Before going any further, let’s be clear that the rational-linear (‘engineering’) approach is no bad thing in the right context. Recognise though that it doesn’t have universal application to all problem types. It has, for example, been successfully deployed to lift living standards for billions. Thanks to these processes, used largely in manufacturing and construction, the general population has been able to enjoy products, produced cheaply and at scale, that were once scarce and only available to the super-wealthy. Incremental and rapid innovation, also in largely linear fashion, has delivered goods and services that would have been unimaginable even 50 years ago. But, as far as problem solving in organisations goes, a one size (approach) does not fit all, especially when there are different types of problem.
Wanted – new approaches!
Interestingly, a growing number of scientists are now questioning whether the fundamental (mechanistic) approaches to science, used by the likes of Newton (quantum mechanics) and Einstein (general relativity) have reached their limits. They explain a lot but, tellingly, not everything is as expected. Apparently, one size/approach does not fit all in this domain either!
Despite nearly a century of research, scientists have been unable to develop a master theory that unifies the two core threads of quantum mechanics and general relativity –i.e., for a mechanistic theory of everything. It is this absence of a universal theory that motivated a quantum physicist, Professor David Deutsch of Oxford University, to publish his embryonic Constructor Theory in 2012. It takes a radically different approach to physics, inspired by information theory and the theory of quantum computation. It embodies deep, radical and revolutionary thinking, highlighting that physicists recognise the limitations of their long-standing rational-linear tradition and are seeking novel explanations to progress. Constructor Theory is one such initiative.
Since there exists both a body of knowledge that ordered and unordered contexts differ fundamentally, and an awareness that complex contexts seem to feature most in organisation failure, managers ought to need little inducement to pay more attention to problem solving in complex contexts. But therein lies the crux of the problem. Their tremendous competence and experience in rational-linear problem-solving, supported by well-tested approaches, methods and tools gives rise to tremendous confidence in the rational-linear approach. Who wouldn’t lean in on that?
An additional barrier to a changed perspective is confirmation bias, where our brains are unconsciously driven to notice what corresponds with our beliefs and ignore the rest. We’re probably still at a point where many leaders don’t even recognise or acknowledge that there are distinctly different problem types, which differ so profoundly and fundamentally from one another that correspondingly different approaches are needed.
In our current contexts where volatility, uncertainty, complexity and ambiguity (VUCA) contexts are hallmarks, we need more than ever for leader-managers to embrace complexity. They will do that only once they develop a competence in dealing effectively with and in complex contexts. Then, over time, from this increased competence, they will develop confidence knowing that their changed approaches are more suited to working in today’s complex contexts. It may not be easy, and there’ll be stumbles along the way, but to better serve organisations and humanity those early steps towards competence and confidence must commence – sooner rather than later.
PRACTICAL IDEAS TO APPLY IN YOUR BUSINESS
5 tips for increasing competence and confidence in complex contexts
- Invest some time in understanding the difference between ordered and unordered contexts and, from that, how problem-solving approaches should vary. UGM recommends Dave Snowden’s continuously updated Cynefin framework as a stand-out in the field and a great starting point.
- Consider context/problem type early on in gaining an understanding of your problem, since how you approach the problem will depend on its type.
- Recognise when you are using linear problem-solving approaches (your likely default) and double check with yourself that the problem you’re trying to solve is not in the unordered domain. If it is, you’ll need to use context appropriate approaches, tools and methods.
- Examine your organisation’s approach to failure. Complex contexts are usually well-served by many safe-to-fail probes/experiments, rather than single ‘bet the house’ moves you might find in a more predictable linear context.
- Recognise and accept that, as a new and now rapidly emerging field, you are likely to find conflicting perspectives at times. The relatively young knowledge base itself can be considered volatile, uncertain, complex and ambiguous. Regardless, it’s from these emergent complexity-focused approaches, rather than the well-refined but inappropriate rational linear tradition, that complex problems will be best addressed into the future.