The Data-Driven problem

and the critical role of intuition in decision making

Photo by Emily Morter on Unsplash

Rethinking the Data-Driven trend

When David Beckham struck a free-kick, Roger Federer returned a serve, or Lewis Hamilton corrects oversteer at 120mph there is very little data-driven thought in the moment. That moment of skill is driven by intuition and instinct, which in turn is driven by hours and hours of purposeful practice and simulation, and perhaps millions of individual repetitions.

Imagine you could pause time. In that imaginary world, imagine you:

• Ask Lewis why he applied that much left hand down or throttle;

• Ask Federer why that angle of return or racket head speed, or;

• Ask Beckham why he placed his standing foot just so or that much spin to the ball.

In the moment. What answer do you think you’d get? I think it would be something like “because”.

It’s currently popular for organisations to become “data-driven”. That we should measure the decide decisions based on insights from ‘the data’, generating complex ‘data models’ to track current performance, and even to predict into the future.

Data > Information > Knowledge > Wisdom after all. (right?)

Before we get too deep into this, remember, I’m a big fan of data - and in the past I’ve evangelised the “data-driven” philosophy. However, there’s always space to re-think.

There are multiple problems with the data-driven approach which I will explore in this post. This is not to say data is bad, but to caution that - without care - it can lead you to errors or omissions.

So, where can the use of data lead us astray?

Measuring what doesn’t matter

Useful data is important. But data is not useful unless it has a purpose. Data can’t have a purpose if we can’t be entirely accurate about what is important to measure as an indicator or ‘signal’. You can’t know what signals we need unless you know what you’re trying to achieve.

Therefore, the first problem with a data-driven approach is ensuring that you are measuring matters.

It’s easy to fall into the trap of measuring what is easily quantifiable rather than what is truly important. For example, if you wanted to understand the penetration of a piece of software into a construction business it would be tempting to measure how many projects or people are using it. But would that tell you if that software was providing the value expected? No, not really. It would be better to measure speed or quality of target processes, and its magnitude, when that software is used - or perhaps users per day, per project.

Metrics should align with the core objectives and values of the organisation, project, or programme. Otherwise, they lead to misleading conclusions and misguided actions.

Taken further, what you measure can quickly become a target, and with the right rewards or consequence become incentives. Some thought needs to be put into the 2nd order effects - does it incentivise the behaviours or outcomes you’re looking for?

Why data models cannot predict the future: Small World vs. Real World, and the Role of Radical Uncertainty

There is a difference between the controlled environment of data models (the “small world”) and the complex, unpredictable nature of the real world. Models are simplifications and, while useful, they cannot capture every variable and nuance, and they are always past looking and stationary. Nassim Nicholas Taleb, in his book “The Black Swan,” highlights how rare and unpredictable events (Black Swans) can have massive impacts, often eluding data models.

No model, ever, can capture everything therefore it can never be the real world.

Radical uncertainty refers to situations where it is impossible to predict outcomes accurately due to the sheer number of variables and their interactions. The world and universe, and the science and behaviour which govern it are complex beyond comprehension. This concept is elaborated by John Kay and Mervyn King in their book “Radical Uncertainty,” where they argue that traditional financial models often fail because they cannot account for the inherent unpredictability of the real world. They emphasise the importance of embracing uncertainty and using judgment and experience alongside data to navigate complex situations.

This is because in any dataset there are gaps.

The Map is not the Territory

The best quick frame I have to remind my self of the ‘gapiness’ of data is “the map is not the territory” https://conceptually.org/concepts/the-map-is-not-the-territory

• Changing the map doesn’t change the territory.

• Though a map may be used to convey the terrain, it will never be so detailed as to show the stone you’ll turn your ankle on.

• We build new roads all the time, even more frequently we have storms which change the coastline forever. Google Maps is great, but it’s ALWAYS out of date in some way.

Bias and Stationarity

Data models can be flawed not only because they are simplifications but also because they rely on historical data. This can lead to a false sense of security, as past or present performance is not indicative of future results. The world is not stationary, things change, but data does not always reflect the underlying shift.

During the 2008 financial crisis, many models based on historical housing and mortgage data and logic failed spectacularly, leading to massive losses.

---

So, what do we do with all this data?

Data-Driven Rehearsal, Data-Informed Execution

While data is invaluable, the key is to strike a balance between data-driven and intuition-driven decision-making. Data should inform decisions, not dictate them. This approach allows for flexibility and the incorporation of human intuition gained through real world experience of the process or scenario.

In practice, this means using data to identify trends, generate hypotheses, and provide evidence, but ultimately allowing for human judgment and creativity to guide the final decision. According to this Harvard Business Review article:

“In today’s highly dynamic business world, many decisions are too complex to rely on metrics or gut feelings alone. The best leaders and decision makers use both data and intuition to their advantage”

Returning to the initial examples of Beckham, Federer, and Hamilton, I see a model of “data-driven rehearsal, data-informed execution.” These athletes use data extensively in their training. They analyse performance metrics, study opponents, and simulate countless scenarios. However, when it comes to executing in the moment, they rely on their instincts and the muscle memory developed through rigorous practice and repetition.

This blend of data and intuition can be applied to business. Data should guide and inform strategies, while intuition, experience, and creativity should drive execution. It’s about leveraging the strengths of both to make well-rounded, effective decisions.

Applying These Concepts to Construction Project Management

For managers in the construction space, the balance between data and intuition is crucial. The construction industry is fraught with complexities and uncertainties that data alone cannot capture. Here’s how the principles discussed can be applied…

Importance of Relevant Metrics

Just as in any business, you must ensure that the metrics you measure truly matter. The metrics you set should be relevant to the objective you’re trying to meet. Whether your objective is to improve safety, accelerate programme, or improve a process - what you measure must matter.

We now know that measures can become targets - which is the subject of Goodhart’s Law. This is expressed simply as: “When a measure becomes a target, it ceases to be a good measure.” In other words, when you set a goal, people will tend to optimise for that objective regardless of the consequences. This leads to problems when we neglect other equally important aspects of a situation. Therefore, make sure your target incentives the correct behaviours and actions.

Dealing with Radical Uncertainty

Construction projects often face unexpected challenges such as weather changes, supply chain disruptions, resourcing and quality issues. These are instances of radical uncertainty that models cannot predict accurately. John Kay and Mervyn King’s insights on radical uncertainty emphasise the importance of flexibility and adaptive strategies in managing these unpredictable elements.

Limitations of Models

While project management software and predictive models can provide valuable insights, they have limitations. Historical data will not always predict future project conditions accurately, especially for unique or unprecedented tasks. You should be wary of over-reliance on data models and ‘dashboards’, ensuring that you incorporate you’re observations and on-the-ground realities into your planning and execution.

Integrating Data with Intuition

Successful construction project management involves data-driven rehearsal and data-informed execution. During the planning phase, data and models should be used to forecast timelines, costs, and resources - as well as plotting out the optimal methodology. However, during execution, you need to rely on your intuition and experience to navigate day-to-day challenges.

For example, what was an optimal work sequence may change in the face of real-time events and observations. Therefore, you need to re-think and reset the planned method in light of this new information.

Construction professionals and managers, like top athletes, benefit from a blend of data and intuition. While data can provide a foundation for planning and analysis, intuition and experience are essential for making real-time decisions and adapting to unexpected challenges. By balancing these elements, you can achieve better outcomes and deliver successful projects.

So, what now?

Where does this leave us? While data is a powerful tool, it should not overshadow the importance of intuition and experience. Measuring the right things, understanding the limitations of models, and maintaining a balance between data and human judgment are crucial for effective decision-making. After all, even the best data can only take us so far; it is our ability to interpret and act on it that makes the difference.

Further Reading

1. Nassim Nicholas Taleb, “The Black Swan: The Impact of the Highly Improbable.”

2. John Kay and Mervyn King, “Radical Uncertainty: Decision-Making Beyond the Numbers.”

3. www.harvardbusiness.org: “Leading the way: Ideas and insights from Harvard Business Publishing Corporate Learning”