Data-Driven Decisions Start with These 4 Questions

Data-driven decision making

Data has become central to how we run and operate our organisations today. The global market intelligence firm International Data Corporation (IDC) has projected spending on data and analytics to reach $274.3 billion by 2023.

However, is this money being spent wisely? It has been estimated, based on case studies, that 85% of big data projects fail. A big part of the problem is the numbers that show up on a computer screen. Once data is pulled in through complex databases and analysed further through analytics software, we rarely ask the source, how it’s been modified, or whether it’s fit for the purpose. The truth is that to get useful answers from data; we can’t just take it at face value. We need to learn how to ask thoughtful questions. In particular, we need to know how it was sourced, what models were used to analyse it, and what was left out. Most of all, we need to go beyond using data simply to optimise operations and leverage it to imagine new possibilities.

How was the data sourced?

In real-world events, such as transactions, diagnostics, and other relevant information, are recorded and stored in massive server farms. However, few bother to ask where the data came from, and unfortunately, the quality and care with which data is gathered can vary widely. In fact, a recent Gartner study recently found that firms lose an average of $15 million per year due to poor data quality. Often data is subject to human error. However, even when the data collection process is automated, there are significant sources of error, such as intermittent power outages in cellphone towers or mistakes in the clearing process for financial transactions. Data that is of poor quality or used in the wrong context can be worse than no data at all. In fact, one study found that 65% of a retailer’s inventory data was inaccurate. Another concern, which has become increasingly important since the EU passed stringent GDPR data standards, is whether there was proper consent when the data was collected. So don’t just assume the data you have is accurate and of good quality. You have to ask where it was sourced from and how it’s been maintained. Increasingly, we need to audit our data transactions with as much care as we do our financial transactions.

How was it analysed?

Even if data is accurate and well maintained, the quality of analytic models can vary widely. Often models are pulled together from open-source platforms, such as GitHub, and re-purposed for a particular task. Before long, everybody forgets where it came from or how it is evaluating a particular data set. As models become more sophisticated and incorporate more sources, we’re also increasingly seeing bigger problems with how models are trained. One of the most common errors is over-fitting, which basically means that the more variables you use to create a model, the harder it gets to make it generally valid. As we do with data, we need to be asking hard questions about our models constantly. Are they suited to the purpose we’re using them for? Are they taking the right factors into account? Does the output truly reflect what’s going on in the real world?

What doesn’t the data tell us?

Data models, just like humans, tend to base judgements on the information that is most available. Sometimes, the data you don’t have can affect your decision-making as much as the data you do have. We commonly associate this type of availability bias with human decisions, but often human designers pass it on to automated systems. So it’s important to ask hard questions about what your data model might be missing. If you are managing what you measure, you need to ensure that what you are measuring reflects the real world, not just the data that’s easiest to collect.

How can we use data to redesign products and business models?

Over the past decade, we’ve learned how data can help us run our businesses more efficiently. Using data intelligently allows us to automate processes, predict when our machines need maintenance, and serve our customers better. It’s data that enables retailers to offer same-day shipping. Data can also become an important part of the product itself. To take one famous example, Netflix has long used smart data analytics to create better programming for less money. This has given the company an important edge over rivals like Disney and Warner Media. It’s been said that data is the new oil, but it’s far more valuable than that. We need to start treating data as more than a passive asset class. If used wisely, it can offer a true competitive edge and take a business in completely new directions. To achieve that, however, you can’t start merely looking for answers. You have to learn how to ask new questions.

In Summary

To summarise, we need to treat data as the lifeblood of an organisation. The quality and time-bound fashion of data is key. We need to remind ourselves and not lose sight of the following questions:

1. How was the data sourced?

2. How was it analysed?

3. What doesn’t the data tell us?

4. How can we use data to redesign products and business models?