$8 billion was erased from Zillow’s market cap, alongside 2,000 jobs, due to the disastrous predictions made by its AI-powered Zestimate home valuation model, which failed to account for pandemic-driven housing market trends. As AI models become increasingly relied upon for high-stakes decision-making, it is imperative to understand the limits of predictive versus causal analysis — and when each is appropriate.
Implementing these techniques in Actable, a no-code data science tool, can help users understand the uses of predictive versus causal analysis.
Predictive analytics has been heralded as a powerful tool for making business decisions. It is often used to identify future trends and optimize outcomes by understanding historical data patterns. For example, retail giant Amazon has utilized predictive analytics for years to improve customer targeting and product recommendations.
Similarly, hospitals have used predictive analytics to identify at-risk patients and readmissions, while banks have employed the technique to detect fraud and prevent financial crimes. In each of these cases, organizations were able to utilize data they already had to make better decisions about the future.
Predictive analytics is best suited for use-cases where trends can be identified and modeled based on historical data. The technique is less effective in situations where there are too many variables at play or when relationships between variables are not well understood. In these cases, causal analysis may provide a better understanding of how different factors influence each other.
In the future, self-driving cars, which are already utilizing predictive analytics to some extent, are expected to become increasingly reliant on the technique as more data is collected.
However, predictive analytics alone is often not enough to make sound business decisions. This was made painfully clear by Zillow’s failed predictions about the pandemic’s effect on the housing market.
Causality must be taken into account when making predictions about the future, otherwise we run the risk of making decisions based on spurious correlations. A causal analysis estimates the effect of a treatment after controlling for different confounders, exposing spurious correlations that should not be used in training a robust predictive model. For example, the homeowner’s age could be correlated to house prices historically, but have no correlation once the workforce dynamic changes (e.g. young people enter the tech industry and earn more).
As a home owner’s age has no causal impact on a house price, predictive models will be more robust if trained without owner ages. Instead, a cause-and-effect analysis might reveal insights into how the house’s area, location, or the economy impact the house price.
Organizations can use causal analysis to understand how different factors influence each other and make more informed decisions about the future. In the case of Zillow, a causal analysis would have revealed that the pandemic would lead to a decrease in demand for housing, and that prices would consequently drop. However, other factors such as the skyrocketing costs of materials and labor might have blindsided the company, leading to greater losses than predicted. This information could have been used to adjust Zillow’s predictions and avoid the billions of dollars in losses suffered by the company.
As we enter a lower-return, higher-inflation future, marked with increased economic uncertainty, the ability to accurately predict causality will become even more valuable. Investors will need to be able to identify which industries and companies are likely to be impacted by various macroeconomic trends. For example, a causal analysis might reveal the extent to which an increase in interest rates will lead to a decrease in demand for new homes.
This information can then be used to make more informed investment decisions, like whether to buy or sell shares of a homebuilding company. As AI models become increasingly relied upon for decision-making, it is imperative that we understand the limits of predictive versus causal analysis — and when each is appropriate.
Both predictive and causal analysis have their place in business decision-making. However, as we enter an age of increased economic uncertainty, the ability to accurately identify causality will become increasingly important. AI models must be designed with this in mind if they are to provide accurate predictions and avoid costly mistakes.