Friday, January 06, 2017

Comments by Andy Haldane, chief economist at the Bank of England, comparing economic forecasts to the famous failure of Michael Fish to predict the October 1987 hurricane have been seized upon by the media. The relevant part of Haldane's commentary comes in the 5 minutes from 15m30s in this video.

A number of points are worth making about this. First, the specific forecasting failure that Haldane compares to Fish is that of the financial crash leading to the Great Recession. Some media outlets have suggested otherwise. Haldane does comment on the Bank's forecasts for the post-referendum period and notes that the economy has been more resilient so far than had been expected, but he continues to expect a relatively tough year in 2017.

Focusing then on the major forecasting failure in 2008, he identifies two contributory factors. The first (extending the analogy with meteorology) is a lack of data. With better data, better forecasts can be produced. The second is arguably more fundamental. As Haldane notes, the forecasting models tend to work well when the economy is close to equilibrium, but perform badly during the (more interesting) periods following a shock. They clearly need to be redesigned, and indeed are being redesigned, better to accommodate such extreme events. Much effort since the crisis has gone into developing macroeconomic models to include imperfectly operating housing markets, and it is likely that this effort will contribute to more successful forecasting in future.

That said, economies are made up of people with free wills, and forecasting in this context can never become an exact science. The forecaster's tools - be they VAR models, neural networks, DSGE models or whatever - allow the evidence to be marshalled systematically in order to produce informed estimates of the likely time paths of key economic variables. But they are informed only by what is known at the time of the forecast, not magically informed by data that are unavailable. That said, data on the vulnerability of the sub-prime sector were available in 2007, and it is certainly fair to say that these should have been given greater heed in forecasts.

However, while many laypeople consider forecasting to be a major part of what economics is all about, that perception is misleading. Most economics is based on generating hypotheses that are then tested on historical data. This allows some stylised facts to be determined, and helps us understand a complex world - for instance: production quotas raise the price of oil; or restricting trade is harmful to growth. The body of economic understanding that has developed in this way over many years is in no way challenged by the fact that (in common with everybody else) economists lack perfect crystal balls.