Economics: Gurus fall short of the gold standard

IT IS the time of year for the Golden Guru award, the ninth such annual consumers' guide to Britain's economic forecasters. The competition aims to assess which forecasting group produced the best predictions of the economy in 1993, and to see whether the overall standard of this burgeoning industry is improving.

This year, the award is newly extended and improved with the help of Professor Robert Fildes of Lancaster University's Centre for Forecasting. We have retained the Guru index for the purposes of historical comparison, but we have added two other ways of comparing forecasts that improve on it.

The winner - the European Commission - comes top on all three measures and is also the first group ever to win the award for a second time. Although the IMF produced even better growth and inflation forecasts last year, it did not predict unemployment and therefore cannot receive a prize.

There is, though, a rather unusual feature of the winner. In the past, the various institutes have accused me of being unfair to them. They have argued that the City forecasting teams produce their predictions every month whereas the London Business School, National Institute of Economic and Social Research and other non- City groups predict only four times a year. By comparing each group's last forecast of the year prior to the year being forecast, the competition compares the institutes' forecasts based on October data with City forecasts based on December data, which inevitably makes for more accurate predictions.

However, timeliness is happily not everything. The NIESR did rather well this year, largely because it was spot on with its 2 per cent growth forecast. Moreover, the winning EC forecast was actually produced further in advance of 1993 than that of any other forecaster: most groups produced their predictions between October and December, whereas the EC produced its forecast in May.

The cynical might argue that the EC merely predicted what most groups predict will happen in the second year of their projections. This is generally a forecast that tends to be broadly in line with the underlying long-term trend of the economy to grow by 2.5 per cent a year. Given that growth was close to trend, the EC was arguably lucky.

Overall, 42 out of the 45 forecasting groups were over-pessimistic about growth, which was perhaps over-compensation for the excessive optimism of the year before. The typical (median) error was one percentage point. Most mainstream forecasters underestimated the severity of the recession and were determined not to be caught out again. There was a lot of worry about high debts.

Almost everyone was also too pessimistic about inflation. Only three groups out of 45 underestimated inflation. In general, the typical error was to overestimate inflation by 1.5 percentage points. This is odd: generally, you would expect that higher-than-expected growth would be accompanied by higher-than-expected inflation. So this is evidence that the growth- inflation mix is better than most economists expected.

Several factors account for the good news on inflation. The world economy was depressed, and the pass-through of rising import prices in the wake of the post- ERM devaluation in September 1992 was much less than most people had thought likely. This in turn helped earnings growth slow to 3.5 per cent in this cycle, compared with a low of 7.5 per cent in the previous cycle.

The mystery deepens when we look at unemployment, where only three out of 45 groups were over- optimistic. In general, the fall in unemployment during 1993 was completely unexpected. The typical error in the forecasts was to over-predict unemployment by 290,000. Even if the forecasting groups had got growth right, they would not have forecast such a big drop in unemployment. So the relationship between unemployment and growth has also changed for the better.

We can also shed some light on two other issues. In general, the City groups seem to have been better at forecasting inflation, while the non-City groups were superior on growth and unemployment. But since any conceivable theoretical view of the world intimately links all three, it is hard to place much confidence in the view that the City will continue to outperform on inflation.

As for the Treasury, there is scant evidence that it is regaining its mid-1980s pre-eminence. Despite all its research effort and its inside knowledge about the intentions of policy makers, it even failed to struggle into the top quartile of forecasters. It was nearly right on unemployment, but then it usually makes an optimistic assumption for its public spending projections. So it was probably right more through luck than judgement.

And now to a summary of the rules: the 45 groups are those whose output is monitored either by HM Treasury, or by Elsevier's Economic Forecasts, or by Consensus Economics' Economic Forecasts.

The idea is to compare each group's forecast for GDP, inflation and unemployment with the official out-turn. Care is taken to compare like with like: some groups forecast the consumer price inflation, some retail inflation and so on.

The Guru index is a simple addition of the difference between forecast and out-turn for growth, inflation and unemployment. The group with the lowest Guru index is the winner. However, this can lead to odd results if any one of the factors - inflation, say - is very different from most forecasts. In those circumstances, the wild outlying forecaster will tend to scoop the prizes even if it has performed badly on growth and unemployment.

Our preferred, first ranking is the simplest: we rank each group according to its accuracy in forecasting growth, inflation and unemployment. The smaller the difference between the forecast and the out-turn for growth, the better the rank. We then repeat the procedure for inflation and unemployment, and we add up the three ranks. The lowest score then wins, and the highest score comes last.

The second ranking compares each group's growth error with the typical (median) error for growth. Thus the EC's error on growth was 0.5 percentage points, and the typical error was 1 per cent. We then take the median of the three ratios, which measures the typical error made by that forecaster compared with other forecasters, and rank the results. This reduces the importance given to the large errors on inflation.

The practical difference between these two methods is shown with the scores for SGST (Societe Generale Strauss Turnbull). It performed poorly in the first ranking because it was badly wrong about inflation.

However, it performed very well in the second ranking, because its growth and unemployment forecasts were good, and its median ranking was therefore good.

The eagle-eyed will have noticed that Professor Wynne Godley, one of the Government's six wise men, is missing from the table. This is because the wise men did not produce their forecasts until February, and Mr Godley does not publish regularly elsewhere. His February forecast was for GDP growth of 0.5 per cent, retail inflation of 3.8 per cent and unemployment of 3.4 million. Had he been eligible, he would have ranked 45th. Two other wise men were also in the bottom half.

Since this is the first year of the new rankings, we can only compare this year's performance with past years by using the Guru index. The average index is a little better, at 3.06, than last year's 3.4. But the forecasting business is still a long way from the good 0.56 to 2.56 scores between 1985 and 1987 before the Lawson boom and bust.

Arts & Entertainment
Ricky Gervais at a screening of 'Muppets Most Wanted' in London last month
tvAs the second series of his divisive sitcom 'Derek' hits screens, the comedian on why he'll never bow to critics who habitually circle his work
News
news
Life & Style
Going down: Google's ambition to build an elevator into space isn't likely to be fulfilled any time soon
techTechnology giant’s scientists say there is no material strong enough for a cable from Earth into orbit
News
David Cameron sings a hymn during the enthronement service of The Most Rev Justin Welby as Archbishop of Canterbury, at Canterbury Cathedral last year
news
VIDEO
Life & Style
From long to Jong: Guy Pewsey outside Mo Nabbach’s M&M Hair Academy in west London before the haircut
fashionThe Independent heads to an Ealing hairdressers to try out the North Korean dictator's trademark do
Sport
Vito Mannone fails to keep out Samir Nasri's late strike
sportMan City 2 Sunderland 2: Keeper flaps at Nasri's late leveller, but Black Cat striker's two goals in 10 minutes had already done damage
Extras
indybest10 best smartphones
News
peopleRyan Gosling says yes, science says no. Take the A-list facial hair challenge
Arts & Entertainment
tvCreator Vince Gilligan sheds light on alternate endings
News
Paul Weller, aka the Modfather, performing at last year’s Isle of Wight Festival in Newport
people
Arts & Entertainment
Play It Forward: the DC Record Fair in Washington, US
musicIndependent music shops can offer a tempting alternative to downloads on Record Store Day
Sport
video
News
Supermarkets are running out of Easter Eggs
Deals make eggs cheaper than normal chocolate
Life & Style
Wasp factory: 1.3 million examples of the Vespa scooter have been sold in the last decade
motoringIconic Italian scooter still revving up millions of sales
Voices
voicesThe Ukip leader on why he's done nothing illegal
Have you tried new the Independent Digital Edition iPad app?
Independent Dating
and  

By clicking 'Search' you
are agreeing to our
Terms of Use.

iJobs Job Widget
iJobs Money & Business

Data Analyst - Financial services, Client data, LEI

£40000 - £50000 per annum + BONUS + BENEFITS: Harrington Starr: A leading, Cit...

Management Consultancy - Operational Research Analysts

£35000 - £50000 per annum + Bonus + Benefits: Pro-Recruitment Group: You must ...

Corporate Actions Consultant - Market data, ISO15022, presales

£45000 - £60000 per annum + BONUS + BENEFITS: Harrington Starr: Corporate Acti...

Prudential Risk/Operational Risk Associate - London

£350 - £400 per day: Harrington Starr: An opportunity has arisen at a FCA regu...

Day In a Page

Ricky Gervais: 'People are waiting for me to fail. If you think it's awful, then just don't watch it'

Ricky Gervais: 'People are waiting for me to fail'

As the second series of his divisive sitcom 'Derek' hits screens, the comedian tells James Rampton why he'll never bow to the critics who habitually circle his work
Google finds a lift into space will never get off the ground as there is no material strong enough for a cable from Earth into orbit

Google finds a lift into space will never get off the ground

Technology giant’s scientists say there is no material strong enough for a cable from Earth into orbit
Westminster is awash with tales of young men being sexually harassed - but it's far from being just a problem in politics

Is sexual harassment a fact of gay life?

Westminster is awash with tales of young men being sexually harassed - but it's far from being just a problem in politics
Kim Jong-un's haircut: The Independent heads to Ealing to try out the dictator's do

Our journalist tries out Kim Jong-un's haircut

The North Korean embassy in London complained when M&M Hair Academy used Kim Jong-un's image in the window. Curious, Guy Pewsey heads to the hair salon and surrenders to the clippers
A History of the First World War in 100 moments: A forgotten naval victory in which even Nature played a part

A History of the First World War in 100 moments

A forgotten naval victory in which even Nature played a part
Vespa rides on with launch of Primavera: Iconic Italian scooter still revving up millions of sales

Vespa rides on with launch of the Primavera

The Vespa has been a style icon since the 1950s and the release this month of its latest model confirms it has lost little of its lustre
Record Store Day: Independent music shops can offer a tempting alternative to downloads

Record Store Day celebrates independent music shops

This Saturday sees a host of events around the country to champion the sellers of well-grooved wax
10 best smartphones

10 best smartphones

With a number of new smartphones on the market, we round up the best around, including some more established models
Homelessness: Why is the supported lodgings lifeline under threat?

Why is the supported lodgings lifeline under threat?

Zubairi Sentongo swapped poverty in Uganda for homelessness in Britain. But a YMCA scheme connected him with a couple offering warmth and shelter
A History of the First World War in 100 Moments: When the world’s biggest shed took over Regent’s Park

A History of the First World War in 100 Moments

When the world’s biggest shed took over Regent’s Park
The pain of IVF

The pain of IVF

As an Italian woman vows to keep the babies from someone else’s eggs, Julian Baggini ponders how the reality of childbirth is often messier than the natural ideal
Supersize art

Is big better? Britain's latest super-sized art

The Kelpies are the latest addition to a growing army of giant sculptures. But naysayers are asking what a pair of gigantic horse heads tells us about Falkirk?
James Dean: Back on the big screen

James Dean: Back on the big screen

As 'Rebel without a Cause' is re-released, Geoffrey Macnab reveals how its star perfected his moody act
Catch-22: How the cult classic was adapted for the stage

How a cult classic was adapted for the stage

More than half a century after it was published 'Catch-22' will make its British stage debut next week
10 best activity books for children

10 best activity books for children

Keep little ones busy this bank holiday with one of these creative, educational and fun books