Читать книгу Inside the Crystal Ball - Harris Maury - Страница 6

If we look at studies of forecast accuracy, we see that economic forecasters have one of the toughest assignments in the academic or workplace world. These studies should remind us how difficult the job is; they shouldn't reinforce a poor opinion of forecasters. If we review the research carefully, we'll see that there's much to learn, both from what works and from what hinders success.

Economists at the Federal Reserve Bank of Cleveland studied the 1983 to 2005 performance of about 75 professional forecasters who participated in the Federal Reserve Bank of Philadelphia's Livingston forecaster survey.7 We examine their year-ahead forecasts of growth rates for real (inflation-adjusted) gross domestic product (GDP) and the consumer price index (CPI). (See Table 1.1.)

Table 1.1 Accuracy of the Year-Ahead Median Economists' Forecasts, 1983–2005

^* Assigned by the author.

Source: Michael F. Bryan and Linsey Molloy, “Mirror, Mirror, Who's the Best Forecaster of Them All?” Federal Reserve Bank of Cleveland, Economic Commentary, March 15, 2007.

If being very accurate is judged as being within half a percentage point of the actual outcome, only around 30 percent of GDP growth forecasts met this test. By the same grading criteria, approximately 39 percent were very accurate in projecting year-ahead CPI inflation. We give these forecasters an “A.” If we award “Bs” for being between one-half and one percentage point of reality, that grade was earned by almost 22 percent of the GDP growth forecasts and just over 30 percent of the CPI inflation projections. Thus, only around half the surveyed forecasters earned the top two grades for their year-ahead real GDP growth outlooks, although almost 7 in 10 earned those grades for their predictions of CPI inflation. (We should note that CPI is less volatile – and thus easier to predict – than real GDP growth.)

Is our grading too tough? Probably not. Consider that real GDP growth over 1983 to 2005 was 3.4 percent. A one-half percent miss was thus plus or minus 15 percent of reality. Misses between one-half and one percent could be off from reality by as much as 29 percent. For a business, sales forecast misses of 25 percent or more are likely to be viewed as problematic.

With that in mind, our “Cs” are for the just more than 17 percent of growth forecasts that missed actual growth by between 1 percent and 1.5 percent, and for the 22 percent of inflation forecasts that missed by the same amount. The remaining 30 percent of forecasters – those whose forecasts fell below our C grade – did not necessarily flunk out, though. The job security of professional economists depends on more than their forecasting prowess – a point that we discuss later.

The CPI inflation part of the test, as we have seen, was not quite as difficult. Throughout 1983 to 2005, the CPI rose at a 3.1 percent annual rate. Thirty-nine percent of the forecasts were within half a percent of reality – as much as a 16 percent miss. Another 30 percent of them earned a B, with misses between 0.5 and 1 percent of the actual outcome, or within 16 to 32 percent of reality. Still, 30 percent of the forecasters did no better than a C.

In forecasting, as in investments, one good year hardly guarantees success in the next. (See Table 1.2.) According to the study, the probabilities of outperforming the median real GDP forecast two years in a row were around 49 percent. The likelihood of a forecaster outperforming the median real GDP forecast for five straight years was 28 percent. For CPI inflation forecasts, there was a 47 percent probability of successive outperformances and a 35 percent probability of beating the median consensus forecast in five consecutive years.

Table 1.2 Probability of Repeating as a Good Forecaster

^* Proportion expected assuming random chance.

Source: Michael F. Bryan and Linsey Molloy, “Mirror, Mirror, Who's the Best Forecaster of Them All?” Federal Reserve Bank of Cleveland, Economic Commentary, March 15, 2007.

Similar results have been reported by Laster, Bennett, and In Sun Geoum in a study of the accuracy of real GDP forecasts by economists polled in the Blue Chip Economic Indicators– a widely followed survey of professional forecasters.8 In the 1977 to 1986 period, which included what was until then the deepest postwar recession, only 4 of 38 forecasters beat the consensus. However, in the subsequent 1987 to 1995 period, which included just one mild recession, 10 of 38 forecasters outperformed the consensus. Interestingly, none of the forecasters who outperformed the consensus in the first period were able to do so in the second!

Perhaps even more important than accurately forecasting economic growth rates is the ability to forecast “yes” or “no” on the likelihood of a major event, such as a recession. The Great Recession of 2008 to 2009 officially began in the United States in January of 2008. By then, the unemployment rate had risen from 4.4 percent in May of 2007 to 5.0 percent in December, and economists polled by the Wall Street Journal in January foresaw, on average, a 42 percent chance of recession. (See Figure 1.1.) Three months earlier, the consensus probability had been 34 percent. And it wasn't until we were three months into the recession that the consensus assessed its probability at more than 50 percent.

Figure 1.1 Unemployment and Consensus Recession Probabilities Heading into the Great Recession of 2008–2009

Source: Bureau of Labor Statistics, The Wall Street Journal.Note: Shaded area represents the recession.

The story was much the same in the United Kingdom (UK). By June of 2008 the recession there had already begun. Despite this, none of the two-dozen economists polled by Reuters at that time believed a recession would occur at any point in 2008 to 2009.9

In some instances, judging forecasters by how close they came to a target might be an unnecessarily stringent test. In the bond market, for example, just getting the future direction of rates correct is important for investors; but that can be a tall order, especially in volatile market conditions. Also, those who forecast business condition variables, such as GDP, can await numerous data revisions (to be discussed in Chapter 5) to see if the updated information is closer to their forecasts. Interest-rate outcomes, however, are not revised, thereby denying rate forecasters the opportunity to be bailed out by revised statistics. Let's grade interest rate forecasters, therefore, on a pass/fail basis, where just getting the future direction of rates correct is enough to pass.

Yet even on a pass/fail test, most forecasters have had trouble getting by. As earlier noted, only 5 of the 34 economists participating in 10 or more of the semiannual surveys of bond rates were directionally right more than half the time. And of those five forecasters, only two – Carol Leisenring of Core States Financial Group and I – made forecasts that, if followed, would have outperformed a simple buy-and-hold strategy employing intermediate-term bonds during the forecast periods. According to calculations discussed in the article, “buying and holding a basket of intermediate-term Treasury bonds would have produced an average annual return of 12.5 percent – or 3.7 percentage points more than betting on the consensus.”10

In their study of forecasters' performance in predicting interest rates and exchange rates six months ahead, Mitchell and Pearce found that barely more than half (52.4 percent) of Treasury bill rate forecasts got the direction right. (See Table 1.3.) Slightly less than half (46.4 percent) of the yen/dollar forecasts were directionally correct. And only around a third of the Treasury bond yield forecasts correctly predicted whether the 30-year Treasury bond yield would be higher or lower six months later.

Table 1.3 Percentages of 33 Economists' Six-Month-Ahead Directional Interest Rate and Exchange Rates Forecasts That Were Correct

Source: Karlyn Mitchell and Douglas K. Pearce, “Professional Forecasts of Interest Rates and Exchange Rates: Evidence from the Wall Street Journal's Panel of Economists,” North Carolina State University Working Paper 004, March 2005.

Although it is easy to poke fun at the forecasting prowess of economists as a group, it is more important to note that some forecasters do a much better job than others. Indeed, the best forecasters of Treasury bill and Treasury bond yields and the yen/dollar were right approximately two-thirds of the time.

Some economic statistics are simply easier to forecast than others. Since big picture macroeconomic variables encompassing the entire U.S. economy often play a key role in marketing, business, and financial forecasting, it is important to know which macro variables are more reliably forecasted. As a rule, interest rates are more difficult to forecast than nonfinancial variables such as growth, unemployment, and inflation.

If we'd like to see why this is so, let's look at economists' track records in forecasting key economic statistics. Consider, in Table 1.4, the relative difficulty of forecasting economic growth, inflation, unemployment and interest rates. In this particular illustration, year-ahead forecast errors for these variables are compared with forecast errors by hypothetical, alternative, “naive straw man” projections. The latter were represented by no-change forecasts for interest rates and the unemployment rate, and the lagged values of the CPI and gross national product (GNP) growth. Displayed in the table are median ratios of errors by surveyed forecasters relative to errors by the “naive straw man.” For example, median errors in forecasting interest rates were 20 percent higher than what would have been generated by simple no-change forecasts. Errors in forecasting unemployment and GNP were about the same for forecasters and their naive straw man opponent. In the case of CPI forecasts, however, the forecasters' errors were only around half as large as forecasts generated by assuming no change from previously reported growth.

Table 1.4 Relative Year Ahead Errors of Forecasters versus “Naive Straw Man”

Note: Short-term and long-term interest rates and unemployment rates are relative to a hypothetical no-change straw man forecast. CPI and GNP growth rates are relative to a same-change straw man forecast.

Source: Twelve individual forecasters' interest rate forecasts, 1982–1991; other variables, 29 individual forecasts, 1986–1991, as published in the Wall Street Journal.Stephen K. McNees, “How Large Are Economic Forecast Errors?” New England Economic Review, July/August 1992.

There are many more examples of forecaster track records, and we examine some of them in subsequent chapters. While critics use such studies to disparage economists' performances, it's much more constructive to use the information to improve your own forecasting prowess.