BOOK REVIEW
Understanding Economic Forecasts
Edited by
David F. Hendry and Neil R. Ericsson
Page Contents
FUTURECASTS online magazine
www.futurecasts.com
Vol. 8, No. 8, 8/1/06
Unfortunately, major unpredictable "shocks" occur all too often  resulting in major forecasting failures, especially for the nearterm forecasts that are so important for policymaking, business and investment purposes.
"Structural breaks appear to explain why it is so hard to reduce forecast error standard deviations: the outcome is sometimes very far from the forecast." 
Forecasting uncertainties arise from
known factors that can only be established as probabilities, and factors that
"we don't know that we don't know." Unpredictable events will always
intrude on the future.

Data that is frequently subject to substantial subsequent revision presents additional problems for shorter term forecasts. 
Forecasting models are far from accurate
representations of complex, dynamic modern economic and commercial systems. Even
models that attempt to closely represent the economy and that provide reasonably accurate
forecasts for one or more periods may suddenly prove unreliable thereafter.
Abrupt shifts arise from technological developments, political turmoil,
legislation and regulation, and similar factors. 
Forecasting failures under or over the expected result are not unusual, but they should not be permitted to persist for a significant duration. 
Forecast models thus must "adapt" to
sudden shifts to minimize the duration for forecast failures. Forecasting
failures under or over the expected result are not unusual, but they should not
be permitted to persist for a significant duration.

Econometric models depend on the accuracy and proper inclusion of significant "deterministic terms" (average levels and yearly or quarterly or other periodic trends)  "observed stochastic variables" (known variables)  and "unobserved errors" (estimates or averaging out of unknown variables).


The most important task is to develop "forecasting models that are more robust to shifts," rather than making improvements in the models themselves. 
Actually, the worst econometric forecasting failures
are caused by unforeseen shifts in the values of average levels or period trends, rather
than shifts in variables. Thus, the unsophisticated models that do not depend on
data that may be erroneous or trends that can shift can often perform better
than more sophisticated models that get tripped up by major variances in the
wide variety of factors that they include. 
Forecasting methods covered by Hendry, excluding guesswork and hunches, include econometric systems, timeseries models, surveys, leading indicators, and extrapolation.
Professional economists generally rely on timeseries and econometric models. These are "causal" models based on the study of underlying causes.


Economic forecasts generally are both multiperiod and
multivariable. They will sometimes miss a rate of change even when they get
the forecast levels right  and vice versa. Therefore, judging the quality of a
forecast depends on what the forecast is being used for. & 

"Confidence intervals" are established according to the calculation of likely outcomes during a certain period of time.
Fan charts and other displays of confidence intervals usefully dispel the illusion of precision when a forecast is presented as a particular number. However, even fan charts fail to reflect uncertainties that "we don't know that we don't know." All forecasting is tentative in nature. 
An estimate of
the uncertainty involved will frequently be included in economic forecasts. These "confidence intervals" are established
according to the calculation of likely outcomes during a certain period of time.
(The chance of rain tomorrow will be x%.) Confidence intervals generally widen as the
forecast time horizon lengthens.

"We don't know the true structure of the economy, the true values of parameters in econometric models, or the exact shocks that hit the economy; and we may never obtain accurate data measurements either." 
Uncertainty is inherent in economic forecasts. With commendable candor, Coyle perceptively explains some of the most prominent aspects of the problem.
Forecasts are generally reported with "spurious precision," Coyle points out. Words like "around," "approximately," "probably," and "might" are not favored. Instead, dramatic terms are employed that routinely overstate the implications of events. Unfortunately, the media generally fail to reflect or explain the uncertainty inherent in economic forecasts.
Ranges of uncertainty are provided with UK Treasury forecasts but are
never reported by the mass media. The Bank of England provides fan charts of inflation and GDP
forecasts displaying the confidence intervals widening into the future.
Unfortunately, these charts are used by the media all too infrequently. 
There are inherent ambiguities in economic and government statistics, and officials are prone to game the system to provide the figures  the "body counts"  desired by their superiors. There have also been substantial changes in recent times in the way various economic statistics are derived. 
The figures lie. (See, Economic
Statistics.) There are inherent ambiguities in economic and government
statistics, and officials are prone to game the system to provide the figures (the "body counts")
desired by their superiors. There have also been
substantial changes in recent times in the way various economic statistics are
derived (especially for dollar productivity and inflation statistics). 
Forecast uncertainty varies with the variable being forecast  the type of forecasting model being used  the economic processes involved  the information available  and the time horizon of the forecast. 
Five major causes of uncertainty are identified.
The first three are "what we don't know that we don't know."
The degree of uncertainty for the last two can be recognized and even
calculated.
A random walk dynamic model for forecasting pound/dollar exchange rates for 1971 to 2000 varies from high confidence levels for one month forecasts to vast forecast uncertainty for two year forecasts. The result of the pound exchange rate crisis of 1992 feeds substantially greater forecast uncertainty into subsequent periods even though in the event exchange rates subsequent to 1992 proved increasingly stable. In this example, the dynamic model proved less reliable than a static model. Redesign of models to reflect more recent data "is a topic of much current research in economics."

Information about forecast uncertainty can be as important as the forecast itself, Ericsson emphasizes. It provides information about likely outcomes and qualifies the forecasts. It provides help in evaluating the models and assists in efforts at improvements. It reveals the importance of the unknown unknowns that impact an economy during a forecast period by indicating if the results were thrown outside the acceptable confidence interval.

Evaluation of forecasts:
"Forecasts are made for a purpose."
& 
Economic models must be evaluated in light of their specific purposes, Clive W. J. Granger, U. Cal. (USA) tells us. Then, the forecast must be evaluated with respect to how well it fulfills its specific purposes.

Because of the complexity of the task and the limits of the art, some probability of error is always involved. 
Generally, models are either theoretical or empirical.
The variety of models is useful since no single model can represent
all the outcomedeterminative variables in a vast modern economic system. The
varying results inform about the range of possibilities that can be forecast.
Many models are designed to deal with just specific segments of an economy.
Evaluation should determine how well a model represents the main economic
features of interest and how well it is performing compared to other models. 
The question is thus: What is the cost of the error rate to the decision makers who use the forecast? The importance of forecast errors that miss the mark in one direction may not be the same as when they miss the mark in the other direction.


The evaluation of error costs is discussed in some detail by Granger.
He provides a mathematical formula for that purpose that takes into
account the asymmetrical nature of error costs. Of course, there may be many
consumers of a particular forecast  some of whom may be unknown to the forecast
provider  and each with its own error cost function. Each consumer must thus
evaluate the error costs of different forecasts for itself  based on the costs
to that consumer of working with the forecast's uncertainty range and error
rate. 
However,
interest rate policy is always an important factor, and so that is the variable
relied upon by the authors. They recognize that the impact of interest rate
changes may be different when moving from an expansion to a recession than when
moving from a recession to an expansion. They highlight the impacts of Bank of
England interest rate decisions. The yield on the 3 month UK Treasury Bill is
the interest rate used. 

The model still does poorly when applied to the 1990s recession, but
at least recognizes the recession when it occurs. However, it's a year early in
its recovery forecast. 
Judgment is required at every step of the process. Other
information  such as survey data and reports from regional agents  are vital
components of the decisionmaking process. (This sounds like a professional
evaluation process by professional economic policy makers.)


The staff produces a preliminary central projection that begins an iterative process of discussion and analysis of results, risks and uncertainties that ultimately produces the final result  including the Bank's probability fan charts of inflation and growth forecasts. 
The forecasting effort is "intensive." Initially,
"provisional assumptions are made about variables exogenous to the core
model." To save time, some assumptions are automatic. Hatch notes that the
starting point for exchange rates is its average during the 15 working days
prior to the meeting, and the short term interest rate is as set by the MPC just
prior to publication of the Inflation Report. However, all assumptions
can be discussed and altered.
The core model is supplemented by various other models including
narrower models of such variables as the labor market showing impacts of such
policy shifts as minimum wage or tax credit changes. Other models incorporate
differing degrees of sophistication based on such factors as the aggregation or disaggregation of
data. Some models are datadriven, while others incorporate economic theory. 
Since the economy takes time to adjust, shortrun tradeoffs do exist between inflation and output as reflected by Philipscurve theory. 
The core model is not designed to reflect the damaging impact
of inflation on longrun forecasts of unemployment and output, although Hatch
notes that persistently high levels of inflation will indeed have damaging
results. However, since the economy takes time to adjust, shortrun tradeoffs do
exist between inflation and output as reflected by Philipscurve theory. & As a model of a relatively small, open economic system, the core model reflects the strong influence on domestic output and inflation of exchange rates and international trade, growth and prices. & 
These models are just tools. The forecasting task still remains an "art" as well as a "science," "especially when iterating between models, data, and economic judgment." 
Supplementary forecasting models are of various types.
The Bank also pays attention to futuresoriented markets that reflect expectations for movements of interest rates, inflation, exchange rates, etc. These, too, appear in the Inflation Report.
These models are just tools. The forecasting task still remains an "art" as well as a "science," "especially when iterating between models, data, and economic judgment." (That's what the publisher of FUTURECASTS has been saying and writing about for 50 years.)

Forecasting the world economy: 
The characteristics and uses of the
world econometric model of the National Institute of Economic and Social
Research (UK) is explained by Institute economist Ray Barrell. & 
Uncertainty constitutes a vital component of these forecasts, so "confidence intervals" are reported with them in the Institute's quarterly "National Institute Economic Review." 
Widely used short and longer term forecasts,
and analyses of the likely impacts of shocks  both positive and negative  and
changes in economic policy, are provided by the Institute. 
There is a great deal of judgment involved in determining how such changes should be reflected in econometric models. 
The National Institute Global Econometric Model (NiGEM) is
widely used. It covers "all OECD countries individually and all nonOECD
countries individually or in blocks." It is constantly being revised to
reflect the continuous flow of EU structural changes. Globalization  the
collapse of the Soviet Union  economic integration and changes in trade policy
 are recent examples of structural changes with worldwide impacts. There is a great deal of judgment involved in determining
how such changes should be reflected in econometric models.
The Institute's analysis and forecasts of the impacts of the Asian
Contagion crisis were far less pessimistic  and thus far more accurate  than
that of many other forecasters. Rapid adjustments and recoveries were accurately
forecast. The global market capitalist system demonstrated its resiliency and
ability to absorb shocks. 
Low unemployment and inflation rate declines in the U.S. in the 1990s puzzle Barrell.
However, Barrell correctly attributes much of this economic success to a strong dollar and the rising demand levels of prosperous consumers. A strong currency does much to dampen inflation (and fend off a variety of other economic and financial ills  like the Asian Contagion). The fly in this picture (actually, the proverbial 800 pound gorilla), is the rising U.S. trade and international payments deficits that Barrell correctly identifies as unsustainable  a significant problem for the future. (It is much worse, now.) However, he expresses puzzlement over the strength of the dollar in the late 1990s.
Another benefit is the "productivity miracle" enjoyed by the U.S. The author asks why other nations are not enjoying similar benefits from modern technology.
Barrell presciently feared the early demise of the 1990s period of
prosperity. He explains how the NiGEM model helps evaluate forecast uncertainty
by running simulations of various kinds of "shocks" and corresponding
interest rate shifts. He demonstrates this analysis using a sudden increase in
inflation above tolerated levels as the "shock." He notes the
importance to the small open economy of the UK of a stable exchange rate for the
pound. Stable exchange rates also substantially increase confidence levels for
economic forecasts by eliminating an important element of risk. 
The old quantitative controls on credit and lending created useful data for forecasters but were obstacles to efficiency and are now gone.
The floating exchange rates of the modern world constitute major and difficult to forecast additional variables that didn't exist in the fixed exchange rate period after WWII. 
Some of the causes of these difficulties are accurately noted by Burns.
Business survey data is frequently useful for qualifying short term forecasts  especially when evaluating the impact of shocks that may not as yet have shown up in the economic data. However, it is hard to include such data in a forecasting model, and survey data, too, has sometimes been in error. Burns mentions the increasing pessimism of Autumn 1998 and winter 19981999 that failed to reflect the booming conditions of 1999.
When forecasts are used as one of the bases for economic policy
shifts, they must cover the additional complication of what the impact of
various policy shifts will be and how long the "lag" time will be
before that impact manifests itself. Interest rate changes, for example, have
historically had their maximum impact between 6 and 8 quarters after the change.
Interest rate changes impact many other variables such as exchange rates,
housing markets, inventories, investments and consumer spending.

A major problem with interest rate policy is that higher interest rates are essential to eventually curb inflation  yet the rise in interest costs is immediately reflected in inflation statistics. Mechanical use of this data  that fails to adjust for this anomaly  has caused interest rate policy to overshoot the mark  staying high until the inflation data has visibly peaked  by which time the economy is already headed in the opposite direction.
Forecast errors can contribute to policy errors that increase
volatility with widespread economic consequences. Volatility generally creates
major problems for public finances and undermines confidence in both forecasts
and policy. 
Please return to our Homepage and email your name and comments.
Copyright © 2006 Dan Blatt