Limits of economic forecasting models

Economic forecasting methods

Economic models

Interpretation of forecasts

Causes of forecast uncertainty

Business cycle forecasting

Bank of England forecasts

Economic forecast errors

New approaches to econometric modeling

The limits of economic forecasting models:

&

  The models are "mis-specified" (missing essential variables - based on the wrong variables - and/or mathematically misrepresented) and are plagued by substantial unanticipated economic shifts.
 &
  New theories and methods of econometric forecasting now acknowledge economic dynamics and sudden shifts, and the imperfections inherent in forecasting models. The editors assert that this enables economists to "account for the different results of competing forecasts."
 &
  The book's editors and contributing economists explain these new developments - their capabilities and continuing limitations - in a clearly written, commendably candid, handy little book, with a minimum of professional jargon and econometric equations, so that the material is readily accessible and useful to the intelligent lay reader and professional economist alike. However, the lay reader must be careful when interpreting the professional jargon that is used - especially when it consists of common words that have professional meanings.
 &

Unfortunately, major unpredictable "shocks" occur all too often - resulting in major forecasting failures, especially for the near-term forecasts that are so important for policy-making, business and investment purposes.

"Structural breaks appear to explain why it is so hard to reduce forecast error standard deviations: the outcome is sometimes very far from the forecast."

  Forecasting uncertainties arise from known factors that can only be established as probabilities, and factors that "we don't know that we don't know." Unpredictable events will always intrude on the future.
 &
  Fortunately, shocks and shifts come in both favorable and unfavorable varieties, and so to a certain extent - especially over longer periods - will average out. Forecasting models are constantly checked to see how well they "characterize the existing evidence," and are solved for average outcomes. They seek to reveal "the average future." Unfortunately, in economics, major unpredictable "shocks" occur all too often - resulting in major forecasting failures, especially for the near-term forecasts that are so important for policy-making, business and investment purposes.
 &
  Forecasters must deal with:

• "Stochastic trends" - changes that are regular and persistent;

• "structural breaks" - changes that are large and sudden;

• inaccurate data; and,

• "mis-specification" of model elements.

  "Structural breaks appear to explain why it is so hard to reduce forecast error standard deviations: the outcome is sometimes very far from the forecast. The 1929 crash and ensuing Great Depression is the classic example of when large forecast errors occur."

  The Great Depression was not unpredictable. There were those who provided reasoned predictions - albeit uncertain as to timing. In fact, the financial press was full of reports of widespread and increasing nervousness among investors and others throughout the eight weeks prior to the crash. British investment trusts quietly abandoned the N.Y. market in September, 1929. See, Great Depression Chronology, "The Crash of '29."
 &
  However, the bulk of the cognizant officials and most prominent economists simply ignored increasing financial distress abroad, and refused to evaluate the weaknesses in the government's trade war economic policies, private over-expansion - especially in agriculture, and, as always, the reckless extent of private leveraging that had grown in some economic and financial sectors during the prior period of prosperity. 

Data that is frequently subject to substantial subsequent revision presents additional problems for shorter term forecasts.

  Forecasting models are far from accurate representations of complex, dynamic modern economic and commercial systems. Even models that attempt to closely represent the economy and that provide reasonably accurate forecasts for one or more periods may suddenly prove unreliable thereafter. Abrupt shifts arise from technological developments, political turmoil, legislation and regulation, and similar factors.
 &
  Recent examples for England of developments with major impacts on economic data and outcomes include the elimination of exchange controls - privatization - and the introduction of interest-bearing checking accounts. Data that is frequently subject to substantial subsequent revision presents additional problems for shorter term forecasts.
 &
  Thus, "naive predictors" that simply extrapolate from previous results may more rapidly reflect such shifts and provide more accurate forecasts over time.
 &

Forecasting failures under or over the expected result are not unusual, but they should not be permitted to persist for a significant duration.

  Forecast models thus must "adapt" to sudden shifts to minimize the duration for forecast failures. Forecasting failures under or over the expected result are not unusual, but they should not be permitted to persist for a significant duration.
 &
  Alternative forecasts or forecasts of average expected outcomes can be used to deal with periodic but rare events.

  The editors refer to oil shocks. When these are caused by conflict, they may indeed be unpredictable, but when they are caused by ordinary supply constraints, any competent economist should be able to see them coming. FUTURECASTS had no trouble predicting in its February, 2001 "Near Futurecast" that this current economic revival would be accompanied and constrained by substantial energy shortages and high oil prices. It was an obvious call.

Forecasting methods:

"Because of the things we don't know [that] we don't know," and because the future is subject to known but apparently random uncertainties, the future is "unpredictable."

  Professional economists don't necessarily provide "predictions," David F. Hendry of Oxford Univ. (UK) explains. He distinguishes between "predictions" and "forecasts." The latter are the results of established techniques, but with no assurance of accuracy for any individual period. "Anything can be forecast, but not everything can be predicted."
 &
  A forecast is more probabilistic than a prediction as those terms are used in this book. Like weathermen, professional economists "forecast" - they don't "predict." Economists forecast "unpredictable" events.

  But weathermen at least attempt to create and apply valid concepts in their forecasting models. They accept their responsibility to forecast weather changes and storms. All too many economists have been mired in unreliable statistics and obviously invalid, ideologically based concepts, and now admit their inability to forecast economic changes and crises.

  "Because of the things we don't know [that] we don't know," and because the future is subject to known but apparently random uncertainties, the future is "unpredictable." The latter uncertainties are called "measurable" uncertainties. It is the former that pose the most intractable problems.

  Many macro-econometric models have been "mis-specified" by reliance on Keynesian concepts that are clearly, grossly invalid. Such models cannot even be said to provide "forecasts." They merely - blindly - yes, stupidly - "project" invalid results of invalid concepts.

  Forecasts may be thrown off by major disturbances that are inherently unforeseeable. A major earthquake or the sudden outbreak of war are factors that can cause forecast failures without implying weakness in the economic forecasting effort.
 &
  However, Hendry includes major changes in equity prices and exchange rate crises as perhaps "inherently unpredictable."

  While the exact timing may be unpredictable, such events are frequently quite predictable within reasonable timeframes - and should be included as possibilities in forecasts by competent economists during those timeframes. What is required is professional analysis and opinion - not blind reliance on mathematical technique and inherently limited and often inaccurate data. Economics is, after all, a "profession" - like law and accounting - a "practical art" - not a "science." Total reliance on mathematical forms of reasoning constitutes professional incompetence.

  Econometric models depend on the accuracy and proper inclusion of significant "deterministic terms" (average levels and yearly or quarterly or other periodic trends) -  "observed stochastic variables" (known variables) - and "unobserved errors" (estimates or averaging out of unknown variables).

  "Relationships involving any of these three components could be inappropriately formulated or inaccurately estimated, or could alter over time in unanticipated ways. Each of the resulting nine types of mistake could induce poor forecast performance through either inaccurate - i.e., biased - or imprecise - i.e., high-variance - forecasts."

The most important task is to develop "forecasting models that are more robust to shifts," rather than making improvements in the models themselves.

  Actually, the worst econometric forecasting failures are caused by unforeseen shifts in the values of average levels or period trends, rather than shifts in variables. Thus, the unsophisticated models that do not depend on data that may be erroneous or trends that can shift can often perform better than more sophisticated models that get tripped up by major variances in the wide variety of factors that they include.
 &
  All models must be subject to frequent adjustment to reflect the economic shifts that impact the models. There is much less excuse for a sequence of poor forecasts after an unforeseen shock than for the initial forecast error caused by the shock.
 &
  Hendry thus asserts that the adequacy of the underlying economic theory is less important than simple errors caused by shifts in yearly, quarterly or other period trends. The most important task is to develop "forecasting models that are more robust to shifts," rather than making improvements in the models themselves. (However, faulty economic theory may be what renders such shifts "unpredictable.")
 &

  Forecasting methods covered by Hendry, excluding guesswork and hunches, include econometric systems, time-series models, surveys, leading indicators, and extrapolation.

• Extrapolation is good only as long as the tendencies persist. It must always fail to predict the turning points in those tendencies, which are inherently the most useful of forecasts.

• Leading indicators (such as interest rate movements and surveys of business and consumer sentiment) are good only so long as they remain leading indicators. The need for frequent changes in the indicators highlights "their inability to capture" many of the underlying economic changes.

• Surveys provide useful information about interviewee plans. However, such plans can change. The reasons for such changes are not always evident from the surveys.

• Time-series models (that generally assume that past relationships will persist) have provided forecasts with reasonable accuracy. These are statistical models based on "past data of the variable in question [that are extrapolated] into the future."

• Econometric forecasting models based on empirical and theoretical knowledge of the functioning of complex economic systems "provide a framework for a progressive research strategy." They provide forecasts and policy advice, and can even "help explain their own failures." 

  To repeat, economics is a profession - not a science. Professional analysis and opinion is not mere "guesswork" and "hunches," and can be far more reliable in the hands of a knowledgeable professional than mathematical forms of analysis based on the faulty theory and the inaccurate statistics generally available to the mathematical economics technicians that rely on them.

  Professional economists generally rely on time-series and econometric models. These are "causal" models based on the study of underlying causes.

  Significantly, Hendry fails to include professional analysis of the impacts of faulty government economic policies or even analysis of ordinary business cycle phenomena like excess leveraging, over-expansion, and continued use of marginal facilities during prosperous times. Of course, much of this will accumulate in the data - but all too frequently, the data in which it accumulates is not adequately represented in forecasting models because it comes into play at infrequent and irregular intervals.
 &
  Accounting for such data requires an effort similar to that being developed for forecasting volcanic eruptions - as Hendry himself notes in a somewhat different context. Unfortunately, much has changed in how statistics were derived in the 1970s. Thus, current statistics are not strictly comparable with those of the 1970s and are less likely to provide any reliable clues about the similar rumblings now beginning to be felt under the economic surface.

"Confidence intervals" are established according to the calculation of likely outcomes during a certain period of time.

Fan charts and other displays of confidence intervals usefully dispel the illusion of precision when a forecast is presented as a particular number. However, even fan charts fail to reflect uncertainties that "we don't know that we don't know." All forecasting is tentative in nature.

  An estimate of the uncertainty involved will frequently be included in economic forecasts. These "confidence intervals" are established according to the calculation of likely outcomes during a certain period of time. (The chance of rain tomorrow will be x%.) Confidence intervals generally widen as the forecast time horizon lengthens.
 &
  The Bank of England provides inflation forecasts with "fan charts" displaying the range of uncertainty widening into the future. Fan charts and other displays of confidence intervals usefully dispel the illusion of precision when a forecast is presented as a particular number. However, even fan charts fail to reflect uncertainties that "we don't know that we don't know." All forecasting is tentative in nature.
 &
  For several decades, economists have been testing and refining their models against computer simulations and actual economic outcomes. The primary problem remains unanticipated shifts in trend rates. Fortunately, many shifts don't have significant impact on forecast results. Even inaccurate data and model errors will frequently have little impact. This is a reflection of the vastness and complexity of modern economic systems. However, these weaknesses working together instead of individually or averaging out can cause forecast failure.
 &
  Again, Hendry refers to major stock market or exchange rate moves or various crises as inherently unpredictable variables that can undermine forecasts. Sometimes, a forecast can undermine itself - as when changes in sales taxes or interest rates or some other trouble are forecast and impact consumer action or policy response.
 &
  Hendry demonstrates the impact of the start of the Industrial Revolution on a 50 year forecast covering 1800 to 1850 using several simple forecasting methods. The sudden growth acceleration threw the models off, but those that included adaptive mechanisms fared much better. With adjustments every 10 years, the 50 year forecast becomes much better. Hendry concludes:

  "While economic forecasts from econometric systems have a poor historic track record and face many potential and real problems, the recently extended theory of economic forecasting offers a vehicle for understanding and learning from failures, and for consolidating our growing knowledge of economic behavior. Consequently, despite their present travails, econometric systems provide the best long-run hope for successful economic forecasting, especially as suitable methods are developed to improve their robustness to unanticipated breaks."

Economic models:

  Models based on economic theory can analyze shifts in economic policy if the policy variables are included in the model. This is a major advantage over pure time-series models that merely assume that past relationships will persist.
 &
  Disadvantages of such "structural" models include the greater time and effort involved, and professional disputes over theory that may impact the analysis.

  "Moreover, the forecasts generated by structural economic models are often no 'better' than those generated by time-series models, at least when assessed statistically in terms of variability and degree of bias from the actual outcomes. Because structural economic models are so costly and the resulting gains in forecasting accuracy are likely to be small, the question naturally arises as to why anyone would adopt a structural modeling approach rather than a time-series approach."

Although time-series forecasts are quick and easy to compute, a structural approach has significant advantages by being able to cope with a variety of different scenarios."

  Simple time-series forecasts will not provide explanations for their results, however. Such explanations are often demanded by business or political or media consumers of the forecasts. When a consumer asks how policy changes or major shifts will affect a forecast, only the structural model will provide answers. By providing a series of forecasts reflecting a number of possible shifts in major exogenous factors, the conditionality of the forecast is highlighted.

  "Forecasts are necessarily conditional on a set of assumptions about the future. These assumptions should be recognized when the forecasts are constructed, and they should be stated clearly when the forecasts are presented. So, although time-series forecasts are quick and easy to compute, a structural approach has significant advantages by being able to cope with a variety of different scenarios."

  The two methods can be used together. Variables outside - "exogenous to" - the structural model can be forecast by time-series methods and the values then included in the structural model, which then forecasts the variables included - "endogenous" - within the model.
 &
  Turner demonstrates how a basic econometric model of the United Kingdom economy can be used to reflect the results of two different disinflation policies - either the adoption of a low, stable growth rate for the money supply as Milton Freedman advocated, or active adjustment of interest rates to target inflation directly as advocated by J. B. Taylor. Both policies arrive at the same result, but the Taylor approach was shown by the model to take effect and stabilize more quickly.

  The Taylor approach, however, is more vulnerable to changes in the definition of "inflation" and the methods by which it is calculated. Although not without problems of their own, money supply data are far less vulnerable to manipulation than inflation data. This is a matter of professional judgment, not economic modeling.

Interpretation of economic forecasts:

"Journalists could improve their own presentations of forecasts, as by avoiding spurious precision and by explaining the uncertainties involved in forecasting."

  Coyle developed the "Golden Guru" forecasting award for the most precise forecast of the UK's "misery index" composed of the rates of inflation, unemployment and economic growth. Unsurprisingly, the winners have been established by very narrow margins. More revealingly, no economist has won more than once. Coyle candidly acknowledges some of the weaknesses of her rough evaluation method.

  "Forecasters face a difficult task; and journalists could improve their own presentations of forecasts, as by avoiding spurious precision and by explaining the uncertainties involved in forecasting."

  Because forecasters tend to "herd together," the average forecast is always in the top few positions. So far, forecasts at one extreme or another have never won. They almost always make a big error on at least one of the three variables.
 &

"We don't know the true structure of the economy, the true values of parameters in econometric models, or the exact shocks that hit the economy; and we may never obtain accurate data measurements either."

  Uncertainty is inherent in economic forecasts. With commendable candor, Coyle perceptively explains some of the most prominent aspects of the problem.

  "Part of that uncertainty is not knowing what has happened, even long after the event. We don't know the true structure of the economy, the true values of parameters in econometric models, or the exact shocks that hit the economy; and we may never obtain accurate data measurements either." 

  Forecasts are generally reported with "spurious precision," Coyle points out. Words like "around," "approximately," "probably," and "might" are not favored. Instead, dramatic terms are employed that routinely overstate the implications of events. Unfortunately, the media generally fail to reflect or explain the uncertainty inherent in economic forecasts.

  The publisher of FUTURECASTS has been explaining just that in articles published over the last 40 years. The mass media is all too often nothing but a conduit for authoritative misinformation.

  Ranges of uncertainty are provided with UK Treasury forecasts but are never reported by the mass media. The Bank of England provides fan charts of inflation and GDP forecasts displaying the confidence intervals widening into the future. Unfortunately, these charts are used by the media all too infrequently.
 &
  When an economy is expanding at about trend rates, forecasts tend to be good. (A straight line graph would work just as well at such times.) It is when the economy accelerates or decelerates or turns that economic forecasts generally fail. Unfortunately, the trend shifts and turning points are precisely the times when good forecasts are most needed. Forecast errors delay appropriate government policy responses.
 &

There are inherent ambiguities in economic and government statistics, and officials are prone to game the system to provide the figures - the "body counts" - desired by their superiors. There have also been substantial changes in recent times in the way various economic statistics are derived.

  The figures lie. (See, Economic Statistics.) There are inherent ambiguities in economic and government statistics, and officials are prone to game the system to provide the figures (the "body counts") desired by their superiors. There have also been substantial changes in recent times in the way various economic statistics are derived (especially for dollar productivity and inflation statistics).
 &
  Benchmarks used to evaluate the quality of government services are particularly prone to manipulation (as experience with U.S. educational performance statistics currently demonstrates). All too often, official forecasts are treated by officials as targets - that must be achieved - by hook or by crook. Coyle explains how surgical waiting periods were reduced by the UK Public Health Service by rushing through a mass of the less complex operations to meet targets.
 &
  Macroeconomic policy suffers from lags that can vary substantially, and lack of precision in the impact that can be achieved. The lags and impact of interest rate policy changes, for example, are particularly critical uncertainties for economic forecasters.
 &

Causes of forecast uncertainty:

Discrepancies between forecasts and outcomes reflect forecast uncertainty.

  The reasons for the forecast uncertainty of econometric models is further explained by Neil R. Ericsson, Federal Reserve System (USA).

  "Economic forecasts typically differ from the realized outcomes, with discrepancies between forecasts and outcomes reflecting forecast uncertainty. Depending upon the degree of forecast uncertainty, forecasts may range from being highly informative to being completely useless for the tasks at hand."

  Measures of forecast uncertainty have been devised to clarify the expected range of outcomes and assess forecast reliability. They reflect the dispersion of possible outcomes relative to the forecasts being made.
 &
  Ericsson provides examples from forecasts of the U.S. trade balance and the Bank of England's inflation forecast fan charts and probability density charts that show the range of outcomes within which the results will fall with a stated degree - 90% in the example - of confidence. He explains how to interpret several types of "histogram" probability density charts.
 &
  There are several ways to measure forecast uncertainty. These measures are typically useful for decisions involving insurance and investments.
 &

Forecast uncertainty varies with the variable being forecast - the type of forecasting model being used - the economic processes involved - the information available - and the time horizon of the forecast.

  Five major causes of uncertainty are identified.

1. Changes in the structure of the economy during the forecast period
2. Error - "mis-specification" - in the model.
3. Errors in the base period data.
4. Inaccurate estimates of such factors as the rate of change in basic interest rates.
5. Unexpected "shocks" to the economy.

  The first three are "what we don't know that we don't know." The degree of uncertainty for the last two can be recognized and even calculated.
 &
  Forecast uncertainty varies with the variable being forecast - the type of forecasting model being used - the economic processes involved - the information available - and the time horizon of the forecast.
 &
  For example, trade balances are relatively small quantities arising from variances in imports and exports - two much larger quantities. Thus, even if imports and exports are fairly accurately forecast, minor deviations can result in major changes in trade balances.
 &
  Some models are simply inherently less precise than others. Some economic processes result in more precise data than others. (Market data is more precise than accounting data.) The limits of available data will vary. The length of the forecast period will impact forecast reliability. Sometimes, lengthened forecast periods will make forecasts less reliable, but sometimes they will reduce uncertainty by permitting cyclical movements to average out.
 &
  Examples are provided by Ericsson with static trend models and dynamic models. A graph of a static trend model of UK real net national income has 95% fixed width confidence intervals extended indefinitely into the future, while a dynamic random walk model shows a substantial fan shaped graph of 95% confidence intervals. Dynamic models frequently imply that forecast uncertainty depends on the forecasting horizon, because dynamic models can accommodate increased possibilities of economic shocks for subsequent periods.

  "More generally, static models commonly imply forecast uncertainty that is time-invariant or nearly so, whereas dynamic models typically imply time-dependent forecast uncertainty, often increasing in the forecast horizon. The trend and random walk models above present static and dynamic relationships as black and white, but in practice a whole spectrum of models exists with both static and dynamic features."

  A random walk dynamic model for forecasting pound/dollar exchange rates for 1971 to 2000 varies from high confidence levels for one month forecasts to vast forecast uncertainty for two year forecasts. The result of the pound exchange rate crisis of 1992 feeds substantially greater forecast uncertainty into subsequent periods even though in the event exchange rates subsequent to 1992 proved increasingly stable. In this example, the dynamic model proved less reliable than a static model. Redesign of models to reflect more recent data "is a topic of much current research in economics."

  How about research into professional evaluation of government budgetary and monetary policies and payments balances? The degree of monetary stability is, after all, not an act of nature like the weather. It is a product of the policy of the monetary authorities.
 &
  The restoration of UK exchange rate stability and strength was no accident. Acknowledgement of Conservative Party government policies after 1992 that were clearly supportive of exchange rate stability and strength could have served to greatly narrow pertinent forecast uncertainty margins.

  Information about forecast uncertainty can be as important as the forecast itself, Ericsson emphasizes. It provides information about likely outcomes and qualifies the forecasts. It provides help in evaluating the models and assists in efforts at improvements. It reveals the importance of the unknown unknowns that impact an economy during a forecast period by indicating if the results were thrown outside the acceptable confidence interval.

  "Measures of forecast uncertainty also provide economists with a way of assessing the importance of unmodeled features of the economy, both directly through the calculated forecast uncertainty, and indirectly through comparison of that calculated uncertainty with the realized distribution of forecast errors."

Evaluation of forecasts:

"Forecasts are made for a purpose."

&

  Economic models must be evaluated in light of their specific purposes, Clive W. J. Granger, U. Cal. (USA)  tells us. Then, the forecast must be evaluated with respect to how well it fulfills its specific purposes.

  "Forecasts are made for a purpose, with those forecasts typically providing the basis for economic decisions and with the resulting forecast errors entailing economic costs. Different models generate different forecasts, and the resulting economic costs have different distributions, which can be compared across models."

Because of the complexity of the task and the limits of the art, some probability of error is always involved.

  Generally, models are either theoretical or empirical.

  "[Theoretical] models are constructed from logic, mathematics, and sets of generally agreed-upon behavior by economic agents in the form of axioms and assumptions. The empirical models arise mainly from the analysis of economic data, possibly at least partially based on economic theory."

  The variety of models is useful since no single model can represent all the outcome-determinative variables in a vast modern economic system. The varying results inform about the range of possibilities that can be forecast. Many models are designed to deal with just specific segments of an economy. Evaluation should determine how well a model represents the main economic features of interest and how well it is performing compared to other models.
 &
  Models can be designed to assist with investment or policy decisions or to provide general or sector forecasts or to test an economic hypothesis. Because of the complexity of the task and the limits of the art, some probability of error is always involved.
 &
  Granger provides the example of a central bank decision to take action against inflationary expectations. Because of lengthy lags in the impact of monetary policy, it may decide to act when inflation rates are forecast to exceed acceptable limits within some reliable confidence level.
 &

  The question is thus: What is the cost of the error rate to the decision makers who use the forecast? The importance of forecast errors that miss the mark in one direction may not be the same as when they miss the mark in the other direction.

  A forecast of the start of a bear market that is too late is generally worse than one that errs on the side of being too soon - except for the bear investor. In forecasting economic turns, FUTURECASTS strives to be within six months and to err on the early side.
 &
  The current FUTURECASTS market forecast is - and has been for several years - that the market will be range-bound, with a bottom more than one third below its top. An austerity recession is now essential to overcome the inflationary impact of recent Keynesian policies in the U.S. - and the longer it is put off, the worse the inflationary problems and the deeper the ultimate austerity recession will have to be.
 &
  Timing is a subject of political forecasting as much as of economic forecasting, since timing is dependent on government policy.

  The evaluation of error costs is discussed in some detail by Granger. He provides a mathematical formula for that purpose that takes into account the asymmetrical nature of error costs. Of course, there may be many consumers of a particular forecast - some of whom may be unknown to the forecast provider - and each with its own error cost function. Each consumer must thus evaluate the error costs of different forecasts for itself - based on the costs to that consumer of working with the forecast's uncertainty range and error rate.
 &
  Thus, evaluating competing forecasts depends critically on the needs of each consumer, and there may be no "best" forecast model. The higher the cost of decision error, the more critical the forecast confidence level becomes.
 &

Business cycle forecasting:

&

  The forecasting of business cycles using econometric models is particularly difficult due to the (fortunately) infrequent nature of the event. Denise R. Osborn and Marianne Sensier, Univ. of Manchester (UK), and Paul W. Simpson, Sheffield Dep't. of Ed. (UK), discuss "regime-switching models" using short-term interest rates as a leading indicator of such economic switches. The interest rate decisions of the Bank of England are thus of primary importance.

  "Large increases in the interest rate raise the probability of switching out of an expansion into a recession a year later, whereas, in a recession, even small decreases in the interest rate help start a recovery."

The authors question whether forecasting tools - particularly forecasting models - are capable of predicting economic turning points.

  Inability to predict business cycle booms and busts has resulted in widespread criticism of professional economists. Written just before 2000, the book notes that economic forecasters missed the extent of the boom in the UK of the late 1980s and the related onset and duration of the UK recession of the early 1990s.
 &
  The respected, independent National Institute of Economic and Social Research failed to forecast declines in real GDP for the UK even as late as August, 1990, and when decline was recognized thereafter, forecast steady recovery throughout the next two years although the recession lingered on in England for some time. Thus, the authors question whether forecasting tools - particularly forecasting models - are capable of predicting economic turning points. However, models that generally fail to forecast economic turning points may nevertheless be the most accurate for periods that do not involve turning points.
 &
  Between WW-II and the end of the century, there were only three periods  when the UK was in recession - suffering at least two quarters of actual decline in GDP. These occurred in the mid-1970s, the early 1980s, and the early 1990s. Not only has the event been comparatively rare, each recession has had different characteristics. (The methods by which the relevant statistics have been derived have also been changing.) Nor is there universal agreement on the precipitating causes in the chain of events leading up to the recessions.

  Were the first two of these recessions "caused" by sharp increases in oil prices - as so many economists stupidly assert - or by the decade of Keynesian economic policies in the U.S. that undermined the value of the dollar, made those sharp increases in oil prices possible, and forced less accommodative monetary policy? It is no mere coincidence that current oil price spikes have come after renewed substantial resort to inflationary Keynesian policies - and that now Fed monetary policy is becoming cautiously less accommodative while commodity prices are soaring .

  The beginnings of recoveries, too, are thus rare events. The economic policies adopted to encourage recovery have varied in type and impact.
 &

  However, interest rate policy is always an important factor, and so that is the variable relied upon by the authors. They recognize that the impact of interest rate changes may be different when moving from an expansion to a recession than when moving from a recession to an expansion. They highlight the impacts of Bank of England interest rate decisions. The yield on the 3 month UK Treasury Bill is the interest rate used.
 &
  The authors discuss both univariate (single variable) linear models and univariate "regime-switching" models. The former can't recognize shifts between economic expansion and recession, and the latter has been generally unsuccessful at regime-switching forecasts. The latter "recognizes" these shifts after they occur, but then fails to forecast the next shift. The problem is the application of a very small - 4% - likelihood of a shift in any particular quarter due to the comparatively rare occurrence of such shifts.
 &
  The authors add short-term interest rate policy to these models. Interest rates and other leading indicator "explanatory" variables are used in multivariate (multi-variable) linear models in the attempt to forecast recessions. Lags for the interest rate indicators of 5 to 7 quarters are also included. "Thus, current or past observations on such variables are used to forecast future output growth." Interest rate increases of at least 3 percentage points within a short period are needed to move the referenced regime-switching model from likely continued expansion to probable decline.
 &

  The model still does poorly when applied to the 1990s recession, but at least recognizes the recession when it occurs. However, it's a year early in its recovery forecast.
 &
  Because there is so little data from the three relatively short post WW-II recessions, the data reflects anomalous and clearly useless results for evaluating recession-to-expansion probabilities from declining interest rates. Several large anomalous interest rate policy shifts during the 1970s and 1980s recessions have to be disregarded to get meaningful results. With that, the data seems to indicate that interest rate declines have a more powerful impact on recession-to-expansion probabilities than interest rate increases have for expansion-to-recession probabilities.
 &
   "[Changes] in the interest rate alone do not appear to have been sufficient to forecast the onset or length of the 1990s recession." Interest rates are thus being supplemented with other financial variables as leading indicators to help with the "exacting task" of forecasting the business cycle.
 &

Bank of England forecasts:

&

The forecasts are published in the Bank's "Inflation Report."

  Efforts to develop models of increasing sophistication and complexity reflecting ever more closely the complexities of the economy have been abandoned, Neal Hatch of the Bank of England explains. The results were invariably poor and did not justify the effort.
 &
  Current efforts concentrate on developing smaller and more compact models. They seek to achieve clarity by simplifying the analytical process. They are used to help explain current conditions and how conditions develop as well as to provide forecasts - insights into future probabilities. They are not good enough to be relied upon mechanically.
 &

  Judgment is required at every step of the process. Other information - such as survey data and reports from regional agents - are vital components of the decision-making process. (This sounds like a professional evaluation process by professional economic policy makers.)
 &
  The Bank uses its "core model" supplemented by other models to produce its probabilities forecasts - its fan charts. Various assumptions and judgments are included in these models. Consideration of other issues and policy judgments along with the forecasts go into the formulation of policy. Thus, the forecasts are that of the Monetary Policy Committee (MPC) - not of its staff.
 &
  The August, 1999 Inflation Report went even further. It included the differing views within the Committee - and "explained how alternative assumptions would shift the profile of forecast inflation." It included a table showing the impact of "alternative assumptions about variables such as earnings, profit margins, exchange rates, and oil prices."

  The inclusion of earnings and profit margins in the complex of variables being considered is another vast improvement over 1970s economic forecasting efforts.

The staff produces a preliminary central projection that begins an iterative process of discussion and analysis of results, risks and uncertainties that ultimately produces the final result - including the Bank's probability fan charts of inflation and growth forecasts.

  The forecasting effort is "intensive." Initially, "provisional assumptions are made about variables exogenous to the core model." To save time, some assumptions are automatic. Hatch notes that the starting point for exchange rates is its average during the 15 working days prior to the meeting, and the short term interest rate is as set by the MPC just prior to publication of the Inflation Report. However, all assumptions can be discussed and altered.
 &
  The core model is continuously adjusted in line with actual data. However, this requires judgment as to whether unexpected data results are mere random variation - perhaps involving measurement errors - that may subsequently unwind. Judgment is also required when including the impacts of unexpected exogenous events - such as a surge in energy prices, a financial crisis, or a major change in taxation. Pertinent assumptions will be revealed in the Inflation Report.
 &
  The staff then produces a preliminary central projection that begins an iterative process of discussion and analysis of results, risks and uncertainties that ultimately produces the final result - including the Bank's probability fan charts of inflation and growth forecasts. These reflect the forecasts and their "forecast error uncertainty" based on prior experience.

  "They allow the MPC to express judgments about the skewness and variance of the forecast errors, and not just about the central case -- i.e., the most likely single outcome."

  The core model is supplemented by various other models including narrower models of such variables as the labor market showing impacts of such policy shifts as minimum wage or tax credit changes. Other models incorporate differing degrees of sophistication based on such factors as the aggregation or disaggregation of data. Some models are data-driven, while others incorporate economic theory.
 &
  The core model itself incorporates more theoretical aspects for longer range forecasts. It is supplemented with survey information - such as consumer and business sentiment - and information from the Bank's regional agencies. These are vital because they are more current than official output data and are considered leading indicators of output.
 &
  The Inflation Report also includes other forecasts with various graphs and charts displaying ranges and probabilities. This forecasting exercise is constantly evolving, and still too recent for meaningful judgment about accuracy. Such judgments are rendered more difficult when economic outcomes are impacted by unpredictable shocks.
 &

Since the economy takes time to adjust, short-run tradeoffs do exist between inflation and output as reflected by Philips-curve theory.

These models are just tools. The forecasting task still remains an "art" as well as a "science," "especially when iterating between models, data, and economic judgment."

  Supplementary forecasting models are of various types.

• Philips-curve models relate price and wage inflation to estimated employment or the output "gap" or other measures of real disequilibrium. They can be important tools for analyzing short-run disequilibria.
• Small-scale macroeconomic models are complete models of the economy in highly aggregated form. They are useful as "test beds."
• Vector autoregressive models can capture the dynamic interactions between particular variables. They are data driven - generally not dependent on theoretical assumptions. Structural versions can be used to investigate the impacts of shocks - both positive and negative - on pertinent variables.
• Optimizing models assume that individuals act to optimize their economic outcomes. They are useful in evaluating the impact of shocks - both positive and negative. The demutualization of Building Societies in England provided widespread windfall gains that could be analyzed by these models.

  The latest econometric modeling approach relies on the "microfoundations" - individual households and firms - of macroeconomic behavior. It relies on an evaluation of how the "representative" household or firm makes its choices. A new generation of macroeconomic models based on microfoundations is now in use by central banks. They include Canada's Quarterly Projection Model, the Bank of England Quarterly Model, and the IMF's Global Economic Model. See, "The Economist," 7/15/06, pp. 67-69.

  The Bank also pays attention to futures-oriented markets that reflect expectations for movements of interest rates, inflation, exchange rates, etc. These, too, appear in the Inflation Report.

  "The Bank uses a suite of models because specific issues often call for new or different tools. It is not possible to construct a single model that will perform all tasks. The Bank's models range in size, complexity, and the role of economic theory. Although a core macroeconometric model helps the MPC make its projections, that model's use depends on various judgments, many of which are informed by other models."

  These models are just tools. The forecasting task still remains an "art" as well as a "science," "especially when iterating between models, data, and economic judgment." (That's what the publisher of FUTURECASTS has been saying and writing about for 50 years.)

  It does, indeed, sound like an exercise in professional judgment - not mere "guesswork" or "hunches." To repeat, economics is a "practical art" - like law and accounting - requiring professional forms of understanding and analysis - resulting in statements of professional opinion on which the professional stakes his reputation.

Forecasting the world economy:

Uncertainty constitutes a vital component of these forecasts, so "confidence intervals" are reported with them in the Institute's quarterly "National Institute Economic Review."

  Widely used short and longer term forecasts, and analyses of the likely impacts of shocks - both positive and negative - and changes in economic policy, are provided by the Institute.
 &
  Uncertainty constitutes a vital component of these forecasts, so "confidence intervals" are reported with them in the Institute's quarterly National Institute Economic Review. Some of this uncertainty is decomposed by source - such as uncertainty about exchange rates, consumption rates and investment rates. Unexpected shocks that impacted forecasts in the 1990s included the Asian Contagion crisis, the reunification of Germany, the sudden growth spurt in the U.S., and the collapse of the Long Term Capital Management (LTCM) hedge fund.
 &

There is a great deal of judgment involved in determining how such changes should be reflected in econometric models.

  The National Institute Global Econometric Model (NiGEM) is widely used. It covers "all OECD countries individually and all non-OECD countries individually or in blocks." It is constantly being revised to reflect the continuous flow of EU structural changes. Globalization - the collapse of the Soviet Union - economic integration and changes in trade policy - are recent examples of structural changes with worldwide impacts. There is a great deal of judgment involved in determining how such changes should be reflected in econometric models.
 &
  Examples provided by Barrell include labor market data through 2000 with forecasts thereafter for Germany, Finland and Sweden.

  The gradual improvements forecast for Germany failed to materialize when expected, but subsequent labor market reforms now seem to have reached the point where they can have some favorable impact. The forecast for the beneficial impacts of labor market reforms in Sweden and Finland have proven closer to the mark through 2005 for Finland, but too pessimistic for Sweden into 2006.

  The Institute's analysis and forecasts of the impacts of the Asian Contagion crisis were far less pessimistic - and thus far more accurate - than that of many other forecasters. Rapid adjustments and recoveries were accurately forecast. The global market capitalist system demonstrated its resiliency and ability to absorb shocks.
 &
  However, only a "bimodal" 60% - 40% forecast favoring avoidance of widespread financial collapse could be provided for the LTCM collapse since the outcome depended crucially on appropriate government policy responses.
 &

  Low unemployment and inflation rate declines in the U.S. in the 1990s puzzle Barrell.

  "Standard economic theory suggests that inflation should increase when the unemployment rate falls below its long-run sustainable level, which might also be the long-run average if markets work well and do not change too much."

  Barrell here exhibits the limits of his professional understanding. Standard economic theory is obviously wrong! Inflation is ultimately a monetary phenomenon - not a labor market phenomenon. Nor is there more than a short-run tradeoff between inflation, output and employment. Ultimately, inflation ALWAYS causes rising rates of unemployment. See, Capital as Purchasing Power, segment on "The determinants of purchasing power," and Understanding Inflation.
 &
  The reason why U.S. unemployment declined in the 1990s was precisely because of the success in squeezing out the last of the 1970s inflationary pressures. The reward was lower nominal interest rates, a strong dollar and reduced risks which, along with a reasonable degree of labor market flexibility, permitted rising employment and growth rates and price stability all at the same time. Nothing "unusual" about that at all!

  However, Barrell correctly attributes much of this economic success to a strong dollar and the rising demand levels of prosperous consumers. A strong currency does much to dampen inflation (and fend off a variety of other economic and financial ills - like the Asian Contagion). The fly in this picture (actually, the proverbial 800 pound gorilla), is the rising U.S. trade and international payments deficits that Barrell correctly identifies as unsustainable - a significant problem for the future. (It is much worse, now.) However, he expresses puzzlement over the strength of the dollar in the late 1990s.

  Just another benefit of declining rates of inflation and the substantial improvement in the federal government's budget in the 1990s due to major reductions in military spending. Again, nothing "unusual" at all.

  Another benefit is the "productivity miracle" enjoyed by the U.S. The author asks why other nations are not enjoying similar benefits from modern technology.

  The multiple factors of economic flexibility provide much of the answer. Moreover, the change in the way productivity is calculated may explain more than the government wants to admit.

  Barrell presciently feared the early demise of the 1990s period of prosperity. He explains how the NiGEM model helps evaluate forecast uncertainty by running simulations of various kinds of "shocks" and corresponding interest rate shifts. He demonstrates this analysis using a sudden increase in inflation above tolerated levels as the "shock." He notes the importance to the small open economy of the UK of a stable exchange rate for the pound. Stable exchange rates also substantially increase confidence levels for economic forecasts by eliminating an important element of risk.
 &

Forecast errors:

&

Sophisticated models can be large and difficult to understand - can be based on assumptions that are implicit rather than explicit - and thus can hide dangerous weaknesses.

"[The] profession appears to have made very little progress in reducing the size of forecast errors over the past 30 years or so, whether for the United Kingdom, the United States, or other industrialized countries."

"The profession appears to have made very little progress in reducing the size of forecast errors over the past 30 years or so."

  Econometric forecasters apply "rigid and mechanistic models of the economy" to produce forecasts that are thrust upon the economic policy-making process.

• Sophisticated models can be large and difficult to understand - can be based on assumptions that are implicit rather than explicit - and thus can hide dangerous weaknesses from policymakers and other customers. It may be difficult to demonstrate the various properties of such models or to understand the roles of the main forces behind a forecast.
• Models may over-emphasize variables that have greater impacts in the short run and inadequately consider variables that have significant longer-term impacts.
• There are suspicions of bias - a tendency to err on the conservative side - so as not to expose the forecasters - so as not to put them out on a limb.
• "Forecasters are also criticized for paying too much attention to their models and not enough to what is actually happening in the economy."

  Burns, too, emphasizes that the greatest risk of forecast error occurs when the economy most sharply shifts course - at turning points or moments of substantial acceleration or deceleration - just when accurate forecasts would be most useful. Forecast errors tend to be multiple - across multiple variables - because of the relationships among variables. Thus substantial errors in forecasting output will produce related errors in government tax receipts, unemployment, and price movements.
 &
  Unexpected shocks may have greater than expected impacts. Burns, too, uses the oil price increases of the 1970s as an example of such an unexpected shock.

  Again, this is a gross misstatement and over-simplification of the causes of the economic problems of the 1970s. More than a decade of accumulated inflationary forces - kept temporarily in check by the then fixed dollar exchange rate - were at work behind those problems and behind the oil price increases themselves.

  However, sometimes - as after the 1987 stock market crash and during the Asian Contagion crisis of the late 1990s - shocks can have surprisingly little apparent economic impact.

  A strong national currency provides significant shielding against such shocks, and provides the government with the strength to deal with any problems they may cause - just two of the many reasons why it is impossible to prosper with a weak currency.

  Inaccurate data can result in forecast errors. For short term forecasts, recent data on which forecast models rely can be notoriously inaccurate and subject to sometimes large subsequent revisions. These revisions are often unfortunately largest precisely when the economy is shifting gears.

"[From] my interpretation of the research evidence as well as from my own investigations, the profession appears to have made very little progress in reducing the size of forecast errors over the past 30 years or so, whether for the United Kingdom, the United States, or other industrialized countries."

The old quantitative controls on credit and lending created useful data for forecasters but were obstacles to efficiency and are now gone.

The floating exchange rates of the modern world constitute major and difficult to forecast additional variables that didn't exist in the fixed exchange rate period after WW-II.

  Some of the causes of these difficulties are accurately noted by Burns.

• The increasingly free market global economic world is increasingly complex.

• The old quantitative controls on credit and lending created useful data for forecasters but were obstacles to efficiency and are now gone.

• Data for the service sectors - which are growing - remain less accurate than for the manufacturing sectors - which are shrinking as a percentage of national output.

• Private sector decision making has become increasingly sophisticated and difficult to model.

• The floating exchange rates of the modern world constitute major and difficult to forecast additional variables that didn't exist in the fixed exchange rate period after WW-II.

  Business survey data is frequently useful for qualifying short term forecasts - especially when evaluating the impact of shocks that may not as yet have shown up in the economic data. However, it is hard to include such data in a forecasting model, and survey data, too, has sometimes been in error. Burns mentions the increasing pessimism of Autumn 1998 and winter 1998-1999 that failed to reflect the booming conditions of 1999.

  However, by 2000, there was indeed a recession in the U.S. Perhaps interpretations of these surveys were just a bit premature. Perhaps the expected recession was artificially delayed by the loose monetary policies leading up to the mythological advent of the millennium bug.

  When forecasts are used as one of the bases for economic policy shifts, they must cover the additional complication of what the impact of various policy shifts will be and how long the "lag" time will be before that impact manifests itself. Interest rate changes, for example, have historically had their maximum impact between 6 and 8 quarters after the change. Interest rate changes impact many other variables such as exchange rates, housing markets, inventories, investments and consumer spending.
 &
  Forecasting errors can thus lead to policy errors with consequences throughout the economy. Because of the lags, these consequences may not get adequately addressed for many months or even for more than a year.

  "As mentioned earlier, forecasting is more difficult when variables move sharply relative to trend. If a significant policy error takes the economy well away from its trend, the initial disturbance can persist for a long time."

  A major problem with interest rate policy is that higher interest rates are essential to eventually curb inflation - yet the rise in interest costs is immediately reflected in inflation statistics. Mechanical use of this data - that fails to adjust for this anomaly - has caused interest rate policy to overshoot the mark - staying high until the inflation data has visibly peaked - by which time the economy is already headed in the opposite direction.

  A more gradual approach is currently being attempted by the U.S. Fed - but the longer it takes for interest rates to rise sufficiently to deal with inflation, the higher they have to go to achieve that objective. Inflation rates have actually risen substantially during the two year period of gradually rising interest rates. If they were calculated using 1970s methods, they would be much higher.
 &
  Commodity price inflation data available from commodity markets can't be fudged - and commodity price inflation has been running at multiple double digit rates for several years. Industrial commodity inflation is currently running astoundingly in excess of 50%.

  Forecast errors can contribute to policy errors that increase volatility with widespread economic consequences. Volatility generally creates major problems for public finances and undermines confidence in both forecasts and policy.
 &

New approach to econometric modeling:

The theory that the "best" model should provide the "best" forecast has not yet been demonstrated.

&

  Confidence in econometric models remains elusive, Hendry and Ericsson note in a final chapter. The theory that the "best" model should provide the "best" forecast has not yet been demonstrated.

  "Many econometric models for forecasting are known to be seriously mis-specified, and the actual economy has been subject to important but unanticipated shifts, so forecast failure has been a relatively common phenomenon. Also, the implications of the above theory are inconsistent with the results of empirical forecasting competitions between many models on numerous time series. Simple methods often outperform better-fitting ones, and pooling of forecasts -- i.e., using an average of forecasts -- can pay."

The ability to rapidly recognize and reflect forecast errors - especially shifts in deterministic terms - data on average levels and period trends - is essential to reduce the impact of such errors and improve subsequent forecasts.

  Simpler more adaptive models thus often outperform more elaborate models, and long term forecasts often turn out to be more accurate than short term forecasts. Business cycle shifts generally average out.

  "A realistic alternative is to construct forecasts that adapt quickly after any mistake is discovered, so that systematic forecast failure does not ensue. Thus, econometric models might be redesigned to capture some of the robustness of the simple models that win forecasting competitions."

  The ability to rapidly recognize and reflect forecast errors - especially shifts in deterministic terms - data on average levels and period trends - is essential to reduce the impact of such errors and improve subsequent forecasts.

  This type of "robustness" will indeed reduce the size of long-term forecast errors, but does absolutely nothing to improve the ability to forecast the shifts and turning points. That must still be done as a matter of professional analysis and opinion - not mathematical doodling often based on "mis-specified" - invalid - theory and unreliable statistics.

  Nevertheless, sophisticated representations of causal variable connections remain essential for policy purposes. Only they can offer "rigorous" estimates of the impacts of policy shifts.

  "Rigorous," yes - as a matter of what is widely expected by the profession. But how accurate do those estimates turn out to be? "Rigor" in the application of invalid concepts is useless. This otherwise commendable book fails to evaluate this important point. Indeed, the accuracy of policy estimates is almost never subject to subsequent evaluation.
 &
  Economist Timothy Kehoe, Univ. of Minn., points out that econometric models drastically underestimated the NAFTA impact on trade flows. They failed to reflect the explosion of exports of goods not previously substantially traded. See, "The Economist," 7/15/06, pp. 68-69.
 &
  Economist Ross McKitrick, Univ. of Guelph (Canada), demonstrated how the way household and company responses are entered can drastically alter an econometric forecast of the response to a tax increase. Id. In short, an economist can easily make an econometric model deliver the result he wants. So much for "rigorous" mathematical economics.