C52 Model Evaluation and Selection

July 1, 2009

In addition to quantitative assessment of economic growth using econometric models, business cycle analyses have been proved to be helpful to practitioners in order to assess current economic conditions or to anticipate upcoming fluctuations. In this paper, we focus on the acceleration cycle in the euro area, namely the peaks and troughs of the growth rate which delimitate the slowdown and acceleration phases of the economy. Our aim is twofold: First, we put forward a reference turning point chronology of this cycle on a monthly basis, based on gross domestic product and industrial production index. We consider both euro area aggregate level and country specific cycles for the six main countries of the zone. Second, we come up with a new turning point indicator, based on business surveys carefully watched by central banks and short-term analysts, in order to follow in real-time the fluctuations of the acceleration cycle.

Download the paper [409.86 KB]

October 1, 2005

This paper explores the usefulness of bagging methods in forecasting economic time series from linear multiple regression models. We focus on the widely studied question of whether the inclusion of indicators of real economic activity lowers the prediction mean-squared error of forecast models of US consumer price inflation. We study bagging methods for linear regression models with correlated regressors and for factor models. We compare the accuracy of simulated out-of-sample forecasts of inflation based on these bagging methods to that of alternative forecast methods, including factor model forecasts, shrinkage estimator forecasts, combination forecasts and Bayesian model averaging. We find that bagging methods in this application are almost as accurate or more accurate than the best alternatives. Our empirical analysis demonstrates that large reductions in the prediction mean squared error are possible relative to existing methods, a result that is also suggested by the asymptotic analysis of some stylized linear multiple regression examples.

Download the paper [416.69 KB]

October 1, 2005

We extend the standard approach to Bayesian forecast combination by forming the weights for the model averaged forecast from the predictive likelihood rather than the standard marginal likelihood. The use of predictive measures of fit offers greater protection against in-sample overfitting and improves forecast performance. For the predictive likelihood we show analytically that the forecast weights have good large and small sample properties. This is confirmed in a simulation study and an application to forecasts of the Swedish inflation rate where forecast combination using the predictive likelihood outperforms standard Bayesian model averaging using the marginal likelihood.

Download the paper [844.77 KB]

October 1, 2005

This paper considers the problem of model uncertainty in the case of multi-asset volatility models and discusses the use of model averaging techniques as a way of dealing with the risk of inadvertently using false models in portfolio management. Evaluation of volatility models is then considered and a simple Value-at-Risk (VaR) diagnostic test is proposed for individual as well as ‘average’ models. The asymptotic as well as the exact finite-sample distribution of the test statistic, dealing with the possibility of parameter uncertainty, are established. The model averaging idea and the VaR diagnostic tests are illustrated by an application to portfolios of daily returns based on 22 of Standard & Poor’s 500 industry group indices over the period 1995-2003. We find strong evidence in support of ‘thick’ modelling proposed in the forecasting literature by Granger and Jeon (2004).

Download the paper [421.05 KB]