Working paper series: Forecast Evaluations

Mincer-Zarnovitz Quantile and Expectile Regressions for Forecast Evaluations under Asymmetric Loss Functions

Posted January 2014

Authors

Kemal Guler
Pin T. Ng
Zhijie Xiao

Abstract

Forecast is pervasive in all areas of applications in business and daily life and, hence, evaluating the accuracy of a forecast is important for both the generators and consumers of forecasts. There are two aspects in forecast evaluation: (1) measuring the accuracy of past forecasts using some summary statistics and (2) testing the optimality properties of the forecasts through some diagnostic tests. On measuring the accuracy of a past forecast, we illustrate that the summary statistics used should match the loss function that was used to generate the forecasts. If there is strong evidence that an asymmetric loss function has been used in the generation of a forecast, then a summary statistic that corresponds to that asymmetric loss function should be used in assessing the accuracy of the forecast instead of the popular RMSE or MAE. On testing the optimality of the forecasts, we demonstrate how the quantile regressions and expectile regressions set in the prediction-realization framework of Mincer and Zarnowitz (1969) can be used to recover the unknown parameter that controls the potentially asymmetric loss function used in generating the past forecasts. Finally, we apply the prediction-realization framework to the Federal Reserve’s economic growth forecast and forecast sharing in a PC manufacturing supply chain. We find that the Federal Reserves values over prediction approximately 1.5 times more costly than under prediction. We also find that the PC manufacturer weighs positive forecast errors (under forecasts) about four times as costly as negative forecast errors (over forecasts).