Application out-of-sample forecasting in model selection on Nigeria exchange rate

In time series, several competing models may adequately fit a given set of data. At times choosing the best model may be easy or difficult. However, there are two major model selection criteria; it could be either in-sample or out-of-sample forecasts. This study was necessitated because Empirical evidence based on out-of-sample model forecast performance is generally considered more trustworthy than evidence based on in-sample model performance which can be more sensitive to outliers and data mining. And also the fact that Out-of-sample forecasts also better reflect the information available to the forecaster in real time was also an added motivation. Hence this study considered data from Nigeria exchange rate (Naira to US Dollar) from January 2002 to December 2018 comprising 204 observations. The first 192 observations were used for model identification and estimation while the remaining 12 observations were holdout for forecast validation. Three ARIMA models; ARIMA (0, 1, 1), ARIMA (1, 1, 2) and ARIMA (2, 1, 0) were fitted tentatively. Base on in-sample information criteria ARIMA (0, 1, 1) was the best model with minimum AIC, SIC and HQ information criteria. However, on the basics of out-of-sample forecast evaluation using RMSE, MSE, MAE, and MAPE, ARIMA (2, 1, 0) perform better than ARIMA (0, 1, 1). The implication of this study is that, a model that is best in the in-sample fitting may not necessary give a genuine forecasts since it is the same data that is used in model identification and estimation that is also use in forecast evaluation.