Annie Liang

Abstract

Economists often use models estimated on data from a particular domain to make predictions in another. We provide a tractable formulation for this "out-of-domain" prediction problem and define the transfer error of a model based on its performance in a new domain. We derive finite-sample forecast intervals that are guaranteed to cover realized transfer errors with a user-selected probability when domains are i.i.d. We apply these intervals in an application to compare the transferability of economic models and black box algorithms, finding that black box algorithms outperform economic models for prediction within a domain, but generalize more poorly across domains.