Years ago I wrote about correcting covariate shift by reweighting your data. Your features come from the wrong distribution (q), you care about a target (p), so you weight every observation by (\beta_i = p(x_i)/q(x_i)) and your estimates are unbiased again. I ended that post by admitting the weights “can be quite a bit off,” and waved at fixing it another day. Here is the more basic question I skipped. Even when the weights are exactly right, what do they cost you? Reweighting buys you...
