Skip to main content
Cross Validated

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

12
  • 2
    $\begingroup$ When you say "you do it the same way on both sets", do you mean: "use the same method to impute missing data in the test set, but NOT the same data"? $\endgroup$
    – colorlace
    Commented May 5, 2018 at 4:22
  • 2
    $\begingroup$ @colorlace Use the past/future analogy. You used the training set in the past, and imputed some values. You now get the test set in the future, and want to impute some of its values; you presumably will use the same method as before applied to the test data (though you are free to incorporate what you learned from the training data) $\endgroup$
    – Henry
    Commented May 5, 2018 at 14:16
  • 1
    $\begingroup$ If you "are free to incorporate what you learned from the training data", then how is that different from just not splitting before imputing. $\endgroup$
    – colorlace
    Commented May 7, 2018 at 22:51
  • 1
    $\begingroup$ Are you suggesting: You can inform your test set imputations with training data, but you can't inform training imputations with test data. ? $\endgroup$
    – colorlace
    Commented May 7, 2018 at 22:54
  • 4
    $\begingroup$ @colorlace: that final point is precisely what I am saying: nothing you do with the training data should be informed by the test data (the analogy is that the future should not affect the past), but what you do do with the test data can be informed by the training data (the analogy is that you can use the past to help predict the future) $\endgroup$
    – Henry
    Commented May 8, 2018 at 19:05