If you replace missing returns (and indeed if you replace anything that can be used as an input of an investment strategy), it is strongly recommended to never use future information (it means: to replace a value at time $t$, do not use any data that have been available after $t$).
If you need a covariance matrix to replace returns of a stock $k$ at date $t$ (they are so many difference way), you are right that you thus should not use any data after $t$. Your covariance matrix should be estimated only using past data.
It will drive you to the traditional problem of estimating covariance matrices (so many possibilities... have a look on stack exchange only).
My advice would be to not really use the full covariance matrix you have in mind but
- create a point in time factor model, the "simplest" being a sliding PCA
- just estimate (from past data) the coefficients of a regression of the past returns of your stock $k$ with your factors as covariates (ie explanatory variables).
[EDIT] following a comment about my "never use future information" recommendation.
Unfortunately, there is often not time reversibility on markets, simply because the arrival of information has an asymmetric effect on price formation (see for instance Marcaccioli, Riccardo, Jean-Philippe Bouchaud, and Michael Benzaquen "Exogenous and endogenous price jumps belong to different dynamical classes" Journal of Statistical Mechanics: Theory and Experiment 2022, no. 2 (2022): 023403).
This is not really a matter a non stationarity (it is indeed worst than that): they are two effects than are layered, first time-revertible dynamics (when no exogenous information occur), and then one that cannot in general be reverted. So no need to take any risk: just use information in the past.
I am not saying it is impossible to make the correct change of variables and projections so that one ends up in a stationary environment, just that it is subtle and so it is better to avoid approaching this kind of problem.
