I have sometimes seen in published work that when estimating covariance matrices, off-diagonal elements are set to 0. For example, in this paper, $N$ neurons are recorded and authors wish to use the $N \times N$ covariance matrix $\Sigma$ of their responses for Linear Discriminant Analysis (LDA). But because the number of data points $K$ is low compared to $N \times N$, they set the off-diagonal elements to 0. From the supplementary section: "... we do not have enough data to obtain reliable covariance estimates ... As such, we assume independence of the stimulus responses within conditions (i.e., we set the off-diagonal entries to zero)."
I am wondering about the implications/justification of the procedure above. For example, let $N = 20$ and $K = 5$, then if we estimate $\Sigma$ with the usual covariance formula, we'll get a singular matrix with rank 5. We can't use this matrix in vanilla LDA. However, if we set the off-diagonal elements of this resulting matrix to 0, we have a full rank matrix that we can use in LDA. In some way it seems like we "gained" information by setting the off-diagonal elements to 0. The original rank-defficient matrix tells us that we don't have enough data for the problem, but this is not the case for the modified matrix.
Can this be seen as just using a strong prior on $\Sigma$, or regularization? Is there some theoretical justification for such a procedure that can helps understand its implications? Is this related to the procedure of fixing the structure of random effect matrices in random effects models (e.g. here and here)?
