What do the two things such that "Data is fixed" and "Parameters vary" in Bayesian statistics mean?

Question

While following the bayesian statistics online, the lecturer said that "Data is fixed" and "Parameters vary" in Bayesian statistics. But the explanations I got doesn't really make me understand what those things mean. The two things sound important to begin with the basic idea of Bayesian statistics. Hope to hear explanations.

Answer 1

Under a frequentist point of view you might have an unknown parameter, say $\theta$, that you want to estimate based on some data you have collected. You assume that this true and unknown parameter is fixed. Your data are expressed through a random variable, say $X$. So, for example you are interested in maximizing a likelihood based on the probability density function $f(X\mid\theta)$. This means that you model your collected data under a belief that the probability function of your data depends on that unknown parameter. You then can estimate that parameter say by maximizing the likelihood of the data (i.e. you consider the data random, in a sense that there are a random realization of the population that you study).

Under a bayesian point of view things are a bit reversed. You do not view the parameter $\theta$ as an unknown constant, i.e. fixed at some value, that you try to estimate. You rather consider that the parameter itself has a marginal distribution $f(\theta)$ which is called a prior. This expresses you prior beliefs regarding the parameter which now is viewed as a random variable since it follows a distribution. Under such a framework you might be interested in modelling $f(\theta\mid X)$, namely update your knowledge for $\theta$ GIVEN the data you have collected. Since now the data are given, they are not random hence the "data is fixed" that your lecturer mentioned.