0
$\begingroup$

I normally use frequentist statistics but I now want to use Bayesian statistics as I want to carry out a two-sample (randomised control trial) test that includes prior information. I have an existing two-group sample and plan to collect a second two-group sample, which I would analyse for group differences, using the group differences from the first sample as the prior. (The reasons I want to do that are here but I don't think most of what's there is relevant for the current question.)

My problem is this. My standard go-to for Frequentist two-sample testing is bootstrapping the confidence interval on the differences for the means. I think this is usually better than a t-test because of having thrown out most (all?) of the parametric assumptions. But my searches for a Bayesian equivalent turn up nothing relevant. There are search results for e.g. non-parametric Bayesian or Bayesian bootstrap but they are about much more complex scenarios than a two-sample test.

I don't yet understand the Bayesian approach well but I don't see why it would inherently need to make assumptions about my data distributions. Are there methods that would give me a Bayes factor for my two-sample comparison without making parametric assumptions about the data distribution? If not, why not?

Some notes:

(1) The question is mainly to help me understand the theory, but I want to get the analysis done, and so I welcome recommendations for R code.

(2) A further complication is that my data is survey data with sample weights.

(3) My existing sample has n=500+500 and my second sample also would.

$\endgroup$
3
  • 1
    $\begingroup$ Theoretical justification for the bootstrap in frequentist analysis isn't exactly a piece of cake, and I remember I have seen asymptotical results that looked like it would need very similar assumptions to the Central Limit Theorem to work - which is the same situation in which a standard t-test will work as well (for large enough samples such as 2*500 appeal to the CLT is fine and there's not much worry about non-normality, unless there's worry for the bootstrap, too). $\endgroup$ Commented May 1 at 10:45
  • 1
    $\begingroup$ Check stats.stackexchange.com/q/447532/7224 $\endgroup$
    – Xi'an
    Commented May 1 at 19:20
  • $\begingroup$ Thanks, both these comments were useful, especially the link - it seems this is a thing, just a bit niche. $\endgroup$
    – Amorphia
    Commented May 3 at 9:12

0