How to construct a confidence interval of the mean, when the distribution is unknown and the sample is small?

Question

I'm looking for the preferred approach to construct a finite sized confidence interval for the population mean, assuming:

• The distribution of the population is unknown
• The sample size is low
• The population standard deviation is unknown

The usual approaches do not work in this setting:

• Using the usual t-distribution to construct the confidence interval is not possible because we do not assume normality
• We can't use the central limit theorem because of the low sample size
• The standard deviation is unknown, so we can't use Chebyshev's inequality

I found out that if we assume the distribution is unimodal & symmetric, we can construct a confidence interval for the population mean from a single value. However, it is unclear to me how to generalize this to higher sample sizes (say, 10 or 15 observations), and I wonder if the unimodal & symmetric assumptions are necessary.

Answer 1

In general this is impossible. Suppose you have a function $f$ for which it is claimed that for any population with a finite mean $\mu$, $f$ applied to a sample of size $n$ from that population will return a finite length interval which is a $100\alpha\%$ confidence interval for $\mu$. Let $I = f(0, ..., 0)$ be the confidence interval $f$ produces when every value in the sample is zero. Pick $c \notin I$. Now suppose the population distribution is as follows:

$$ P(X = 0) = 1 - P(X = \frac{c}{1 - \alpha^{\frac {1}{2n}}}) = \alpha^{\frac 1 {2n}} $$

Then the mean of this distribution is $c$ which does not belong to $I$. But with probability $> \alpha$, a sample of size $n$ from this distribution will consist of $n$ zeroes, so will give you an interval which does not contain the true population mean.

So $f$ does not work on this distribution.

(This argument assumes $f$ is deterministic. It can be tweaked to work even if $f$ is random.)