4
$\begingroup$

Let's say that I have a bunch of independent samples, $X_1, X_2, \dots, X_n$ and that they all follow Exponential($\theta_i$) distributions. (So they all have pdf $f(x_i)=\theta_i\exp(-\theta_iy_i)$.) I don't know if all the $\theta_i$s are equal or not, so I will assume the worst and say they are not for generalization purposes. How do I find the maximum likelihood estimate of this?

Here's my work so far:

$L = L(\theta_1, \theta_2, \dots, \theta_n | x_1, x_2, \dots, x_n)=\prod \theta_i \exp(-\sum \theta_iy_i)$

$\ln(L)=\sum\ln(\theta_i) - \sum\theta_ix_i$

$d/d\theta_i=\sum\frac{1}{\theta_i} - \sum x_i = 0$

$\sum \theta_i = \sum \frac{1}{x_i}$

Here's where I'm stuck - how can I say anything about a single $\theta_i$? Is $\hat{\theta}_i=\frac{1}{x_i}$??

Improve this question
$\endgroup$
6
  • $\begingroup$ Hint: Can you solve this problem for the case $n=1$? What is your solution? $\endgroup$
    – whuber ♦
    Commented May 8, 2015 at 16:45
  • $\begingroup$ In that case I get $\hat{\theta}=\frac{1}{x}$, as I had thought... But can I generalize that and say that $\theta_i=\frac{1}{x_i}$ without a proof...? $\endgroup$
    – user76623
    Commented May 8, 2015 at 17:22
  • $\begingroup$ Since the $X_i$ are independent, do you think it possible to improve on the estimates of the $\theta_i$? That is, what kind of information does $X_j$ give you about $\theta_i$ for $i\ne j$? To obtain rigor, any correct intuition you might have about the answer can be translated into properties of the likelihood function. (Incidentally, your formula for $d/d\theta_i$ is incorrect--and that might be confusing you. The correct expression has no summations over all the $i$.) $\endgroup$
    – whuber ♦
    Commented May 8, 2015 at 18:11
  • $\begingroup$ I'm afraid you lost me and I don't understand... How is the expression for $d/d\theta_i$ incorrect? If I have a single $\ln(\theta_i)$, then the derivative of that will be $1/\theta_i$. I have a sum of $\ln(\theta_i)$, so the derivative becomes $1/\sum\theta_i$. Along the same lines, if I have a single $\theta_i*x_i$ then the derivative will be $x_i$. Because it's a sum, the derivative is $\sum x_i$. I must be missing something fundamental here, because I can't find any alternative. $\endgroup$
    – user76623
    Commented May 8, 2015 at 19:05
  • $\begingroup$ You are not reading the notation correctly. To illustrate, suppose $n=2$. Then $\log(L) = (\log(\theta_1) +\log(\theta_2)) - (\theta_1x_1 + \theta_2x_2)$. When you differentiate with respect to $\theta_1$, all you get is $1/\theta_1 - x_1$: there is no sum because $\theta_1$ appears in one and only one term of each part. $\endgroup$
    – whuber ♦
    Commented May 8, 2015 at 19:27

1 Answer 1

Reset to default
2
$\begingroup$

$\ln(L)=\sum\ln(\theta_i) - \sum\theta_ix_i$

$\partial/\partial\theta_i=\frac{1}{\theta_i} - x_i = 0 $ $\implies$ $\hat\theta_i = \frac{1}{x_i}$

And $\partial^2/\partial^2\theta_i=\frac{-1}{\theta_i^2} < 0\ \forall\ \theta_i$ so $\hat\theta_i=\frac{1}{x_i}$ is indeed where the maxima occurs.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.