I'm a bit lost on understanding this formula in my bioinformatics text, and I appreciate any tips or advice.
Mutual Information, $\operatorname{MI}(X; Y)$ is: $$ \mu = \sum_x \sum_y p(xy) \log \left( \frac{p(xy)}{p(x) p(y)} \right). $$ If $X$ and $Y$ are independent r.v.'s, then $\operatorname{MI} = 0$.
By the way, It's part of this larger picture
Thank You very much.
Well, without a little more background on which parts you understand and which parts you don't, it's hard to gauge the level at which to pitch the explanation. However, the situation is this:
We have two discrete random variables X and Y, which in this case represent the nucleotide at a a particular position in a DNA sequence, so $X$ and $Y$ take on the values A, C, G, and T.
In the equation above, $p(xy)$ denotes ${\rm Pr}[X = x,\; Y = y]$, i.e. the probability that X takes on the value x and Y takes on the value Y. $p(x)$ denotes ${\rm Pr}[X = x]$, the probability that $X$ takes on the value $x$ without regard to $Y$, and similarly for $p(y)$. So, in other words, we could write this in more readable form as
$${\rm MI}(X; Y) = \sum_{x \in \{A, C, G, T\}} \sum_{y \in \{A, C, G, T\}} {\rm Pr}[X = x,\; Y = y] \log \left( \frac{{\rm Pr}[X = x, Y = y]}{{\rm Pr}[X = x] {\rm Pr}[Y = y]}\right)$$
If $X$ and $Y$ are independent, then
$${\rm Pr}[X = x,\; Y = y] = {\rm Pr}[X = x] {\rm Pr}[Y = y]$$
so we have
$$\log \left( \frac{{\rm Pr}[X = x, Y = y]}{{\rm Pr}[X = x] {\rm Pr}[Y = y]}\right) = \log 1 = 0$$
for every possible choice of $x$ and $y$, and therefore when we calculate $MI(X; Y)$ we're just adding up sixteen zeros, which explains why it says that $MI(X; Y) = 0$ when $X$ and $Y$ are independent.
If you're still getting used to probabilities and statistics, you're not going to pick something like this up on the first try. Try drawing out a 4x4 grid with the various possibilities, filling it in with different probabilities, and calculating the mutual information to see how it works.
EDIT: You can try it out at
https://docs.google.com/spreadsheet/ccc?key=0AhBSLKlaRyzedHhHX2d4LXlPR2lmMmVORzg3ZjBleUE&hl=en_US
Just make sure only to edit the yellow cells. You can also check out the Wikipedia article:
