Conditional probabilities and independence:

Conditional probabilities give us the formal tools which allow us to talk about dependencies between events. We could model the patterns of language use in the Conan-Doyle story using word-confetti, but that would leave out the evident fact that ``Holmes'' follows ``Sherlock'' just as night follows day. The formal statement of this fact is that the conditional probability of the nth word being ``Holmes'' if the n-1th is ``Sherlock'' appears to be 1 for Conan-Doyle stories. The notation for this is

P(W_n = holmes | W_n-1 = sherlock) = 1

We also have notation for the joint event of the n-1th word being ``Sherlock'' and the nth ``Holmes''. This is:

P(W_n = holmes,W_n-1 = sherlock)

Because we are absolutely certain of the identity of the next word when we have seen the ``Sherlock'', it follows that:

P(W_n = holmes,W_n-1 = sherlock) =

$$P(W_{n-1} = sherlock) \times P(W_{n} = holmes \vert W_{n-1} = sherlock) =$$

P(W_n-1 = sherlock)

In general, for any pair of words, we will have:

$$P(W_{n} = w_{n},W_{n-1} = w_{n-1}) = P(W_{n-1} = w_{n-1}) \times P(W_{n} = w_{n}\vert W_{n-1} = w_{n-1})$$

which is usually written more compactly:

$$P(w_{n},w_{n-1} ) = P(w_{n-1}) \times P(w_{n}\vert w_{n-1})$$

While it is true that P(holmes_n| sherlock_n-1) = 1, it is definitely not true that P(sherlock_n-1| holmes_n) = 1, because the word ``holmes'' occurs frequently in contexts where it is preceded by something other than ``sherlock''. If someone tells us that the 354th word of the story is ``holmes'' (I haven't checked), then we cannot be certain that the 353rd is ``sherlock''. There is a better than even chance, but we cannot be sure. It remains the case that $(P(sherlock_{n-1},holmes_{n}) \equiv (P(sherlock_{n-1}\vert holmes_{n}) \times P(holmes_{n}$.