Bossy Lobster

A blog by Danny Hermes; musing on tech, mathematics, etc.

Edit on GitHub

Bayes' Law Primer

I'm currently writing a blog post that uses Bayes' Law but don't want to muddy the post with a review in layman's terms. So I have something to link, here is a short description and a chance to flex my teaching muscles before the school year starts.

Bayes' Law

For those who aren't sure, Bayes' Law tells us that the probability event XX occurs given we know that event YY has occurred can easily be computed. It is written as Pr(XY)\text{Pr}(X \, | \, Y) vertical bar is meant like the word "given", in other words, the event XX is distinct from the event (XY),(X \, | \, Y), i.e. XX given YY. Bayes' law, states that

Pr(XY)=Pr(X and Y both occur)Pr(Y).\text{Pr}(X \, | \, Y) = \frac{\text{Pr}(X \text{ and } Y \text{ both occur})}{\text{Pr}(Y)}.

This effectively is a re-scaling of the events by the total probability of the given event: Pr(Y)\text{Pr}(Y).

For example, if XX is the event that a 33 is rolled on a fair die and YY is the event that the roll is odd. We know of course that Pr(Y)=12\text{Pr}(Y) = \frac{1}{2} since half of the rolls are odd. The event X and Y both occurX \text{ and } Y \text{ both occur} in this case is the same as XX since the roll can only be 33 if the roll is already odd. Thus

Pr(X and Y both occur)=16\text{Pr}(X \text{ and } Y \text{ both occur}) = \frac{1}{6}

and we can compute the conditional probability

Pr(XY)=1/61/2=13.\text{Pr}(X \, | \, Y) = \frac{1 / 6}{1 / 2} = \frac{1}{3}.

As we expect, one out of every three odd rolls is a 33.

Bayes' Law Extended Form

Instead of considering a single event Y,Y, we can consider a range of nn possible events Y1,Y2,,YnY_1, Y_2, \ldots, Y_n occur. We require that one of these YY-events must occur and that they cover all possible events that could occur. For example Y1Y_1 is the event that H2O is vapor, Y2Y_2 is the event that H2O is water and Y3Y_3 is the event that H2O is ice.

In such a case we know that since the YY-events are distinct

Pr(X)=Pr(X and Y1 both occur)+Pr(X and Y2 both occur)+Pr(X and Y3 both occur).\begin{aligned} \text{Pr}(X) &= \text{Pr}(X \text{ and } Y_1 \text{ both occur}) \\ &+ \text{Pr}(X \text{ and } Y_2 \text{ both occur}) \\ &+ \text{Pr}(X \text{ and } Y_3 \text{ both occur}). \end{aligned}

Using Bayes' law, we can reinterpret as

Pr(X and Yj both occur)=Pr(XYj)Pr(Yj)\text{Pr}(X \text{ and } Y_j \text{ both occur}) = \text{Pr}(X \, | \, Y_j) \cdot \text{Pr}(Y_j)

and the above becomes

Pr(X)=Pr(XY1)Pr(Y1)+Pr(XY2)Pr(Y2)+Pr(XY3)Pr(Y3).\begin{aligned} \text{Pr}(X) &= \text{Pr}(X \, | \, Y_1) \cdot \text{Pr}(Y_1) + \text{Pr}(X \, | \, Y_2) \cdot \text{Pr}(Y_2) \\ &+ \text{Pr}(X \, | \, Y_3) \cdot \text{Pr}(Y_3). \end{aligned}

The same is true if we replace 33 with an arbitrary number of events nn.