Bayes Factor
In two minutes
Probably one of the most important formulas in probability, the Bayes theorem(and, as an extension, Bayes factor) helps us update our existing beliefs based on newly acquired data. Here’s the proof of Bayes Theorem in two minutes, if you are interested in it.
To understand the Bayes factor, we’ll first need to understand the difference between probability and odds and how they are related.
Probabilities vs. Odds
Consider a fair die, with this, the probability of getting an even number is given as follows:
i.e. the count of times an even number can occur divided by the total number of outcomes.
And the odds of getting an even number is given as follows:
i.e. the number of times an even number can occur to(often represented by a colon) the number of times an even number cannot occur.
Steps
Forming a hypothesis: We start by forming a hypothesis, say, there’s a 50% percent chance it’ll rain today.
In other words, the odds of it raining today is 1:1
Collecting Data: Next is collecting data on our hypothesis.
You realize that you are in southern California! You look at some statistics and find out that it only rains 1 day out of 10(i.e. 10% chance of rain) in SoCal.
Calculate Bayes Factor: The ratio of the probability of our belief occurring to the probability of the belief not occurring given the data is called the Bayes factor. It’s the amount of information we’ve learned about the hypothesis(it’ll rain today) from the data(we’re in SoCal).
Updating our Belief: Finally, we use the newly acquired data to update our hypothesis
As a result, our prior belief that it’s as likely to rain today gets updated to a lower chance of rain because we know we are in SoCal. This is called the posterior.
Okay, Now What?
We could chain multiple Bayes factors to repeatedly update our hypothesis based on any new data that we acquire.
Let’s say you look outside and you see clouds! And you estimate that the probability of raining when there are clouds is about 75%(a 3:1 odds).
That’s how simple it is to incorporate new data!
Note that the prior highly influences our results, so if you believe it’s highly likely to rain today(say a 10:1 odds) the results will be very different. But if you think about it, it’s how our minds work! not everyone has the same belief and this construct helps with that!
