4 Rules of Probability
In two minutes
Probability and statistics are two sides of the same coin. Understanding both and learning to apply them is crucial to understanding how the world works.
Most probability questions can be answered by applying a few basic rules repeatedly.
The following 3 rules apply to probabilities of individual events
Rule 0: The probability for an event occurring always ranges from 0 to 1.
Rule 1: The Complement Rule
If P(event) is the probability of the event happening. Then, the probability of an event not happening is equal to 1 minus P(event).
Rule 2: The Rule of Equally Likely Outcomes
If there are n possible outcomes and they are equally likely. Then P(event) is equal to the number of outcomes in the event divided by n.
The following 2 rules help estimate the probability of multiple events in terms of individual events
Rule 3: The Addition Rule
Take two events A and B. A and B are mutually exclusive if they cannot occur at the same time. Then, the probability of A or B happening is simply the sum of their individual probabilities.
But if they are not mutually exclusive, then it is given by adding both individual probabilities and subtracting the shared probability to control for double-counting
Rule 4: The Multiplication Rule
Now, consider two events A and B. A and B are independent when knowing that A will occur does not impact the probability of occurrence of B. If independent, the probability of both A and B occurring is given by multiplying their individual probabilities.
And if they are not independent, the probability of both events occurring is got by multiplying the probability of A with the probability of B when A is true, like below
Which is just re-ordering the below formula.
Here’s a more intuitive explanation of why that is true
These rules are used extensively when working with Bayes Theorem and using Bayes Factor to update beliefs. Probability also plays an important role in Hypothesis Testing.
