Probability Theory for Machine Learning — Overview
Sometimes it can be impossible to say what will happen from one minute to the next. But certain events are more likely to occur than others, and that’s where probability theory comes into play. Probability lets you predict the future by assessing how likely outcomes are, and knowing what could happen helps you make informed decisions.
Probability is a mathematical framework for reasoning about uncertainty.
Probability indicates how likely an occurrence is, or how unlikely. In stats-speak, an event is any occurrence that has a probability attached to it — in other words, an event is any outcome where you can say how likely it is to occur. Probability is measured on a scale of 0 to 1. If an event is impossible, it has a probability of 0. If it’s an absolute certainty, then the probability is 1. A lot of the time, you’ll be dealing with probabilities somewhere in between.
Probability of an event happening = Number of ways it can happenTotal number of outcomes
S is known as the possibility space, or sample space. Possible events are all subsets of S. It’s a shorthand way of referring to all of the possible outcomes.
Mutually Exclusive Events
When two events are Mutually Exclusive it is impossible for them to happen together.
Example : The probability of a card in a deck being King and a Queen is zero
P(King and Queen) = 0
P(King or Queen) = (1/13) + (1/13) = 2/13
Non-Mutually Exclusive Events are events that can happen at the same time
In Non-Mutually Exclusive Events, if you add P(A) and P(B), you count the intersection part twice. You should never count an element twice to find out how many you have. If you have counted it twice, there is an easy way to correct for that: subtract it once.
Complimentary Events: Two events are said to be complementary when one event occurs if and only if the other does not take place. Here both the events are mutually exclusive. A’ is the complementary event of A. It’s the probability that event A does not occur.
Conditional Probabilities
Conditional probabilities measure the probability of one event occurring relative to another occurring. If we want to express the probability of one event happening given another one has already happened, we use the “|” symbol to mean “given.”
P(A and B) is the probability of getting both A and B. With this probability, you can make no assumptions about whether one of the events has already occurred. You have to find the probability of both events happening without making any assumptions.
P(A | B) is the probability of event A given event B. In other words, you make the assumption that event B has occurred, and you work out the probability of getting A under this assumption.
Law of Total Probability
Law of Total Probability gives a way of finding the total probability of a particular event based on conditional probabilities.
Bayes’ Theorem
Bayes’ Theorem is one of the most difficult aspects of probability.It gives you a means of finding reverse conditional probabilities, which is really useful if you don’t know every probability up front.
Applying Law of total probability
If two events A and B are independent, then
P(A | B) = P(A)
If this holds for any two events, then the events must be independent. Also
P(A ∩ B) = P(A) x P(B)
I can understand how you are feeling right now, This much knowledge in probability is quite sufficient for Machine Learning.
