Eugene is a qualified control/instrumentation engineer Bsc (Eng) and has worked as a developer of electronics & software for SCADA systems.
What Is Probability Theory?
Probability theory is an interesting area of statistics concerned with the odds or chances of an event happening in a trial, e.g. getting a six when a dice is thrown or drawing an ace of hearts from a pack of cards. To work out odds, we also need to have an understanding of permutations and combinations. The math isn't terribly complicated, so read on and you might be enlightened!
What's covered in this guide:
- Equations for working out permutations and combinations
- Expectation of an event
- Addition and multiplication laws of probability
- General binomial distribution
- Working out the probability of winning a lottery
Before we get started let's review a few key terms.
- Probability is a measure of the likelihood of an event occurring.
- A trial is an experiment or test. E.g., throwing a dice or a coin.
- The outcome is the result of a trial. E.g., the number when a dice is thrown, or the card pulled from a shuffled pack.
- An event is an outcome of interest. E.g., getting a 6 in a dice throw or drawing an ace.
What Is the Probability of an Event?
There are two types of probability, empirical and classical.
If A is the event of interest, then we can denote the probability of A occurring as P(A).
This is determined by carrying out a series of trials. So, for instance, a batch of products is tested and the number of faulty items is noted plus the number of acceptable items.
If there are n trials
and A is the event of interest
Then if event A occurs x times
P(A) = x / n
Example: A sample of 200 products is tested and 4 faulty items are found. What is the probability of a product being faulty?
So x = 4 and n = 200
Therefore P(faulty item) = 4 / 200 = 0.02 or 2%
If we do the trial again with a different number of products, we can expect 2% of them to be faulty.
This is a theoretical probability which can be worked out mathematically.
If A is the event, then
P(A) = Number of ways the event can occur / The total number of possible outcomes
Example 1: What are the chances of getting a 6 when a dice is thrown?
In this example, there is only 1 way a 6 can occur and there are 6 possible outcomes, i.e. 1, 2, 3, 4, 5 or 6.
So P(6) = 1/6
Example 2: What is the probability of drawing a 4 from a pack of cards in one trial?
There are 4 ways a 4 can occur, i.e. 4 of hearts, 4 of spades, 4 of diamonds or 4 of clubs.
Since there are 52 cards, there are 52 possible outcomes in 1 trial.
So P(4) = 4 / 52 = 1 / 13
What Is the Expectation of an Event?
Once a probability has been worked out, it's possible to get an estimate of how many events will likely happen in future trials. This is known as the expectation and is denoted by E.
If the event is A and the probability of A occurring is P(A), then for N trials, the expectation is:
E = P(A) N
For the simple example of a dice throw, the probability of getting a six is 1/6.
So in 60 trials, the expectation or number of expected 6's is:
E = 1/6 x 60 = 10
Remember, the expectation is not what will actually happen, but what is likely to happen. In 2 throws of a dice, the expectation of getting a 6 (not two sixes) is:
E = 1/6 x 2 = 1/3
However, as we all know, it's quite possible to get 2 sixes in a row, even though the probability is only 1 in 36 (see how this is worked out later). As N becomes larger, the actual number of events which happen will get closer to the expectation. So for example when flipping a coin, if the coin isn't biased, the number of heads will be closely equal to the number of tails.
Success or Failure?
The probability of an event can range from 0 to 1.
P(Event) = Number of ways the event can occur / The total number of possible outcomes
So for a dice throw
P(getting a number between 1 and 6 inclusive) = 6 / 6 = 1 (since there are 6 ways you can get "a" number between 1 and 6, and 6 possible outcomes)
P(getting a 7) = 0 / 6 = 0 (there are no ways the event 7 can occur in any of the 6 possible outcomes)
P(getting a 5) = 1 / 6 (only 1 way of getting a 5)
If there are 999 failures in 100 samples
Empirical probability of failure = P(failure) = 999/1000 = 0.999
A probability of 0 means that an event will never happen.
A probability of 1 means that an event will definitely happen.
In a trial, if event A is a success, then failure is not A (not a success)
P(A) + P(not A) = 1
Independent and Dependent Events
Events are independent when the occurrence of one event doesn't affect the probability of the other event.
So if a card is drawn from a pack, the probability of an ace is 4/52 = 1/13.
If the card is replaced, the probability of drawing an ace is still 1/13.
Two events are dependent if the occurrence of the first event affects the probability of occurrence of the second event.
If an ace is drawn from a pack and not replaced, there are only 3 aces left and 51 cards remaining, so the probability of drawing a second ace is 3/51.
For two events A and B where B depends on A, the probability of Event B occurring after A is denoted by P(B|A).
Mutually Exclusive and Non-Exclusive Events
Mutually exclusive events are events that cannot occur together. For instance in the throwing of a dice, a 5 and a 6 can't occur together. Another example is picking coloured sweets out of a jar. if an event is picking a red sweet, and another event is picking a blue sweet, if a blue sweet is picked, it can't also be a red sweet and vice versa.
Mutually non-exclusive events are events that can occur together. For instance when a card is drawn from a pack and the event is a black card or an ace card. If a black is drawn, this doesn't exclude it from being an ace. Similarly if an ace is drawn, this doesn't exclude it from being a black card.
Addition Law of Probability
Mutually exclusive events
For mutually exclusive (they can't occur simultaneously) events A and B
P(A or B) = P(A) + P(B)
Example 1: A sweet jar contains 20 red sweets, 8 green sweets and 10 blue sweets. If two sweets are pickets are picked out, what is the probability of picking a red or a blue sweet?
The event of picking out a red sweet and picking out a blue sweet are mutually exclusive.
There are 38 sweets in total, so:
P(red) = 20/38 = 10/19
P(blue) = 10/38 = 5/19
P(red or blue) = P(red) + P(blue) = 10/19 + 5/19 = 15/19
Example 2 : A dice is thrown and a card is drawn from a pack, what is the possibility of getting a 6 or an ace?
There is only one way of getting a 6, so:
P(getting a six) is 1/6
There are 52 cards in a pack and four ways of getting an ace. Also drawing an ace is an independent event to getting a 6 (the earlier event doesn't influence it).
P(getting an ace) is 4/52 = 1/13
P(getting a six or an ace) = P(getting a six) + P(getting an ace)
= 1/6 + 1/13 = (13 + 6)/78 = 19/78
Remember in these type of problems, how the question is phrased is important. So the question was to determine the probability of one event occurring "or" the other event occurring and so the addition law of probability is used.
Mutually non-exclusive events
If two events A and B are mutually non-exclusive, then:
P(A or B) = P(A) + P(B) - P(A and B)
..or alternatively in set theory notation where "U" means the union of sets A and B and "∩" means the intersection of A and B:
P(A U B) = P(A) + P(B) - P(A ∩ B).
We effectively have to subtract the mutual events that are "double counted". You can think of the two probabilities as sets and we are removing the intersection of the sets and calculating the union of set A and set B.
Example 3: A coin is flipped twice. Calculate the probability of getting a head in either of the two trials.
In this example we could get a head in one trial, in the second trial or in both trials.
Let H1 be the event of a head in the first trial and H2 be the event of a head in the second trial
P(H1) = 1/2 and P(H2) = 1/2 (there is only one way a head can occur in each trial and two possible outcomes)
and P(H1 or H2) = P(H1) + P(H2) - P(H1 and H2)
There are four possible outcomes, HH, HT, TH and TT and only one way heads can appear twice. So P(H1 and H2) = 1/4
So P(H1 or H2) = P(H1) + P(H2) - P(H1 and H2) = 1/2 + 1/2 - 1/4 = 3/4
For more information on mutually non-exclusive events, see this article:
Taylor, Courtney. "Probability of the Union of 3 or More Sets." ThoughtCo, Feb. 11, 2020, thoughtco.com/probability-union-of-three-sets-more-3126263.
Multiplication Law of Probability
For independent (the first trial doesn't affect the second trial) events A and B
P(A and B) = P(A) x P(B)
Example: A dice is thrown and a card drawn from a pack, what is the probability of getting a 5 and a spade card?
P(getting a 5) = number of ways of getting a 5 / total number of outcomes
There are 52 cards in the pack and 4 suits or groups of cards, aces, spades, clubs and diamonds. Each suit has 13 cards, so there are 13 ways of getting a spade.
So P(drawing a spade) = number of ways of getting a spade / total number of outcomes
= 13/52 = 1/4
So P(getting a 5 and drawing a spade)
= P(getting a 5) x P(drawing a spade) = 1/6 x 1/4 = 1/24
Again it's important to note that the word "and" was used in the question, so the multiplication law was used.
Engineering Mathematics by K.A. Stroud is an excellent math textbook for both engineering students and anyone with a general interest in mathematics. The material has been written for part 1 of BSc. Engineering Degrees and Higher National Diploma courses.
A wide range of topics are covered including matrices, vectors, complex numbers, calculus, calculus applications, differential equations and series. The text is written in the style of a personal tutor, guiding the reader through the content, posing questions and encouraging them to provide the answer. Personally, I've found it really easy to follow.
It also covers a more in-depth treatment of probability theory than what has been covered in this article plus a section on statistics.
This book basically makes learning mathematics fun!
Note: Second hand 1987 editions of this text book are available on Amazon for only about $6
Summary of Probability Rules
The probability of an event has a value between 0 and 1 inclusive:
0 ≤ P(A) ≤ 1
The sum of all probabilities adds up to 1
If Ā is the compliment of A, or "not" A, i.e. event A not occurring, P(Ā) is the probability of A not occurring (or Ā occurring):
P(Ā) + P(A) = 1
It follows from rule 2 that the probability of an event not occurring is 1 - the probability of it occurring:
P(Ā) = 1 - P(A)
For two events A and B:
P(A and B) = P(A) x P(B)
For mutually exclusive events A and B
P(A or B) = P(A) + P(B)
For non mutually exclusive events:
P(A or B) = P(A) + P(B) - P(A and B)
= P(A) + P(B) - P(A) x P(B)
Permutations and Combinations
To solve more difficult problems and derive an expression for the probability of a general binomial distribution, we need to understand the concept of permutations and combinations. I won't go into the mathematics of the derivation, but basically the expression is derived from the equation for working out combinations.
A Permutation Is an Arrangement
A permutation is a way of arranging a number of objects. So, for instance, if you have the letters A, B, and C then all the possible permutations are:
ABC, ACB, BAC, BCA, CAB, CBA
Note that BA is a different permutation to AB.
If you have n objects, there are n factorial number of ways of arranging them, written as n!
n! = n x (n-1) x (n-2) .... x 3 x 2 x 1
The reason for this is because for the first position, there are n choices, and for each of these choices, there are (n-1) choices for the second place (because 1 choice was used up for the first place), and for each of the choices in the first two places, (n-3) choices for the third place and so on.
In the example above, the 3 letters A, B, C could be arranged in 3! = 3 x 2 x 1 = 6 ways
In general, if n objects are selected r at a time then, the number of permutations is:
n! / (n-r)!
This is written as nPr
Example: 2 letters are chosen from the set of letters A, B, C, D. How many ways can the 2 letters be arranged?
There are 4 letters so n =4 and r = 2
nPr = 4P2 = 4! / (4 - 2)! = 4! / 2! = 4 x 3 x 2 x 1 / 2 x 1 = 12
A Combination Is a Selection
A combination is a way of selecting objects from a set without regard to the order of the objects. So again if we have the letters A, B and C and select 3 letters from this set, there is only 1 way of doing this, i.e. select ABC.
If we select 2 letters at a time from ABC, all the possible selections are:
AB, AC, and BC
Remember, BA is the same selection as AB etc.
In general, if you have n objects in a set and make selections r at a time, the total possible number of selections is:
nCr = n! / ((n - r)! r!)
Example: 2 letters are chosen from the set ABCD. How many combinations are possible?
There are 4 letters so n = 4 and r = 2
nCr = 4C2 = 4! / ( (4 - 2)! x 2!) = 4! / (2! x 2!)
= 4 x 3 x 2 x 1 / ( (2 x 1) x (2 x 1) ) = 6
General Binomial Distribution
In a trial, an event could be getting heads in a coin throw or a six in a throw of a dice.
If the occurrence of an event is defined as a success, then
Let the probability of success be denoted by p
Let the probability of non-occurrence of the event or failure be denoted by q
p + q = 1
Let the number of successes be r
And n is the number of trials
Example: What are the chances of getting 3 sixes in 10 throws of a dice?
There are 10 trials and 3 events of interest, i.e. successes so:
n = 10
r = 3
The probability of getting a 6 in a dice throw is 1/6, so:
p = 1/6
The probability of not getting a dice throw is:
q = 1 - p = 5/6
P(3 successes) = 10! / ((10 - 3)! 3!) x (5/6)(10 - 3) x (1/6)3
= 10! / (7! x 3!) x (5/6)7 x (1/6)3
= 3628800 / (5040 x 6) x (78125 / 279936) x (1/216)
Note that this is the probability of getting exactly three sixes and not any more or less.
Winning the Lottery! How to Work out the Odds
We would all like to win the lottery, but the chances of winning are only slightly greater than 0. However "If you're not in, you can't win" and a slim chance is better than none at all!
Take, for example, the California State Lottery. A player must choose 5 numbers between 1 and 69 and 1 Powerball number between 1 and 26. So that is effectively a 5 number selection from 69 numbers and a 1 number selection from 1 to 26. To calculate the odds, we need to work out the number of combinations, not permutations, since it doesn't matter what way the numbers are arranged to win.
The number of combinations of r objects is nCr = n! / ((n - r)! r!)
n = 69
r = 5
nCr = 69C5 = 69! / ( (69 - 5)! 5!) = 69! / (64! 5!) = 11,238,513
So there are 11,238,513 possible ways of picking 5 numbers from a choice of 69 numbers.
Only 1 Powerball number is picked from 26 choices, so there are only 26 ways of doing this.
For every possible combination of 5 numbers from the 69, there are 26 possible Powerball numbers, so to get the total number of combinations, we multiply the two combinations.
So the total possible number of combinations = 11,238,513 x 26 = 292,201,338 or roughly 293 million and the probability of winning is 1 in 293 million.
Stroud, K.A., (1970) Engineering Mathematics (3rd ed., 1987) Macmillan Education Ltd., London, England.
This article is accurate and true to the best of the author’s knowledge. Content is for informational or entertainment purposes only and does not substitute for personal counsel or professional advice in business, financial, legal, or technical matters.
Questions & Answers
Question: I am playing a game with 5 possible outcomes. It is assumed that the outcomes are random. For sake of his argument let us call the outcomes 1, 2, 3, 4 and 5. I have played the game 67 times. My outcomes have been: 1 18 times, 2 9 times, 3 zero times, 4 12 times and 5 28 times. I am very frustrated in not getting a 3. What are the odds of not getting a 3 in 67 tries?
Answer: Since you carried out 67 trials and the number of 3s was 0, then the empirical probability of getting a 3 is 0/67 = 0, so the probability of not getting a 3 is 1 - 0 = 1.
In a greater number of trials there may be an outcome of a 3 so the odds of not getting a 3 would be less than 1.
Question: If you have nine outcomes and you need three specific numbers to win without repeating a number how many combinations would there be?
Answer: It depends on the number of objects n in a set.
In general, if you have n objects in a set and make selections r at a time, the total possible number of combinations or selections is:
nCr = n! / ((n - r)! r!)
In your example, r is 3
Number of trials is 9
The probability of any particular event is 1/nCr and the expectation of the number of wins would be 1/(nCr) x 9.
Question: Each Sign has twelve different possibilities, and there are three signs. What are the odds that any two people will share all three signs? Note: the signs can be in different aspects, but at the end of the day each person is sharing three signs. For example, one person could have Pisces as Sun sign, Libra as Rising and Virgo as Moon sign. The other party could have Libra Sun, Pisces Rising, and Virgo moon.
Answer: There are twelve possibilities, and each can have three signs = 36 permutations.
But only half of these are a unique combination (e.g., Pisces and Sun is the same as Sun and Pisces)
so that's 18 permutations.
The probability of a person getting one of these arrangements is 1/18
The probability of 2 people sharing all three signs is 1/18 x 1/18 = 1/324
Question: What if someone challenged you to never roll a 3? If you were to roll the dice 18 times, what would be the empirical probability of never getting a three?
Answer: The probability of not getting a 3 is 5/6 since there are five ways you can not get a 3 and there are six possible outcomes (probability = no. of ways event can occur / no of possible outcomes). In two trials, the probability of not getting a 3 in the first trial AND not getting a 3 in the second trial (emphasis on the "and") would be 5/6 x 5/6. In 18 trials, you keep multiplying 5/6 by 5/6 so the probability is (5/6)^18 or approximately 0.038.
Question: I have a 12 digit keysafe and would like to know what is the best length to set to open 4,5,6 or 7?
Answer: If you mean setting 4,5,6 or 7 digits for the code, 7 digits would of course have the greatest number of permutations.
© 2016 Eugene Brennan
Eugene Brennan (author) from Ireland on May 08, 2019:
Not offhand. However I did a quick Google search for "games of chance probability books" and several were listed. Maybe you could check them out on Amazon and there might be customer reviews.
maurrice on May 08, 2019:
Thank you Eugene for this tutorial. Very Interesting! Do you recommend any book which goes into more detail, ideally exploring games of chance, sports books etc?
Eugene Brennan (author) from Ireland on April 30, 2019:
The probability of the event is 1/6, so in 60 trials, the probability of that event is 1/6 + 1/6 + 1/6....... 60 times.
It's an "or" situation, so it's the probability of that event occurring in trial 1 or trial 2 or trial 3 etc up to trial 60.
So you add the probabilities.
If for instance you throw a dice and the event is getting a 6. Then if the question was "what is the expectation of getting a 6 in each trial", then you would multiply the probabilities because it's an "and" situation.
So it's the probability of a 6 in trial 1 and a 6 in trial 2 etc
= 1/6 x 1/6, 60 times = 1/6 ^ 60
Probability = number of ways event can occur / number of possible outcomes.
So taking the dice example again:
In two trials there's 12 ways you can get a 6:
1) 6 in the first trial and 6 other numbers in the second trial (6 possibilities)
2) 6 in the second trial and 6 other numbers in the first trial (6 possibilities)
The number of outcomes is 6 x 6 = 36
Since if you get 1 in the first trial, you can get 1 to 6 in the second trial
If you get 2 in the first trial, you can get 1 to 6 in the second trial and so on.
So probability = 12/36 = 1/3
So you get the same answer as by adding the probabilities because it’s an “or” situation
1/6 + 1/6 = 1/3
maurrice on April 29, 2019:
From the following section: What Is the Expectation of an Event?
Why is the answer calculated as 1/6 x 60?
Isn't it the same probability per trial, i.e.:
1st trial = 1/6 chance of getting any number
2nd trial = 1/6 chance of getting any number
and so on...
Therefore, why is it not calculated as (1/6)^60? What am I missing out/confusing, please?
Ekki on November 29, 2018:
Thank you so much for this article. It was most helpful. It answered questions that bothered me since the days in college!
Eugene Brennan (author) from Ireland on January 24, 2016:
Larry Rankin from Oklahoma on January 24, 2016:
Wonderful insight into odds.
Eugene Brennan (author) from Ireland on January 21, 2016:
Thanks LM, I learned this stuff in school over 30 years ago, but it was refreshing to revisit it!
LM Gutierrez on January 21, 2016:
Thanks for sharing and reiterating the basic mathematics we learn in our early years of schooling! Actually, this topic is very useful in real life even if you engange in a field which does not deal much on numbers such as mine. I agree with Jodah, well-researched hub!
Eugene Brennan (author) from Ireland on January 18, 2016:
Thanks Jodah and well spotted! That's what I get for racing through the proof reading!
John Hansen from Gondwana Land on January 18, 2016:
It's nice to know these equations and the odds of throwing certain numbers of dice, drawing a certain card etc. Very well researched hub , Eugene. However under the heading "Probability of an Event" it says; "There are two types of probability, empirical and empirical."(should the second one be "classical"?)