Bayes Theorem – Explanation, Formula, Proof, Examples

Bayes Theorem: Bayes’ theorem is one of the most important concepts in Mathematics and Statistics which helps us decipher the world of probability in a better way. Probability is defined as the chance of occurrence of any particular event. It is named after the famous British Mathematician Thomas Bayes. It is a potent tool in probability theory that offers a methodical approach to revising probability estimates in light of fresh data. In this article, we will explore different aspects of this theorem including its statement, proof, and solved examples.

Bayes Theorem

The Bayes theorem defines the likelihood that an event connected to any condition will occur. It is also taken into account in the case of conditional probability. The Bayes theorem is frequently referred to as the “causes” formula of probability. It helps in determining the chance of occurrence of an event in light of another event that has already occurred. Let us understand this concept through an example:

Suppose there are 3 bags, and each bag has some green balls and some yellow balls inside. if a green marble is selected by chance. How likely is it that this green ball came from the first bag? We employ Bayes’ Theorem in situations like this. It is utilized for determining the likelihood that a specific event will occur based on other circumstances.

Bayes Theorem Statement

This theorem can be better understood by its formal statement. It is used to find the conditional probability of an event, say X, when another relevant event, say Y, has already occurred. The probability of the occurrence of an Event X, given the condition that an event Y has already occurred, will be given by the product of the probability of the event Y when X has already occurred and probability of X divided by the probability of the event Y. This theorem is also known as Bayes Rule or Bayes Law.

We can expand the Bayes Theorem statement for multiple events too. Let us suppose there are E1, E2, E3,……., En events connected to a sample space S, where each event has a non-zero chance of occurring and together they constitute a partition of S. If A is any event connected to S, then the Bayes theorem states that

P(E_i|A) = P(E_i)P(A|E_i) / ∑ P(E_k)P(A|E_k), where k can take values from 1 to n

Bayes Theorem Interpretation

From the above statement of the Bayes rule, we can say that it is related to the conditional probability, i.e., when an occurrence of an event depends on another event. In other words, the Bayes theorem basically describes the likelihood of an event based on knowledge of potential relevant conditions in advance. It is important to note that the probability of events must fulfill certain conditions to be able to qualify for the bayes theorem applicability. These conditions are:

•  Events participating in Bayes Theorem must be mutually exhaustive, i.e., A ∩ B must be Φ, where A and B are two associated events.
• Probability of all events must lie in the range of 0 to 1 (end values included).
• The sample space should be obtained by adding together all the events of the partition.

Bayes Theorem Formula

The Bayes’ theorem formula is the mathematical expression of the statement of Bayes Rule. It is a mathematical formula that estimates the likelihood that an event will occur based on knowledge of circumstances that may be connected to the event.

The Bayes Theorem Formula is given by:

P(H|E) = P(H ∩ E)/P(E)

OR

P(H|E) = P(H) . P(E|H)/P(E)

where, P(H|E) = probability of event H if event E has already occurred

P(E) = probability of Event E

P(H) = probability of event H

P(E|H) = probability of event E if event H has already occurred

Bayes Theorem Proof

We can prove the validity of Bayes rule using the conditional probability concepts. The proof of this theorem is given below:

Let E1, E2, E3,……., En events be connected to a sample space S. Suppose Ei denotes the partition of a sample space S. If an event A occurs, then we can write event A in terms of Ei.

By using the intersection rule, we can write

A = A ∩ S

=> A ∩ ($E_1,E_2,E_3,...,E_n$)

By using the distributive property of unions and intersections, we get:

A = (A ∩ E1) ∪ (A ∩ E2) ∪ (A ∩ E3)….∪ ( A ∩ En)

Taking probability on both sides

P(A) = P[(A ∩ E1) ∪ (A ∩ E2) ∪ (A ∩ E3)….∪ ( A ∩ En)]

As all the above pairs are disjoint. we know that two disjoint sets A and B can be expanded using the formula

P (A∪B) = P(A) + P(B)

Hence, P(A) = P(A ∩ E1) + P(A ∩ E2) + P(A ∩ E3)….+ P( A ∩ En) ———(i)

As per the multiplication theorem of dependent events,

P(A) = P(E). P(A|E1) + P(E). P(A|E2) + P(E). P(A E3)……+ P(A|En)

Using this rule in equation (i), we get:

Hence, total probability of P(A) =

P(E_i)P(A|E_i)………………………. (ii)

where i = 1, 2, 3, …., n

As we knwo by conditional probability that,

P(Ei|A) = P(Ei ∩ A)/P(A) ————(iii)

As we know that the conditional probability of P(A|E_i) is given by:

P(Ei ∩ A) = P(A|Ei) . P(Ei) ————–(iv)

putting the values of equation (ii) and equation (iv) in equation (iii), we get:

P(Ei|A) = P(A|Ei) . P(Ei)/

P(E_i)P(A|E_i)

Hence proved

Bayes Theorem Application in Real Life

This theorem improves our capacity to make wise decisions in the face of ambiguity and has broad applications in a variety of domains, including artificial intelligence and medical diagnosis. Some of the applications of bayes theorem in real-life are given below.

• Medical Diagnosis
• Spam Filtering
• Criminal Investigations
• Weather Forecasting
• Machine Learning

Bayes Theorem Solved Examples

Some of the solved questions on bayes theorem has been given below. These solved examples will help students to understand this topic in a better way.

Example 1: A task has been taken up by a person. With and without rain, there is a 0.44 and 0.95 chance that the job will be finished on schedule, respectively. Find the likelihood that the project will be finished on time if the probability of rain is 0.45.

Solution: Let E1 represent the timely completion of the mining project, and E2 represent rain.

We’ve got

P(A) = 0.45,

P(no rain) is equal to P(B) 

As P(B) = 1 – P(A) = 1 – 0.45 = 0.55.

The probability multiplication law states that

P(E1) = 0.44

P(E2) = 0.95

According to the total probability theorem, since occurrences A and B divide the sample space S, we have:

P(E) = P(E1) P(A) + P(E2) P(B)

=> 0.45 × 0.44 + 0.55 × 0.95

=> 0.198 + 0.5225 = 0.7205

Thus, there is a 0.7205 percent chance that the project will be finished on schedule.

Example 2: There are 4 white and 6 black balls in bag I, and 4 white and 3 black balls in bag II. A random ball is taken out of one of the bags, and it turns out to be black. Calculate the likelihood that it came from Bag I.

Solution: Assume that A is the event of drawing a black ball and that E1 and E2 are the events of selecting bag I and bag II, respectively.

So, P(E1) = P(E2) in case of random events

Hence, P(E1) = P(E2) = 1/2

Hence, P(A|E1) also equals P(drawing a black ball from Bag I) = 6/10

= 3/5.

P(A|E2) will be equal to P(drawing a black ball from the bag II)

= 3/7

So, the likelihood of drawing a black ball from bag I out of two bags can be calculated using Bayes’ theorem by using the formula:

P(E1|A) = [P(A|E1).P(E1)]/P(A|E1).P(E1) + P(A|E2).P(E2)

Putting the values of each term and solving it, we get

P(E1|A) = 7/12

Example 3: A man is known to tell the truth two out of three times. He throws a die and announces that the result is a four. Calculate the likelihood that the obtained number is a four.

Solution: Let A represent the situation where the man reports getting number 4.

Let E1 represent the occurrence of four and E2 represent its complementary event, i.e., 4 does not occur.

Therefore,

The probability that four will occur = P(E1) = 1/6

P(E2) = The probability that four does not occur = 1 – probability that 4 occurs.

So, P(E2) = 1 – P(E1)

or, P(E2) = 1 – 1/6 = 5/6

Additionally, P(A|E1)= Probability that the man reports four when it is indeed a four = 2/3.

P(A|E2) = likelihood that the man reports four when it is not a 4 = 1/3

Probability that the result is indeed a four, i.e., P(E1|A), according to Bayes’ theorem will be.

P(E1|A) = [P(A|E1).P(E1)]/P(A|E1).P(E1) + P(A|E2).P(E2)

on putting the respective values and solving the equation, we get

P(E1|A) = 2/7

Sharing is caring!