What is Bayes' Theorem?
- Authors
- Name
- Amit Shekhar
- Published on
Bayes' Theorem is a rule that tells us how to update our belief when we get new evidence, turning a starting guess into a better, more informed guess.
In this blog, we will learn about Bayes' Theorem. We will also see the simple idea of prior and posterior behind it, the formula and each of its parts, a real example with numbers, why the answer surprises us, and where it is used.
We will cover the following:
- What is Bayes' Theorem?
- The simple idea behind it
- The formula, decomposed
- A real example with numbers
- Why the answer surprises us
- Where Bayes' Theorem is used
I am Amit Shekhar, Founder @ Outcome School, I have taught and mentored many developers, and their efforts landed them high-paying tech jobs, helped many tech companies in solving their unique problems, and created many open-source libraries being used by top companies. I am passionate about sharing knowledge through open-source, blogs, and videos.
I teach AI and Machine Learning at Outcome School.
Let's get started.
What is Bayes' Theorem?
Bayes' Theorem is a rule that tells us how to update our belief when we get new evidence.
We start with a guess. Then we see some new information. Bayes' Theorem tells us how to change our guess into a better, more informed guess.
We do this all the time in real life. Let's say we think it will not rain today. Then we look outside and see dark clouds. Now we change our mind and expect rain. We just updated our belief based on new evidence. Bayes' Theorem is the exact math for doing this.
The simple idea behind it
Before jumping into the formula, we must understand the core idea in plain words.
We have a belief before seeing evidence. We call this the prior.
Then we see some evidence.
After seeing the evidence, we form a new belief. We call this the posterior.
So, the whole idea is a simple flow:
Prior belief + New evidence = Updated belief
(before) (after)
Bayes' Theorem takes our belief from "before" to "after" in a correct way.
Planning your move into AI?
No matter which tech domain you work in today, with your effort, you can transition to these roles:
- AI Engineer
- Forward Deployed Engineer
- AI Agent Engineer
- Harness Engineer
- Voice AI Engineer
- AI Architect
- LLM Engineer
- LLM Inference Engineer
- Gen AI Engineer
- MLOps Engineer
- ML Engineer
- Deep Learning Engineer
Our AI and Machine Learning Program at Outcome School takes you there.
The formula, decomposed
Now, let's see the actual formula.
P(A | B) = (P(B | A) * P(A)) / P(B)
Let's break every piece down.
Here, A is the thing we want to know about, like "the person has a disease". And B is the evidence we saw, like "the test came back positive".
P(A | B)is what we want. It reads as "the probability of A given B". This is the updated belief. This is the posterior.P(A)is our belief before any evidence. This is the prior.P(B | A)is how likely the evidence is when A is true. This is the likelihood.P(B)is how likely the evidence is overall. This is the evidence.
So, in plain words, the formula says:
Posterior = (Likelihood * Prior) / Evidence
The best way to understand this is by taking an example with real numbers.
A real example with numbers
Let's say there is a disease, and it is quite rare.
- Only 1 out of every 100 people has the disease. So the prior is 1%.
- There is a test for it. When a person has the disease, the test correctly shows positive 99% of the time.
- But the test is not perfect. When a person does not have the disease, the test still wrongly shows positive 5% of the time.
Now, a person takes the test, and it comes back positive. What is the chance that this person actually has the disease?
Most people quickly say 99%, because the test is "99% accurate". Let's see if that is true.
The best way to see this clearly is to imagine 10,000 people and count.
10,000 people
/ \
1% have 99% do not
disease have disease
(100) (9,900)
/ \ / \
test + test - test + test -
(99) (1) (495) (9,405)
Let's read this diagram.
Out of 10,000 people, 100 have the disease. The test catches 99 of them. So we get 99 true positives.
The other 9,900 people do not have the disease. But the test wrongly flags 5% of them. And 5% of 9,900 is 495. So we get 495 false positives.
Now, let's count everyone who got a positive result:
Positive results = 99 (real) + 495 (false) = 594 total
Out of these 594 people who tested positive, only 99 actually have the disease.
So the real chance of having the disease after a positive test is:
99 / 594 = 0.1667 = about 16.7%
A quick note for you
No matter which tech domain you work in, get familiar with these topics:
- LLM
- RAG
- MCP
- Agent
- Fine-tuning
- Quantization
We put it all together in one video:
AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, and Quantization
No need to stop reading - bookmark it and watch later when you get time. Future you will thank you.
Now, let's get back to the topic.
Why the answer surprises us
The test is "99% accurate", but a positive result means only about a 16.7% chance of actually being sick. This surprises almost everyone.
Here is why this happens. The disease is very rare. So there are very few truly sick people to begin with. Even a small mistake rate on the huge healthy group creates many false positives. These false positives outnumber the true positives.
This is the main lesson of Bayes' Theorem. The prior matters a lot. When something is rare, even a good test gives many false alarms. We must combine the evidence with the prior, not look at the evidence alone.
Let's confirm this with the formula so we see it matches our counting.
- Prior,
P(A)= 0.01 - Likelihood,
P(B | A)= 0.99 - Evidence,
P(B)= the total chance of a positive result = (0.01 * 0.99) + (0.99 * 0.05) = 0.0099 + 0.0495 = 0.0594
P(A | B) = (0.99 * 0.01) / 0.0594 = 0.0099 / 0.0594 = 0.1667
We get about 16.7%, the same answer as our counting. It works perfectly.
Stay updated: Subscribe to our newsletter to get our latest AI and Machine Learning blogs straight to your inbox.
Where Bayes' Theorem is used
Let's see where Bayes' Theorem helps us in the real world.
- Spam filters. The filter starts with a belief about spam. Then it sees words in the email as evidence and updates the belief that the email is spam.
- Medical testing. As we saw, it tells doctors the real chance of a disease after a test result.
- Machine learning. A whole family of models called Naive Bayes uses this rule to classify things like text and reviews.
- Everyday reasoning. Any time we update a guess after new information, we are using the idea behind Bayes' Theorem.
Frequently Asked Questions
What is the difference between P(A | B) and P(B | A)?
They are not the same. P(A | B) is the probability of A given B, which is the posterior, our updated belief after seeing the evidence. P(B | A) is the likelihood, which tells us how likely the evidence is when A is true. In the disease example, P(B | A) is 99%, but P(A | B) is only about 16.7%.
How do we calculate the evidence P(B) in Bayes' Theorem?
P(B) is the total chance of seeing the evidence. We add the chance of the evidence when A is true and the chance of the evidence when A is not true. In the disease example, the total chance of a positive result is (0.01 * 0.99) + (0.99 * 0.05) = 0.0594.
Can we solve a Bayes' Theorem problem without the formula?
Yes. We can imagine a large group, like 10,000 people, and count how many fall into each case. In the disease example, counting gives 99 real positives out of 594 positive results, which is about 16.7%. The formula gives the same answer, so counting is a simple way to check our work.
How is Bayes' Theorem used in machine learning?
A whole family of models called Naive Bayes uses Bayes' Theorem to classify things like text and reviews. Spam filters use the same idea. The filter starts with a belief about spam, sees the words in the email as evidence, and updates its belief that the email is spam.
That's it for now.
Thanks
Amit Shekhar
Founder @ Outcome School
You can connect with me on:
Follow Outcome School on: