How does a Recommendation System work?
- Authors
- Name
- Amit Shekhar
- Published on
A Recommendation System is software that looks at what we like and suggests other things we are likely to enjoy, using methods like content-based and collaborative filtering.
In this blog, we will learn about how a Recommendation System works. We will also see the main techniques behind it, the problems that arise, and how big platforms like Netflix and YouTube put it all together.
We will cover the following:
- What is a Recommendation System?
- The goal of a Recommendation System
- Content-based filtering
- Collaborative filtering
- The user-item interaction matrix
- Explicit vs implicit feedback
- Matrix factorization and embeddings
- The cold-start problem
- The modern two-stage design: candidate generation and ranking
- How big platforms combine everything
I am Amit Shekhar, Founder @ Outcome School, I have taught and mentored many developers, and their efforts landed them high-paying tech jobs, helped many tech companies in solving their unique problems, and created many open-source libraries being used by top companies. I am passionate about sharing knowledge through open-source, blogs, and videos.
I teach AI and Machine Learning at Outcome School.
Let's get started.
What is a Recommendation System?
A Recommendation System is software that looks at what we like and suggests other things we are likely to enjoy.
It picks items for us from a very large list, so that we do not have to search through everything ourselves.
We use recommendation systems every day, even when we do not notice them.
- When we open Netflix, the rows of movies and shows are picked for us.
- When we watch one video on YouTube, the next suggestions appear on the side.
- When we shop on Amazon, we see "Customers who bought this also bought" sections.
- When we listen to Spotify, it builds a playlist of songs we have never heard but end up enjoying.
In all of these cases, no human is sitting and choosing items for us one by one. A recommendation system is doing the work behind the scenes.
So, why do we even need this? Let's understand the real problem first.
Suppose a platform has more than ten thousand movies and shows. If we had to scroll through all of them to find one good movie, we would give up before we even started. There are simply too many choices.
This is called the problem of too much choice. When there are thousands or millions of items, finding the right one becomes very hard.
So, here comes the Recommendation System to the rescue. Instead of showing us everything, it shows us the few items that we are most likely to enjoy. It saves our time and makes the experience pleasant.
The goal of a Recommendation System
Before we go into the techniques, we must be very clear about the goal.
The goal of a Recommendation System is to predict what a user will like, and then show the most relevant items at the top.
Let's break this into two parts.
Step 1: Predict. The system tries to guess how much we will like an item that we have not seen yet. For example, it tries to guess that we will give a particular movie 4.5 stars out of 5, even though we have never watched it.
Step 2: Rank and show. Once it has guessed a score for many items, it sorts them from highest score to lowest score. Then it shows the top few to us.
Now, the question is: how does the system predict what we will like? It all starts with two classic ideas, and we will build on top of them from there. Let's learn each one in detail.
Content-based filtering
Content-based filtering recommends items that are similar to the items we already liked, based on the features of those items.
Here, the key word is features. A feature is a property that describes an item.
Let's say we are talking about movies. The features of a movie can be:
- Its genre, such as action, comedy, or horror.
- Its actors.
- Its director.
- Its language.
- Its release year.
Now, suppose we watched and loved an action movie starring a particular actor. Content-based filtering will look at the features of that movie. It will then search for other movies that share those features, such as other action movies with the same actor. It will recommend those to us.
Let's see this with a tiny example. Assume that we describe each movie using just two features, "action level" and "romance level", each scored from 0 to 1.
Movie A (we loved it): action = 0.9, romance = 0.1
Movie B: action = 0.8, romance = 0.2 -> very similar
Movie C: action = 0.1, romance = 0.9 -> very different
Here, we can see that Movie B has feature values close to Movie A. So, the system recommends Movie B. Movie C is very far in its features, so it is not recommended.
This idea of "how close are two items" is called similarity. The system measures how similar two items are by comparing their feature values. Items that are close are similar. Items that are far apart are different. We have a detailed blog on Cosine Similarity that explains one popular way to measure this closeness.
Advantage: Content-based filtering works even if no other user has interacted with an item. As long as we know the features of the item, we can recommend it.
Disadvantage: It tends to keep recommending the same type of thing. If we only watched action movies, it will keep showing action movies, and we may never discover a great comedy. This problem is called being stuck in a filter bubble, which means we only see more of what we already know.
Now, it's time to learn about a very different idea.
Collaborative filtering
Collaborative filtering recommends items by using the behavior of many users, not the features of the items.
The word "collaborative" means working together. Here, the system uses the combined behavior of a huge crowd of users to help each single user.
The best part is that it does not need to know anything about the item itself. It does not need to know the genre, the actor, or the director. It only needs to know who liked what.
There are two types of collaborative filtering. Let's understand both.
User-based collaborative filtering is the idea of "people like you also liked this".
Let's say there are two users, Riya and Sam. They have both watched and liked the same nine movies. So, Riya and Sam have very similar taste.
Now, Sam has also watched and liked one more movie that Riya has not seen yet. Since Riya and Sam are so similar, the system guesses that Riya will like that movie too. So, it recommends it to Riya.
This is the meaning of "people like you also liked this". The system finds users similar to us and recommends what they enjoyed.
We can visualize this as below:
Riya liked: [Movie1] [Movie2] ... [Movie9]
Sam liked: [Movie1] [Movie2] ... [Movie9] [Movie10]
↓
Same nine movies, so same taste
↓
Riya has not seen Movie10 yet
↓
Recommend [Movie10] to Riya
Here, we can see that Riya and Sam liked the same nine movies, so the system treats them as similar users. Sam liked one extra movie, Movie10, that Riya has not seen. Since their taste matches, the system recommends Movie10 to Riya.
Item-based collaborative filtering is the idea of "people who liked this also liked that".
Here, instead of finding similar users, we find similar items based on how people behave.
Let's say a large number of people who bought a phone also bought a particular phone cover. The system notices this pattern. So, when a new customer buys that phone, it recommends the same phone cover, because people who liked one thing also liked the other.
Note: Item-based collaborative filtering does not look at the features of the phone or the cover. It only notices that the same people tend to like both. This is what makes it different from content-based filtering, which compares actual item features.
Let me tabulate the differences between these approaches for your better understanding.
| Approach | What it uses | Core idea |
|---|---|---|
| Content-based filtering | Features of items | Recommend items similar to what we liked |
| User-based collaborative | Behavior of similar users | People like us also liked this |
| Item-based collaborative | Behavior across items | People who liked this also liked that |
Now, the next question is: how does the system actually store all this "who liked what" information? For this, we must learn about a structure.
The user-item interaction matrix
To do collaborative filtering, the system needs to record every interaction between users and items in an organized way. This is done using a user-item interaction matrix.
A user-item interaction matrix is a big table where each row is a user, each column is an item, and each cell tells us how that user interacted with that item.
Let's see a tiny example with four users and four movies. A number means the rating the user gave. A dash means the user has not interacted with that movie yet.
Movie1 Movie2 Movie3 Movie4
Riya 5 4 - -
Sam 5 - 4 -
Aman - - 5 4
Neha 1 2 - 5
Here, we can see that Riya rated Movie1 with 5 stars and Movie2 with 4 stars. The dash means we simply do not have that information yet.
In the real world, this matrix is enormous. A big platform can have millions of users and thousands of titles. Most users have watched only a tiny fraction of all titles.
This means almost all the cells in the matrix are blank. A matrix that is mostly empty is called a sparse matrix. "Sparse" simply means "mostly empty".
So, the real job of a recommendation system becomes very clear now. The system must fill in the blank cells. If it can guess the rating Aman would give to Movie1, then it can decide whether to recommend Movie1 to Aman.
This is how the abstract goal of "predicting preference" turns into a very concrete task: fill in the empty cells of the matrix.
Planning your move into AI?
No matter which tech domain you work in today, with your effort, you can transition to these roles:
- AI Engineer
- Forward Deployed Engineer
- AI Agent Engineer
- Harness Engineer
- Voice AI Engineer
- AI Architect
- LLM Engineer
- LLM Inference Engineer
- Gen AI Engineer
- MLOps Engineer
- ML Engineer
- Deep Learning Engineer
Our AI and Machine Learning Program at Outcome School takes you there.
Explicit vs implicit feedback
Before we learn how to fill the matrix, we must understand where the numbers in the matrix come from. The data comes in two types of feedback.
Explicit feedback is when the user directly tells us their opinion.
For example, giving a movie 5 stars, clicking a thumbs up, or writing a review. The user has clearly and intentionally expressed how they feel. This data is very clean and accurate.
But, here is the problem. Most users never bother to rate anything. We watch many videos but rarely click like or dislike. So, explicit feedback is rare and the matrix stays very empty.
So, here comes implicit feedback into the picture.
Implicit feedback is what we learn from the user's behavior, without the user telling us anything directly.
For example:
- Did the user watch the full movie or stop after two minutes?
- How long did they listen to a song?
- Did they click on the item?
- Did they add it to the cart?
Here, the user never said "I like this". But their behavior gives us a strong hint. If someone watched a whole movie twice, they most likely enjoyed it. If they skipped it after one minute, they did not.
Implicit feedback is not always reliable. We call such data noisy, which means it can sometimes point us in the wrong direction. For example, someone may have left a video playing while they walked away from the screen.
Let me tabulate the differences between explicit and implicit feedback for your better understanding.
| Feedback type | What it is | Example | Quality |
|---|---|---|---|
| Explicit | User states their opinion | Rating, like, review | Accurate but rare |
| Implicit | We infer from behavior | Watch time, clicks, purchases | Plenty but noisy |
So, implicit feedback is plentiful but we must be careful with it. Many modern systems rely mostly on implicit feedback, because there is so much more of it.
A quick note for you
No matter which tech domain you work in, get familiar with these topics:
- LLM
- RAG
- MCP
- Agent
- Fine-tuning
- Quantization
We put it all together in one video:
AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, and Quantization
No need to stop reading - bookmark it and watch later when you get time. Future you will thank you.
Now, let's get back to the topic.
Now that we understand the matrix and the feedback, it's time to learn a technique that fills the blank cells.
Matrix factorization and embeddings
Earlier we saw that the system must fill the blank cells of a huge, mostly empty matrix. A popular technique to do this is called matrix factorization.
Matrix factorization breaks the big user-item matrix into two smaller tables, one describing users and one describing items, in a way that lets us rebuild the original ratings.
Imagine that every person's taste can be described by a few hidden traits. For the sake of understanding, let's say there are just two traits: how much someone likes "action" and how much someone likes "romance".
So, we can describe each user with two numbers, like below:
Riya -> [action: 0.9, romance: 0.1]
Neha -> [action: 0.2, romance: 0.9]
A small list of numbers like this is called a vector. And in machine learning, when this vector captures the hidden meaning of something, we call it an embedding. So, an embedding is a list of numbers that captures the hidden traits of a user or an item.
In the same way, we can describe each movie with the same two traits, like below:
Movie1 (action film) -> [action: 0.95, romance: 0.05]
Movie4 (love story) -> [action: 0.1, romance: 0.9]
To guess how much a user will like a movie, we multiply the matching numbers and add them up. This operation is called the dot product.
Let's calculate how much Riya will like Movie1.
Riya = [0.9, 0.1]
Movie1 = [0.95, 0.05]
dot product = (0.9 x 0.95) + (0.1 x 0.05)
= 0.855 + 0.005
= 0.86 (high score -> recommend it)
Now let's calculate how much Riya will like Movie4, the love story.
Riya = [0.9, 0.1]
Movie4 = [0.1, 0.9]
dot product = (0.9 x 0.1) + (0.1 x 0.9)
= 0.09 + 0.09
= 0.18 (low score -> do not recommend it)
Here, we can see that the dot product is high when the user's tastes and the item's traits point in the same direction, and low when they do not. When the user embedding and the item embedding are aligned, the user is likely to like the item.
So, the whole trick of matrix factorization is to learn one embedding for each user and one embedding for each item, such that their dot product matches the known ratings. The two smaller tables we mentioned in the definition are exactly these: one table holding every user embedding, and one table holding every item embedding.
We can picture this split as below:
Big user-item matrix (mostly empty)
+---------------------------------+
| M1 M2 M3 M4 ... |
| ---------------------------- |
| 5 4 - - ... |
| 5 - 4 - ... |
| - - 5 4 ... |
+---------------------------------+
↓
broken into two small tables
↓
User embeddings Item embeddings
+-------------------+ +-------------------+
| Riya -> [.9, .1] | | M1 -> [.95, .05] |
| Neha -> [.2, .9] | | M4 -> [.1, .9 ] |
+-------------------+ +-------------------+
\ /
\ /
↓ ↓
dot product fills the blank cell
Here, we can see that the big and mostly empty matrix is replaced by two compact tables, one for users and one for items. To find any rating, even a blank one, we take the dot product of that user's embedding and that item's embedding. So, two small tables can approximately rebuild the entire huge matrix.
In our example we hand-picked "action" and "romance". In reality, the system does not know these traits in advance. It learns them automatically from the data.
The system starts with random numbers in every embedding. Then it looks at the ratings we already know, checks how wrong its guesses are, and slowly adjusts the numbers to reduce the error. After many rounds, the embeddings become meaningful on their own. We have a detailed blog on Gradient Descent that explains this step by step.
The system discovers hidden patterns of taste by itself, and that too without being explicitly programmed with what those patterns are. The traits it finds may not even have names we can describe, but they work.
Once the embeddings are learned, filling a blank cell is easy. We take the dot product of that user's embedding and that item's embedding.
To learn Embeddings, Gradient Descent, and Loss Functions in depth, check out our AI and Machine Learning Program at Outcome School.
The cold-start problem
Now, there is one big problem that every recommendation system must face. It is called the cold-start problem.
The cold-start problem happens when we have a brand new user or a brand new item with no interaction history, so the system has nothing to learn from.
There are two cases here.
New user: A person just signed up. They have not watched, rated, or clicked anything. Collaborative filtering cannot help, because it works on past behavior, and there is none. We do not know who is similar to this user.
New item: A new movie was just added. Nobody has watched it yet. So, item-based collaborative filtering cannot help, because no behavior pattern exists for it.
Now, how do real systems handle this? Let's see the common ways.
For a new user, the system can do the following:
- Ask a few quick questions when signing up, such as "pick three genres you enjoy". This gives some starting signal.
- Show the most popular items to everyone at first. Popular items are a safe guess for anybody.
- Use whatever we know, such as the country, the device, or the age group, to guess rough preferences.
For a new item, the system can do the following:
- Use content-based filtering, because even a brand new movie has features such as genre, actor, and director. We can recommend it to people who liked similar movies.
- Show it to a small group of users first to gather some early behavior, and then expand if the response is good.
Here, we can notice something important. Content-based filtering and collaborative filtering help each other. Collaborative filtering is powerful once we have data, and content-based filtering helps us when we have no data. This is why real systems combine both. A system that uses more than one technique together is called a hybrid system.
Stay updated: Subscribe to our newsletter to get our latest AI and Machine Learning blogs straight to your inbox.
The modern two-stage design: candidate generation and ranking
Now, let's move to how big platforms run their recommendations. Everything we learned so far is the foundation. Modern systems put it all together using a two-stage design.
Why two stages? Let's understand the problem first.
YouTube has a huge number of videos. When we open the app, the system cannot carefully score every single video for us. That would take far too long.
So, the work is split into two stages, like below:
Billions of items
↓
[ Stage 1: Candidate Generation ]
narrow down to a few hundred
↓
[ Stage 2: Ranking ]
carefully order the few hundred
↓
Top items shown to the user
Let's understand each stage.
Stage 1 is candidate generation. The job of this stage is to quickly narrow down from millions or billions of items to just a few hundred reasonable candidates.
This stage values speed over perfection. It does not need to find the perfect order. It only needs to throw away the huge mass of clearly irrelevant items and keep a small pile of promising ones.
Here, embeddings are very useful. The system takes the user's embedding and quickly finds items whose embeddings are close to it, using fast similarity search. In a fraction of a second, billions become a few hundred.
Note: A big platform often runs several candidate generators at once. One fetches items based on collaborative filtering, another based on what is trending, and another based on the user's recent activity. All their results are pooled together into one candidate list.
Stage 2 is ranking. Now we have a small shortlist of a few hundred candidates. The job of this stage is to carefully score and order them, so the very best items appear at the top.
Because the list is now small, we can afford to use a much more powerful and expensive model here. This is usually a deep learning model, which means a large neural network that can learn very complex patterns.
The ranking model looks at many signals together for each candidate, such as:
- The user's embedding and the item's embedding.
- The time of day and the device being used.
- How fresh or popular the item is.
- The user's recent watch history.
- How long similar users watched this item.
It combines all of these and predicts a final score, often the chance that we will click and stay engaged. Then it sorts the candidates by this score and shows us the top few.
This two-stage idea is the key to scale. The first stage is fast and rough to handle huge numbers. The second stage is slow and precise to handle a small shortlist. Together they give us results that are both quick and accurate. We have a detailed blog on how a Reranker works that covers the same two-stage idea in search.
This is how modern recommendation systems serve a huge number of people in real time.
If we want to go deep into Neural Networks, Model Deployment and Serving, and AI and ML System Design, we can explore our AI and Machine Learning Program at Outcome School, where we learn them from the ground up.
How big platforms combine everything
Now, let's tie everything together and see the full picture in a real platform.
A big platform does not pick only one technique. It uses all of them together, each where it is strongest.
Let's walk through one quick real-world flow. Suppose we open YouTube.
First, several candidate generators run together and pull a few hundred videos, some from our watch history, some trending, some from similar users.
Then, the deep-learning ranker scores each of those few hundred videos using our embedding, the time, the device, and many other signals.
After that, the videos are sorted from highest score to lowest score.
Finally, the top videos appear on our home screen, freshly chosen for us in a fraction of a second.
Every person sees a different home screen, because every person has a different embedding and a different history.
This is how a Recommendation System works.
Frequently Asked Questions
What is the difference between content-based filtering and item-based collaborative filtering?
Content-based filtering compares the actual features of items, such as genre, actor, and director. Item-based collaborative filtering does not look at item features at all. It only notices that the same people tend to like both items, as in "people who liked this also liked that".
What is a hybrid recommendation system?
A hybrid recommendation system uses more than one technique together, usually content-based filtering and collaborative filtering. Collaborative filtering is powerful once we have data, and content-based filtering helps when we have no data, such as for a brand new item. This is why real systems combine both.
Why do recommendation systems rely mostly on implicit feedback?
Recommendation systems rely mostly on implicit feedback because there is so much more of it. Most users never bother to rate anything, so explicit feedback like ratings and reviews is rare. Implicit feedback such as watch time, clicks, and purchases is plentiful, but it is noisy, so we must be careful with it.
What is a filter bubble in a recommendation system?
A filter bubble means we only see more of what we already know. It happens with content-based filtering, which tends to keep recommending the same type of thing. If we only watched action movies, it will keep showing action movies, and we may never discover a great comedy.
That's it for now.
Thanks
Amit Shekhar
Founder @ Outcome School
You can connect with me on:
Follow Outcome School on:
