How does Cosine Similarity work?

Cosine Similarity is a way to measure how similar two things are by looking at the angle between their vectors, giving a number between -1 and 1.

In this blog, we will learn about how Cosine Similarity works. We will also see what a vector is, how the angle between two vectors measures their similarity, the formula with a step-by-step numeric walkthrough, why we use the angle and not the distance, where it is used in the real world, and how it differs from Euclidean Distance.

We will cover the following:

• What is Cosine Similarity?
• What is a vector?
• The idea of direction and angle
• Where does the word "Cosine" come from?
• The Cosine Similarity formula
• A step-by-step numeric walkthrough
• Cosine Similarity in code
• Why we use the angle and not the distance
• Where Cosine Similarity is used
• Cosine Similarity vs Euclidean Distance

I am Amit Shekhar, Founder @ Outcome School, I have taught and mentored many developers, and their efforts landed them high-paying tech jobs, helped many tech companies in solving their unique problems, and created many open-source libraries being used by top companies. I am passionate about sharing knowledge through open-source, blogs, and videos.

I teach AI and Machine Learning at Outcome School.

Let's get started.

What is Cosine Similarity?

Cosine Similarity is a way to measure how similar two things are by looking at the angle between them.

In simple words, it gives us a single number that tells us whether two things point in the same direction or in different directions.

This number is always between -1 and 1.

• When the number is close to 1, the two things are very similar.
• When the number is close to 0, the two things are not related.
• When the number is close to -1, the two things are opposite.

What is a vector?

To compare things with numbers, we first turn each thing into a vector.

A vector is a list of numbers.

Let's say we want to describe a fruit using two features, sweetness and roundness. We can write a fruit as a list of two numbers.

• An apple could be [8, 9], which means sweetness is 8 and roundness is 9.
• A lemon could be [3, 8], which means sweetness is 3 and roundness is 8.

Here, each number in the list is called a feature. The apple is now a vector [8, 9]. The lemon is now a vector [3, 8].

In the real world, a vector can have hundreds or thousands of numbers. For the sake of understanding, we will use small vectors with only two or three numbers.

The idea of direction and angle

Now, let's understand direction and angle.

Let's say we draw a graph with two lines, one going right and one going up. The line going right is the sweetness. The line going up is the roundness.

Now, we place our apple [8, 9] on this graph. We start from the corner and move 8 steps to the right and 9 steps up. Then we draw an arrow from the corner to that point.

This arrow is the direction of the apple.

We do the same for the lemon [3, 8]. We move 3 steps right and 8 steps up, then draw an arrow.

Now we have two arrows. The angle is the gap between these two arrows.

Let's visualize this as below:

roundness
   ^
 9 |                * apple [8, 9]
 8 |      * lemon [3, 8]
   |
   |
   |
   +-------------------------> sweetness
   0      3         8

Here, we can see the two points. The lemon arrow goes from the corner 0 to the point [3, 8], and the apple arrow goes from the corner 0 to the point [8, 9]. The small gap between these two arrows is the angle that Cosine Similarity measures.

• When two arrows point in almost the same direction, the angle between them is small.
• When two arrows point in very different directions, the angle between them is large.

Cosine Similarity looks at this angle to decide how similar the two things are.

A small angle means the things are similar. A large angle means the things are different.

Where does the word "Cosine" come from?

Now, the question is, why do we call it Cosine Similarity and not Angle Similarity?

The reason is that we do not directly use the angle. We use the cosine of the angle.

Cosine is a simple math function that takes an angle and gives back a number between -1 and 1.

Cosine gives us exactly the values we need:

• When the angle is 0 degrees, which means the arrows point in the exact same direction, the cosine is 1.
• When the angle is 90 degrees, which means the arrows are at a right angle and unrelated, the cosine is 0.
• When the angle is 180 degrees, which means the arrows point in the exact opposite direction, the cosine is -1.

So, cosine turns the angle into a clean similarity score. A score of 1 means most similar, and a score of -1 means most opposite.

We can picture these three cases as below:

angle 0 degrees         angle 90 degrees        angle 180 degrees

  A B                     A
  | |                     |                     B <--- ---> A
  ↓ ↓                     |
                          +----> B

cosine = 1              cosine = 0              cosine = -1
most similar            unrelated               most opposite

Here, we can see that as the angle grows from 0 degrees to 90 degrees and then to 180 degrees, the cosine value smoothly drops from 1 to 0 and finally to -1.

This is why we call it Cosine Similarity.

Planning your move into AI?

No matter which tech domain you work in today, with your effort, you can transition to these roles:

• AI Engineer
• Forward Deployed Engineer
• AI Agent Engineer
• Harness Engineer
• Voice AI Engineer
• AI Architect
• LLM Engineer
• LLM Inference Engineer
• Gen AI Engineer
• MLOps Engineer
• ML Engineer
• Deep Learning Engineer

Our AI and Machine Learning Program at Outcome School takes you there.

The Cosine Similarity formula

Now, let's see the formula.

The Cosine Similarity between two vectors A and B is calculated like below:

Cosine Similarity = (A dot B) / (|A| * |B|)

This formula has two parts, the top part and the bottom part. Let's understand each part.

The top part: A dot B

This is called the dot product of the two vectors.

The dot product means we multiply matching numbers from both vectors and then add the results.

Let's say A is [a1, a2] and B is [b1, b2]. Then the dot product is calculated like below:

A dot B = (a1 * b1) + (a2 * b2)

Here, we multiply the first number of A with the first number of B, then multiply the second number of A with the second number of B, and finally add both results.

The bottom part: |A| * |B|

The symbol |A| is called the magnitude of the vector A.

The magnitude is the length of the arrow we drew earlier.

We calculate the magnitude by squaring every number, adding them, and taking the square root. For A = [a1, a2], the magnitude is calculated like below:

|A| = sqrt(a1^2 + a2^2)

So, the bottom part is just the length of arrow A multiplied by the length of arrow B.

The full formula divides the dot product by the product of the two lengths. This division removes the effect of length and keeps only the direction. We will see why this is important soon.

A quick note for you

No matter which tech domain you work in, get familiar with these topics:

• LLM
• RAG
• MCP
• Agent
• Fine-tuning
• Quantization

We put it all together in one video:

AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, and Quantization

No need to stop reading - bookmark it and watch later when you get time. Future you will thank you.

Now, let's get back to the topic.

A step-by-step numeric walkthrough

The best way to learn this is by taking an example.

Let's say we have two vectors:

• A = [8, 9]
• B = [3, 8]

Now, let's calculate the Cosine Similarity step by step.

Step 1: Calculate the dot product A dot B.

A dot B = (8 x 3) + (9 x 8)
        = 24 + 72
        = 96

Here, we multiplied 8 with 3 to get 24, then multiplied 9 with 8 to get 72, and added them to get 96.

Step 2: Calculate the magnitude of A.

|A| = square root of (8^2 + 9^2)
    = square root of (64 + 81)
    = square root of 145
    = 12.04

Here, we squared 8 to get 64, squared 9 to get 81, added them to get 145, and took the square root to get 12.04.

Step 3: Calculate the magnitude of B.

|B| = square root of (3^2 + 8^2)
    = square root of (9 + 64)
    = square root of 73
    = 8.54

Here, we did the same steps for B and got 8.54.

Step 4: Put everything into the formula.

Cosine Similarity = 96 / (12.04 x 8.54)
                  = 96 / 102.82
                  = 0.93

The result is 0.93. This number is very close to 1.

So, the apple and the lemon are quite similar in our example, because both are round fruits with some sweetness.

This is how we calculate Cosine Similarity by hand.

Cosine Similarity in code

Now, let's see the same calculation in code. We will write it in Python because it is simple to read.

We can write the code as below:

import math

def cosine_similarity(a, b):
    dot_product = 0
    for i in range(len(a)):
        dot_product += a[i] * b[i]

    magnitude_a = math.sqrt(sum(x * x for x in a))
    magnitude_b = math.sqrt(sum(y * y for y in b))

    return dot_product / (magnitude_a * magnitude_b)

vector_a = [8, 9]
vector_b = [3, 8]

result = cosine_similarity(vector_a, vector_b)
print(result)

Here, we can see what the code does:

• First, we calculate the dot_product by multiplying matching numbers and adding them.
• Then, we calculate magnitude_a and magnitude_b by squaring every number, adding them, and taking the square root.
• Finally, we divide the dot product by the product of the two magnitudes.

This will print the following:

0.9330948702273482

It works perfectly. The code gives us the same 0.93 that we calculated by hand.

Stay updated: Subscribe to our newsletter to get our latest AI and Machine Learning blogs straight to your inbox.

Why we use the angle and not the distance

Now, the next big question is, why do we care about the angle and not the plain distance between the two points?

Let's understand this with a simple example.

Suppose we have two documents about cars.

• Document A is a short note that says "car" 1 time and "engine" 1 time, so its vector is [1, 1].
• Document B is a long article that says "car" 10 times and "engine" 10 times, so its vector is [10, 10].

Both documents talk about the exact same topic in the same proportion. Only the length is different.

If we measure the plain distance between [1, 1] and [10, 10], the distance is large, so they look very different. But that is wrong, because they are about the same thing.

But both vectors point in the exact same direction. The angle between them is 0. So, the Cosine Similarity is 1, which means they are perfectly similar.

We can see it as below:

engine
   ^
10 |                   * B [10, 10]
   |                 .
   |               .
   |             .
   |           .
   |         .
   |       .
   |     .
   |   .
 1 | * A [1, 1]
   +----------------------> car
   0 1                 10

Here, we can see that arrow A is short and arrow B is long, but both lie on the exact same line, so the angle between them is 0. The plain distance between the two arrow tips is large, yet the direction is identical, which is why Cosine Similarity gives them a score of 1.

So, Cosine Similarity ignores the size and looks only at the direction. This is why it works so well for text, where a long document and a short document can still be about the same topic. We have a detailed blog on Semantic Search that explains step by step how this idea powers search by meaning.

Where Cosine Similarity is used

Now, let's see where we use Cosine Similarity in the real world.

• Search engines: to find documents that match what we typed.
• Recommendation systems: to suggest movies or products similar to what we already liked.
• Chatbots and AI assistants: to find the most relevant answer to our question.
• Plagiarism checkers: to find documents that are very similar to each other.
• Embedding search in LLM apps: text is turned into vectors called embeddings, and Cosine Similarity is used to find which pieces of text are closest in meaning.

To learn Embeddings, Vector Databases, and RAG in depth, check out our AI and Machine Learning Program at Outcome School.

Cosine Similarity vs Euclidean Distance

There is another popular way to compare vectors called Euclidean Distance.

Euclidean Distance is the straight-line distance between two points. It measures how far apart the two arrow tips are, not the angle between the arrows.

Let me tabulate the differences between Cosine Similarity and Euclidean Distance for your better understanding.

Here, we can see that both measure similarity in different ways. We pick one based on our use case.

When the size of the vector does not matter and only the direction matters, we use Cosine Similarity. When the actual distance matters, we use Euclidean Distance.

We have a detailed blog on how a Vector Database works that compares Cosine Similarity, dot product, and Euclidean Distance in depth.

Now, we have understood how Cosine Similarity works.

Frequently Asked Questions

Can Cosine Similarity be negative?

Yes. Cosine Similarity is always between -1 and 1, so it can be negative. A negative score means the two vectors point in different directions, and a score of -1 means they point in the exact opposite direction, which means the two things are opposite.

What does a Cosine Similarity of 0 mean?

A Cosine Similarity of 0 means the two things are not related. It happens when the angle between the two vectors is 90 degrees, so the arrows are at a right angle to each other. A score close to 1 means similar, and a score close to -1 means opposite.

What is the difference between the dot product and Cosine Similarity?

The dot product is only the top part of the Cosine Similarity formula. We multiply matching numbers from both vectors and add the results. Cosine Similarity then divides the dot product by the product of the two vector lengths, which removes the effect of length and keeps only the direction.

Does Cosine Similarity work for vectors with thousands of numbers?

Yes. In the real world, a vector can have hundreds or thousands of numbers, and the formula stays the same. We multiply matching numbers and add them for the dot product, and we square every number, add them, and take the square root for each magnitude.

That's it for now.

Thanks

Amit Shekhar
Founder @ Outcome School

You can connect with me on:

• X
• LinkedIn
• GitHub

Follow Outcome School on:

• X
• YouTube
• LinkedIn
• GitHub

Read all of our high-quality blogs here.