How does In-Context Learning work?

Authors
How does In-Context Learning work?

In-Context Learning is the ability of a large language model to learn a new task just from the examples and instructions in our prompt, without changing the model.

In this blog, we will learn about how In-Context Learning works. We will also see why we need it, how it works, how zero-shot, one-shot, and few-shot learning differ, how the context window limits it, how it compares with fine-tuning, and how to get the best results from it.

We will cover the following:

  • What is In-Context Learning?
  • Why do we need In-Context Learning?
  • How does In-Context Learning work?
  • Zero-shot, One-shot, and Few-shot learning
  • A worked example, step by step
  • The role of the context window
  • In-Context Learning vs Fine-tuning
  • Best practices for In-Context Learning

I am Amit Shekhar, Founder @ Outcome School, I have taught and mentored many developers, and their efforts landed them high-paying tech jobs, helped many tech companies in solving their unique problems, and created many open-source libraries being used by top companies. I am passionate about sharing knowledge through open-source, blogs, and videos.

I teach AI and Machine Learning at Outcome School.

Let's get started.

What is In-Context Learning?

In-Context Learning is the ability of a large language model to learn a new task just from the examples and instructions we place inside our prompt, without changing the model itself.

Here, context means the text we provide in the prompt. Means, the model picks up the pattern from whatever we show it in the prompt, and then it answers in the same pattern.

The model does not change at all. Its internal knowledge stays exactly the same before and after. It only uses the examples in front of it to decide how to respond.

Why do we need In-Context Learning?

Let's understand the problem first.

Suppose we want a model to do a new task, like sorting customer reviews into "Happy" and "Unhappy".

Traditionally, we would collect thousands of reviews, label each one by hand, and then train the model on them. Training means feeding examples to the model again and again so that it adjusts its internal settings. This process is slow, costly, and needs powerful machines.

The issue with this approach is that it is expensive and time-consuming for every small task.

So, here comes In-Context Learning to the rescue.

With In-Context Learning, we do not train anything. We just write a few examples inside the prompt and ask the model to follow the same pattern. The model does the new task immediately, without any training.

How does In-Context Learning work?

Now, let's understand the working part.

To understand this, we must first know one simple fact about a large language model. The only thing the model does is predict the next word, again and again, based on all the text it has seen so far.

Let's say we type "The sky is". The model looks at these three words and predicts the most likely next word, which is "blue". Then it looks at "The sky is blue" and predicts the next word, and so on.

When we put examples in the prompt, those examples become part of the text the model is reading. The model treats our examples as a strong hint about what kind of word should come next.

The best way to learn this is by taking an example.

Suppose we send this prompt to the model:

Translate English to French.

English: cat
French: chat

English: dog
French: chien

English: house
French:

Here, we have given two examples first. We showed that "cat" becomes "chat" and "dog" becomes "chien". Then we asked for "house" and left the French part blank.

The model reads all of this as one long piece of text. It notices the clear pattern. Every English word is followed by its French translation. So, when it reaches the blank space after "house", it predicts the French word "maison".

The result will be:

maison

Let's visualize this flow as below:

       Prompt we send (the context)
+-------------------------------------+
|  English: cat   ->  French: chat    |
|  English: dog   ->  French: chien   |
|  English: house ->  French: ______  |
+-------------------------------------+
                   ↓
    Model reads it all as one text
                   ↓
        +---------------------+
        |  Spot the pattern:  |
        |  English -> French  |
        +---------------------+
                   ↓
        Predict the next word
                   ↓
             +----------+
             |  maison  |
             +----------+

Here, we can see that the two examples set up a clear pattern of English mapped to French. The model reads everything as one long text, spots that pattern, and then predicts the next word maison to continue it.

It works perfectly. We never trained the model. We simply showed the pattern, and the model continued it.

This is how In-Context Learning works. The examples in the context set up a pattern, and the model completes that pattern by predicting the next words.

Planning your move into AI?

No matter which tech domain you work in today, with your effort, you can transition to these roles:

  • AI Engineer
  • Forward Deployed Engineer
  • AI Agent Engineer
  • Harness Engineer
  • Voice AI Engineer
  • AI Architect
  • LLM Engineer
  • LLM Inference Engineer
  • Gen AI Engineer
  • MLOps Engineer
  • ML Engineer
  • Deep Learning Engineer

Our AI and Machine Learning Program at Outcome School takes you there.

Zero-shot, One-shot, and Few-shot learning

Now, let's learn the three common styles of In-Context Learning. The word shot here simply means one example. So, the number of shots means the number of examples we give.

Zero-shot means we give zero examples. We only give the instruction.

Let's see the prompt for zero-shot as below:

Translate the word "house" from English to French.

Here, we gave no example at all. We just gave a clear instruction. The model uses what it already knows and answers "maison". This works when the task is common and easy.

One-shot means we give exactly one example.

Let's see the prompt for one-shot as below:

English: cat
French: chat

English: house
French:

Here, we gave one example, "cat" becomes "chat". This single example helps the model understand the exact format we want. Then it answers "maison".

Few-shot means we give a few examples, usually two or more.

Let's see the prompt for few-shot as below:

English: cat
French: chat

English: dog
French: chien

English: bird
French: oiseau

English: house
French:

Here, we gave three examples. More examples make the pattern even clearer, so the model becomes more reliable. This is similar to the example we saw earlier.

Let's picture the three styles side by side as below:

   Zero-shot            One-shot             Few-shot

+--------------+     +--------------+     +----------------+
| (no example) |     | cat -> chat  |     | cat -> chat    |
+--------------+     +--------------+     | dog -> chien   |
| house -> ?   |     | house -> ?   |     | bird -> oiseau |
+--------------+     +--------------+     +----------------+
                                          | house -> ?     |
                                          +----------------+

fewer examples, weaker pattern  --->  more examples, stronger pattern

Here, we can see that as we move from zero-shot to few-shot, we add more examples to the prompt. More examples make the pattern stronger, so the model becomes more reliable.

If the task is simple, zero-shot is enough. If the task is tricky or needs a special format, few-shot is better.

A quick note for you

No matter which tech domain you work in, get familiar with these topics:

  • LLM
  • RAG
  • MCP
  • Agent
  • Fine-tuning
  • Quantization

We put it all together in one video:

AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, and Quantization

No need to stop reading - bookmark it and watch later when you get time. Future you will thank you.

Now, let's get back to the topic.

A worked example, step by step

Now, let's take a real use case to see In-Context Learning in action with a clear step-by-step walkthrough.

Suppose we want to sort customer messages into "Happy" or "Unhappy". This task is called sentiment classification. Sentiment means the feeling behind the text.

We will use a few-shot prompt as below:

Classify the message as Happy or Unhappy.

Message: The food was amazing and the staff was kind.
Sentiment: Happy

Message: I waited two hours and my order never came.
Sentiment: Unhappy

Message: Best purchase I have made this year.
Sentiment: Happy

Message: The product broke on the very first day.
Sentiment:

Let's walk through what happens inside, step by step.

Step 1: The model reads the instruction. It learns that the goal is to label each message as "Happy" or "Unhappy".

Step 2: The model reads the first example. A positive message maps to "Happy". The model notices this link.

Step 3: The model reads the second example. A negative message maps to "Unhappy". The pattern is now clearer.

Step 4: The model reads the third example. Another positive message maps to "Happy". The pattern is now strong.

Step 5: The model reaches the final message, "The product broke on the very first day." This message sounds negative. So, the model predicts the next word as "Unhappy".

The result will be:

Unhappy

The role of the context window

Now, there is an important limit we must understand.

The context window is the maximum amount of text the model can read at one time. It includes our examples, our instruction, our question, and the model's own answer.

We can picture the context window as below:

+================ Context Window ================+
|                                                |
|   [ Instruction ]                              |
|   [ Example 1 ]                                |
|   [ Example 2 ]                                |
|   [ Example 3 ]                                |
|   [ Our question ]                             |
|   [ Model's answer ]                           |
|                                                |
+================================================+
   Everything must fit inside this fixed space.

Here, we can see that the instruction, our examples, our question, and the model's answer all share one fixed space.

If we put too many examples in the prompt, we may cross the context window limit. When that happens, either the request fails or some text gets cut, and the model can no longer see it.

So, we must keep our examples short and relevant. We must include only the examples that help. We have a detailed blog on Context Engineering that covers how to decide what goes into the context.

In-Context Learning lives entirely inside the context window. The moment the conversation ends, the model forgets everything we taught it in the prompt. It does not remember it next time, because the model itself never changed.

Stay updated: Subscribe to our newsletter to get our latest AI and Machine Learning blogs straight to your inbox.

In-Context Learning vs Fine-tuning

Now that we have learned about In-Context Learning, it's time to compare it with another approach called fine-tuning, so we know when to use which one.

Fine-tuning is the process of training a model further on our own examples so that its internal settings change permanently. After fine-tuning, the model keeps the new behavior, even without examples in the prompt.

Let me tabulate the differences between In-Context Learning and Fine-tuning for your better understanding.

In-Context LearningFine-tuning
We give examples inside the prompt.We train the model on examples.
The model does not change.The model changes permanently.
No training cost, works instantly.Needs time, data, and computing power.
The model forgets after the chat ends.The model keeps the new behavior.
Limited by the context window size.Not limited by the context window.
Great for quick tasks and experiments.Great for large, repeated, specialized tasks.

So, when do we use which one?

We use In-Context Learning when we want a quick result, when we have only a few examples, and when we do not want to spend money on training. It works well for trying ideas fast.

We use fine-tuning when we have a large number of examples, when the task is very specific to our business, and when we will repeat the same task millions of times.

We choose based on our use case.

To master Fine-tuning, Parameter-Efficient Fine-Tuning (PEFT), and Prompt Engineering, we can check out our AI and Machine Learning Program at Outcome School, where we learn them in depth.

Best practices for In-Context Learning

Now, let's learn a few simple tips to get the best results from In-Context Learning.

Give clear instructions. Always start with a plain instruction that tells the model exactly what to do. A clear instruction removes confusion.

Use a consistent format. Keep every example in the same shape. If the first example uses "Message:" and "Sentiment:", every example must use the same labels. The model follows the format we set.

Pick good examples. Choose examples that cover the different cases the model will face. If we expect both happy and unhappy messages, we must show both kinds.

Keep it short. Remember the context window. Include only the examples that truly help. Do not fill the prompt with too many examples.

Order matters. Sometimes the last example influences the answer more than the first. So, place a strong, clear example near the end when needed.

We have a detailed blog on Prompt Engineering that explains how to write good prompts step by step.

Now, we have understood how In-Context Learning works.

Frequently Asked Questions

Does In-Context Learning change the model's weights?

No. In-Context Learning does not change the model at all. Its internal knowledge stays exactly the same before and after. The model only uses the examples in the prompt to decide how to respond, so the moment the conversation ends, it forgets everything we taught it in the prompt.

How many examples should we give in a few-shot prompt?

Few-shot usually means two or more examples. More examples make the pattern clearer, so the model becomes more reliable. But every example takes space in the context window, so we must keep the examples short and relevant, and include only the ones that truly help.

When is a zero-shot prompt enough?

A zero-shot prompt is enough when the task is simple and common, like translating a single word. The model uses what it already knows and follows the clear instruction. When the task is tricky or needs a special format, a few-shot prompt with examples works better.

Does the order of examples in the prompt matter?

Yes. Sometimes the last example influences the answer more than the first one. So, it helps to place a strong, clear example near the end of the prompt. We must also keep every example in the same format, because the model follows the format we set.

That's it for now.

Thanks

Amit Shekhar
Founder @ Outcome School

You can connect with me on:

Follow Outcome School on:

Read all of our high-quality blogs here.