How does an LLM work?
- Authors
- Name
- Amit Shekhar
- Published on
An LLM works by predicting the next word. It turns text into tokens and embeddings, passes them through a Transformer using attention, and repeats word by word.
In this blog, we will learn about how an LLM works. We will also see how text is turned into tokens and embeddings, how the Transformer and attention work, how an LLM generates an answer word by word, how it is trained, what the context window and hallucination are, and where it is used in the real world.
We will cover the following:
- What is an LLM?
- What does it really do? (Predicting the next word)
- How a computer turns words into numbers (Tokens)
- How a token becomes meaning (Embeddings)
- The Transformer
- Attention
- How an LLM generates a full answer, word by word
- How an LLM is trained
- Pretraining and Fine-tuning
- The Context Window
- Why an LLM sometimes makes things up (Hallucination)
- Real-World Applications
I am Amit Shekhar, Founder @ Outcome School, I have taught and mentored many developers, and their efforts landed them high-paying tech jobs, helped many tech companies in solving their unique problems, and created many open-source libraries being used by top companies. I am passionate about sharing knowledge through open-source, blogs, and videos.
I teach AI and Machine Learning at Outcome School.
Let's get started.
What is an LLM?
An LLM is a model that has read a huge amount of text and learned to understand and produce human language.
Let's decompose the term:
LLM = Large Language Model
Let's understand each word.
Large means it has learned from a very large amount of text, such as public web pages, books, articles, and code. It is also large because it has a huge number of internal numbers that it adjusts while learning. These numbers are called parameters, and an LLM can have billions of them.
Language means it works with human language, like English, Hindi, Spanish, and etc. It reads our words and writes words back to us.
Model means it learns patterns from examples instead of being given exact rules.
Let's say a child reads thousands of storybooks over many years. After a while, the child can finish a sentence we start, write a new story, and answer questions. Nobody gave the child a rulebook for language. The child simply learned the patterns from all that reading.
An LLM learns in a very similar way. We show it an enormous amount of text, and it slowly learns how language works, and that too without us writing the exact rules for it.
What does it really do? (Predicting the next word)
At its core, an LLM does only one simple thing. It predicts the next word.
We give it some text, and it predicts the most likely next word that should come after it.
Let's take an example. Suppose we give it this text:
The sky is
The LLM looks at these words and predicts what word is most likely to come next. It will probably predict:
The sky is -> blue
The LLM has read so much text that it knows "blue" is a very common word to come after "The sky is".
Note: An LLM does not just pick one word. It calculates a probability for many possible next words, and then chooses one. For our example, it may calculate something like below:
blue -> 70%
clear -> 15%
grey -> 10%
falling -> 5%
Here, we can see that each possible next word gets a percentage, which is its probability. The word "blue" has the highest chance, so it will most likely be chosen. We have a detailed blog on Top-k and Top-p Sampling that explains how an LLM picks the next word from these probabilities.
Now, the next big question is: if an LLM only predicts one word, how does it write full paragraphs and long answers?
The answer is simple. It does this prediction again and again. We will understand this fully in the section on generating an answer.
How a computer turns words into numbers (Tokens)
An LLM cannot work with words directly. It only works with numbers.
So, the first step is to turn our text into numbers. To do this, we first break the text into small pieces. Each small piece is called a token.
A token is a small chunk of text. It can be a whole word, a part of a word, or even a single character.
Let's take an example. Suppose we have the sentence "I love learning". An LLM may break it into tokens like below:
"I love learning" -> [ "I" ] [ " love" ] [ " learn" ] [ "ing" ]
Here, we can see that the sentence was broken into four tokens. Notice that the word "learning" was split into two tokens, "learn" and "ing". This is because the LLM breaks text into common chunks, not always full words.
After breaking the text into tokens, each token is given a number, called a token ID. So our sentence becomes a list of numbers like below:
[ "I" ] -> 40
[ " love" ] -> 1842
[ " learn" ] -> 4673
[ "ing" ] -> 278
Here, we can see that every token is now just a number. The whole sentence "I love learning" becomes the list of numbers [40, 1842, 4673, 278]. This process of breaking text into tokens is called tokenization. These token IDs are just for the sake of example, as every LLM has its own list of tokens.
So, when we say an LLM predicts the next word, it really predicts the next token. We will keep saying "word" just for the sake of understanding.
How a token becomes meaning (Embeddings)
Now, a token ID like 1842 is just a label. The number itself does not carry any meaning. The number for "love" is not bigger or smaller than the number for "hate" in any meaningful way.
Now, the question is: how does the LLM understand what a word means? So, here comes the embedding to the rescue.
An embedding is a list of numbers that represents the meaning of a token.
Let's say each token is turned into a list of numbers like below:
"king" -> [ 0.8, 0.6, 0.1, ... ]
"queen" -> [ 0.8, 0.6, 0.9, ... ]
"apple" -> [ 0.1, 0.2, 0.7, ... ]
Here, we can see that "king" and "queen" have very similar numbers, because their meanings are similar. The word "apple" has very different numbers, because its meaning is very different. Words with similar meaning get similar numbers.
The LLM learns these embeddings on its own while it is trained. We do not set these numbers by hand. The network figures out, from reading lots of text, that "king" and "queen" are related, and it gives them similar numbers.
So, an embedding is how a plain token gets a meaning that the computer can work with.
Planning your move into AI?
No matter which tech domain you work in today, with your effort, you can transition to these roles:
- AI Engineer
- Forward Deployed Engineer
- AI Agent Engineer
- Harness Engineer
- Voice AI Engineer
- AI Architect
- LLM Engineer
- LLM Inference Engineer
- Gen AI Engineer
- MLOps Engineer
- ML Engineer
- Deep Learning Engineer
Our AI and Machine Learning Program at Outcome School takes you there.
The Transformer
Now, let's learn about the engine that powers every modern LLM. This engine is called the Transformer.
The Transformer is the architecture that an LLM uses to process language and predict the next word.
The Transformer was introduced in 2017 in a famous research paper titled "Attention Is All You Need". Before the Transformer, language models read text one word at a time, in order, which was slow and often lost track of earlier words. The Transformer can look at all the words at once and figure out how they relate to each other.
An LLM is built by stacking many Transformer layers one on top of another, like below:
input tokens (as embeddings)
↓
+-----------------------+
| Transformer layer 1 |
+-----------------------+
↓
+-----------------------+
| Transformer layer 2 |
+-----------------------+
↓
...
↓
+-----------------------+
| Transformer layer N |
+-----------------------+
↓
prediction for the next token
Here, we can see that our tokens enter at the top as embeddings. They pass through many Transformer layers, one after another. Each layer improves the understanding of the text a little more. At the very end, the network produces the prediction for the next token.
So, the Transformer is the engine, and the most important part inside this engine is called attention. We will learn about attention next.
Attention
Now, let's learn about the single most important idea inside a Transformer. It is called attention.
Attention is the mechanism that lets each word look at the other words in the sentence and decide which ones are important for its meaning.
The best way to learn this is by taking an example. Consider this sentence:
The animal did not cross the street because it was too tired.
Now, what does the word "it" refer to? Does "it" mean the animal, or the street? We humans know instantly that "it" means the animal, because a street cannot be tired.
For the LLM to understand this sentence correctly, the word "it" must pay attention to the word "animal". This is exactly what attention does. It connects "it" to "animal".
Let's picture this connection like below:
The animal did not cross the street because it was too tired
^ |
|____________________________________________________|
"it" pays strong attention to "animal"
Here, we can see that the word "it" looks back at all the earlier words and pays the strongest attention to "animal". This is how the LLM understands that "it" means the animal and not the street.
So, every word in the sentence does this. In an LLM, every word looks at itself and all the words before it, and decides how much attention to pay to each one. This way, the LLM builds an understanding of how all the words relate to each other.
To learn LLM Internals, Tokenization, Embeddings, the Attention Mechanism, and the Transformer Architecture, and even build a Large Language Model (LLM) from scratch, check out our AI and Machine Learning Program at Outcome School.
How an LLM generates a full answer, word by word
Now, let's connect everything and see how an LLM writes a full answer. Remember, an LLM only predicts the next word. So, how does it write a long answer?
The answer is that it predicts one word, adds it to the text, and then predicts the next word again. It repeats this loop until the answer is complete. This process is called autoregression, which simply means it feeds its own output back into itself.
Let's take an example. Suppose we ask it to complete "The capital of France is".
Step 1: The LLM reads "The capital of France is" and predicts the next word.
The capital of France is -> Paris
Step 2: Now it adds "Paris" to the text and predicts the next word again.
The capital of France is Paris -> .
Step 3: It keeps going until it decides the answer is complete and produces a special "stop" token.
We can picture this whole loop like below:
"The capital of France is" -> predict -> "Paris"
"The capital of France is Paris" -> predict -> "."
"The capital of France is Paris." -> predict -> [stop]
Here, we can see that each predicted word is added back to the text, and the LLM uses the longer text to predict the next word. This repeats again and again. This is exactly why an LLM seems to "type" the answer to us one word at a time.
So, a long answer is just next-word prediction repeated many times, very fast.
A quick note for you
No matter which tech domain you work in, get familiar with these topics:
- LLM
- RAG
- MCP
- Agent
- Fine-tuning
- Quantization
We put it all together in one video:
AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, and Quantization
No need to stop reading - bookmark it and watch later when you get time. Future you will thank you.
Now, let's get back to the topic.
How an LLM is trained
When an LLM is brand new, all its internal numbers, the parameters, are just random. So its first predictions are pure nonsense. The question is: how does it learn?
It learns by playing a simple game on a huge amount of text. The game is called next-word prediction. Let's understand it step by step.
Step 1: We take a real sentence from the training text, for example "The sun rises in the east". We hide the last word and show the rest to the LLM.
Show this: "The sun rises in the"
Hide this: "east"
Step 2: The LLM predicts the next word. At the start, it predicts something random and wrong, for example "banana".
Step 3: We compare its prediction with the correct word, which is "east". The difference between the prediction and the correct answer is called the loss, also called the error. A big loss means the LLM was very wrong.
Step 4: The LLM slightly adjusts its internal numbers so that next time it is a little more likely to predict "east". This adjustment is done by methods called backpropagation and gradient descent. Means, the network traces back its mistake and nudges every number a tiny bit in the direction that reduces the error.
We can picture the full training loop like below:
hide a word
↓
predict
↓
compare with the correct word
↓
measure the loss
↓
adjust the numbers
↓
repeat
Here, we can see that the LLM plays this guessing game on trillions of tokens, over and over. Each time, its numbers get a little better, and its predictions get a little more accurate. Slowly, the random network turns into a model that predicts language very well.
This is how an LLM learns, and the best part is that nobody labels this data by hand. The text itself provides the questions and the answers, because the correct next word is already there in the text.
Pretraining and Fine-tuning
Now, training an LLM happens in two main stages. Let's understand both.
Pretraining is the first stage. This is the big next-word prediction game we just learned, played on a massive amount of text from the internet, books, and code. After pretraining, the LLM knows a lot about language and the world, but it is just a raw text-completion machine. It does not yet know how to be a helpful assistant.
Fine-tuning is the second stage. Here, we further train the LLM on carefully chosen examples that show it how to follow instructions and be helpful, honest, and safe.
Pretraining gives the LLM its raw knowledge, and fine-tuning teaches it how to be useful.
Let's picture the two stages like below:
Huge internet text (trillions of tokens)
↓
+---------------+
| Pretraining |
+---------------+
↓
Raw LLM that knows language and facts
↓
+---------------+
| Fine-tuning |
+---------------+
↓
Helpful assistant that follows instructions
Here, we can see that pretraining comes first and builds raw knowledge, and fine-tuning comes after and shapes the LLM into a helpful assistant. A popular fine-tuning method uses human feedback, where humans rate the answers and the LLM learns to prefer the better ones. This method is called RLHF, which stands for Reinforcement Learning from Human Feedback.
So, the AI assistants we use every day are based on LLMs that have been both pretrained and fine-tuned.
Stay updated: Subscribe to our newsletter to get our latest AI and Machine Learning blogs straight to your inbox.
The Context Window
Now, let's learn about an important limit of every LLM. It is called the context window.
The context window is the maximum amount of text, measured in tokens, that an LLM can look at in one go.
We can think of it as the LLM's short-term memory. It can only pay attention to the text that fits inside this window.
Let's say an LLM has a context window of 100,000 tokens. This means it can read and consider about that many tokens of text at once, which includes both our question and its answer. If a conversation grows longer than the window, the application must drop or shorten the oldest text, and the LLM can no longer see it.
We can picture the context window like below:
[=================== context window ===================]
our question + earlier chat + the answer being written
|<-- everything here is visible to the LLM at once -->|
text older than this falls out and is forgotten
Here, we can see that everything inside the window is visible to the LLM, and anything that falls outside is forgotten. This is why a very long chat can sometimes make the LLM forget what we said at the very beginning.
Many modern LLMs have very large context windows, which means they can read entire books or large codebases at once. But the limit always exists.
Why an LLM sometimes makes things up (Hallucination)
Now, let's understand one important weakness of LLMs. Sometimes an LLM gives an answer that sounds confident and correct but is actually false. This is called a hallucination.
A hallucination is when an LLM produces information that is wrong or made up, while sounding completely confident.
Why does this happen? Let's remember our core idea. An LLM predicts the next most likely word. It does not look up facts in a database, and it does not truly "know" things the way a human does. It produces text that sounds right based on the patterns it learned.
So, if we ask about something it has not learned well, it may still produce a confident, smooth-sounding answer, because its job is to predict likely-sounding words, not to verify the truth.
Let's say we ask it for a book written by a certain author. If the LLM is unsure, it may invent a realistic-sounding book title that does not actually exist, simply because that title fits the pattern of a likely answer.
This is why we must always verify important facts that an LLM gives us, especially names, numbers, dates, and quotes.
Note: To reduce hallucinations, modern systems often connect the LLM to trusted sources of information. One popular method is called RAG, which stands for Retrieval-Augmented Generation. The system first fetches facts from a trusted source and then gives them to the LLM to use while answering. This way, the LLM answers based on real information instead of guessing.
Real-World Applications
Now, let's discuss where LLMs are used in our daily life. We use them more often than we realize.
- AI assistants: When we ask a question and get a written answer, an LLM is generating it.
- Writing help: Drafting emails, essays, and summaries.
- Coding help: Writing code, finding bugs, and explaining how code works.
- Translation: Converting text from one language into another.
- Summarization: Turning a long document into a short summary.
- Customer support: Answering common questions automatically for companies.
- Search: Giving direct answers instead of just a list of links.
Now we must have understood how an LLM works, from tokens and embeddings to attention, next-word prediction, and training.
Frequently Asked Questions
Does an LLM look up facts in a database?
No. An LLM does not look up facts in a database. It predicts the next most likely word based on the patterns it learned from a huge amount of text. This is why it can sometimes produce a confident answer that is wrong, and why we should verify important facts like names, numbers, dates, and quotes.
Does an LLM predict words or tokens?
An LLM really predicts tokens, not whole words. A token is a small chunk of text, which can be a whole word, a part of a word, or a single character. For example, "learning" can be split into "learn" and "ing". We often say "word" only to keep the explanation simple.
Why does an LLM show its answer one word at a time?
An LLM shows its answer one word at a time because it really generates it that way. It predicts one word, adds it to the text, and uses the longer text to predict the next word. It repeats this loop until it produces a special stop token, so a long answer is just next-word prediction repeated many times, very fast.
Who labels the data used to train an LLM?
Nobody labels the pretraining data by hand. The text itself provides both the questions and the answers, because the correct next word is already there in the text. During training, a word is hidden, the LLM predicts it, the loss is measured, and its parameters are adjusted, again and again on trillions of tokens.
Why does an LLM forget the start of a long chat?
An LLM forgets the start of a long chat because of its context window. The context window is the maximum number of tokens it can look at in one go. When a conversation grows longer than this window, the oldest text is dropped or shortened, and the LLM can no longer see it.
That's it for now.
Thanks
Amit Shekhar
Founder @ Outcome School
You can connect with me on:
Follow Outcome School on: