How does flow matching train a generative model?

Flow matching trains a generative model by teaching it the direction and speed to move from a random noise point toward a real data point, like a photo. During training, it draws a straight line between a noise point and a real data point and asks the model to predict the movement along that line.

In this blog, we will learn about how flow matching trains a generative model. We will also see what velocity and a flow mean, how a straight line between noise and data gives a simple training target, why training on single pairs still works, how we generate new samples with small steps, how flow matching compares with diffusion, and where it works well and where it fails.

We will cover the following:

• The core idea: moving from noise to data
• What is velocity?
• What is a flow?
• The problem: we do not know the path
• The trick: draw a straight line
• How flow matching trains the model
• A step-by-step example with numbers
• Why training on single pairs still works
• How we generate new samples
• Code example
• Flow matching vs diffusion
• Where it works well and where it fails

I am Amit Shekhar, Founder @ Outcome School, I have taught and mentored many developers, and their efforts landed them high-paying tech jobs, helped many tech companies in solving their unique problems, and created many open-source libraries being used by top companies. I am passionate about sharing knowledge through open-source, blogs, and videos.

I teach AI and Machine Learning at Outcome School.

Let's get started.

The core idea: moving from noise to data

A generative model learns from examples and then creates new examples that look like them. To a computer, a photo is a long list of numbers. So, creating a new photo means creating a new list of numbers that looks like a real photo.

Most modern generative models start from noise. Noise is a list of random numbers. As an image, it looks like the static on an old TV screen with no signal.

Random numbers are very easy to create. We can make as many as we want.

So, the plan is:

• Start with a random noise point.
• Slowly move it, step by step, until it becomes a realistic photo.

+----------------------------------------+
| Start: noise (random static)           |
+----------------------------------------+
                    ↓
+----------------------------------------+
| Journey: many small steps              |
+----------------------------------------+
                    ↓
+----------------------------------------+
| End: data (a real-looking cat photo)   |
+----------------------------------------+

Here, we can see that creating an image becomes a journey. The model's job is to guide this journey.

For the sake of understanding, let's imagine every data point as a dot on a 2D map. In reality, a photo has millions of numbers, which means millions of directions, but the idea stays exactly the same.

What is velocity?

To guide a journey, we need to know the velocity.

Velocity means the direction and the speed of movement.

Let's say we are at position 2 on a number line, and our velocity is +3 per second. After 1 second, we will be at position 5. If our velocity is -1, we move left to position 1.

On a 2D map, velocity is an arrow. The arrow points in the direction we must move, and its length shows how fast.

Position (2, 1) + velocity arrow (3, 1)
                   ↓
After 1 second, we reach (5, 2)

Here, we added the velocity arrow to the current position to get the next position.

What is a flow?

Now, imagine that every point on the map has its own arrow. It is like a river, where the water at every spot moves in a certain direction with a certain speed.

A flow is a smooth movement of points that follows these arrows over time.

If we drop a leaf anywhere in the river, the water carries it along a path. In the same way, if we drop a noise point on our map, the arrows carry it along a path.

We will use a time number t that goes from 0 to 1:

• At t = 0, the point is pure noise.
• At t = 1, the point is real-looking data.
• In between, the point is partly noise and partly data.

So, a generative model based on flows needs to learn one thing: the arrow (velocity) at every position and every time.

We will call this learned function v(x, t). It takes a position x and a time t, and returns the arrow.

If we learn v(x, t) well, we can drop any noise point into the river, follow the arrows from t = 0 to t = 1, and get new data.

The problem: we do not know the path

How do we teach the model the right arrows? We do not know the "correct" path from noise to data. Nobody tells us which noise point should become which cat photo.

Older methods, called continuous normalizing flows, tried to learn these arrows by simulating the whole journey again and again during training and checking how likely the real data was at the end. This was very slow and hard to scale to big images.

We need a much simpler way. So, here comes flow matching to the rescue.

The trick: draw a straight line

Flow matching was introduced in 2022 in the paper "Flow Matching for Generative Modeling" by Yaron Lipman and colleagues. A very similar idea called rectified flow was published around the same time by Xingchao Liu and colleagues.

Instead of searching for the path, we decide the path ourselves.

Step 1: Pick one random noise point. Call it x0.

Step 2: Pick one real data point, for example one real cat photo. Call it x1.

Step 3: Draw a straight line from x0 to x1.

   x0 (noise)  *-------------------------------*  x1 (real photo)
              t=0            t=0.5            t=1
                               *
                              xt (halfway mix)

Any point on this line at time t is a simple mix of the two:

xt = (1 - t) * x0 + t * x1

Here, at t = 0, we get x0, which is pure noise. At t = 1, we get x1, which is the real photo. At t = 0.5, we get half noise and half photo.

If we move along this straight line at a steady speed from t = 0 to t = 1, what is the velocity?

It is simply the difference between the end and the start:

velocity = x1 - x0

The velocity is the same at every point on this line, because it is a straight line with steady speed.

This gives us a clear training target. At the point xt, the correct arrow is x1 - x0. No simulation is needed.

Planning your move into AI?

No matter which tech domain you work in today, with your effort, you can transition to these roles:

• AI Engineer
• Forward Deployed Engineer
• AI Agent Engineer
• Harness Engineer
• Voice AI Engineer
• AI Architect
• LLM Engineer
• LLM Inference Engineer
• Gen AI Engineer
• MLOps Engineer
• ML Engineer
• Deep Learning Engineer

Our AI and Machine Learning Program at Outcome School takes you there.

How flow matching trains the model

Now, we have everything. Training goes like below:

Step 1: Pick a real data point x1 from the training data.

Step 2: Create a random noise point x0.

Step 3: Pick a random time t between 0 and 1.

Step 4: Compute the mixed point xt = (1 - t) * x0 + t * x1.

Step 5: Ask the model to predict the velocity: v(xt, t).

Step 6: Compare the prediction with the target x1 - x0. The error is the squared difference, which we call the loss.

Step 7: Update the model parameters a little to reduce the loss.

We repeat this millions of times, with different photos, different noise, and different times.

loss = average of (v(xt, t) - (x1 - x0))^2

Here, the loss is small when the predicted arrow is close to the target arrow. Training makes it smaller and smaller.

Flow matching turns training into a simple regression problem, which means predicting a number and comparing it with a known correct number.

A step-by-step example with numbers

The best way to learn this is by taking an example. Let's use a 1D number line to keep it very simple.

Let's say:

• Noise point x0 = -2
• Real data point x1 = 6
• Random time t = 0.25

Step 1: Compute the mixed point.

   xt = (1 - 0.25) * (-2) + 0.25 * 6
      = 0.75 * (-2) + 1.5
      = -1.5 + 1.5
      = 0

So, at t = 0.25, the point is at position 0.

Step 2: Compute the target velocity.

   target = x1 - x0 = 6 - (-2) = 8

So, the correct arrow is +8. Means, move to the right with speed 8.

Step 3: Check the model's guess.

Let's say the model predicts v(0, 0.25) = 5.

   loss = (5 - 8)^2 = 9

The loss is 9. The model is too slow. So, training updates the model to predict a bigger number next time at this position and time.

Step 4: Check that the target makes sense.

From position 0 at t = 0.25, if we move with velocity 8 for the remaining time of 0.75, we travel 8 * 0.75 = 6. So, we reach 0 + 6 = 6, which is exactly x1.

A quick note for you

No matter which tech domain you work in, get familiar with these topics:

• LLM
• RAG
• MCP
• Agent
• Fine-tuning
• Quantization

We put it all together in one video:

AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, and Quantization

No need to stop reading - bookmark it and watch later when you get time. Future you will thank you.

Now, let's get back to the topic.

Why training on single pairs still works

Now, the next big question is:

During training, we pair a random noise point with a random photo. But the same noise point could be paired with a cat photo in one step and a dog photo in another step. So, the target arrow at the same position keeps changing. How can the model learn one correct arrow?

The answer is: the model learns the average arrow.

When a model is trained with squared error, and it sees many different targets for the same input, the best answer it can give is the average of those targets.

Let's say, at some position, half the training pairs say "go up" and half say "go right". The model learns "go up and to the right", the average direction.

   pair A says:  ^  (toward a cat photo)
   pair B says:  >  (toward a dog photo)
   model learns: /  (average direction)

Here, we can see that the model blends all the possible directions at that point.

The flow matching paper proved that following these averaged arrows carries noise to the full data distribution, which means the whole collection of realistic data. Training with these per-pair targets is called conditional flow matching: each training target is computed for one specific pair, but the model ends up learning the overall flow.

Because of this averaging, the real paths are not perfectly straight. But they are often much straighter than the paths in older methods, and this helps a lot during generation.

How we generate new samples

Once training is done, we generate new samples as below:

Step 1: Create a new random noise point x. Set t = 0.

Step 2: Choose a number of steps, for example 4. So, each step size is dt = 1 / 4 = 0.25.

Step 3: At each step, ask the model for the arrow, and move a little:

x = x + v(x, t) * dt

t = t + dt

Step 4: After the last step, t = 1, and x is our new sample.

This way of moving with small steps is called the Euler method. It is the simplest way to follow arrows.

Let's continue our 1D example. Just for the sake of understanding, assume all our training data is the single point 6, and the model has learned well:

   t=0.00   x=-2.0   model arrow=+8   -->  x = -2.0 + 8*0.25 = 0.0
   t=0.25   x= 0.0   model arrow=+8   -->  x =  0.0 + 8*0.25 = 2.0
   t=0.50   x= 2.0   model arrow=+8   -->  x =  2.0 + 8*0.25 = 4.0
   t=0.75   x= 4.0   model arrow=+8   -->  x =  4.0 + 8*0.25 = 6.0
   t=1.00   x= 6.0   done

Here, we can see that the point travels from noise at -2 to data at 6 in just 4 steps.

Because the paths are fairly straight, we need only a small number of steps. A curved path needs many small steps to follow it correctly, while a straight path can be covered in a few big steps.

Stay updated: Subscribe to our newsletter to get our latest AI and Machine Learning blogs straight to your inbox.

Code example

Let's see the full training step and the generation loop in PyTorch as below:

import torch
import torch.nn as nn

model = nn.Sequential(nn.Linear(3, 128), nn.ReLU(), nn.Linear(128, 2))  # input: x (2 numbers) + t
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)

def train_step(x1):                       # x1: a batch of real 2D points
    x0 = torch.randn_like(x1)             # random noise
    t = torch.rand(x1.shape[0], 1)        # random time between 0 and 1
    xt = (1 - t) * x0 + t * x1            # point on the straight line
    target = x1 - x0                      # correct velocity
    pred = model(torch.cat([xt, t], dim=1))
    loss = ((pred - target) ** 2).mean()
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    return loss.item()

Here, we have:

• model is a small neural network. It takes the position xt (2 numbers) and the time t (1 number), and returns a velocity arrow (2 numbers).
• torch.randn_like(x1) creates noise of the same shape as the data.
• torch.rand picks a random time for each example.
• xt and target are exactly the two formulas we learned above.
• loss is the squared difference between the predicted arrow and the target arrow.
• loss.backward() and optimizer.step() update the model parameters to reduce the loss.

Now, let's see the generation code as below:

@torch.no_grad()
def generate(num_samples, steps=20):
    x = torch.randn(num_samples, 2)       # start from noise
    dt = 1.0 / steps
    for i in range(steps):
        t = torch.full((num_samples, 1), i * dt)
        x = x + model(torch.cat([x, t], dim=1)) * dt   # follow the arrow
    return x

Here, we start from random noise and take 20 small steps. At each step, the model gives the arrow, and we move the points a little. At the end, the points look like the training data.

For real images, the small network is replaced by a big network, often a Transformer, and the 2 numbers become millions of numbers. But the training code stays almost the same.

To learn PyTorch, Neural Networks, and Loss Functions from the ground up, check out our AI and Machine Learning Program at Outcome School.

Flow matching vs diffusion

Flow matching is closely related to diffusion models. A diffusion model also starts from noise, but it learns by adding noise to data over many steps and predicting the noise to remove.

Let me tabulate the differences between flow matching and diffusion for your better understanding.

Note: The two ideas are closely related. With the right choices, a diffusion model can be rewritten as a kind of flow matching model. So, flow matching is best seen as a cleaner and simpler way of training the same family of models.

If we want to go deep into Diffusion Models and the rest of Generative AI, we have a complete program - check out our AI and Machine Learning Program at Outcome School, where we cover them in depth.

Where it works well and where it fails

Flow matching works well because:

• Training is a simple regression with a very clear target.
• It needs no simulation of the full journey during training.
• Straighter paths allow good results with fewer steps, which makes generation faster.
• It scales to big image, video, and audio models.

Flow matching has limits because:

• The real paths are not perfectly straight, because of the averaging we saw. So, very few steps, like 1 or 2, still give lower quality without extra tricks.
• Generation still needs many calls to the model, unlike some methods that create a sample in one step.
• The quality still depends heavily on the network size, the data, and the compute used.

To make paths even straighter, researchers use an extra step called reflow. They generate pairs using the trained model and train again on those pairs. This allows generation in very few steps.

Now we must have understood how flow matching trains a generative model.

Frequently Asked Questions

Is flow matching the same as rectified flow?

They are very similar ideas. Flow matching was introduced in 2022 in the paper "Flow Matching for Generative Modeling" by Yaron Lipman and colleagues. A very similar idea called rectified flow was published around the same time by Xingchao Liu and colleagues, so the two are closely linked.

What is conditional flow matching?

Conditional flow matching is training with per-pair targets. Each training target is computed for one specific pair of a noise point and a real data point, which is simply x1 - x0. Even so, the model ends up learning the overall flow, because it learns the average arrow at every position, and following these averaged arrows carries noise to the full data distribution.

Which image models use flow matching?

Stable Diffusion 3, Flux, and Meta Movie Gen use flow matching. Older models like Stable Diffusion 1 and 2 and DALL-E 2 use diffusion. Flow matching scales to big image, video, and audio models, and for real images the small network is replaced by a big network, often a Transformer.

What is reflow in flow matching?

Reflow is an extra step that makes the paths even straighter. Researchers generate pairs using the trained model and then train again on those pairs. This allows generation in very few steps, which matters because with plain flow matching, using only 1 or 2 steps still gives lower quality.

That's it for now.

Thanks

Amit Shekhar
Founder @ Outcome School

You can connect with me on:

• X
• LinkedIn
• GitHub

Follow Outcome School on:

• X
• YouTube
• LinkedIn
• GitHub

Read all of our high-quality blogs here.