How does Data Augmentation work?

Data Augmentation is a technique where we create new training examples from our existing data by making small changes to it, like flipping an image or rewording text.

In this blog, we will learn about how Data Augmentation works. We will also see why we need it, how it creates new training examples from images and text, a simple code example, a step-by-step numeric walkthrough, and when to use it.

We will cover the following:

• What is Data Augmentation?
• Why do we need Data Augmentation?
• How does Data Augmentation work?
• Data Augmentation for images
• A simple code example
• Data Augmentation for text
• A step-by-step numeric walkthrough
• When to use Data Augmentation

I am Amit Shekhar, Founder @ Outcome School, I have taught and mentored many developers, and their efforts landed them high-paying tech jobs, helped many tech companies in solving their unique problems, and created many open-source libraries being used by top companies. I am passionate about sharing knowledge through open-source, blogs, and videos.

I teach AI and Machine Learning at Outcome School.

Let's get started.

What is Data Augmentation?

Data Augmentation is a technique where we create new training examples from our existing data by making small changes to it.

The word Augmentation means to increase something or make it larger.

So, Data Augmentation means increasing the amount of training data we have, without going out and collecting brand new data.

We take one example, make a few small changes to it, and now we have a new example. We do this again and again. From a small set of data, we end up with a much larger set.

Now, let's understand why we need it.

Why do we need Data Augmentation?

A model learns from examples. The more good examples it sees, the better it learns.

Let's say we want to build a model that can look at a photo and tell us whether it contains a cat or a dog. To teach this model, we need thousands of cat photos and thousands of dog photos.

Collecting thousands of labeled photos is hard. It takes a lot of time, money, and effort.

Suppose we only managed to collect 500 cat photos and 500 dog photos. That is a small amount of data.

When a model trains on a small amount of data, a big problem arises. The model starts to memorize the exact photos instead of learning the general idea of what a cat looks like.

This problem has a name. It is called overfitting. Means, the model performs very well on the training data but performs poorly on new data it has never seen before.

Let us understand overfitting with a simple comparison.

Suppose a student prepares for an exam by memorizing the answers to only 10 specific questions. If the exam asks those exact 10 questions, the student scores full marks. But if the exam asks even slightly different questions, the student fails. The student memorized instead of truly understanding.

A model that overfits behaves exactly like this student.

So, how do we help the model truly understand instead of memorize? The answer is to give it more varied examples.

So, here comes Data Augmentation to the rescue. It gives us more varied examples from the data we already have.

We have a detailed blog on Dropout in Neural Networks that explains another way to reduce overfitting in depth.

How does Data Augmentation work?

The core idea is simple. We take an existing example and apply a small transformation to it to create a new example.

A transformation is just a small change we make to the data. The change is small enough that the meaning stays the same, but the data looks different to the model.

Let's say we have one photo of a cat. We can do the following with it:

• We can flip the photo from left to right. It is still a cat.
• We can rotate the photo by a small angle. It is still a cat.
• We can make the photo a little brighter or darker. It is still a cat.
• We can zoom into the photo slightly. It is still a cat.

From one single cat photo, we just created four new cat photos. Each one looks different to the model, but to us, all of them are clearly cats.

Let's visualize this as below:

                          +----------------+
                          | One cat photo  |
                          | (label: cat)   |
                          +----------------+
                                  |
        +----------------+--------+---------+----------------+
        |                |                  |                |
        ↓                ↓                  ↓                ↓
   +----------+     +----------+       +----------+     +----------+
   | Flipped  |     | Rotated  |       | Brighter |     | Zoomed   |
   | cat      |     | cat      |       | cat      |     | cat      |
   | (cat)    |     | (cat)    |       | (cat)    |     | (cat)    |
   +----------+     +----------+       +----------+     +----------+

Here, we can see that one cat photo turns into four new cat photos. Each photo looks different to the model, but the label stays "cat" for all of them.

This is the core idea of Data Augmentation. The label does not change, but the data looks new.

A label is the correct answer we attach to an example. For a cat photo, the label is "cat". When we flip or rotate a cat photo, the answer is still "cat", so the label stays the same.

Each new photo looks like a slightly different camera angle, a slightly different lighting, or a slightly different position of the cat. In the real world, cats appear in all these conditions. So, by showing these variations to the model, we teach it that a cat is a cat no matter how it is flipped, rotated, or lit.

This is how the model learns the general idea of a cat instead of memorizing exact photos.

Planning your move into AI?

No matter which tech domain you work in today, with your effort, you can transition to these roles:

• AI Engineer
• Forward Deployed Engineer
• AI Agent Engineer
• Harness Engineer
• Voice AI Engineer
• AI Architect
• LLM Engineer
• LLM Inference Engineer
• Gen AI Engineer
• MLOps Engineer
• ML Engineer
• Deep Learning Engineer

Our AI and Machine Learning Program at Outcome School takes you there.

Now, let's see the most common types of transformations we use for images.

Data Augmentation for images

For images, we have many transformations. The common ones are as below:

• Flipping: We flip the image horizontally or vertically.
• Rotation: We rotate the image by a small angle, such as 10 or 20 degrees.
• Cropping: We cut out a smaller part of the image and use it.
• Zooming: We zoom in or out a little.
• Brightness change: We make the image brighter or darker.
• Adding noise: We add small random dots to the image, like a slightly grainy photo.
• Shifting: We move the image a little to the left, right, up, or down.

Each of these creates a new example, and the label stays the same. A flipped cat is still a cat.

Let me map each transformation to the real-world situation it imitates, for your better understanding.

Here, we can see that every transformation copies a condition that happens naturally in the real world. So, the model gets ready for these conditions before it ever sees a real new photo.

Data Augmentation also plays a big role in self-supervised learning. We have a detailed blog on Contrastive Learning that explains how two augmented versions of the same image help a model learn without labels.

Note: We must apply these changes carefully. The change must not break the meaning. For example, if we are recognizing the digit 6, we must not flip it upside down, because an upside-down 6 looks like a 9. So, the type of transformation depends on our use case.

A quick note for you

No matter which tech domain you work in, get familiar with these topics:

• LLM
• RAG
• MCP
• Agent
• Fine-tuning
• Quantization

We put it all together in one video:

AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, and Quantization

No need to stop reading - bookmark it and watch later when you get time. Future you will thank you.

Now, let's get back to the topic.

Now that we have understood the idea, let's see a simple code example.

A simple code example

The best way to learn this is by taking an example. Let's see the code for image Data Augmentation using Python and a popular library called Keras.

Keras gives us ready-made tools for Data Augmentation. So, we do not write the flipping and rotating steps ourselves.

We can write the code as below:

from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Create an augmentation generator with a few transformations
data_generator = ImageDataGenerator(
    rotation_range=20,        # rotate the image up to 20 degrees
    horizontal_flip=True,     # flip the image left to right
    brightness_range=[0.8, 1.2],  # make it darker or brighter
    zoom_range=0.2,           # zoom in or out by 20 percent
    width_shift_range=0.1     # shift the image left or right
)

Here, we have created an object called data_generator. We told it the kind of changes we want to make to our images.

Let us understand each line.

• rotation_range=20 tells it to rotate each image by a random angle up to 20 degrees.
• horizontal_flip=True tells it to randomly flip images from left to right.
• brightness_range=[0.8, 1.2] tells it to randomly make images darker or brighter.
• zoom_range=0.2 tells it to randomly zoom in or out by up to 20 percent.
• width_shift_range=0.1 tells it to randomly shift the image sideways.

Note: ImageDataGenerator is the legacy API, which means it is the older way. It is deprecated and not recommended for new code. The modern Keras approach uses augmentation layers like tf.keras.layers.RandomFlip("horizontal"), RandomRotation(0.1), RandomZoom(0.2), and RandomBrightness(0.2). The idea stays the same, only the tool is newer.

Now, we feed our original images to this data_generator. Every time the model asks for an image during training, the data_generator takes an original image, applies these random changes, and hands over a new, slightly different image.

We can picture the flow as below:

   +-----------------+      +--------------------+      +------------+
   | Original image  | ---> |   data_generator   | ---> |   Model    |
   | from our folder |      | (random changes:   |      | (training) |
   |                 |      |  flip, rotate,     |      |            |
   |                 |      |  zoom, brightness) |      |            |
   +-----------------+      +--------------------+      +------------+
                                      |
                                      ↓
                            Fresh image each time,
                            never saved to disk

Here, we can see that the data_generator sits between our stored images and the model. It takes one original image, applies random changes on the fly, and hands a new image to the model every time.

So, the model almost never sees the exact same image twice. It sees a new variation each time.

This is how, from a small set of images, the model effectively trains on a much larger and more varied set.

One more thing to notice. We did not save thousands of new images on our computer. The changes happen on the fly during training. This saves us a lot of storage space.

If we want to go deep into Machine Learning Fundamentals, Deep Learning, and Model Development and Training, we can check out our AI and Machine Learning Program at Outcome School, which covers them from the ground up.

Now, let's learn how Data Augmentation works for text.

Data Augmentation for text

Data Augmentation is not only for images. We can also use it for text.

The idea stays the same. We take a sentence and make a small change to create a new sentence with the same meaning.

Let's say we have this sentence for a model that detects positive reviews: "The movie was very good and I really enjoyed it." The label for this sentence is "positive".

We can create new versions of this sentence using the following techniques.

• Synonym replacement: We replace a word with a word that means the same thing. For example, we replace "good" with "nice". The new sentence becomes: "The movie was very nice and I really enjoyed it."
• Word shuffling: We change the order of a few words slightly, while keeping the meaning clear.
• Back translation: We translate the sentence into another language and then translate it back into the original language. The result is usually a sentence with the same meaning but slightly different words. For example, the sentence may come back as: "The film was really good and I enjoyed it a lot."
• Random insertion: We add a small extra word that does not change the meaning.

From one sentence, we just created several new sentences. All of them carry the same positive feeling. So, the label stays positive.

Note: With text, we must be even more careful. Changing one important word can flip the meaning completely. For example, replacing "good" with "bad" changes a positive review into a negative one. So, we must always check that the meaning is preserved.

This is how Data Augmentation works for text.

Stay updated: Subscribe to our newsletter to get our latest AI and Machine Learning blogs straight to your inbox.

A step-by-step numeric walkthrough

Let us see how Data Augmentation grows our dataset with simple numbers, just for the sake of understanding.

Step 1: We start with 1,000 original images.

Step 2: For each image, we create a flipped version. Now we have the original 1,000 plus 1,000 flipped, which is 2,000 images.

Step 3: For each of these, we create a rotated version. Now we have 2,000 plus 2,000, which is 4,000 images.

Step 4: For each of these, we create a brightness-changed version. Now we have 4,000 plus 4,000, which is 8,000 images.

So, from just 1,000 original images, we ended up with 8,000 training images. And we did not collect a single new photo.

The growth looks like below:

    Start            + Flip           + Rotate        + Brightness
  +-------+         +-------+         +-------+         +-------+
  | 1,000 |  ---->  | 2,000 |  ---->  | 4,000 |  ---->  | 8,000 |
  +-------+         +-------+         +-------+         +-------+
   images            images            images            images

Here, we can see that the dataset doubles at every step. We start with 1,000 images, and after three transformations we reach 8,000 images, without collecting a single new photo.

When to use Data Augmentation

Let me tabulate when Data Augmentation helps the most, for your better understanding.

So, based on our use case, we decide which transformations to apply and how many.

Note: We usually apply augmentation only to the training data, not to the test data, because we must check the model on real, unchanged examples.

This is how we use Data Augmentation to solve the interesting problem of having too little data.

Frequently Asked Questions

Does Data Augmentation change the label of an example?

No. Data Augmentation does not change the label. The change we make is small enough that the meaning stays the same. A flipped or rotated cat photo is still a cat, so its label stays "cat", and a reworded positive review still carries the label "positive".

Do we apply Data Augmentation to the test data?

No. We usually apply augmentation only to the training data, not to the test data. We must check the model on real, unchanged examples, so the test data stays exactly as it was collected.

Do we need to save the augmented images on disk?

No. With tools like Keras, the changes happen on the fly during training. Every time the model asks for an image, the generator applies random changes to an original image and hands over a fresh one. We do not save thousands of new images, which saves a lot of storage space.

Can Data Augmentation hurt a model?

Yes, if the transformation changes the meaning. For example, flipping the digit 6 upside down makes it look like a 9, and replacing "good" with "bad" turns a positive review into a negative one. So, we pick only the transformations that keep the meaning for our use case.

That's it for now.

Thanks

Amit Shekhar
Founder @ Outcome School

You can connect with me on:

• X
• LinkedIn
• GitHub

Follow Outcome School on:

• X
• YouTube
• LinkedIn
• GitHub

Read all of our high-quality blogs here.