What is a Multilayer Perceptron (MLP) and How Does It Work?

A Multilayer Perceptron (MLP) is a neural network made of layers of simple units called perceptrons, where every unit in one layer is connected to every unit in the next layer. It works by passing numbers forward through these layers, where each unit multiplies, adds, and applies a simple rule, until the last layer gives the answer.

In this blog, we will learn about what a Multilayer Perceptron (MLP) is and how it works. We will also see what a single perceptron is and the XOR problem it cannot solve, how stacking layers fixes that problem, how numbers flow through an MLP with a worked example, why we need a non-linear activation, how an MLP learns its weights, and where MLPs are used today.

We will cover the following:

What is a Perceptron?
The Limit of a Single Perceptron: The XOR Problem
What is a Multilayer Perceptron (MLP)?
Forward Pass: Solving XOR with an MLP
Why Do We Need a Non-Linear Activation?
How Does an MLP Learn?
Where are MLPs Used Today?

I am Amit Shekhar, Founder @ Outcome School, I have taught and mentored many developers, and their efforts landed them high-paying tech jobs, helped many tech companies in solving their unique problems, and created many open-source libraries being used by top companies. I am passionate about sharing knowledge through open-source, blogs, and videos.

I teach AI and Machine Learning at Outcome School.

Let's get started.

What is a Perceptron?

A perceptron is the simplest unit of a neural network. It takes some numbers as input, multiplies each one by a weight, adds them up with a bias, and then applies a simple rule to decide the output.

Let's understand each part:

Inputs: the numbers we give to it, for example x1 and x2.
Weights: one number for each input. A weight tells how important that input is.
Bias: one extra number that is added at the end. It shifts the result up or down.
Activation function: the rule that turns the sum into the output.

The perceptron was introduced by Frank Rosenblatt in 1958. The original perceptron uses the simplest activation function, called the step function. If the sum is greater than 0, the output is 1. Otherwise, the output is 0.

The formula is:

output = step(w1 * x1 + w2 * x2 + b)

Let's take a perceptron with w1 = 1, w2 = 1, and b = -1.5, and give it the inputs x1 = 1 and x2 = 1. Let's see this as a diagram:

         x1 = 1, x2 = 1
                ↓
┌───────────────────────────────┐
│ Multiply by weights, add bias │
│ sum = 1*1 + 1*1 + (-1.5)      │
│     = 0.5                     │
└───────────────────────────────┘
                ↓
┌───────────────────────────────┐
│ Step function                 │
│ 0.5 > 0, so step(0.5) = 1     │
└───────────────────────────────┘
                ↓
           output = 1

Here, we can see that the two inputs are multiplied by their weights, added together with the bias b = -1.5, and then the step function gives the final output.

This perceptron behaves like the AND rule. It gives 1 only when both inputs are 1. Let's check all four cases:

x1	x2	sum	output
0	0	0 + 0 - 1.5 = -1.5	0
0	1	0 + 1 - 1.5 = -0.5	0
1	0	1 + 0 - 1.5 = -0.5	0
1	1	1 + 1 - 1.5 = 0.5	1

If we change only the bias to b = -0.5, the same perceptron behaves like the OR rule. It gives 1 when at least one input is 1.

So, by choosing the weights and the bias, one perceptron can follow simple rules like AND and OR.

The Limit of a Single Perceptron: The XOR Problem

Now, let's take a rule called XOR (exclusive OR). XOR gives 1 when exactly one of the two inputs is 1. It gives 0 when both are the same.

x1	x2	XOR
0	0	0
0	1	1
1	0	1
1	1	0

Let's place these four cases on a grid as a diagram:

  x2
   │
 1 │   (0,1)        (1,1)
   │   XOR = 1      XOR = 0
   │
 0 │   (0,0)        (1,0)
   │   XOR = 0      XOR = 1
   │
   └──────────────────────────── x1
         0            1

Here, we can see that the two 1s sit on one diagonal, and the two 0s sit on the other diagonal.

Now, the question is: why does this matter for a perceptron?

A single perceptron computes w1 * x1 + w2 * x2 + b and checks if it is greater than 0. The points where this sum is exactly 0 form a straight line on the grid. Everything on one side of the line gives 1, and everything on the other side gives 0. Means, a single perceptron can only split the grid into two parts with one straight line.

Let's see the AND rule with the line of our AND perceptron (x1 + x2 - 1.5 = 0) as a diagram:

  x2
   │         ╲
 1 │   0      ╲  1
   │           ╲
   │            ╲
   │             ╲
 0 │   0         0╲
   │               ╲
   └──────────────────── x1
       0         1

Here, we can see that the line keeps the single 1 at (1,1) on one side and the three 0s on the other side. So, for AND, one straight line is enough. Similarly, for OR, one straight line can cut off the corner (0,0) from the other three points.

But for XOR, we need the two 1s on one side and the two 0s on the other side. No single straight line can do this, because the 1s and the 0s sit on crossing diagonals. Any line that puts (0,1) and (1,0) on one side also puts at least one of (0,0) or (1,1) on that same side.

Hence, a single perceptron cannot solve XOR, no matter which weights and bias we choose.

This limit was pointed out in 1969 in the book "Perceptrons" by Marvin Minsky and Seymour Papert, and it is often linked to a slowdown in neural network research in the 1970s. We needed a solution for that, and stacking perceptrons in layers was the answer.

Planning your move into AI?

No matter which tech domain you work in today, with your effort, you can transition to these roles:

AI Engineer
Forward Deployed Engineer
AI Agent Engineer
Harness Engineer
Voice AI Engineer
AI Architect
LLM Engineer
LLM Inference Engineer
Gen AI Engineer
MLOps Engineer
ML Engineer
Deep Learning Engineer

Our AI and Machine Learning Program at Outcome School takes you there.

What is a Multilayer Perceptron (MLP)?

A Multilayer Perceptron (MLP) is a neural network in which perceptrons are arranged in layers, and the output of one layer becomes the input of the next layer.

An MLP has three kinds of layers:

Input layer: it holds the input numbers. It does no calculation.
Hidden layers: one or more layers in the middle. They are called hidden because we never see their outputs directly. We only see the final answer.
Output layer: the last layer. It gives the final answer.

Each perceptron in these layers is also called a neuron.

In an MLP, every neuron in one layer is connected to every neuron in the next layer. Such a layer is called a fully connected layer (also called a dense layer). Each connection has its own weight, and each neuron has its own bias.

Let's see an MLP with 2 inputs, 2 hidden neurons, and 1 output as a diagram:

       Input layer (2 inputs)
   ┌──────────┐      ┌──────────┐
   │    x1    │      │    x2    │
   └──────────┘      └──────────┘
                 ↓
   4 weights: x1->h1, x1->h2,
              x2->h1, x2->h2
                 ↓
       Hidden layer (2 neurons)
   ┌──────────┐      ┌──────────┐
   │    h1    │      │    h2    │
   │  + bias  │      │  + bias  │
   └──────────┘      └──────────┘
                 ↓
   2 weights: h1->y, h2->y
                 ↓
       Output layer (1 neuron)
            ┌──────────┐
            │    y     │
            │  + bias  │
            └──────────┘

Here, we can see that x1 connects to both h1 and h2, and x2 also connects to both h1 and h2. Then, both h1 and h2 connect to y. Every neuron connects to every neuron in the next layer, and the numbers flow from top to bottom. Because the numbers only move forward, from the input to the output, an MLP is a type of feed-forward neural network.

Let's count the numbers that this small MLP needs to learn:

Input to hidden: 2 inputs * 2 hidden neurons = 4 weights, plus 2 biases = 6 numbers.
Hidden to output: 2 hidden neurons * 1 output = 2 weights, plus 1 bias = 3 numbers.
Total: 9 numbers.

These weights and biases are called the parameters of the MLP.

In practice, a fully connected layer is computed in one go with matrix multiplication. We put all the inputs in a list (a vector), all the weights of the layer in a table (a matrix), multiply them, add the biases, and apply the activation to every value. Here, . means matrix multiplication.

hidden = activation(x . W1 + b1)

y = activation(hidden . W2 + b2)

Real MLPs can have many hidden layers and thousands of neurons in each layer, but the idea stays the same.

A quick note for you

No matter which tech domain you work in, get familiar with these topics:

LLM
RAG
MCP
Agent
Fine-tuning
Quantization

We put it all together in one video:

AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, and Quantization

No need to stop reading - bookmark it and watch later when you get time. Future you will thank you.

Now, let's get back to the topic.

Forward Pass: Solving XOR with an MLP

Now, let's build an MLP that solves XOR. The process of passing the input forward through the layers to get the output is called the forward pass.

We will use the MLP from above (2 inputs, 2 hidden neurons, 1 output), with the step function as the activation, and we will pick the weights by hand.

Here, we must notice that XOR is 1 when "at least one input is 1" (OR) but "not both inputs are 1" (not AND). So, we can build it as below:

Hidden neuron h1 will compute OR: h1 = step(x1 + x2 - 0.5)
Hidden neuron h2 will compute AND: h2 = step(x1 + x2 - 1.5)
Output neuron y will compute "h1 and not h2": y = step(h1 - h2 - 0.5)

Let's see the forward pass for the input x1 = 1, x2 = 1 as a diagram:

       x1 = 1, x2 = 1
              ↓
┌───────────────────────────┐
│ h1 = step(1 + 1 - 0.5)    │
│    = step(1.5) = 1        │
│ h2 = step(1 + 1 - 1.5)    │
│    = step(0.5) = 1        │
└───────────────────────────┘
              ↓
┌───────────────────────────┐
│ y = step(1 - 1 - 0.5)     │
│   = step(-0.5) = 0        │
└───────────────────────────┘
              ↓
          output = 0

Here, we can see that both hidden neurons fire, the AND neuron cancels the OR neuron, and the output is 0, which is the correct XOR answer for (1, 1).

Let me tabulate all four cases:

x1	x2	h1 (OR)	h2 (AND)	y = step(h1 - h2 - 0.5)	XOR
0	0	0	0	step(-0.5) = 0	0
0	1	1	0	step(0.5) = 1	1
1	0	1	0	step(0.5) = 1	1
1	1	1	1	step(-0.5) = 0	0

All four outputs match XOR. Two layers solved the problem that one perceptron could not solve.

Here, the hidden layer changed the two inputs into two new features: "is at least one input 1?" and "are both inputs 1?". With these features, one straight line in the output neuron is enough.

Let's see the same forward pass in Python as below:

import numpy as np

def step(z):
    return (z > 0).astype(int)

W1 = np.array([[1, 1],
               [1, 1]])
b1 = np.array([-0.5, -1.5])
W2 = np.array([1, -1])
b2 = -0.5

for x in [[0, 0], [0, 1], [1, 0], [1, 1]]:
    h = step(np.array(x) @ W1 + b1)
    y = step(h @ W2 + b2)
    print(x, "->", h, "->", y)

Here, we have:

step returns 1 where the value is greater than 0, and 0 otherwise.
W1 holds the 4 weights from the inputs to the hidden layer, and b1 holds the 2 hidden biases.
W2 holds the 2 weights from the hidden layer to the output, and b2 is the output bias.
np.array(x) @ W1 + b1 is the fully connected layer: multiply by the weights and add the bias.

This will print the following:

[0, 0] -> [0 0] -> 0
[0, 1] -> [1 0] -> 1
[1, 0] -> [1 0] -> 1
[1, 1] -> [1 1] -> 0

It works perfectly.

Why Do We Need a Non-Linear Activation?

As we have seen, an activation function is the rule each neuron applies to its sum. The step function is one example. It is called non-linear because its output is not just the input multiplied by a number and shifted. It bends the result.

Now, the question is: what if we remove the step function from our XOR MLP and just pass the plain sum forward?

Then, our hidden neurons compute:

h1 = x1 + x2 - 0.5
h2 = x1 + x2 - 1.5

And the output computes:

y = h1 - h2 - 0.5

Let's put h1 and h2 inside y:

y = (x1 + x2 - 0.5) - (x1 + x2 - 1.5) - 0.5 = 0.5

The x1 and x2 cancel out, and y is always 0.5, for every input. This is a straight-line rule again (here, a flat one), and it cannot be XOR.

This is true in general. Every layer without an activation does "multiply by weights and add a bias". For example, if the first layer does h = 2 * x + 1 and the second layer does y = 3 * h + 4, then together they do y = 3 * (2 * x + 1) + 4 = 6 * x + 7. This is again just one "multiply and add", which one layer can do alone. So, without a non-linear activation, any number of layers collapses into a single layer, and we are back to one straight line.

The non-linear activation lets each extra layer bend and combine the straight lines from the previous layer into a new shape. In our XOR MLP, the step function turned the two lines of h1 and h2 into the 0 or 1 features "is at least one input 1?" and "are both inputs 1?", and that is what made XOR possible.

Stay updated: Subscribe to our newsletter to get our latest AI and Machine Learning blogs straight to your inbox.

How Does an MLP Learn?

In our XOR example, we picked the weights by hand. But a real MLP has thousands or even billions of weights, so the MLP learns them from examples. This is called training.

Let's see the training loop as a diagram:

┌──────────────────────────────────┐
│ 1. Start with random weights     │
└──────────────────────────────────┘
                 ↓
┌──────────────────────────────────┐
│ 2. Forward pass: input -> output │
└──────────────────────────────────┘
                 ↓
┌──────────────────────────────────┐
│ 3. Loss: how far is the output   │
│    from the correct answer?      │
└──────────────────────────────────┘
                 ↓
┌──────────────────────────────────┐
│ 4. Backpropagation: find how     │
│    each weight affected the loss │
└──────────────────────────────────┘
                 ↓
┌──────────────────────────────────┐
│ 5. Nudge each weight a little    │
│    to make the loss smaller      │
└──────────────────────────────────┘
                 ↓
        repeat from step 2

Here, the loss is a single number that tells how wrong the output is. Backpropagation goes backward from the output to the input and finds, for every weight, whether increasing it would make the loss go up or down. Then, we change each weight a little in the direction that lowers the loss. This is called gradient descent.

We repeat this loop many times over many examples, and slowly the weights settle on values that give correct answers.

We have a detailed blog on L1 and L2 Loss Functions that explains how the loss is calculated in depth.

One more thing to notice: the step function jumps suddenly from 0 to 1, so a tiny change in a weight usually does not change the output at all. Backpropagation cannot learn from that. That is why real MLPs use smooth non-linear activations, such as ReLU (which keeps positive numbers and turns negative numbers into 0) or sigmoid (which squeezes any number into the range 0 to 1). They still bend the result, and a small change in a weight gives a small change in the output.

If we train an MLP on the four XOR examples, it usually finds weights that solve XOR on its own. They will not be our hand-picked numbers, but they will do the same job.

To learn Backpropagation, Gradient Descent, and Activation Functions, and to build a Neural Network from scratch, check out our AI and Machine Learning Program at Outcome School.

Where are MLPs Used Today?

MLPs are one of the most widely used building blocks in AI. Few important places where they are used:

Tabular data: predicting a house price from size, rooms, and location, or predicting whether a customer will leave.
Final classification layers: many image and text models end with an MLP that turns the learned features into the final answer, for example "cat" or "dog".
Recommendation systems: combining user features and item features to predict what a user will like.
Inside Transformers: every Transformer block, the building block of LLMs (Large Language Models), has a feed-forward network, which is an MLP with one hidden layer. After Self Attention mixes information between tokens, this MLP processes each token on its own. Its hidden layer is usually about 4 times wider than the token's vector, so it expands the vector, applies a non-linear activation, and then shrinks it back to the original size. A large share of an LLM's parameters live in these MLP blocks.

Now we must have understood what a Multilayer Perceptron (MLP) is, how it works, and where it is used.

Frequently Asked Questions

What is the difference between a perceptron and a Multilayer Perceptron?

A perceptron is a single unit that multiplies its inputs by weights, adds a bias, and applies an activation function. A Multilayer Perceptron arranges many perceptrons in layers, where the output of one layer becomes the input of the next. A single perceptron can only split the data with one straight line, while an MLP can solve problems like XOR.

Is an MLP a feed-forward neural network?

Yes. An MLP is a type of feed-forward neural network, because the numbers only move forward, from the input layer through the hidden layers to the output layer. Every neuron in one layer is connected to every neuron in the next layer, and such a layer is called a fully connected layer or a dense layer.

Why don't real MLPs use the step function?

The step function jumps suddenly from 0 to 1, so a tiny change in a weight usually does not change the output at all, and backpropagation cannot learn from that. That is why real MLPs use smooth non-linear activations such as ReLU or sigmoid. They still bend the result, and a small change in a weight gives a small change in the output.

How many parameters does an MLP have?

An MLP has one weight for every connection and one bias for every neuron, and together these are its parameters. For example, an MLP with 2 inputs, 2 hidden neurons, and 1 output has 4 weights and 2 biases in the hidden layer, plus 2 weights and 1 bias in the output layer, which gives 9 parameters in total.

That's it for now.

Thanks

Amit Shekhar
Founder @ Outcome School

You can connect with me on:

Follow Outcome School on:

Read all of our high-quality blogs here.