Linear Regression vs Logistic Regression
- Authors
- Name
- Amit Shekhar
- Published on
I am Amit Shekhar, Founder @ Outcome School, I have taught and mentored many developers, and their efforts landed them high-paying tech jobs, helped many tech companies in solving their unique problems, and created many open-source libraries being used by top companies. I am passionate about sharing knowledge through open-source, blogs, and videos.
Join Outcome School and get high paying tech job: Outcome School
Before we begin, we’d like to mention that we’ve launched our YouTube channel. Subscribe to the Outcome School YouTube Channel.
In this blog, we will learn about Linear Regression vs Logistic Regression in Machine Learning.
In machine learning, the two fundamental algorithms you will frequently encounter are Linear Regression and Logistic Regression. Although they sound similar, they are designed for fundamentally different tasks. Understanding their differences is crucial for choosing the right one. Let's explore what they are, how they work, and when to use them.
But before diving into the algorithms, it's important to understand the difference between regression and classification.
Regression
It predicts a continuous value.
A continuous value is numeric and can take any value. These values are measurable such as temperature, height, weight, or house prices.
Example: Predicting house prices, temperature, or stock prices.
Classification
It predicts a categorical value.
A categorical value, on the other hand, represents discrete groups or classes. These values are descriptive and usually non-numeric, like "yes/no," "red/blue/green," or "spam/not-spam." Categorical values are used in classification problems, where the aim is to assign inputs to a specific category or class.
Example: Predicting whether an email is spam or not, or whether a patient has a disease or not.
Now it's time to jump into our main topic: Linear Regression vs Logistic Regression. Let's start with Linear Regression.
Linear Regression
Linear Regression is one of the most basic algorithms in supervised learning. It is used to predict a continuous value based on one or more input features.
Most importantly, it predicts a continuous value using a linear relationship between inputs and output.
Linear Regression uses a simple linear equation for the input variables (x) and the output variable (y):
y = w1 × x1 + w2 × x2
x1,x2are input features.w1,w2are weights.yis the predicted continuous value.
The algorithm finds the line (or hyperplane in multiple dimensions) that best fits the data by minimizing the Mean Squared Error (MSE) between predicted and actual values.
Linear Regression fits a straight line to predict a continuous numeric value, with no limits on the output range, it can go below 0 or above 1.
And finally, as a prediction, we get a continuous value.
Example: Predicting house prices based on size and location.
This was all about the Linear Regression. Now it's time to learn Logistic Regression.
Logistic Regression
Logistic Regression, despite its name, is used for classification problems in supervised learning, not regression. It predicts the probability that a given input belongs to a particular class.
It starts with the same linear combination of inputs:
z = w1 × x1 + w2 × x2 // notice we’re using z instead of y
Then applies the sigmoid (logistic) function:
y = σ(z) = 1 / (1 + e^(-z))
This converts the raw value z into a probability between 0 and 1.
That probability is then converted into a class:
- If probability ≥ 0.5 → Class 1
- If probability < 0.5 → Class 0
The model is trained using Cross-Entropy loss function. This loss function is mainly used for classification problems.
Logistic Regression uses an S-shaped curve (Sigmoid) to predict the probability, which is always restricted to the range [0, 1].
And finally, as a prediction, we get probabilities, that works well for classification.
You might be wondering: if the algorithm is used for classification, why is it called Logistic Regression, especially the "Regression" part of the name?
The name comes from the way the model is built, not the task it performs.
Logistic Regression actually starts by doing something that looks exactly like linear regression. It takes the input features and forms a linear combination:
z = w1 × x1 + w2 × x2
This part is literally a regression-style calculation, nothing about classification yet.
Then comes the twist: instead of using z directly, the model passes it through the logistic (sigmoid) function to convert it into a probability between 0 and 1:
y = σ(z) = 1 / (1 + e^(-z))
That probability is then used to decide the final class.
So even though the final outcome is classification, the model's internal mechanics are:
- Linear regression-like equation
- Followed by a logistic transformation using sigmoid function.
- Producing probabilities for classification
Because the core computation is regression-based, the name "Logistic Regression" remained, even though we use it for classification.
That's why the name is Logistic Regression.
So, this was all about the Logictic Regression.
Differences Between Linear Regression and Logistic Regression
Let's summarize the differences in a tabular format.
| Linear Regression | Logistic Regression |
|---|---|
| Regression: Continuous value | Classification: Probability (0–1) → Class |
| y = w1 × x1 + w2 × x2 | z = w1 × x1 + w2 × x2 → y = sigmoid(z) |
| Loss Function: Mean Squared Error | Loss Function: Cross-Entropy |
| Examples: House price, temperature | Examples: Spam detection, disease prediction |
| Use it when the target is continuous | Use it when the target is categorical |
I hope this explanation helped.
You can find the complete code that contains the implementation of linear regression and logistic regression from scratch. Check Build your own X - Machine Learning
Prepare yourself for Machine Learning Interview: Machine Learning Interview Questions
That's it for now.
Thanks
Amit Shekhar
Founder @ Outcome School
You can connect with me on:
Follow Outcome School on: