Outcome School

Regularization In Machine Learning

Authors
  • Amit Shekhar
    Name
    Amit Shekhar
    Published on
Regularization In Machine Learning

I am Amit Shekhar, Co-Founder @ Outcome School, I have taught and mentored many developers, and their efforts landed them high-paying tech jobs, helped many tech companies in solving their unique problems, and created many open-source libraries being used by top companies. I am passionate about sharing knowledge through open-source, blogs, and videos.

Join Outcome School and get high paying tech job: Outcome School

Before we start, I would like to mention that, I have released a video playlist to help you crack the Android Interview: Check out Android Interview Questions and Answers.

In this blog, we will learn about the Regularization In Machine Learning.

Regularization is a technique which is used to solve the overfitting problem of the machine learning models.

Solving overfitting problem with regularization

What is overfitting?

Overfitting is a phenomenon which occurs when a model learns the detail and noise in the training data to an extent that it negatively impacts the performance of the model on new data.

So the overfitting is a major problem as it negatively impacts the performance.

Regularization technique to the rescue.

Generally, a good model does not give more weight to a particular feature. The weights are evenly distributed. This can be achieved by doing regularization.

There are two types of regularization as follows:

  • L1 Regularization or Lasso Regularization
  • L2 Regularization or Ridge Regularization

L1 Regularization or Lasso Regularization

L1 Regularization or Lasso Regularization adds a penalty to the error function. The penalty is the sum of the absolute values of weights.

l1 regularization

p is the tuning parameter which decides how much we want to penalize the model.

L2 Regularization or Ridge Regularization

L2 Regularization or Ridge Regularization also adds a penalty to the error function. But the penalty here is the sum of the squared values of weights.

l2 regularization

Similar to L1, in L2 also, p is the tuning parameter which decides how much we want to penalize the model.

This is Regularization.

That's it for now.

Thanks

Amit Shekhar
Co-Founder @ Outcome School

You can connect with me on:

Follow Outcome School on:

Read all of our high-quality blogs here.