Machine Learning News Hubb
Advertisement Banner
  • Home
  • Machine Learning
  • Artificial Intelligence
  • Big Data
  • Deep Learning
  • Edge AI
  • Neural Network
  • Contact Us
  • Home
  • Machine Learning
  • Artificial Intelligence
  • Big Data
  • Deep Learning
  • Edge AI
  • Neural Network
  • Contact Us
Machine Learning News Hubb
No Result
View All Result
Home Machine Learning

Understanding Bias, Variance, and Bias-Variance Tradeoff | by Azma Syed | Sep, 2022

admin by admin
September 9, 2022
in Machine Learning


‘To err is human…’, a famous quote by Alexander Pope, means it is normal for people to make mistakes. So, it goes without saying that we expect computers to be error-free. But when you step into the world of machine learning, the first of the many concepts you must master is prediction errors — Bias and Variance.

Where does Prediction Error come from?

A supervised machine learning task involves training an algorithm such that it learns to map inputs to an output variable based on some training examples also known as a labeled data set. The model trains itself on the input variables (X) and tries to predict some target variable (y).

But these predictions aren’t 100% accurate. This difference between the actual and predicted values is the prediction error and becomes a basis of model evaluation and acts as an indicator to improve the model.

What is model accuracy?

When building a predictive model, one of the most important metrics you use to evaluate it is ‘accuracy’. It is calculated by dividing the number of correct predictions by the total number of predictions.

The accuracy of a machine learning algorithm is subject to various kinds of errors. To make your models as accurate as possible it is important to understand these errors and learn to find an optimal solution known as the Bias-Variance tradeoff.

Bias

The difference between the values predicted by a supervised machine learning model and the actual values of the target variable present in the data set is called Bias.

Having high bias in your model means it gives huge errors on the training as well as the testing data. It means your algorithm performs poorly on both the data it has and hasn’t seen. A highly biased model is one that has failed to recognize the patterns in the data and is an overly simple one. This is also known as the problem of Underfitting.

Recognizing Bias

  • Both training and testing error is high

Causes of Bias

  • A very simple model such as a linear model

Reducing Bias

  • Getting additional features
  • Adding polynomial features such as x1 2 , x22 , x1x2 , etc.
  • Decreasing Λ (lambda) also known as the Regularization Constant
  • Boosting

Variance

Sometimes, a machine learning model performs extremely well on the training data. But when it is introduced to the testing data, the accuracy of the model falls down considerably. This is known as Variance.

Having high variance in your model means it didn’t just recognize the patterns in the data but learned the data points too well. This happens because your algorithm is highly complex and performs much better on the data it has seen than on data it hasn’t seen. It becomes too familiar with the data it was trained on, essentially learning it, but fails to predict accurately when new data is presented. This is the case of Overfitting.

Recognizing Variance

  • Testing error is much higher than the training error

Causes of Variance

  • A very complex model such as a fourth-degree polynomial equation

Reducing Variance

  • Getting more training examples
  • Using smaller sets of features (removing x3, x4, x5, …)
  • Increasing Λ (lambda) also known as the Regularization Constant
  • Bagging

Bias-Variance Tradeoff

So far we have learned that bias refers to model error whereas variance refers to inconsistency in the accuracy of a model when applied to new data sets. The best model is the one that has low bias (low error and high accuracy) and low variance (consistent accuracy on new data sets).

Achieving this ideal model involves a tradeoff between the two errors known as the Bias-Variance Tradeoff. This is because you can’t have an algorithm that is too simple and too complex at the same time. The tradeoff in the complexity of the model helps achieve an optimal model for our predictive needs.

Final Thoughts

  • A highly simple model underfits the data and fails to learn from it. Such a model has high bias.
  • A highly complex model overfits the data and learns too much from it. Such a model has high variance.
  • Bias-Variance Tradeoff helps us pick ‘just the right model’ where both bias and variance errors are balanced.

Remember that model building is an iterative process. It is highly unlikely to get the perfect model in the first go. By playing around with the data you will eventually be able to build the best fit model. But don’t forget — the lingering irreducible error will always be there!



Source link

Previous Post

Power of Artificial Intelligence. Artificial Intelligence or Machine… | by ARBELO KHUSHK | Sep, 2022

Next Post

Broadcasting Operations in Pandas Explained | by Seungjun (Josh) Kim | Sep, 2022

Next Post

Broadcasting Operations in Pandas Explained | by Seungjun (Josh) Kim | Sep, 2022

Clear your mind with clear BOW. Develop transparent and performant text… | by Sam Hardy | Sep, 2022

How Docker runs machine learning on NVIDIA GPUs, AWS Inferentia and other AI accelerators | by Shashank Prasanna | Medium

Related Post

Artificial Intelligence

Dates and Subqueries in SQL. Working with dates in SQL | by Michael Grogan | Jan, 2023

by admin
January 27, 2023
Machine Learning

ChatGPT Is Here To Stay For A Long Time | by Jack Martin | Jan, 2023

by admin
January 27, 2023
Machine Learning

5 steps to organize digital files effectively

by admin
January 27, 2023
Artificial Intelligence

Explain text classification model predictions using Amazon SageMaker Clarify

by admin
January 27, 2023
Artificial Intelligence

Human Resource Management Challenges and The Role of Artificial Intelligence in 2023 | by Ghulam Mustafa Shoaib | Jan, 2023

by admin
January 27, 2023
Deep Learning

Training Neural Nets: a Hacker’s Perspective

by admin
January 27, 2023

© 2023 Machine Learning News Hubb All rights reserved.

Use of these names, logos, and brands does not imply endorsement unless specified. By using this site, you agree to the Privacy Policy and Terms & Conditions.

Navigate Site

  • Home
  • Machine Learning
  • Artificial Intelligence
  • Big Data
  • Deep Learning
  • Edge AI
  • Neural Network
  • Contact Us

Newsletter Sign Up.

No Result
View All Result
  • Home
  • Machine Learning
  • Artificial Intelligence
  • Big Data
  • Deep Learning
  • Edge AI
  • Neural Network
  • Contact Us

© 2023 JNews - Premium WordPress news & magazine theme by Jegtheme.