Model Evaluation/Bias_and_Variance_using_Python.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Bias and Variance using Python.ipynb",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gL0AffZGOZxZ"
      },
      "source": [
        "# **Bias and Variance in Machine Learning**\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O5iX_J1BY8UO"
      },
      "source": [
        "We use the terms bias and variance or bias-variance trade-off to describe the performance of a machine learning model. In this article, I will introduce you to the concept of bias and variance in machine learning\n",
        "\n",
        "When training a machine learning model, it is very important to understand the bias and variance of predictions of your model. It helps in analyzing prediction errors which help us in training more accurate machine learning models. In this article, I’ll walk you through how to calculate bias and variance using Python."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2vGnZiIXuWdv"
      },
      "source": [
        "In machine learning, you must have heard that the model has a **high variance or high bias**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HCJMFdRIyEE_"
      },
      "source": [
        "**High bias**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Uf9-H-oMuum6"
      },
      "source": [
        "To understand what bias and variance are, suppose we have a point estimator of a parameter or function. Then, **the bias is usually defined as the difference between the expected value of the estimator and the parameter we want to estimate.**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yNYPasTnyj3W"
      },
      "source": [
        "high bias is proportional to the **underfitting**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2Q-c8Ae5yNEh"
      },
      "source": [
        "Bias is the difference between predicted values and expected results. A machine learning model with a low bias is a perfect model and a model with a high bias is expected with a high error rate on the training and test sets."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OG4K7aCUvDcK"
      },
      "source": [
        "If the bias is greater than zero, we also say that the estimator is positively biased, if the bias is less than zero, the estimator is negatively biased, and if the bias is exactly zero, the estimator is unbiased. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YRBAgEMEzC4W"
      },
      "source": [
        "**Variance**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UvtrnQBozdsd"
      },
      "source": [
        "Variance as the difference between the expected value of the estimator squared minus the expectation squared of the estimator. A machine learning model with high variance indicates that the model may work well on the data it was trained on, but it will not generalize well on the dataset it has never seen before."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IHDYldyNzI4l"
      },
      "source": [
        " In general, one could say that a high variance is proportional to the **overfitting**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8w8p0-KTQWwo"
      },
      "source": [
        "**Bias and Variance using Python**\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "brQLUbEU0Hpe"
      },
      "source": [
        "You must be using the scikit-learn library in Python for implementing most of the machine learning algorithms. But it does not have any function to calculate the bias and variance of your trained model. So to calculate the bias and variance of your model using Python, you have to install another library known as mlxtend. You can easily install it in your system by using the pip command:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ixuoUgX4aQl8"
      },
      "source": [
        "!pip install mlxtend"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YjfUljli1FiK",
        "outputId": "02b8d63f-23ac-4547-eb93-760afa9b8b29"
      },
      "source": [
        "!pip install mlxtend --upgrade"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: mlxtend in /usr/local/lib/python3.7/dist-packages (0.18.0)\n",
            "Requirement already satisfied: matplotlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from mlxtend) (3.2.2)\n",
            "Requirement already satisfied: joblib>=0.13.2 in /usr/local/lib/python3.7/dist-packages (from mlxtend) (1.0.1)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from mlxtend) (57.2.0)\n",
            "Requirement already satisfied: scikit-learn>=0.20.3 in /usr/local/lib/python3.7/dist-packages (from mlxtend) (0.22.2.post1)\n",
            "Requirement already satisfied: scipy>=1.2.1 in /usr/local/lib/python3.7/dist-packages (from mlxtend) (1.4.1)\n",
            "Requirement already satisfied: pandas>=0.24.2 in /usr/local/lib/python3.7/dist-packages (from mlxtend) (1.1.5)\n",
            "Requirement already satisfied: numpy>=1.16.2 in /usr/local/lib/python3.7/dist-packages (from mlxtend) (1.19.5)\n",
            "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.0.0->mlxtend) (2.4.7)\n",
            "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.0.0->mlxtend) (2.8.1)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.0.0->mlxtend) (1.3.1)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.0.0->mlxtend) (0.10.0)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from cycler>=0.10->matplotlib>=3.0.0->mlxtend) (1.15.0)\n",
            "Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.24.2->mlxtend) (2018.9)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kNEwf4BrQqhB"
      },
      "source": [
        "Now let’s train a machine learning model and then we will see how we can calculate its bias and variance using Python:\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K7Vcp8aQ0o3z"
      },
      "source": [
        "from mlxtend.evaluate import bias_variance_decomp\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "from sklearn.linear_model import LinearRegression\n",
        "from sklearn.utils import shuffle\n",
        "from sklearn.metrics import mean_squared_error\n",
        "\n",
        "data = pd.read_csv(\"https://raw.codeproxy.net/amankharwal/Website-data/master/student-mat.csv\")\n",
        "data = data[[\"G1\", \"G2\", \"G3\", \"studytime\", \"failures\", \"absences\"]]\n",
        "\n",
        "predict = \"G3\"\n",
        "x = np.array(data.drop([predict], 1))\n",
        "y = np.array(data[predict])\n",
        "\n",
        "from sklearn.model_selection import train_test_split\n",
        "xtrain, xtest, ytrain, ytest = train_test_split(x, y, test_size=0.2)\n",
        "\n",
        "linear_regression = LinearRegression()\n",
        "linear_regression.fit(xtrain, ytrain)\n",
        "y_pred = linear_regression.predict(xtest)"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZOpJ6on01UyC"
      },
      "source": [
        "So till now, we have trained a machine learning model by using the linear regression algorithm, below is how we can calculate its bias and variance using Python:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gjnWuL-k1Tpm",
        "outputId": "a6c993d9-e5f6-44d9-fd2e-c28387585c35"
      },
      "source": [
        "mse, bias, variance = bias_variance_decomp(linear_regression, xtrain, ytrain, xtest, ytest, \n",
        "                                           loss='mse', num_rounds=200, random_seed=123)\n",
        "print(\"Average Bias : \", bias)\n",
        "print(\"Average Variance : \", variance)"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Average Bias :  4.910302451198915\n",
            "Average Variance :  0.05685635558630853\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nmU-Bn7q1gm-"
      },
      "source": [
        "Bias is the difference between predicted values and expected results. Variance is the variability of your model’s predictions over different sets of data. I hope you liked this article on how to calculate the bias and variance of a machine learning model. Feel free to ask your valuable questions in the comments section below."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MIXjbz4hOO6i"
      },
      "source": [
        "[Bias and Variance using Python](https://thecleverprogrammer.com/2021/05/20/bias-and-variance-using-python/)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kP2erWOV1vHB"
      },
      "source": [
        "[Bias and Variance in Machine Learning](https://thecleverprogrammer.com/2020/12/28/bias-and-variance-in-machine-learning/)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YPEeaJW9170J"
      },
      "source": [
        "[Overfitting and Underfitting in Machine Learning\n",
        "](https://thecleverprogrammer.com/2020/09/04/overfitting-and-underfitting-in-machine-learning/)"
      ]
    }
  ]
}