2020-06-16 18:27:43 +02:00
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Coverage plots"
]
},
{
"cell_type": "code",
"execution_count": 1,
2020-06-23 21:53:50 +02:00
"metadata": {},
2020-06-16 18:27:43 +02:00
"outputs": [],
"source": [
"%matplotlib inline\n",
"import numpy as np\n",
"from matplotlib import pyplot as plt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let us consider the following function which applies a linear model to the given data. \n",
"Specifically, given a \"model\" vector containing the model coefficients $(a,b)$ and a $n \\times 2$ \"data\" matrix containing the data points to be classified, the function outputs a vector $\\mathbf{z}$, $|\\mathbf{z}| = n$ of booleans where $z_i$ is `True` if $a \\cdot x_{i,1} + b \\cdot x_{i,2} \\geq 0$, it is `False` otherwise."
]
},
{
"cell_type": "code",
"execution_count": 2,
2020-06-23 21:53:50 +02:00
"metadata": {},
2020-06-16 18:27:43 +02:00
"outputs": [],
"source": [
"def apply_linear_model(model, data):\n",
" return np.dot(data, np.transpose(model)) > 0"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let us define `data` by generating $1000$ points drawn uniformly from $\\mathcal{X} = [-100,100]^2$."
]
},
{
"cell_type": "code",
"execution_count": 3,
2020-06-23 21:53:50 +02:00
"metadata": {},
2020-06-16 18:27:43 +02:00
"outputs": [
2020-06-23 21:53:50 +02:00
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/user/.local/lib/python3.7/site-packages/ipykernel_launcher.py:1: DeprecationWarning: This function is deprecated. Please call randint(-100, 100 + 1) instead\n",
" \"\"\"Entry point for launching an IPython kernel.\n"
]
},
2020-06-16 18:27:43 +02:00
{
"data": {
"text/plain": [
2020-07-03 19:08:23 +02:00
"array([[-60, -58],\n",
" [-54, 99],\n",
" [ 95, 99],\n",
2020-06-23 21:53:50 +02:00
" ...,\n",
2020-07-03 19:08:23 +02:00
" [ -3, -80],\n",
" [ 45, -64],\n",
" [ 14, 59]])"
2020-06-16 18:27:43 +02:00
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"data = np.random.random_integers(-100,100,[1000,2])\n",
"data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"and let target_labels be the labeling output by applying `apply_linear_model` with our target model: $4x -y > 0$"
]
},
{
"cell_type": "code",
"execution_count": 4,
2020-06-23 21:53:50 +02:00
"metadata": {},
2020-06-16 18:27:43 +02:00
"outputs": [],
"source": [
"target_model = [4.,-1.]\n",
"target_labels = apply_linear_model(target_model, data)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"By using matplotlib.pyplot module it is easy to plot the generated points onto a 2D plot:"
]
},
{
"cell_type": "code",
"execution_count": 5,
2020-06-23 21:53:50 +02:00
"metadata": {},
2020-06-16 18:27:43 +02:00
"outputs": [
{
"data": {
"text/plain": [
2020-07-03 19:08:23 +02:00
"<matplotlib.collections.PathCollection at 0x7f068e473b90>"
2020-06-16 18:27:43 +02:00
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
2020-07-03 19:08:23 +02:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD4CAYAAAAEhuazAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOxdZ5gUxdauyT3dM7PkJCIIiIFgQJAsKkYMKCqiIBjQew0YUMEIIogJRTGgCKgYMYACSlAEFURyBlkkg+Auedk4Xd+Pd8vu6a7qMDO7cL+d93n6Ydntrq6ucOrk46GUkgwyyCCDDCoWvMe6AxlkkEEGGZQ/MsQ/gwwyyKACIkP8M8gggwwqIDLEP4MMMsigAiJD/DPIIIMMKiD8x7oDTlCtWjVav379Y92NDDLIIIP/KSxZsiSHUlqd97f/CeJfv359snjx4mPdjQwyyCCD/yl4PJ6tor9l1D4ZZJBBBhUQGeKfQQYZZFABkSH+GWSQQQYVEBnin0EGGWRQAfE/YfD9XwalhKxcSciRI4Sccw4hkuTu+c2bCdm+nZCmTQmpUkV8XzxOyNKlhKgq3uN3OLOHDhGyYgX6F40SctZZhChKan2pCMjOJmT3bkKaNSOkUqVj3RuX2L2bkI0bCWnUiJA6dcr2XUVFhCxZgoV/5pmEeDzHri8ZJIJSmvJFCBlHCNlLCFmt+10VQsgsQsjG0n8rl/7eQwh5nRCSTQhZSQg52679c845h5YnNm6k9KefKP3nn9Ta2bCB0kaNKFUUSmMxSqNRSj/7zNmzBw5QeuGFlEoSpVlZ+PfRRylVVfO98+dTWrMm2o9GKa1WjdKffxa3nZtL6Zw5lD7yCNr1eiklBP+Gw5SOHp14/8GDlF50Ef4Wi1EaDFLauzel8bjjofh/g9xcStu318ZCkigdPJg/L5RSWlxM6W+/UbpgAaUlJeXbVxOKiii95RZtUYVClPbsid+XBaZMwXtiMUojEUpPOonSVaus+1JQQOnChZT++iulhYVl06/jCZs3g9j8/XeZNE8IWUxFdFv0BzcXIaQjIeRsA/F/kRAysPTngYSQF0p/vpwQ8n3pIXAeIWShXfvlRfz376f0/POxsRnBHTBAvLGtUFJCad26lHo8GGV2yTKIwezZ2j7goVs37Af9s4pC6YQJifcdOACCr7+PEOw14+GlqpQ++SS+Kxw2P8OuYDDx8Lj2WnNfCKG0ShVKV650Pzb/y7jkEkoDAfO8fPGF+d6ffsIYsUO5enXQtGOGxx83T3woBC4g3cjOxmI3Lprq1UHURX1RFCzeWAybcOrU9PfNKUpKwFnNnYtDKZ04coTSyy5LPPzuuivtHFWZE3+8g9Q3EP8NhJDapT/XJoRsKP15DCHkJt59oqu8iP9VV4Hw6dejJFE6dqz7tn76iU+UPR5KfT7MtyxTeuaZlO7enfjsgQN8YksIpU2bJt77/vvYL8b7wmFK33gj8d7PPuPfy7vat8czBw+ax8S4l1NhHPfupXTGDO0gVFVKlyyhdOZMHMbHE/bsEc9Lq1aJ9+7dyx/rSATzWy7YsIHSH36gdMcO/D8rSzyRM2em992PP24+JQnBphgzxvlClGVKt25Nb9/0yM6m9LvvKP38cxB5tpgXLaK0Vi30NxbD9e236XvvrbeCuBi/9ZVX0vcOeuyI/wHdzx72f0LIVEJIe93ffiSEtOS0148QspgQsrhevXppHRAerAiu30/p22+DGy4udtbeF1/wiT+v7fPOwzP791P6/feUTp5sXhfsql078T0vvIA2eIfM008n3nvuuc72GyGUVq6MZ7Zvt5YSYrHkmDNV1dRO7CBs1kxTkzHJ66WX3LddVli/HsSbNw4NGiTe+/rr/HFTFEpHjKB02jTQ5jLBoUOUXnBBogjbt6/1hIfDON3Shdtu47/H5+MfCqIrGKR06ND09Yvh6FFKL788kbPx+7Hwf/iB0kqV+GO0ZUvq7y4sFBObNNO6Y078S/+/n7og/vqrPDj/bdusiZzHo+nT58+3b2/nTmuO2bimnnoKe5SpR5keXn95vZT26pX4noUL+dK1opj1/g0aON9zdevimXgcB47oPp4qygk+/tgZ8xcKpZ8p1WP7djB0y5fb31tcDDWOsY+BAKX33JN479NPm1V+bB35/aDJ4TClF18MDUBa0bOnmbjIMr9D7JIkSl97LX19+OIL8Unp9rrvvvT1i+Guu8QEOBTi9z0YpPTZZ1N/98GD/A1OCIhMGpFR+ziAHZHTX7GY/YYtLuYTZd4ly2ZO3+Ph79UqVbBHZ84EA6GqlHbvnkhIZRlMjdFWce+9YLyc9KlSJUo3bcJz334rPsgkCTYrt2jZ0vnel2WtL+lCPE5pv37Y57EY3tGyJaU5Ofj77t2QwBYsSBzHzz5LpKOhEFRfO3cmtj93rrPDTZIoveOONH5Yfr5zrsN4PfVU8u/NzYWBd+5c6MqLiiht3dr5JhAtzEgEapl0Ih4Xi9aE4HQW9efee1N//8SJ4neffjruWb+e0m++oXTt2pRedayI/0sGg++LpT9fYTD4/mHXdnnp/L/7zhlxjEYp/fRT67ZmzXLO+PDUNowwWDEIlSpROm8e9tqHH1LaqROlHTtSOm4cXz21axckFyd98ngoPfVUjfAtXEhp1aqJB5KiUPrgg5SuWEHpl19ivTrFySe7o0v6vqQDb75ppkuBAKWXXgqvKnYoRCKUNm4Maf/XXyn96iusk+uvh55/0CC+tkRVKb3ySufSTdrsfPv3u1Or6K+vvtLa2bwZk/rHH/YD//LLmtgajVJ6wgmUrlmDg2j0aErbtoXrmtViu/NOSm+4wczFdO6cnJuUquLk/uors81g4UJrKUh0RSKpG6DXr7c+eGrWhCGYuZMx8TAvL6nXlYe3z6eEkN2EkGJCyA5CyO2EkKqlKp2NhJDZhJAqpfd6CCFvEkI2EUJW2al8KC1fV88XXxQTY3aFw5S+9Rbm49tvYVj94ANKly7V2nGi8/d4sL5PO43/92DQ/jCKRik9fNj59+3di32oP1RE+0BREr15Cgrw3R06YH1++ikMw7KsrdMrr3TmGHH//e5olLEvqeLUU/nv8fvNh4LXi7lgTiihEKUPPQT6wmjM55/DdkgppPpvvgEj/N57oF/GMTeug7Q5k6gqpaec4p6web0wNpaUaMbIWAwD37w5RJtZsyidNCnRLXHuXL76pG5d84l21ln8j7/kEq3vX34JYtepEwYvGW+CXbvAQbMJkySoeXJzNWlENBlWC/DCC1M/pR97zHpTB4Pmw0GSzHpFhygXzr8sr/Ik/sXFIGhW8yNJcJkMBhPXUChEabt2sLft2WN9wHs8lF5zDYhE1ar8e5xKIR9/7P47f/oJ7qQdO1Javz6/7ViM0l9+EbfRu7d530sSuGE77NkDNRsbI0ZgRd8Zi0HKSRfq1BHPi1Na8N57MFJHIpgHSQKTy5g2dv34I94pOuw8njTHAMydK36Z1aJ6+mlw6sbTz+/H5DDOPhSi9LnnIB2IRJto1OzXunAhBov1jbWZomrDBN4GVhRKzznHvUrM44EXwrhx9gdRSQkOyE8/hTGJhzvvdPd+dslyUkORIf4usX+/9R6pVUv8t2AQjBOl2B9WXPWff4J48JiQYJDS22+3VxuEw/BESgUvvcQ3dkcicIrgoaREbC+rWhUM3BdfWLs15ubCkeP88ynt0wdG13vu4Y99KETp119rdo65cyn96CNK161L7pvvvJMv4bmRRipVspcS2TgeOiRmBtLK+TNcd524Q7xOKwoCUEQiEY8YNWwo/nssRun06eZ+ZWfDSNWkCbgHEZFMFrt3ixem6PL54OUjstA/+aT9e9evB0fBgjpEQUJTpiRnCPd6k9J7Zoi/S+zbJ97Ufr/9htfrcLt356+pGjUofecdMSPSogUIbOfO1gdROKypG5LF4cNQPTGGz+vFz+PHi58pLLSWnNkeCIedRzXz+sIupoWoUgWBopEIrnAY+nenLrgM2dnwuGHEnql7eve2ltiSuSIR2GS6duWvhZYt3fXdEXr3FneoVavEAZZ
2020-06-16 18:27:43 +02:00
"text/plain": [
2020-06-23 21:53:50 +02:00
"<Figure size 432x288 with 1 Axes>"
2020-06-16 18:27:43 +02:00
]
},
2020-06-23 21:53:50 +02:00
"metadata": {
"needs_background": "light"
},
2020-06-16 18:27:43 +02:00
"output_type": "display_data"
}
],
"source": [
"colors = ['r' if l else 'b' for l in target_labels]\n",
"plt.scatter(data[:,0], data[:,1], color=colors)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally let us now generate at random 100 linear models with coefficients in $[-5,5]$:"
]
},
{
"cell_type": "code",
"execution_count": 6,
2020-07-03 19:08:23 +02:00
"metadata": {
"scrolled": true
},
2020-06-16 18:27:43 +02:00
"outputs": [
{
"data": {
"text/plain": [
2020-07-03 19:08:23 +02:00
"array([[ 0.9118242 , -0.90045672],\n",
" [-3.94382217, -3.98527811],\n",
" [-1.18697017, 0.80016719],\n",
" [-1.85839332, 4.98952722],\n",
" [ 1.07261938, -0.90803895],\n",
" [-4.53758697, -4.30257501],\n",
" [-1.17461754, -2.39901897],\n",
" [ 2.34774061, -4.72969268],\n",
" [ 2.47747169, 2.77914928],\n",
" [-1.11617222, -4.16037991],\n",
" [ 1.4716821 , 0.88162217],\n",
" [ 1.78297491, -1.44883212],\n",
" [ 1.58199674, 4.6052321 ],\n",
" [ 3.99068967, -4.28056417],\n",
" [-0.16221614, -2.24741286],\n",
" [ 2.32933961, 0.28596881],\n",
" [-4.70438519, 2.44675435],\n",
" [-3.29578251, -3.86079991],\n",
" [-2.15187954, -3.38596945],\n",
" [-1.31823703, 1.6351425 ],\n",
" [-3.13072307, 1.32378202],\n",
" [ 2.76175045, -1.78814111],\n",
" [ 2.73156012, -2.64332344],\n",
" [ 1.64996223, 1.83230706],\n",
" [ 3.31936639, 0.2228602 ],\n",
" [ 2.26694906, 0.63193375],\n",
" [-2.51414076, 4.51560257],\n",
" [ 0.27215252, -1.67282386],\n",
" [-3.64167257, -0.14559744],\n",
" [-4.30424639, -3.37589157],\n",
" [ 3.01745299, 3.51058308],\n",
" [ 1.35223951, -3.06364559],\n",
" [ 4.94253085, -1.60716143],\n",
" [-4.36074161, -3.10693624],\n",
" [ 3.10628154, 3.49373291],\n",
" [-2.74311538, -4.29366027],\n",
" [-1.89979198, 1.41734176],\n",
" [ 1.9159884 , 1.23531441],\n",
" [-2.15457615, 1.1728522 ],\n",
" [ 4.60642972, 3.51823611],\n",
" [ 1.59513489, 0.56356173],\n",
" [-0.32910123, 1.31288732],\n",
" [ 1.36686363, 0.96076635],\n",
" [-3.9091037 , 0.96514774],\n",
" [ 4.37669631, -0.8778982 ],\n",
" [-3.13000071, -2.59206421],\n",
" [ 0.85730862, 3.96159211],\n",
" [ 2.91165311, 2.24727293],\n",
" [ 2.16991404, -3.35593884],\n",
" [-0.38522275, -1.67180888],\n",
" [-1.91436601, 3.62229527],\n",
" [ 1.31583377, -1.93048586],\n",
" [ 0.52322948, 0.91378549],\n",
" [ 0.69736315, 3.05799437],\n",
" [-2.33259618, 4.23093531],\n",
" [-0.01882034, -3.16737335],\n",
" [-1.85567722, -0.16700837],\n",
" [ 4.74309296, 3.57241682],\n",
" [ 0.96709141, -1.3653478 ],\n",
" [-2.98210548, -0.11106027],\n",
" [-3.86461267, 3.62193573],\n",
" [ 2.83976749, 2.94566098],\n",
" [ 3.76245288, -2.64933837],\n",
" [ 4.58809654, 1.23109222],\n",
" [ 4.84968707, -2.75644381],\n",
" [-1.54471238, 4.83523772],\n",
" [ 1.89738986, 3.61006974],\n",
" [ 1.89077461, 3.96448192],\n",
" [ 0.58264712, -3.48158676],\n",
" [-3.70699049, -1.55128007],\n",
" [-1.74431095, -1.26414456],\n",
" [ 4.95881191, -3.89363783],\n",
" [-3.49425476, 4.69333757],\n",
" [-1.13661494, -4.86289907],\n",
" [-0.80047881, -2.36304971],\n",
" [-2.22814782, -1.71573374],\n",
" [ 1.93181752, -2.84184699],\n",
" [-2.01459345, 3.04690045],\n",
" [ 1.77370361, 2.63596514],\n",
" [-1.62391354, 3.9170375 ],\n",
" [-1.16831826, -0.35730506],\n",
" [ 2.81017534, 4.68734215],\n",
" [ 3.50859446, 3.53556171],\n",
" [-3.00404934, -0.31632676],\n",
" [-3.19738369, -0.50324866],\n",
" [-1.14409139, 3.06816086],\n",
" [-4.67354814, 1.12585223],\n",
" [ 2.62801894, -2.11531302],\n",
" [-3.26599429, -2.09618265],\n",
" [-1.77991357, -3.54630238],\n",
" [ 1.83623843, -2.97438757],\n",
" [-1.90333658, 0.66363691],\n",
" [ 2.61705961, 0.10912733],\n",
" [ 1.76458691, 1.21896092],\n",
" [-2.5188483 , 0.77614823],\n",
" [ 1.75016557, 2.0592426 ],\n",
" [ 4.82096292, -0.0816393 ],\n",
" [-2.66030292, -2.54501908],\n",
" [ 1.90560799, -1.66171268],\n",
" [-1.30050042, -1.94071811]])"
2020-06-16 18:27:43 +02:00
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"models = (np.random.rand(100,2) - 0.5) * 10\n",
"models"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Exercise"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. Write a function that, taken two list of labellings build the corresponding confusion matrix [[1](#hint1)];\n",
"1. For each model in `models` plot the [FP,TP] pairs on a scatter plot;\n",
"1. Just looking at the plot: which is the best model in the pool?\n",
"1. Find the model with the best accuracy [[2](#hint2)] and compare it with the target model, is it close? Is it the model you would have picked up visually from the scatter plot?\n",
"1. If everything is ok, you should have found a pretty good model for our data. It fits the data quite well and it is quite close to the target model. Did you expect this? If so, why? If not so, why not?\n",
"\n",
"<a name=\"hint1\">Hint 1:</a> it may be helpful to have a way to map TRUE to 0, FALSE to 1 and to use these values as indices in the confusion matrix. \n",
"\n",
"<a name=\"hint2\">Hint 2:</a> one way to proceed is to build a function `accuracy`, use the `map` function to calculate the accuracies of all the models, and then apply the `numpy.argmax` to retrieve the index of the best model."
]
},
2020-07-03 19:08:23 +02:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Exercise 1\n",
"Write a function that, taken two list of labelling, builds the confusion matrix"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[416. 99.]\n",
" [ 77. 408.]]\n"
]
}
],
"source": [
"def confusion_matrix(labels1, labels2):\n",
" assert len(labels1) == len(labels2), \"Label arrays must be of same length\"\n",
" confusion_matrix = np.zeros((2,2))\n",
" for i in range(len(labels1)):\n",
" confusion_matrix[1 - labels1[i], 1 - labels2[i]] += 1\n",
" return confusion_matrix\n",
"\n",
"print(confusion_matrix(apply_linear_model(target_model, data), apply_linear_model(models[0], data)))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Exercise 2\n",
"For each model in models plot the [FP,TP] pairs on a scatter plot\n"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.collections.PathCollection at 0x7f065f9cecd0>"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAb2klEQVR4nO3df2zc9Z3n8ec7jgGH7nUCuFyYJE32GgVR5RJXFgSlfxS6JbTXAy+CZRGo0QldpFtWKj/O3WSPa2HFKqkiFba6Fbf0qJaqCAKUNVnCbsglWd21InSdtUPIQo6ASsgATVri7N3iso79vj/mM85kPN/54ZnxzPf7fT0kyzOf+Y79/Qbz9sfv7/vz/pi7IyIiyTWv3ScgIiKtpUAvIpJwCvQiIgmnQC8iknAK9CIiCTe/3ScAcMkll/iyZcvafRoiIrFy4MCBX7l7b7XjOiLQL1u2jOHh4XafhohIrJjZu7Ucp9SNiEjCKdCLiCScAr2ISMIp0IuIJJwCvYhIwnVE1c1sDI3k2LbrCO+PjXNZpofB9SsZ6Mu2+7RERDpOLAP90EiOzc8fYnxiEoDc2Dibnz8EoGAvIlKiptSNmf3CzA6Z2aiZDYexi8xst5m9FT4vDONmZt83s6Nm9pqZfaHZJ71t15HpIF8wPjHJtl1Hmv2tRERir54c/TXuvsbd+8PzTcAed18B7AnPAb4KrAgfG4FHm3WyBe+Pjdc1LiKSZo3cjL0ReCI8fgIYKBr/keftBzJmtqiB7zPDZZmeusZFRNKs1kDvwMtmdsDMNoaxS939g/D4Q+DS8DgLvFf03uNhrGmuubx8a4eocRGRNKv1ZuwX3T1nZp8BdpvZm8UvurubWV17EoZfGBsBli5dWs9b2ffmybrGRUTSrKYZvbvnwucTwF8BVwK/LKRkwucT4fAcsKTo7YvDWOnXfMzd+929v7e3vpm4cvQiIrWrGujN7EIz+63CY+A64HVgB7AhHLYBeCE83gF8I1TfrAVOF6V4mkI5ehGR2tUyo78U+KmZHQR+Dux0978FtgJfMbO3gN8JzwFeAt4BjgI/AP6g2Sc9uH4lPd1d54z1dHcxuH5ls7+ViEjsVc3Ru/s7wOoy478Gvlxm3IG7mnJ2EQqLorQyVkSkuliujIV8sFdgFxGpTk3NREQSLrYz+kaoIZqIpEnqAr0aoolI2qQudaOGaCKSNqkL9FpsJSJpk7pAH7Woap4ZQyMzFvCKiMRe6gJ9ucVWAJPu3L19lL4/eVkBX0QSxfLrm9qrv7/fh4eH5+z7DY3kuO+Zg0xWufZMTzcP3PB53aQVkY5kZgeK9giJlLoZPeSra6Zq+AU3Nj7B4LMHNcMXkVhLZaCH2hugTUy5KnJEJNZSG+ijcvXl5MbGWbd1L8s37WTd1r2a4YtIrKQ20A/0Zdly0yoyPd01HZ8bG8c5u8BKwV5E4iK1gR7ywX70O9fxyK1rWLigtoAPWmAlIvGS6kBfMNCXZeTb+YBvVtt7tMBKROJCgb7IQF82vw16DbSblYjEhQJ9iVoCuHH2Bq1y9SLS6VK5YKqS0u6WAN3zjE9dMJ9TH09gzJz093TP4zcTU2p5LCJzSgumZqlQjZPN9GBANtPDtltWM/Lt68hmespmdsYnpqYrcu7ZPsr9Q4fm+KxFRKKlrh99LaK2KazlBqwDT+4/Rv9nL9LMXkQ6gmb0daj1BqyDyi9FpGMo0NdhcP3Kmo9V+aWIdAoF+jrUk4q5LNPD0EhOrRNEpO0U6OuUrbH88prLe9n8/CG1ThCRtlN5ZZ2GRnLcs3206rqqbKaHXJn0zTyDqfBm9bsXkUaovLJFBvqy3L52KZU6JWQzPZE5+qmi3xDqdy8ic0GBfhYeGljFw7euKdv5sqe7i8H1K9XvXkQ6hgL9LBV3vixeXLXlplUM9GXr6nevCh0RaSUtmGpQ1OKqwti2XUd4f2yceWaRe9SqQZqItJICfQsV/xIYGskx+NxBJibPDfbd82y6Pn9oJDf9i0F9c0SkWWpO3ZhZl5mNmNmL4flyM3vVzI6a2XYzOy+Mnx+eHw2vL2vNqcfLQF+WbTevPmeDk0xPN9tuWc1AX3a6mZrKMUWk2eqZ0X8TeAP4V+H5d4GH3f1pM/vvwJ3Ao+HzKXf/nJn9fjju1iaec2xFpXkgn+Ip7pgJZ3ey0qxeRBpR04zezBYD/w74H+G5AdcCz4VDngAGwuMbw3PC618Ox0sFUTdkdaNWRBpVa+rmEeBbwFR4fjEw5u5nwvPjQGHamQXeAwivnw7HSwVRN2R1o1ZEGlU10JvZ14ET7n6gmd/YzDaa2bCZDZ88ebKZXzqWypVjFmryRUQaUUuOfh1wg5l9DbiAfI7+z4CMmc0Ps/bFQOGuYQ5YAhw3s/nAp4Ffl35Rd38MeAzyLRAavZC4Ky3HLK26UUWOiMxW1UDv7puBzQBm9iXgP7v77Wb2LHAz8DSwAXghvGVHeP5KeH2vd0JDnRiIullbur1hoSKn8B4RkUoaWRn7R8C9ZnaUfA7+8TD+OHBxGL8X2NTYKUqlihwRkWrqWjDl7n8H/F14/A5wZZljfgPc0oRzkyCq8iY3Ns66rXuVxhGRitTrJgYqVd5oYZWIVKNAHwPVGqQpjSMilajXTQwUV+SU28wEtLBKRKJph6mYWbd1b9lgbwZetHPV11cvYt+bJ1WOKZJg2mEqoaLSOF6yc9WP9x9TgzQRARToY2egL8uWm1ZNb3bSVWMbofGJSb713MHWnpyIdCSlbmJu+aadVTcqL2WAk98RSykdkfhS6iYlZtP0rPCLQSkdkXRQoI+5wfUr6e6afRdolWaKJJ8CfcxF7VxVD5VmiiSb6ugToFwztNt/8Ao/e/ujmt6vnvciyaYZfUI9+R+vZsVnLqx6nHreiySfAn2C7b73Szxy65pz0jqQr7oBWLigm/Pnz+Oe7aOs27pXN2VFEkqpm4SbTY97iN4ARUTiR4E+paJ63D+w4zCfnJnSJiciCaLUTUpFVdqMjU9okxORhFGgT6l6K21UgikSXwr0KVWuOVpPd9eMG7cFKsEUiS/l6FOquMd98U1X4JybtFBbCebQSE43cEU6lAJ9ikVV5EB9VTeVKngU7EXaT4FeZqj0C6CcqAqebbuOKNCLdADl6KVhUdsbRo2LyNxSoJeGRW1+UuumKCLSWgr00rDJiM1rosZFZG4p0EvDshGll1HjIjK3FOilYeVq8ru7jH/+5AzLN+0s2zBtaCTHuq17I18XkeZR1Y00rLQmP7Ogm1MfTzA2PgHkb8re+8wow+9+xL43T864SZsbG2cwbFyuKh2R5tPm4NJ0V/zXv+Hjiam637dwQTcj376uBWckkkzaHFzaZjZBHuDUxxNNPhMRAQV6EZHEU6CXpptt+Xy9m5qLSG2qBnozu8DMfm5mB83ssJk9GMaXm9mrZnbUzLab2Xlh/Pzw/Gh4fVlrL0E6ze1XLa37Pd3zjAdu+HwLzkZEapnRfwJc6+6rgTXA9Wa2Fvgu8LC7fw44BdwZjr8TOBXGHw7HSYo8NLCKO9YunV4Z22XGHWuXcsfapZSb7Gd6utl2y2pV3Ii0SF1VN2a2APgp8J+AncC/dvczZnY18IC7rzezXeHxK2Y2H/gQ6PUK30hVN+mhdsYizVNr1U1NdfRm1gUcAD4H/DnwNjDm7mfCIceBwv+tWeA9gPBL4DRwMfCrkq+5EdgIsHRp/X/qSzzV2xmzVvoFIhKtppux7j7p7muAxcCVwOWNfmN3f8zd+929v7e3t9EvJylW6IefGxvHOdsPX6ttRfLqWhnr7mNmtg+4GsiY2fwwq18MFP6vygFLgOMhdfNp4NdNPGdJuHpn57X0w9eMX9KslqqbXjPLhMc9wFeAN4B9wM3hsA3AC+HxjvCc8PreSvl5kWKzmZ1HbVxeGNeMX9KultTNImCfmb0G/D2w291fBP4IuNfMjpLPwT8ejn8cuDiM3wtsav5pS1JVmp1Hidq4vDA+m68pkiRVUzfu/hrQV2b8HfL5+tLx3wC3NOXsJHWqzc7LGVy/suKG5rP5moVUT25snC4
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"confusion_matrices = [confusion_matrix(target_labels, apply_linear_model(model, data))for model in models]\n",
"\n",
"fp, tp = list(map(lambda cm: cm[1,0], confusion_matrices)), list(map(lambda cm: cm[0,0], confusion_matrices))\n",
"plt.scatter(fp, tp)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Exercise three\n",
"By looking at the plot, which is the best model?\n",
"\n",
"Answer: The best model is the one at the top left corner, which has the highest TP/FP ratio"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Exercise four\n",
"Find the model with the best accuracy and compare it with the target model, is it close? Is it the model you would have picked up visually from the scatter plot?"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Plot best: [ 4.37669631 -0.8778982 ], Accuracy best: [ 4.37669631 -0.8778982 ], with accuracy: 0.986\n"
]
}
],
"source": [
"def accuracy(confusion_matrix):\n",
" return sum(confusion_matrix.diagonal()) / sum(sum(confusion_matrix))\n",
"\n",
"models_acc = list(map(lambda m: (m[0], accuracy(m[1])), zip(models, confusion_matrices)))\n",
"models_acc = sorted(models_acc, key=lambda ma: ma[1], reverse=True)\n",
"\n",
"plot_best = models[np.argmax([t / f for t, f in zip(tp, fp)])]\n",
"accuracy_best = models_acc[0]\n",
"\n",
"print(f'Plot best: {plot_best}, Accuracy best: {accuracy_best[0]}, with accuracy: {accuracy_best[1]}')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Exercise 5\n",
"Yes, because the models were generated with a uniform distributions of variables of range [-5;5]"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(array([ 4.37669631, -0.8778982 ]), 0.986)"
]
},
"execution_count": 38,
"metadata": {},
"output_type": "execute_result"
}
],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
2020-06-16 18:27:43 +02:00
{
"cell_type": "code",
"execution_count": null,
2020-06-23 21:53:50 +02:00
"metadata": {},
2020-06-16 18:27:43 +02:00
"outputs": [],
"source": []
}
],
"metadata": {
"anaconda-cloud": {},
"kernelspec": {
2020-06-23 21:53:50 +02:00
"display_name": "Python 3",
2020-06-16 18:27:43 +02:00
"language": "python",
2020-06-23 21:53:50 +02:00
"name": "python3"
2020-06-16 18:27:43 +02:00
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
2020-06-23 21:53:50 +02:00
"version": 3
2020-06-16 18:27:43 +02:00
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
2020-06-23 21:53:50 +02:00
"pygments_lexer": "ipython3",
"version": "3.7.7"
2020-06-16 18:27:43 +02:00
}
},
"nbformat": 4,
"nbformat_minor": 1
}
