handouts/handout_10.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 10 - Priors on Function Spaces: Gaussian Processes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Objectives:\n",
    "\n",
    "+ Express prior knowledge/beliefs about model outputs using Gaussian process (GP)\n",
    "\n",
    "+ Sample functions from the probability measure defined by GP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Readings:\n",
    "\n",
    "Please read the following before lecture: \n",
    "\n",
    "+ [Chapter 1 from C.E. Rasmussen's textbook on Gaussian processes](http://www.gaussianprocess.org/gpml/chapters/RW1.pdf).\n",
    "\n",
    "+ (Optional video lecture?) [Neil Lawrence's video lecture on Introduction to Gaussian processes](https://www.youtube.com/watch?v=ewJ3AxKclOg)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Modeling prior knowledge in Gaussian processes\n",
    "\n",
    "An experienced scientist or engineer typically has some knowledge about a function of interest $f(\\cdot)$ even before observing it anywhere. For example, he/she might know that $f(\\cdot)$ cannot exceed, or be smaller than, certain values or that it is periodic or that it shows translational invariance. Such knowledge is known as the *prior knowledge*. \n",
    "Prior knowledge may be *precise*, e.g., the response is twice differentiable, or it may be vague, e.g., the probability that the periodicity is $T$ is $p(T)$. When one is dealing with vague prior knowledge, he/she may refer to it as *prior belief*. Almost always, prior knowledge a field quantity is a _prior belief_.\n",
    "\n",
    "\n",
    "Prior beliefs about $f(\\cdot)$ can be modeled by a probability measure on the space of functions from $\\mathcal{X}$ to $\\mathbb{R}$.\n",
    "A Gaussian process (GP) is a great way to represent this probability measure."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Introduction to Gaussian Processes.\n",
    "In many engineering problems we have to deal with functions that are unknown.\n",
    "For example, in oil reservoir modeling, the permeability tensor or the porosity  of\n",
    "the ground are, generally, unknown quantities.\n",
    "Therefore, we would like to treat them as if they where random.\n",
    "That is, we have to talk about probabilities on function spaces.\n",
    "Such a thing is achieved via the theory of *random fields*.\n",
    "However, instead of developing the generic mathematical theory of random fields,\n",
    "we concentrate on a special class of random fields, the *Gaussian random fields*\n",
    "or *Gaussian processes*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A Gaussian process (GP) is a generalization of a multivariate Gaussian distribution to\n",
    "*infinite* dimensions.\n",
    "It essentially defines a probability measure on a function space.\n",
    "When we say that $f(\\cdot)$ is a GP, we mean that it is a random variable that is actually\n",
    "a function.\n",
    "Mathematically, we write:\n",
    "\\begin{equation}\n",
    "f(\\cdot) | m(\\cdot), k(\\cdot, \\cdot) \\sim \\mbox{GP}\\left(f(\\cdot) | m(\\cdot), k(\\cdot, \\cdot) \\right),\n",
    "\\end{equation}\n",
    "where \n",
    "$m:\\mathbb{R}^d \\rightarrow \\mathbb{R}$ is the *mean function* and \n",
    "$k:\\mathbb{R}^d \\times \\mathbb{R}^d \\rightarrow \\mathbb{R}$ is the *covariance function*.\n",
    "So, compared to a multivariate normal we have:\n",
    "\n",
    "+ A random function $f(\\cdot)$ instead of a random vector $\\mathbf{x}$.\n",
    "+ A mean function $m(\\cdot)$ instead of a mean vector $\\boldsymbol{\\mu}$.\n",
    "+ A covariance function $k(\\cdot,\\cdot)$ instead of a covariance matrix $\\mathbf{\\Sigma}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But, what does this definition actually mean? Actually, it gets its meaning from the multivariate Gaussian distribution. Here is how: \n",
    "\n",
    "+ Let $\\mathbf{x}_{1:n}=\\{\\mathbf{x}_1,\\dots,\\mathbf{x}_n\\}$ be $n$ points in $\\mathbb{R}^d$.\n",
    "+ Let $\\mathbf{f}\\in\\mathbb{R}^n$ be the outputs of $f(\\cdot)$ on each one of the elements of $\\mathbf{x}_{1:n}$, i.e.,\n",
    "$$\n",
    "\\mathbf{f} =\n",
    "\\left(\n",
    "\\begin{array}{c}\n",
    "f(\\mathbf{x}_1)\\\\\n",
    "\\vdots\\\\\n",
    "f(\\mathbf{x}_n)\n",
    "\\end{array}\n",
    "\\right).\n",
    "$$\n",
    "+ The fact that $f(\\cdot)$ is a GP with mean and covariance function $m(\\cdot)$ and $k(\\cdot,\\cdot)$, respectively, *means* that the vector of outputs $\\mathbf{f}$ at\n",
    "the arbitrary inputs in $\\mathbf{X}$ is the following multivariate-normal:\n",
    "$$\n",
    "\\mathbf{f} | \\mathbf{x}_{1:n}, m(\\cdot), k(\\cdot, \\cdot) \\sim \\mathcal{N}\\left(\\mathbf{f} | \\mathbf{m}(\\mathbf{x}_{1:n}), \\mathbf{K}(\\mathbf{x}_{1:n}, \\mathbf{x}_{1:n}) \\right),\n",
    "$$\n",
    "with mean vector:\n",
    "$$\n",
    "\\mathbf{m}(\\mathbf{x}_{1:n}) =\n",
    "\\left(\n",
    "\\begin{array}{c}\n",
    "m(\\mathbf{x}_1)\\\\\n",
    "\\vdots\\\\\n",
    "m(\\mathbf{x}_n)\n",
    "\\end{array}\n",
    "\\right),\n",
    "$$\n",
    "and covariance matrix:\n",
    "$$\n",
    "\\mathbf{K}(\\mathbf{x}_{1:n},\\mathbf{x}_{1:n}) = \\left(\n",
    "\\begin{array}{ccc}\n",
    "k(\\mathbf{x}_1,\\mathbf{x}_1) & \\dots & k(\\mathbf{x}_1, \\mathbf{x}_n)\\\\\n",
    "\\vdots & \\ddots & \\vdots\\\\\n",
    "k(\\mathbf{x}_n, \\mathbf{x}_1) & \\dots & k(\\mathbf{x}_n, \\mathbf{x}_n)\n",
    "\\end{array}\n",
    "\\right).\n",
    "$$\n",
    "\n",
    "Now that we have defined a Gaussian process (GP), let us talk about we encode our prior beliefs into a GP. \n",
    "We do so through the mean and covariance functions. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interpretation of the mean function.\n",
    "What is the meaning of $m(\\cdot)$?\n",
    "Well, it is quite easy to grasp.\n",
    "For any point $\\mathbf{x}\\in\\mathbb{R}^d$, $m(\\mathbf{x})$ should give us the value we believe is more probable for \n",
    "$f(\\mathbf{x})$.\n",
    "Mathematically, $m(\\mathbf{x})$ is nothing more than the expected value of the random variable $f(\\mathbf{x})$.\n",
    "That is:\n",
    "\\begin{equation}\n",
    "m(\\mathbf{x}) = \\mathbb{E}[f(\\mathbf{x})].\n",
    "\\end{equation}\n",
    "\n",
    "The mean function can be any arbitrary function. Essentially, it tracks generic trends in the response as the input is varied. In practice, we try and make a suitable choice for the mean function that is easy to work with. Such choices include: \n",
    "\n",
    "+ zero, i.e.,\n",
    "$$\n",
    "m(\\mathbf{x}) = 0.\n",
    "$$\n",
    "\n",
    "+ a constant, i.e.,\n",
    "$$\n",
    "m(\\mathbf{x}) = c,\n",
    "$$\n",
    "where $c$ is a parameter.\n",
    "\n",
    "+ linear, i.e.,\n",
    "$$\n",
    "m(\\mathbf{x}) = c_0 + \\sum_{i=1}^dc_ix_i,\n",
    "$$\n",
    "where $c_i, i=0,\\dots,d$ are parameters.\n",
    "\n",
    "+ using a set of $m$ basis functions (generalized linear model), i.e.,\n",
    "$$\n",
    "m(\\mathbf{x}) = \\sum_{i=1}^mc_i\\phi_i(\\mathbf{x}),\n",
    "$$\n",
    "where $c_i$ and $\\phi_i(\\cdot)$ are parameters and basis functions.\n",
    "\n",
    "+ generalized polynomial chaos (gPC), i.e., \n",
    "using a set of $d$ polynomial basis functions upto a given degree $\\rho$\n",
    "$m(\\mathbf{x}) = \\sum_{i=1}^{d}c_i\\phi_i(\\mathbf{x})$ \n",
    "where the basis functions $\\phi_i$ are mutually orthonormal with respect to some \n",
    "measure $\\mu$:\n",
    "$$\n",
    "\\int \\phi_{i}(\\mathbf{x}) \\phi_{j}(\\mathbf{x}) d\\mu(\\mathbf{x}) = \\delta_{ij}\n",
    "$$\n",
    "\n",
    "+ and many other possibilities. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interpretation of the covariance function.\n",
    "What is the meaning of $k(\\cdot, \\cdot)$?\n",
    "This concept is considerably more challenging than the mean.\n",
    "\n",
    "Let's try to break it down:\n",
    "\n",
    "+ Let $\\mathbf{x}\\in\\mathbb{R}^d$. Then $k(\\mathbf{x}, \\mathbf{x})$ is the variance of the random variable $f(\\mathbf{x})$, i.e.,\n",
    "$$\n",
    "\\mathbb{V}[f(\\mathbf{x})] = \\mathbb{E}\\left[\\left(f(\\mathbf{x}) - m(\\mathbf{x}) \\right)^2 \\right].\n",
    "$$\n",
    "In other words, we believe that there is about $95\\%$ probability that the value of\n",
    "the random variable $f(\\mathbf{x})$ fall within the interval:\n",
    "$$\n",
    "\\left((m(\\mathbf{x}) - 2\\sqrt{k(\\mathbf{x}, \\mathbf{x})}, m(\\mathbf{x}) + 2\\sqrt{k(\\mathbf{x},\\mathbf{x})}\\right).\n",
    "$$\n",
    "\n",
    "+ Let $\\mathbf{x},\\mathbf{x}'\\mathbb{R}^d$. Then $k(\\mathbf{x}, \\mathbf{x}')$ tells us how the random variable $f(\\mathbf{x})$ and\n",
    "$f(\\mathbf{x}')$ are correlated. In particular, $k(\\mathbf{x},\\mathbf{x}')$ is equal to the covariance\n",
    "of the random variables $f(\\mathbf{x})$ and $f(\\mathbf{x}')$, i.e.,\n",
    "$$\n",
    "k(\\mathbf{x}, \\mathbf{x}') = \\mathbb{C}[f(\\mathbf{x}), f(\\mathbf{x}')]\n",
    "= \\mathbb{E}\\left[\n",
    "\\left(f(\\mathbf{x}) - m(\\mathbf{x})\\right)\n",
    "\\left(f(\\mathbf{x}') - m(\\mathbf{x}')\\right)\n",
    "\\right].\n",
    "$$\n",
    "\n",
    "Essentially, a covariance function (or covariance kernel) defines a nearness or similarity measure on the input space. We cannot choose any arbitrary function of two variables as a covariance kernel. How we go about choosing a covariance function is discussed in great detail [here](http://www.gaussianprocess.org/gpml/chapters/RW4.pdf). We briefly discuss some properties of covariance functions here and then we shall move onto a discussion of what kind of prior beliefs we can encode through the covariance function. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Properties of the covariance function\n",
    "\n",
    "+ There is one property of the covariance function that we can note right away.\n",
    "Namely, that for any $\\mathbf{x}\\in\\mathbb{R}^d$, $k(\\mathbf{x}, \\mathbf{x}) > 0$.\n",
    "This is easly understood by the interpretation of $k(\\mathbf{x}, \\mathbf{x})$ as the variance\n",
    "of the random variable $f(\\mathbf{x})$.\n",
    "\n",
    "+ $k(\\mathbf{x}, \\mathbf{x}')$ becomes smaller as the distance between $\\mathbf{x}$ and $\\mathbf{x}'$ grows.\n",
    "\n",
    "+ For any choice of points $\\mathbf{X}\\in\\mathbb{R}^{n\\times d}$, the covariance matrix: $\\mathbf{K}(\\mathbf{X}, \\mathbf{X})$ has\n",
    "to be positive-definite (so that the vector of outputs $\\mathbf{f}$ is indeed a multivariate\n",
    "normal distribution).\n",
    "\n",
    "\n",
    "### Encoding prior beliefs in the covariance function. \n",
    "\n",
    "+ **Modeling regularity**. The choice of the covariance function controls the regularity properties of the functions sampled from the probability induced by the GP. For example, if the covariance kernel chosen is the squared exponential kernel, which is infinitely differentiable, then the functions sampled from the GP will also be infinitely differentiable. \n",
    "\n",
    "+ **Modeling invariance** If the covariance kernel is invariant w.r.t. a transformation $T$, i.e., $k(\\mathbf{x}, T\\mathbf{x}')=k(T\\mathbf{x}, \\mathbf{x}')=k(\\mathbf{x}, \\mathbf{x}')$ then samples from the GP will be invariant w.r.t. the same transformation. \n",
    "\n",
    "+ Other possibilities include periodicity, additivity etc. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Squared exponential covariance function\n",
    "\n",
    "Squared expnential (SE) is the most commonly used covariance function.\n",
    "Its formula is as follows:\n",
    "$$\n",
    "k(\\mathbf{x}, \\mathbf{x}') = v\\exp\\left\\{-\\frac{1}{2}\\sum_{i=1}^d\\frac{(x_i - x_i')^2}{\\ell_i^2}\\right\\},\n",
    "$$\n",
    "where $v,\\ell_i>0, i=1,\\dots,d$ are parameters.\n",
    "The interpretation of the parameters is as follows:\n",
    "\n",
    "+ $v$ is known as the *signal strength*. The bigger it is, the more the GP $f(\\cdot)$ will vary\n",
    "about the mean.\n",
    "\n",
    "+ $\\ell_i$ is known as the *length scale* of the $i$-th input dimension of the GP.\n",
    "The bigger it is, the smoother the samples of $f(\\cdot)$ appear along the $i$-th input dimension.\n",
    "\n",
    "Let's experiment with this for a while:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  \u001b[1mrbf.       \u001b[0;0m  |  value  |  constraints  |  priors\n",
      "  \u001b[1mvariance   \u001b[0;0m  |    1.0  |      +ve      |        \n",
      "  \u001b[1mlengthscale\u001b[0;0m  |    0.3  |      +ve      |        \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a1c9b8090>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " /Users/ibilion/anaconda/lib/python2.7/site-packages/matplotlib/figure.py:1743: UserWarning:This figure includes Axes that are not compatible with tight_layout, so its results might be incorrect.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAAEMCAYAAADwJwB6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X1YVHXeP/D3zMAwMygPMgiKljxkutaGihr9WCvZu70v\n0yWNNDHb3dtSMLN7y1/W7S8y8xZ3t8vVHqQVq03cco3utpL2rmvXovWBFVJy1dSMUVEEGZ5lmAdm\n5vcHnMERBnmYOecMvF/X5XXJmTMzH8rhzef7/Z7vUTidTieIiIh8TCl1AURENDQwcIiISBQMHCIi\nEgUDh4iIRBEgdQHeZDabcfz4cURGRkKlUkldDhHRkGO321FTU4PbbrsNGo3G7bFBFTjHjx/H4sWL\npS6DiGjI+9Of/oSkpCS3Y4MqcCIjIwG0f6PR0dESV0NENPRUVVVh8eLFrp/H1xpUgSMMo0VHR2PM\nmDESV0NENHR1N63BRQNERCQKBg4REYmCgUNERKKQJHCOHTuGlJQUj4/v3bsXqampSExMxPLly2E0\nGkWsjoiIfEHUwHE6nSgoKMB//Md/wGazdXvOqVOn8OKLL2Lz5s0oLi6GXq/H888/L2aZRETkA6IG\nzptvvomdO3ciMzPT4zmffvopUlNTcccdd0Cj0WD16tX4xz/+wS6HiMjPibos+sEHH0RmZiYOHz7s\n8Zzy8nJMnjzZ9XV4eDhCQ0NhMBig1+vFKJNIEgVfnUFR2UUolQosnHUr7rpttNQlEXmVqIEzcuTI\nG57T2traZTsErVaL1tZWX5VFJLmGZjNe2V0Ku6P99lQVV5rx4ctzoVAoJK6MyHtkt0pNo9HAbDa7\nHWttbYVOp5OoIiLf+6LkPOwOJybfEgl9qBYVV5rxr3IOI9PgIrvAiY+Ph8FgcH1dV1eHxsZGxMfH\nS1gVkW8VFrf/m0+/ezz+fcY4AMBnxYYenkHkf2QXOHPmzMEXX3yB0tJSWCwWbN68GTNnzkR4eLjU\npRH5hOFyI06eq0WwJhAzE8fg/jtjAbR3PVabXeLqiLxHFoGTnZ2N7OxsAMDEiRPx8ssvY+3atUhO\nTsaVK1eQk5MjcYVEvvP3by4AAFKn3gSNOgAJY8Ixfmw4mk1WHP6uSuLqiLxHks07Z8yYgX/+85+u\nr9evX+/2+OzZszF79myxyyKSxMlztQCAGRM7dzifPjEaZyrq8d35WqT8OEaq0oi8ShYdDtFQdrqi\nHgAw/qYRrmO3jm0fQj7T8RjRYMDAIZJQQ7MZV+pN0AYFYOzIYa7jt3aEz2kGDg0iDBwiCQmBkhAT\nBpWy8+N4U9RwBAWqcLm2BY0tFqnKI/IqBg6RhIQhs1tvcl+FqVIqccuYMADA9+xyaJBg4BBJ6HRF\nHQBg/NgRXR4b3zGPw2E1GiwYOEQScnU4Y7teZ3ZrRwhx4QANFgwcIomYLW04X9UMlVKB+JiwLo8L\nw2ynL9SJXRqRTzBwiCRy/koTHE6na4HA9eJHt4fQ+epm2B0Oscsj8joGDpFELl25CgCIiRze7eOa\noADoQ7Vosztwpd4kZmlEPsHAIZLIJWN74IyJHObxHOGxSzVXRamJyJcYOEQSEQInRu85cGKEwDEy\ncMj/MXCIJCJ0LTE9dDhCGF1kh0ODAAOHSCIXa5oBAGM8zOEAnYHDITUaDBg4RBJosztwubYFADAq\nItjjeTEj28OIQ2o0GDBwiCRwpd4Eu8OJyDAtNGrPdwnp7HCaxSqNyGcYOEQS6M0KNQCICNEgKFCF\nxhYrmk1WMUoj8hkGDpEEhEUAo/We528AQKFQuBYVVHJYjfwcA4dIAsIQWU8r1ATCogKuVCN/x8Ah\nkkBvh9QAzuPQ4MHAIZLAZWP7CrXRPaxQE4zSt59T2bGqjchfMXCIJHCloX1vtKjwGwdOVLiu/Tnc\nT438HAOHSGRtdgdqG81QKAB9qPaG5wuBU9PQ6uvSiHyKgUMkstomMxxOJ0YM1yAg4MYfwZHscGiQ\nYOAQiUwIDiFIbmREiAYqpQJ1zWZYbXZflkbkUwwcIpEJgRPVy8BRKZWI6Bh6q2nksBr5LwYOkciq\n+9jhAFw4QIMDA4dIZH0dUgOAkWEdCwcYOOTHGDhEIhMCJzKsD4HTEU7VDQwc8l8MHCKR9XUOB+BK\nNRocRA2ckydPIj09HYmJiUhLS0NZWVm3533wwQdITU3F1KlT8fDDD+P48eNilknkU8JFn30aUgtv\nXzTAwCF/JlrgWCwWZGZmYv78+SgpKcGSJUuQlZWFlhb37TpOnTqFV155BTt27EBJSQlmzZqFp556\nSqwyiXzK4XC6LuCMDLvxRZ8CLhqgwUC0wCkuLoZSqURGRgYCAwORnp4OvV6PoqIit/POnz8Ph8MB\nu90Op9MJpVIJjUYjVplEPlV/1Yw2uwOhwUE93njteq4hNe42QH6s9//iB8hgMCA+Pt7tWGxsLMrL\ny92OpaSkYNy4cbj//vuhUqkQHByMnTt3ilUmkU/1Z4Ua0LkFTm1jK9rsDgSoOP1K/ke0f7Umkwla\nrfsQgkajgdlsdjtmsViQkJCAgoICHD16FL/4xS+wcuXKLucR+SNX4PRhOA0AAgNUGBGigd3hRG0T\nPwvkn0QLHK1W2yU0zGYzdDr33/Ref/11REdH4/bbb0dQUBCeeOIJ2Gw2HDx4UKxSiXxGmL/R9zFw\ngM5rcYzcbYD8lGiBExcXB4PB4HbMYDAgISHB7VhlZSWs1s57tysUCqhUKqhUKlHqJPKluo7uJKIX\nu0RfLyKkfS6zloFDfkq0wElOTobVakV+fj5sNhsKCgpgNBqRkpLidt4999yDgoICnDhxAm1tbXjn\nnXdgt9sxdepUsUol8pnapvawEMKjL4SQ4pAa+SvRFg2o1Wrk5eVh3bp12Lx5M26++Wbk5uZCp9Mh\nOzsbALB+/XosXLgQTU1NePLJJ9HU1ISJEydix44dGDbsxrfiJZK72sb2sNCHsMOhoUe0wAGACRMm\nYPfu3V2Or1+/3vV3hUKBZcuWYdmyZWKWRiQKV4cTOpAOh4FD/olrK4lE5JrDGVCHwyE18k8MHCKR\nOJ2dS5pHDGgOhx0O+ScGDpFIWlptsNjs0AYFQKcJ7PPz2eGQv2PgEInE6BpO699WTdd2OE6n02t1\nEYmFgUMkkjrXkui+z98AgC4oAEGBKpitdpgsbd4sjUgUDBwikQxk/gZoX8EZcc2eakT+hoFDJBIh\nJPqzJFrgmsfhxZ/khxg4RCKpHcCSaAE7HPJnDBwikXR2OAMInI4Op44dDvkhBg6RSISQGDF8AENq\nvBaH/BgDh0gkwpCannM4NEQxcIhEUjvAZdHXPpdzOOSPGDhEInA4nJ1Dav1cFg10dkfscMgfMXCI\nRNDUYoHd4cRwnRrqwP7fTJAdDvkzBg6RCAa6rY1gxDVzOA4Ht7ch/8LAIRKBa0n0AOZvAEAdqEKI\nTg27w4mmFos3SiMSDQOHSASuiz4HsEJNILyGkfM45GcYOEQi8MaCAQHncchfMXCIROCNJdGCCK5U\nIz/FwCESgTc27hS4OhzuNkB+JqAvJxsMBtTW1kKpVCIyMhJjx471VV1Eg4o3Nu4U8M6f5K9uGDjf\nfPMNdu3ahQMHDqCpqcl1XKFQIDQ0FD/5yU+waNEiTJkyxaeFEvmzOi8tiwa4nxr5L4+Bc/78eWRn\nZ6OqqgqpqanYunUr4uPjERYWBofDgfr6epw6dQqlpaV45plnMGbMGKxfvx6xsbFi1k/kF7w6h+Na\nNMAOh/yLx8BZs2YNnnjiCfzkJz/p9vFRo0Zh1KhRuPfee7F69Wrs27cPzz33HP785z/7rFgif9TW\n5kDDVQsUCiBseNCAX69z0QA7HPIvHgNn9+7dvX4RhUKB1NRUpKameqUoosGk/qoZTmf7bQkCVANf\np8MOh/yVx8CxWq39ekG1Wt3vYogGI29egwMAocPUUCkVaGyxwNZmR2BA//dmIxKTx8C54447+vxi\nCoUCJ0+eHFBBRION0bWtjXcCR6VUIny4BsbGVtQ1WxAVrvPK6xL5msfAcTqdeO211xAaGtqrF2po\naMBTTz3ltcKIBovObW0GvmBAEBHSHji1ja0MHPIbHgNnypQpmDFjBkJCQnr1Qo2NjZg8ebLXCiMa\nLDpXqHmnwwE6wquingsHyK94nMH84x//CI1GA6vV2qs/Wq0Wf/zjH3t8s5MnTyI9PR2JiYlIS0tD\nWVlZt+eVlpZi3rx5mDx5MubOnYtDhw4N6JskklLnHI73OhxhPsjIhQPkR0Sbw7FYLMjMzERmZiYe\neughfPzxx8jKysLf/vY3BAcHu86rrq5GVlYWNmzYgPvuuw+FhYV48sknsX//fmg03vsNkUgswmoy\nb2xrIxC6pTp2OORHRJvDKS4uhlKpREZGBgAgPT0d7777LoqKijB79mzXeR9//DHuuusu/OxnPwMA\nzJkzB7GxsVAque0b+SdvXvQpEOaD6riBJ/kR0eZwDAYD4uPj3Y7FxsaivLzc7diJEycQFRWFJ554\nAqWlpRg3bhzWrl3L5dbkt3zS4QznfmrkfzwGznvvvQcAaGlpwYkTJ1BVVeWaq4mKisKPfvQj6HSd\nq2NCQ0Ndz+mOyWSCVuv+G55Go4HZ7P6BaWxsxNdff43XXnsNW7ZswZ49e7Bs2TJ8/vnnve62iOSk\nrrkjcIZ7edEAuNsA+RePgWOxWJCTk4MPP/wQDocDYWFhUKvVsFqtaGhoQEBAAB566CE8++yzveo+\ntFptl3Axm81uoQW0Xzg6c+ZMpKSkAAAWL16Mt956C0eOHMG9997bn++RSDIWmx3NJitUSgVCgge+\nrY1ghGsOhx0O+Q+PEyMbNmzA4cOH8fbbb+Pbb7/FgQMH8OWXX+LAgQP49ttvkZeXh4MHD2LDhg29\neqO4uDgYDAa3YwaDAQkJCW7HYmNju+xy4HA44HQ6e/s9EclG3TVLopVKhdde13WLAnY45Ec8Bs5f\n//pXbNq0CdOmTUNAgHsjFBAQgOnTpyMnJweff/55r94oOTkZVqsV+fn5sNlsKCgogNFodHUygrS0\nNOzfvx9fffUVHA4H8vPzYbFYMGPGjH58e0TS6py/8d6CAQAYrlMjMECJFnMbzJY2r742ka94DByN\nRgOTydTjk5uamnq9ekytViMvLw+FhYWYPn06du3ahdzcXOh0OmRnZyM7OxsA8KMf/Qi5ubnYsmUL\npk6dio8++ghvvvmm29JpIn8h7DIwwovzN0D7JQjCa9Y2c1iN/IPHOZyHH34Yq1evxsqVKzFt2jSM\nHDkSQUFBsFqtuHLlCr755hts2bIFCxYs6PWbTZgwodtdqNevX+/2dUpKSpfOh8gfuZZEe7nDAdrn\ncarrTahrMiNGP8zrr0/kbR4DZ+XKlQgLC8OOHTuwbt06KBSd489OpxMxMTF4/PHH8ctf/lKMOon8\nUq0X7/R5PddKtUbO45B/6PEW04888ggeeeQRGI1GVFdXo7W1FRqNBtHR0dDr9WLVSOS3hDDwRYcT\nwZVq5Gd6DByBXq9nwBD1g7fvhXMtrlQjfzOg/WLq6up4l0+iHvhip2iB686f7HDITwwocDQaDebN\nm+etWogGHV8tiwY6uyYGDvmLAQWOTqfDypUrvVUL0aDji407BVw0QP6mxzkcp9OJAwcOoKyszLWX\nmkajQVRUFKZMmYLk5GSx6iTyOyazDWarHUGBKgRrejVd2ifc3ob8jcdPQUVFBbKyslBZWYlJkyZB\nr9dDrVajubkZ5eXleOuttzB27Fhs27YNMTExYtZM5BeuXRJ97WUF3tI5h8MOh/yDx8DJzs5GXFwc\nPvjggy67PAPtuz8/99xzyM7OxltvveXTIon8kS+XRANAsCYAQYEqmK12mMw26DSBPnkfIm/xOIdz\n9OhRPPXUU92GDdA+f7Nq1SocOXLEZ8UR+TNfXvQJtG9vE8GFA+RHPAbOqFGjbhgm//znPxEREeH1\noogGA19uayPovPMnh9VI/jwOqT399NNYvXo1Dh482GUvtZqaGpSWluLzzz/Hb37zGzHrJfIbvtq4\n81ojeOdP8iMeA+ff/u3f8P7772PXrl3Iz89HdXU1zGYzgoKCEB0djcTERLz//vu47bbbxKyXyG90\nzuH4LnCE1zaywyE/4DFwzp49iwkTJmDjxo29eiG73d7tDdWIhqrOORzfDamNCBGG1NjhkPx5nMOZ\nO3cuGhoaev1C9fX1mDt3rleKIhoMhBDw6RwOFw2QH/HY4TidTmRnZyMoqHf3YbdYLF4rimgwEIbU\nfLFxp4AXf5I/8Rg4fd0jTa1W44EHHhhwQUSDgdPp7BxS8+GiAW5vQ/7EY+AsXboUsbGxUKlUvXoh\nYQ6HiIAmkxVtdgeCNYHQBHl/WxuBEGZ1vM00+YEe53AaGxt7/UKcwyHq1Dl/47vuBrhmx+jGVjid\nTp++F9FAeW0Ox2zmb1hEAteSaB/O3wCAThMIXVAATJY2tLTaMEyn9un7EQ0E53CIfECMJdGCESEa\nmGquwthkZuCQrHkMnJycHDHrIBpUfL1x57UiQrS4WHMVtY2tGBcd4vP3I+qvAd2AjYi6V9vs2407\nr+VaGs2FAyRzDBwiHxD2NvPlNTgCYWECl0aT3DFwiHzAl7eWvh63tyF/wcAh8gFRO5yO9zCywyGZ\nG1DgOBwOVFZWora21lv1EA0KxkYTACAyzPcdjr5jYQIDh+RuQJdA19XVYdasWUhLS0N9fT3efPNN\nKJVsmmhoa2tzoL7ZAqVCgXAfbmsjEELNyHvikMwNKHBCQ0Oxc+dOTJ8+HRcuXGDYEKHz3jQjQjQI\nUPn+MxEZqmt/346uikiuPH4aerNNRmBgIGJiYgAAN910k/eqIvJjxob2wBFjOA0AwkOCoFQoUN9s\nga3NLsp7EvWHx8BZtGgRKioqenzyn/70pz7tn3by5Emkp6cjMTERaWlpKCsr6/H8Q4cOYcKECWhp\naen1exBJraZjLkUvwkWfAKBSKq9ZGs1hNZIvj4Gj1+uRlpaGPXv2dHns0qVL+MUvfoENGzb0egsc\ni8WCzMxMzJ8/HyUlJViyZAmysrI8hkljYyP+67/+ixsSkt+paRBvwYBACLcaLhwgGfMYOK+//jqe\ne+455OTkIDMzE3V1dQCA9957D3PnzkVVVRXy8/Pxwgsv9OqNiouLoVQqkZGRgcDAQKSnp0Ov16Oo\nqKjb89etW4fZs2f341sikpYwpKYP04n2nq7AaeA8DslXjzOaCxYswEcffYTm5mbcf//9yMjIwMaN\nG7Fw4UJ88sknSEpK6vUbGQwGxMfHux2LjY1FeXl5l3M/+eQTNDU1YdGiRb1+fSK5ELqMSJGG1IBr\nVqo1sMMh+brhEppx48Zh9uzZaG5uxpEjRzB58mQsXbq017ctEJhMJmi17h9AjUbT5bYGlZWV2Lp1\nKzZu3Nin1yeSC7EXDQCd3RSH1EjOegycc+fOISMjA5s2bcKKFSvcup29e/f26Y20Wm2XcDGbzdDp\nOocdHA4H1qxZg1//+teIiorq0+sTyYVwAaZexMCJ5MWf5Ac8Bs6OHTvwwAMPwGKx4MMPP8SKFSsw\nceJEfPDBB1iwYAHWrFmDJ598ste7DMTFxXW5BbXBYEBCQoLr66qqKnz77bdYt24dkpKS8POf/xwA\ncPfdd6O0tLQ/3x+R6GqEDidUvDkcDqmRP/AYOK+++iqysrKwZ88ejB8/3nU8MDAQzzzzDPLz83Hq\n1CnMmTOnV2+UnJwMq9WK/Px82Gw2FBQUwGg0IiUlxXXO6NGjcezYMZSWlqK0tBSffPIJAKCoqKhP\n80VEUrHa7GhssUClVCBsWN+GnQdC6Ka4aIDkzGPgfPTRR1i+fDlUKlW3j0+ZMgUff/wx7rvvvl69\nkVqtRl5eHgoLCzF9+nTs2rULubm50Ol0yM7ORnZ2dv++AyIZMV5z4zWlUiHa+3buNsAOh+TL49Y2\nTU1NN3yyTqfDSy+95Pr6yJEjmDJlisfzJ0yYgN27d3c5vn79+m7PHzNmDE6fPn3DOojkQooVagAQ\nNiwIKqUCjS1WWGx2BAV2/4sikZQ8djjbtm3DihUrUFxcDIfD4fEFHA4HvvrqKzz++ON44403fFIk\nkb+QYoUaACiVCtftrNnlkFx57HDy8vLw2WefYcOGDaipqcGMGTOQkJCA8PBwOBwO1NfX49SpUygr\nK0NkZCQyMzNx//33i1k7kewIcygRInc4QHtXdaXehJp6E2L0w0R/f6Ib6XG36NmzZ2P27Nk4dOgQ\nvv76a5SUlKCurg5KpRJ6vR6TJk3CY489xgl9og7V9e2BEzVCvBVqgpHhOpw4V4sr9Vw4QPLUq9sT\nJCcnIzk5udvH7HbuTkskqK7rCJzwYNHfWwi5agYOyVSvbtbR3QaeAHD06NFeb95JNBRU17dvRhsV\nLn6HI4QcA4fkqleBs3HjRrz77ruur5uamvDCCy8gIyOD98Ehuoarw5FgSM3V4dTxdh4kT70aUtux\nYwdWrFgBk8mEUaNG4Te/+Q1CQkLwhz/8ATNnzvR1jUR+we5wuFaIjRRxp2iB0FWxwyG56lXgJCUl\nYefOnXj88cdRW1uL//zP/8SvfvUrBAYG+ro+Ir9R22iG3eHEiOEaqCW4DiZqBIfUSN48Bs7+/fu7\nHFuxYgVycnJw8eJFHD582HVztGu3pyEaqqRcoQYA+lANVEoF6prMsNrskoQeUU88Bs5jjz3m8Ul7\n9uxxLSRQKBT47rvvvF8ZkZ8R5k6kWDAAtN9qWh+qRXW9CVcaTBgTOVySOog88Rg4p06dErMOIr/X\n2eGIvyRaMDJch+p6E6rrGDgkP71apUZEN9Z5DY40HQ7Aa3FI3hg4RF4iXOEvaeB0XIvD3QZIjhg4\nRF4iXPQ5UhYdDq/FIflh4BB5idSr1IBrrsWpY4dD8sPAIfKCtrb2iz4VCnFvLX09XotDcsbAIfKC\nqvoWOJ1AZJgOAQHSfaxGdQTO5VoOqZH8MHCIvKDS2P4DXur70IQPD4I2KADNJiuaWiyS1kJ0PQYO\nkRdcMl4FAIyWOHAUCgVGR7R3OUIIEskFA4fICyo7AidGL91FnwIh9ISaiOSCgUPkBZdqhMCR/tbO\nMZHtNVxi4JDMMHCIvKBSJkNq19bAwCG5YeAQeUFlrYwCJ4JDaiRPDByiATKZbahvtkAd0L5bs9Q4\npEZyxcAhGiChkxgVMQxKpULiauBapXa5tgUOh1Piaog6MXCIBuhSx/Lj0TJYoQYAOk0gwocHwdbW\nectrIjlg4BANkDB0JQxlyUEMFw6QDDFwiAZITivUBLwWh+SIgUM0QBevNAOQxzU4gjEd3VZFR21E\nciBq4Jw8eRLp6elITExEWloaysrKuj1vz549uO+++zBlyhQ8+OCDKC0tFbNMoj45X90EABgXHSpx\nJZ1ujgoB0FkbkRyIFjgWiwWZmZmYP38+SkpKsGTJEmRlZaGlxX2/p+LiYmzevBlbt25FaWkpHnnk\nEWRmZqK+vl6sUol6zWKzo9LYAqVC4eoq5ODm6I7AqWLgkHyIFjjFxcVQKpXIyMhAYGAg0tPTodfr\nUVRU5HZeVVUVli5diokTJ0KpVGLevHlQqVQ4e/asWKUS9VrFlWY4nE7ERA6DOlAldTkuQodzobqZ\nS6NJNgLEeiODwYD4+Hi3Y7GxsSgvL3c79sADD7h9/c0336ClpaXLc4nk4EJHByF0FHIxTKeGPlQL\nY2MrqupaZLWggYYu0Tock8kErdb9KmyNRgOz2ezxOWfPnsWqVauwatUqjBgxwtclEvXZOSFwouQV\nOEBnTec4rEYyIVrgaLXaLuFiNpuh03V/O979+/dj0aJFWLx4MZYtWyZGiUR9dq6qEQAwTmYdDgCM\nG8XAIXkRLXDi4uJgMBjcjhkMBiQkJHQ598MPP8SqVavw4osvYsWKFWKVSNRn56vblx3LMXCEYb4L\nXKlGMiFa4CQnJ8NqtSI/Px82mw0FBQUwGo1ISUlxO+/QoUN46aWXsH37dsyZM0es8oj6zOl04nxH\nhyO3ORzgmiG1ywwckgfRAketViMvLw+FhYWYPn06du3ahdzcXOh0OmRnZyM7OxsAkJeXB5vNhscf\nfxyTJ092/fn666/FKpWoV4yNrWgxtyE0WI2wYUFSl9OF0HUJw35EUhNtlRoATJgwAbt37+5yfP36\n9a6/v/3222KWRNRvwtzITVEhUCik3yX6etEjghEUqEJtkxnNJiuG69RSl0RDHLe2IeqnMxXtFyPf\nMiZM4kq6p1QqEDe6ffcDoVYiKTFwiPpJ+CE+fmy4xJV4JtR25iIDh6THwCHqp9MX6gAAt46V7zVi\nt97UXtvpCwwckh4Dh6gfLDY7zlU1QalQICFGnkNqwDUdDofUSAYYOET98MOlBtgdTtwcPRyaIFHX\n3vTJLTFhUCiA8soGWG12qcuhIY6BQ9QPp13zN/IdTgPabzd908gQ2B1OlFdyeTRJi4FD1A9nKoT5\nG/kuGBBwWI3kgoFD1A/CJLycV6gJbr2pvcbTHSFJJBUGDlEfWW12V7cgrAKTswkdNR431EpcCQ11\nDByiPvruQh0sNjtiR4XKckub690ep4dKqcDpC3UwmW1Sl0NDGAOHqI+OnrkCAJh8S6TElfSOThOI\n8WPDYXc48a9yo9Tl0BDGwCHqo7LvhcAZKXElvSfUerSjdiIpMHCI+sDucODbH2oAAIl+FDhCrWXf\n10hcCQ1lDByiPjh7qQFXW20YFRGM6BHBUpfTa4kJ7cN/xw1G2Np4AShJg4FD1Aed8zf+090AQPhw\nDWJHhcBis+PkOa5WI2kwcIj6YP+xSwCAaROiJa6k74Sa9/+rUuJKaKhi4BD10lWTFd+cuQKlQoGU\n20dLXU6fzbxjDACgqOyixJXQUMXAIeqlg8cr0WZ3IPGWSIQN10hdTp9NHR+FYdpAGC434kJ1k9Tl\n0BDEwCHqpa86OoO7OzoFfxMQoETKj2MAAEXfsssh8TFwiHrBarPj4PH2uY+Zif4ZOEBnWBYdZeCQ\n+Bg4RL2w78gFtJhtGD82HGMih0tdTr8l3zYaGrUK3/5Qg3OXebsCEhcDh6gXPiz6HgDw4MxbJK5k\nYII1gfh6tftOAAALAklEQVT3GbEAgA+//l7iamioYeAQ3cD3F+tRdrYGwZoA/PuMcVKXM2Dp97SH\n5t6D5Wi1tElcDQ0lDByiG9jz5RkAwP3JcdBpAiWuZuBuHTsCt8fpcbXVhr8WG6Quh4YQBg5RD8or\nG/HpgR+gVCjw0D3jpS7Hax5OvRUAkLf3X7xlAYmGgUPUg60FR2B3ODFvZgLGjQqVuhyv+enUmzFp\nXASMja3Y+flJqcuhIYKBQ+TB30rP4+DxSgRrArFs7o+lLserlEoFfr1gCgBg1xff4YfKBokroqGA\ngUPUjR8qG7D+3WIAQNYDd2BEiP/tLHAjdySMxJy74mCx2fF/t32NZpNV6pJokGPgEF3nfHUTnn7t\nK7Ra2nDftJux4N7BM3dzvTWLpuGWMeGouNKMZ974Ck0tFqlLokFM1MA5efIk0tPTkZiYiLS0NJSV\nlXV73t69e5GamorExEQsX74cRiNvi0u+53Q6UVR2EUs3fY7K2hZMGheB//fonVAoFFKX5jOaoAD8\nbsVM6EO1OPp9DR777Rc4db5O6rJokBItcCwWCzIzMzF//nyUlJRgyZIlyMrKQktLi9t5p06dwosv\nvojNmzejuLgYer0ezz//vFhl0hDUZnfgH8cu4skt+7B6WxEaW6xI+XEMcp/5KbRBAVKX53Mx+mF4\n57mfIW50KAyXm/Doxr9i3TsHceyHGjidTqnLo0FEtE9TcXExlEolMjIyAADp6el49913UVRUhNmz\nZ7vO+/TTT5Gamoo77rgDALB69WokJyfDaDRCr9f7vM5mkxW2Nke3jznh+cPX4+eyh8c8vWZPr9fj\nW/Xzif393nxTS/9es6fntdkdaDW3odXahqYWKy7XtuBy7VVcqG7GsR9qYOq4ADJYE4jlaT/GQ/eM\nR4Bq6Iw4R0cE4+3nfobtnx7Dn/9+GoWHDCg8ZMCI4RrcHqdHzMhhiIkYhqgROmiDAqFRqxCkVkGj\nDoA6QAkFFEBHI6hQKIS/QqHAdY91fN3xd5KfoECVz643Ey1wDAYD4uPj3Y7FxsaivLzc7Vh5eTkm\nT57s+jo8PByhoaEwGAw+D5z3/34Km//8jU/fg+QpdlQI5iTHYe7/iUe4H956wBuCNYH49UNTkX73\neHz09ff438PnUNPQyp2lh5gAlRKvPnWvT24yKFrgmEwmaLVat2MajQZms9ntWGtrKzQa9w+8VqtF\na2urz2scGabDyDAtbPbuOxyg87czDw/256Ee5wg8PdTjL4c9vV7/ntZzjX5Qi0qlgDYoELqgAARr\nAhEdEYzREcEYpR+G22IjEBmm66GaoWXsyOFYlT4FTz44Geerm3HmQh0u17Wg0ngVV+pNMFvtMFvb\nYLHZYbbaYbXZO7tLZ3uvLDSiTuc1vbOzs5PmSJ18DdepERoc5JPXFi1wtFptl3Axm83Q6dw/6J5C\n6PrzfCF16k1InXqTz9+HyB8oFAqMiw7BuOgQqUuhQUK0Qeq4uDgYDO77NhkMBiQkJLgdi4+Pdzuv\nrq4OjY2NXYbjiIjIv4gWOMnJybBarcjPz4fNZkNBQQGMRiNSUlLczpszZw6++OILlJaWwmKxYPPm\nzZg5cybCw8PFKpWIiHxAtMBRq9XIy8tDYWEhpk+fjl27diE3Nxc6nQ7Z2dnIzs4GAEycOBEvv/wy\n1q5di+TkZFy5cgU5OTlilUlERD6icA6ihfYXL15Eamoq/v73v2PMGP+9DTARkb/q6efw0LnQgIiI\nJMXAISIiUTBwiIhIFINqoyi73Q4AqKqqkrgSIqKhSfj5K/w8vtagCpyamhoAwOLFiyWuhIhoaKup\nqcHNN9/sdmxQrVIzm804fvw4IiMjoVKppC6HiGjIsdvtqKmpwW233dZlm7JBFThERCRfXDRARESi\nYOAQEZEoGDhERCQKBg4REYmCgUNERKJg4BARkSgYOEREJAoGjggKCgowY8YMqcuQnW3btuGee+5B\nUlISlixZgjNnzkhdkqROnjyJ9PR0JCYmIi0tDWVlZVKXJBulpaV46KGHMHXqVPz0pz/F7t27pS5J\ndoxGI5KTk/Hll19KXYpHDBwfq6iowKZNm6QuQ3b+53/+Bx9//DHy8/NRXFyM5ORkLF++HA6HQ+rS\nJGGxWJCZmYn58+ejpKQES5YsQVZWFlpaWqQuTXKNjY1YsWIFHn30UZSUlGDr1q3YvHkzDh48KHVp\nsrJ27Vo0NDRIXUaPGDg+ZLfb8eyzz2LhwoVSlyI79fX1yMzMxNixYxEQEIBHH30UlZWVQ3bj1eLi\nYiiVSmRkZCAwMBDp6enQ6/UoKiqSujTJVVZW4u6778bcuXOhVCoxadIkzJgxA0eOHJG6NNl4//33\nodVqMWrUKKlL6dGg2rxTbG1tbTCZTF2OK5VKDBs2DNu3b8ctt9yCmTNnoqCgQIIKpdXTf5+lS5e6\nHdu3bx/CwsIQHR0tVnmyYjAYEB8f73YsNjYW5eXlElUkHxMnTsTvfvc719eNjY0oLS1FWlqahFXJ\nh8FgwDvvvIM9e/Zg/vz5UpfTIwbOABw+fBi/+tWvuhyPiYnBq6++ik8++QQFBQU4fvy4BNVJr6f/\nPvv27XM778UXX8T69euhVA7NpttkMkGr1bod02g0MJvNElUkT83NzcjMzMSkSZMwa9YsqcuRXFtb\nG5599lmsXbsWYWFhUpdzQwycAbjrrrtw+vTpLsfNZjPS09OxYcMGBAcHS1CZPHj673Otv/zlL3jp\npZfwwgsvYO7cuSJVJj9arbZLuJjNZuh0Ookqkp+KigrXMOyWLVuG7C8n19q2bRsmTpyIu+++W+pS\neoX/x3zg+PHjqKiowPLly5GUlITMzEw0NjYiKSkJlZWVUpcnG2+88QZycnKwbds22Q8F+FpcXBwM\nBoPbMYPBgISEBIkqkpcTJ05gwYIFSElJwbZt27psez9UffbZZygsLERSUpLr58vTTz+N7du3S11a\n95zkc8XFxc7p06dLXYasFBQUOKdNm+Y8e/as1KXIgsVicaakpDh37tzptFqtzg8++MB55513Olta\nWqQuTXI1NTXOO++80/mHP/xB6lJk795773Xu27dP6jI8YodDkti+fTtaWlqQnp6OyZMnu/788MMP\nUpcmCbVajby8PBQWFmL69OnYtWsXcnNzOaSG9uvY6urqkJub6/Zv5fe//73UpVEf8QZsREQkCnY4\nREQkCgYOERGJgoFDRESiYOAQEZEoGDhERCQKBg4REYmCgUMkobNnz+L222/v9srwlStXYuHChbDb\n7RJURuR9DBwiCSUkJOCJJ57A66+/7rYz9J49e3DgwAH89re/hUqlkrBCIu/hhZ9EEmtra8OCBQug\nVqvx3nvv4cKFC5g3bx6effZZLFq0SOryiLyGgUMkA6dPn8aDDz6I559/HoWFha77KRENJgwcIpl4\n/fXXkZubi2HDhmHv3r2IjIyUuiQir+IcDpFMzJo1C21tbYiNjYVer5e6HCKvY+AQyYDVasXzzz+P\npKQkHDt2DO+9957UJRF5HQOHSAZee+01XL58Gb///e/x6KOP4pVXXkFFRYXUZRF5FedwiCR25MgR\nPPLII8jJyUFaWhrMZjN+/vOfIyoqCjt37oRCoZC6RCKvYIdDJCGTyYQ1a9bgnnvuQVpaGgBAo9Hg\nv//7v1FSUsKhNRpUGDhEEtq0aROamprw0ksvuR2fNm0aMjIyOLRGgwqH1IiISBTscIiISBQMHCIi\nEgUDh4iIRMHAISIiUTBwiIhIFAwcIiISBQOHiIhEwcAhIiJR/H+tE9SW3Kf1lQAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10cc520d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import seaborn as sns # Comment this out if you don't have it\n",
    "sns.set_style('white')\n",
    "sns.set_context('talk')\n",
    "import GPy\n",
    "# The input dimension\n",
    "dim = 1\n",
    "# The variance of the covariance kernel\n",
    "variance = 1.\n",
    "# The lengthscale of the covariance kernel\n",
    "ell = 0.3\n",
    "# Generate the covariance object\n",
    "k = GPy.kern.RBF(dim, variance=variance, lengthscale=ell)\n",
    "# Print it\n",
    "print k\n",
    "# and plot it\n",
    "k.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 1: Plotting a covariance function\n",
    "Remember:\n",
    "> The covariance function $k(x,x')$ measures the similarity of $f(x)$ and $f(x')$.\n",
    "\n",
    "The interactive tools provided, draw $k(\\mathbf{x}, \\mathbf{x}'=0)$ in one and two dimensions. \n",
    "Use them to answer the following questions:\n",
    "+ What is the intuitive meaning of $\\ell$?\n",
    "+ What is the intuitive meaning of $v$?\n",
    "+ There are many other covariance functions that we could be using. Try changing ``RBF`` to ``Exponential``. What changes do you nottice.\n",
    "+ Repeat the previous steps on a 2D covariance function.\n",
    "+ If you still have time, try a couple of other covariances, e.g., ``Matern32``, ``Matern52``.\n",
    "+ If you still have time, explore ``help(GPy.kern)``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAAEMCAYAAADwJwB6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAG+JJREFUeJzt3X98jXXjx/H3OfvhbLltasMSml8llB+zmWhMffPAUvIj\nP9Lt69eMW92SH18RhXHrTqQp6q7Mr5gbd9KduklFYi3ck+lWi2mNsZm1tc3OzvePbnskDmPnXNcx\nr+fj4dHjXOdcn/N2Ym+f67rO57I4HA6HAABwM6vZAQAANwYKBwBgCAoHAGAICgcAYAgKBwBgCAoH\nAGAIUwrnwIED6tixY/njvLw8jRkzRm3btlXnzp21bt06M2IBANzI28g3czgcWr9+vebOnSsvL6/y\n7dOmTZO/v7927dqlw4cPa8SIEWrSpIlatWplZDwAgBsZOsN57bXXtHz5csXGxpZvKygo0Mcff6xx\n48apWrVquvvuu9WzZ09t3LjRyGgAADczdIbz6KOPKjY2Vnv27CnfdvToUXl7e6tevXrl20JDQ7V1\n69YKjVlUVKTU1FQFBwdfMGsCABjPbrcrOztbLVq0kM1mu+A5QwunVq1aF20rLCy8KJTNZlNRUVGF\nxkxNTdWgQYNckg8A4BorV65UWFjYBdsMLZxL8fPzU3Fx8QXbioqK5O/vX6H9g4ODJf36m6tTp47L\n8wEAKi4rK0uDBg0q/9n8W6YXToMGDXTu3DllZmbq1ltvlSSlp6ercePGFdr//GG0OnXq6LbbbnNb\nTgBAxV3qFIfp38OpXr26unbtqr/+9a/65ZdfdODAAW3evFkxMTFmRwMAuJDphSNJL7zwgkpLSxUV\nFaVx48bpmWee0T333GN2LACAC5lySC0iIkJffvll+ePAwEAtXLjQjCgAAIN4xAwHAFD1UTgAAENQ\nOAAAQ1A4AABDUDgAAENQOAAAQ1A4AABDUDgAAENQOAAAQ1A4AABDUDgAAENQOAAAQ1A4AABDUDgA\nAENQOAAAQ1A4AABDUDgAAENQOAAAQ1A4AABDUDgAAENQOAAAQ1A4AABDUDgAAENQOAAAQ1A4AABD\nUDgAAENQOAAAQ1A4AABDUDgAAENQOAAAQ1A4AABDUDgAAEN4TOGkpKSod+/eatOmjR588EG99957\nZkcCALiQRxSO3W7XmDFjNHLkSKWkpGj27NmaPHmyjh8/bnY0AICLeEThnD17Vjk5ObLb7XI4HLJY\nLPLx8ZGXl5fZ0QAALuJtdgBJqlmzpgYOHKjx48frmWeeUVlZmWbPnq2QkBCzowEAXMQjZjhlZWWy\n2WxauHCh9u3bp9dee01z5sxRWlqa2dEAAC7iEYWzdetWHThwQN26dZOvr686d+6szp07a+PGjWZH\nAwC4iEcUzk8//aSSkpILtnl7e3MOBwCqEI8onA4dOujQoUNav369HA6H9uzZo48++kjdunUzOxoA\nwEU84qKBO+64Q4sWLdLChQs1e/Zs3XrrrZo3b55atmxpdjQAgIt4ROFIUnR0tKKjo82OAQBwE484\npAYAqPooHACAISgcAIAhKBwAgCEoHACAISgcAIAhKBwAgCEoHACAISgcAIAhKBwAgCEoHACAISgc\nAIAhKBwAgCEoHACAISgcAIAhKBwAgCEoHACAISgcAIAhKBwAgCEoHACAISgcAIAhKBwAgCEoHACA\nISgcAIAhvK/mxenp6Tp9+rSsVquCg4NVr149d+UCAFQxVyycr776SitWrNDOnTt19uzZ8u0Wi0UB\nAQHq1KmTBgwYoDZt2rg1KADg+ua0cI4eParp06crKytLXbt21cKFC9WoUSMFBgaqrKxMubm5SktL\nU3Jysp5++mnddtttev755xUaGmpkfgDAdcJp4UyaNEljxoxRp06dLvl8SEiIQkJC1KVLF02YMEHb\ntm3T5MmT9e6777otLADg+uW0cNasWVPhQSwWi7p27aquXbu6JBQAoOpxWjglJSXXNKCvr+81hwEA\nVF1OC+eee+656sEsFou++eabSgUCAFRNTgvH4XDolVdeUUBAQIUGOnPmjJ588kmXBQMAVC1OC6dN\nmzaKiIhQjRo1KjRQXl6eWrdufc1BsrKy9Nxzz2nv3r2qXr26hg8friFDhlzzeAAAz+K0cN5++21J\nFT+X4+fnV77P1XI4HIqLi1NERIQWL16sH374QYMGDVKLFi34fg8AVBEecQ5n//79OnnypCZMmCAv\nLy81adJEa9asUc2aNa96LACAZ/KIczgHDx5UkyZNNH/+fL333nuqXr26YmNj9cgjj1zTeAAAz+MR\n53Dy8vL05Zdfqn379tq+fbtSU1M1fPhw1atXT2FhYdc0JgDAszgtnFWrVkmSCgoKdPDgQWVlZamk\npER+fn6qXbu27rrrLvn7+5e/PiAgoHyfq+Xr66uAgACNGjVK0q9l9+CDD+pf//oXhQMAVYTTwiku\nLlZ8fLzWr1+vsrIyBQYGytfXVyUlJTpz5oy8vb3Vt29fTZw4sdJf9gwNDZXdbpfdbpeXl5ckyW63\ny+FwVGpcAIDncHo/nFmzZmnPnj3629/+pv3792vnzp3avn27du7cqf3792vZsmXatWuXZs2aVekQ\n9957r2w2mxYvXqzS0lKlpKToo48+Urdu3So9NgDAMzgtnA8++EBz585Vu3bt5O194UTI29tb4eHh\nio+P14cffljpEDabTYmJiTpw4IA6dOigCRMm6Nlnn1WrVq0qPTYAwDM4PaRms9lUWFh42Z3Pnj0r\nq9U1Nw1t0KCB3nzzTZeMBQDwPE4L57HHHtOECRM0duxYtWvXTrVq1VK1atVUUlKikydP6quvvtLL\nL7+sfv36GZkXAHCdclo4Y8eOVWBgoN544w3NmDFDFoul/DmHw6G6detqxIgR+uMf/2hETgDAde6y\nt5gePHiwBg8erFOnTunEiRP65ZdfZLPZVKdOHQUFBRmVEQBQBVy2cM4LCgqiYAAAlVKpM/45OTnc\n5RMAUCGVKhybzcZ6ZwCACqlU4fj7+2vs2LGuygIAqMIuew7H4XBo586d2rdvX/laajabTbVr11ab\nNm0UGRlpVE4AwHXOaeFkZGRo9OjRyszMVPPmzRUUFCRfX1/l5+fr+++/15tvvql69eopISFBdevW\nNTIzAOA65LRwpk+froYNG2rdunXy8/O76PnCwkJNnjxZ06dPZ4UAAMAVOT2H8/XXX+vJJ5+8ZNlI\nv56/GTdunFJSUtwWDgBQdTgtnJCQkCuWyZdffqlbbrnF5aEAAFWP00Nq48eP14QJE7Rr166L1lLL\nzs5WcnKyPvzwQ82bN8/IvACA65TTwnnggQe0evVqrVixQomJiTpx4oSKiopUrVo11alTR61atdLq\n1avVokULI/MCAK5TTgvnyJEjuvPOOzVnzpwKDWS325Wenq7GjRu7LBwAoOpweg4nJiZGZ86cqfBA\nubm5iomJcUkoAEDV43SG43A4NH36dFWrVq1CAxUXF7ssFACg6nFaOFe7Rpqvr68efvjhSgcCAFRN\nTgtn2LBhCg0NlZeXV4UGOn8OBwCAS7nsOZy8vLwKD8Q5HADA5bjsHE5RUZHLQgEAqh7O4QAADOG0\ncOLj443MAQCo4ip1AzYAACqKwgEAGILCAQAYgsIBABiiUoVTVlamzMxMnT592lV5AABVlNOr1Coi\nJydH0dHR6tWrl3Jzc/Xaa6/JamXSBAC4WKUKJyAgQMuXL1d4eLiOHTtG2QAAnHLaEA6H44o7+/j4\nqG7dupKk+vXruy4VAKDKcVo4AwYMUEZGxmV3XrlyJeunAQAqxGnhBAUFqVevXlq7du1Fz/344496\n4oknNGvWrKteAgcAcGNyWjiLFy/W5MmTFR8fr9jYWOXk5EiSVq1apZiYGGVlZSkxMVHTpk1zaaBT\np04pMjJS27dvd+m4AABzXfaigX79+ik8PFxTp05Vjx49FBoaqgMHDujxxx/XU089VeGVpK/G1KlT\nr+rW1gCA68MVLyu7/fbb1b17d+Xn5yslJUWtW7fWsGHD3FI2q1evlp+fn0JCQlw+NgDAXJctnB9+\n+EEDBw7U3LlzFRcXpw0bNig/P189evTQ5s2bXRokPT1db731lmbMmOHScQEAnsFp4bzxxht6+OGH\nVVxcrPXr1ysuLk7NmjXTunXr1K9fP02aNEl/+tOfXLLKQGlpqSZOnKipU6cqMDCw0uMBADyP08JZ\ntGiRRo8erbVr16pp06bl2318fPT0008rMTFRaWlp6tmzZ6VDJCQkqFmzZoqKiqr0WAAAz+T0ooEN\nGzaoUaNGTnds06aNNm3apHnz5lU6xJYtW5Sdna0tW7ZIkn7++WeNHz9eo0eP1siRIys9PgDAfE4L\n5+zZs1fc2d/fXzNnzix/nJKSojZt2lx1iH/+858XPI6Ojta0adPUpUuXqx4LAOCZnBZOQkKCfHx8\nNGTIEIWHhztdJ62srEyffvqpVq5cqbKyMr355ptuCwsAuH45LZxly5Zpy5YtmjVrlrKzsxUREaHG\njRurZs2aKisrU25urtLS0rRv3z4FBwcrNjZWPXr0cEmobdu2uWQcAIDnuOwXP7t3767u3bvriy++\n0Keffqq9e/cqJydHVqtVQUFBat68uYYPH66wsDCj8gIArlMVuj1BZGSkIiMjL/mc3W53aSAAQNVU\noRvYXGoBT0n6+uuvWbwTAFAhFSqcOXPm6J133il/fPbsWU2bNk0DBw7kPjgAgAqp0CG1N954Q3Fx\ncSosLFRISIjmzZunGjVq6PXXX9d9993n7owAgCqgQoUTFham5cuXa8SIETp9+rSeeuopDR06VD4+\nPu7OBwCoIpwWzueff37Rtri4OMXHx+v48ePas2dP+W2oO3bs6L6EAIAqwWnhDB8+3OlOa9euLb+Q\nwGKx6NChQ65PBgCoUpwWTlpampE5AABVXIWuUgMAoLIoHACAISgcAIAhKBwAgCEoHACAISgcAIAh\nKBwAgCEoHACAISgcAIAhKBwAgCEoHACAISgcAIAhKBwAgCEoHACAISgcAIAhKBwAgCEoHACAISgc\nAIAhKBwAgCEoHACAISgcAIAhKBwAgCEoHACAITymcJKTk9W3b1+1bdtW999/v9asWWN2JACAC3mb\nHUCS8vLyFBcXp2nTpqlHjx46dOiQhg4dqvr166tDhw5mxwMAuIBHzHAyMzMVFRWlmJgYWa1WNW/e\nXBEREUpJSTE7GgDARTyicJo1a6b58+eXP87Ly1NycrLuvPNOE1MBAFzJIwrnt/Lz8xUbG6vmzZsr\nOjra7DgAABfxqMLJyMjQY489poCAAC1evFhWq0fFAwBUgsf8RD948KD69eunjh07KiEhQTabzexI\nAAAX8oir1E6dOqXhw4dr6NChGjlypNlxAABu4BEznKSkJOXk5GjJkiVq3bp1+a8FCxaYHQ0A4CIe\nMcOJjY1VbGys2TEAAG7kETMcAEDVR+EAAAxB4QAADEHhAAAMQeEAAAxB4QAADEHhAAAMQeEAAAxB\n4QAADEHhAAAMQeEAAAxB4QAADEHhAAAM4RGrRQM3sqycAh0+liuHw6E6N9+kpvVqymq1mB0LcDkK\nBzCBw+HQjn3HtWzzv/VtRu4Fz91cw6b+Xe7QwAfulM2Xv6KoOvjTDBjsTH6Rpv1tl3Yf/EmS5F/N\nW3c3CpaPt1WHM3J1MrdQSzbt14bP/qM5IzupZcMgkxMDrkHhAAY6cjxX41/doZ9OFyjgJl+NjLlb\nD3dqLF8fL0m/znz2pp3QwqQUfZuRq5HzP9KUweF66N5GJicHKo+LBgCDHDp6WqNe/Fg/nS5Qi9Bb\ntGp6D/WLvqO8bCTJYrEovFkdvTOlm/pH36FSe5leeGe33t122MTkgGtQOIABjvx4RmMWbNPZwhJ1\nuruuXp/wgGrV9Hf6em9vqyY8FqYJj4VJkl5ck6ykT741Ki7gFhQO4GbZZwr11KLtyi8sUdQ9t2le\nbKcLZjWX0z/6Dk0e1E6SNH91snbsO+7OqIBbUTiAGxUWndOfX/lEJ3IL1bJhkGaNuFc+3hUrm/Me\njWqqkTEtVeZwaOqyz5X6/Sk3pQXci8IB3KTUXqb/W/q5Dmfkql6tP+ilMVHXfJnz8J4t9dC9jVR8\nzq7xr36ijJP5Lk4LuB+FA7iBw+HQi2uStTM1UwE3VdPLf+qswD/Yrnk8i8WiKYPC1b55iHLzi/XU\nou0683OxCxMD7kfhAG6Q+OE3Wr/jP/L1tuqvY6NUv3aNSo/p7W3V3FGd1LReTR07ma9nEnao+Jzd\nBWkBY1A4gItt+eJ7vfL3fZKkmf/bQfc0CnbZ2DfZfLRgbGfVCvTTviPZev7tL1RW5nDZ+IA7UTiA\nC31+4Ec9/85uSdKf+7bR/WENXP4etWr66+VxXXSTzVtb9x7Vkk37Xf4egDtQOICL7P8uW5Nf/0z2\nMoee6HaXBj7QzG3v1eS2moof1UleVove/uCgErd+47b3AlyFwgFc4PCxHP35lU9UfM6uh+5tpDGP\ntHL7e0Y2v1XTnmgvSVqU9DWrEcDjUThAJe0/clKjXvy4/IudUwaHy2Ix5vYCPSIbauLAX78Y+uKa\nZL31QaocDs7pwDNROEAlfPBluuJe+pcKis6pa5v6ih/VUd5exv616tu56X9LTkrYsF9zVuzRuVKu\nXoPnYbVo4Br8UlyqRetTlPTJfyRJve9rrGcGtDO8bM7rfV8TVffz0fNv79bGz47oux/PaMbQSJdc\njg24CoUDXAWHw6HPDvyol9elKONkvry9rBrfv636RDUx7DCaM//T7nbdFvwHPbPkU/37+1Ma9MIW\nDevRUgPuv1PVKrh2G+BOHnNI7ZtvvlGfPn3UqlUr9erVS/v27TM7ElCutLRM21KOafhfturpV3co\n42S+Gt0aoLf/70H17dzU9LI5767bb9Hq6d31YHgDFZXY9eqGfer97D+UuPUb5RWwMgHM5REznOLi\nYsXGxio2NlZ9+/bVpk2bNHr0aH388ce66aabzI6HG1BZmUPpWXn693entP+7bH22/7jyCkokSTX8\nfTWsZ0v17dzkqhfiNEKNm6pp1vCOiunQSAuTUvSf42e0KOlrJWzYr4hmddT2jtpq0TBIzRrczC2s\nYSiP+NO2e/duWa1WDRw4UJLUp08fvfPOO9qxY4e6d+/u1vc+kVuotKM5v9t68VU+v7/w51LXAV10\nddAlXvT7TZe6oqgir/n9iyqU55JjX3LHCoz9+3Gv/Jld+vP43Xtd42d2rWOfs5ep4JdzKiw6p7OF\nJTqRW6gTOYXKyinQL8WlF7w2NKSGHo1qqpgODeVv87l4MA8TcVeIVjzbXbtSM/XutsPacyhLO1Mz\ntTM1U5LkZbWo9s03qXZNf9W+2V81q9vkV81Ltmre8vP1ll81b/l4WSWLRVaLZLVYpP/+12qxyGK1\nyKJf13nzkAkeXCA40E/NGtzilrE9onDS09PVqNGFt9ANDQ3V999/f8V97fZfr8bJysq6pvceMf8j\nncwtvKZ9UbUF1bDpjgY36876N+ueRkGqX7uGLBaLck6d0O//ieLJbr9ZmtSnqc78XF8p32Yr7dhp\nHT6Wq6NZ+TpemKvj3GIHv7PoyS5qcI0XnJz/WXz+Z/NveUThFBYWys/P74JtNptNRUVFV9w3Oztb\nkjRo0CC3ZMON62dJP0j60OQcgNH+d+CiSo+RnZ2tBg0uXNrJIwrHz8/vonIpKiqSv7/zW/Ce16JF\nC61cuVLBwcHy8vK84+kAcCOx2+3Kzs5WixYtLnrOIwqnYcOGWrFixQXb0tPT1bNnzyvua7PZFBYW\n5q5oAICr9PuZzXkecVl0ZGSkSkpKlJiYqHPnzikpKUmnTp1Sx44dzY4GAHARi8NDFl5KS0vTjBkz\ndPjwYTVo0EAzZsxQq1buXwARAGAMjykcAEDV5hGH1AAAVR+FAwAwBIUDADAEhQMAMASF4yZJSUmK\niIgwO4bHSUhIUOfOnRUWFqbHH39c3377rdmRTMdK6ZeXnJysvn37qm3btrr//vu1Zs0asyN5pFOn\nTikyMlLbt283O4pTFI4bZGRkaO7cuWbH8Dh///vftWnTJiUmJmr37t2KjIzUqFGjVFZWZnY005xf\nKb13797au3evHn/8cY0ePVoFBQVmR/MIeXl5iouL05AhQ7R3714tXLhQL730knbt2mV2NI8zdepU\nnTlzxuwYl0XhuJjdbtfEiRPVv39/s6N4nNzcXMXGxqpevXry9vbWkCFDlJmZec0Lr1YFv10p3cfH\nR3369FFQUJB27NhhdjSPkJmZqaioKMXExMhqtap58+aKiIhQSkqK2dE8yurVq+Xn56eQkBCzo1yW\nRyxtcz0pLS1VYeHFq0tbrVZVr15dS5cuVZMmTXTfffcpKSnJhITmutznM2zYsAu2bdu2TYGBgapT\np45R8TxOZVZKvxE0a9ZM8+fPL3+cl5en5ORk9erVy8RUniU9PV1vvfWW1q5dq969e5sd57IonKu0\nZ88eDR069KLtdevW1aJFi/SPf/xDSUlJSk1NNSGd+S73+Wzbtu2C1z333HN6/vnnZbXeuBPtyqyU\nfqPJz89XbGysmjdvrujoaLPjeITS0lJNnDhRU6dOVWBgoNlxrojCuUodOnTQ4cOHL9peVFSkPn36\naNasWTf0XUqdfT6/tXHjRs2cOVPTpk1TTEyMQck8U2VWSr+RZGRklB+Offnll2/of6T8VkJCgpo1\na6aoqCizo1QI/9dcJDU1VRkZGRo1apTCwsIUGxurvLw8hYWFKTMz0+x4HuPVV19VfHy8EhISPH76\nb4SGDRsqPT39gm3p6elq3LixSYk8z8GDB9WvXz917NhRCQkJstlsZkfyGFu2bNH777+vsLCw8p81\n48eP19KlS82OdmkOuMXu3bsd4eHhZsfwKElJSY527do5jhw5YnYUj1FcXOzo2LGjY/ny5Y6SkhLH\nunXrHO3bt3cUFBSYHc0jZGdnO9q3b+94/fXXzY5yXejSpYtj27ZtZsdwihkODLN06VIVFBSoT58+\nat26dfmv7777zuxopvH19dWyZcv0/vvvKzw8XCtWrNCSJUs4pPZfSUlJysnJ0ZIlSy74M7NgwQKz\no+EasFo0AMAQzHAAAIagcAAAhqBwAACGoHAAAIagcAAAhqBwAACGoHAAExw5ckQtW7a85DfCx44d\nq/79+8tut5uQDHAfCgcwQePGjTVmzBgtXrz4gpWh165dq507d+ovf/mLvLy8TEwIuB5f/ARMUlpa\nqn79+snX11erVq3SsWPH9Mgjj2jixIkaMGCA2fEAl6NwABMdPnxYjz76qKZMmaL333+//J5KQFVE\n4QAmW7x4sZYsWaLq1atr8+bNCg4ONjsS4BacwwFMFh0drdLSUoWGhiooKMjsOIDbUDiAiUpKSjRl\nyhSFhYXpwIEDWrVqldmRALehcAATvfLKK/rpp5+0YMECDRkyRC+++KIyMjLMjgW4BedwAJOkpKRo\n8ODBio+PV69evVRUVKSHHnpItWvX1vLly2WxWMyOCLgUMxzABIWFhZo0aZI6d+6sXr16SZJsNptm\nz56tvXv3cmgNVRKFA5hg7ty5Onv2rGbOnHnB9nbt2mngwIEcWkOVxCE1AIAhmOEAAAxB4QAADEHh\nAAAMQeEAAAxB4QAADEHhAAAMQeEAAAxB4QAADPH/ET13w7KLCK8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a1c9d9250>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from ipywidgets import interactive\n",
    "def plot_kernel(variance=1., ell=0.3):\n",
    "    k = GPy.kern.RBF(dim, variance=variance, lengthscale=ell)\n",
    "    k.plot()\n",
    "    plt.ylim(0, 10)\n",
    "interactive(plot_kernel, variance=(1e-3, 10., 0.01), ell=(1e-3, 10., 0.01))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAAEMCAYAAADwJwB6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8FHX+x/HXtmTTe0GaoSgQpLcgnpwiCIgRBVQEJAoY\nAbs/mqcHHvUsdyjtRJQiyAkoeopyKApnQCF0UECkJUBCOmmbbfP7Y2aTAAaRbHYDfJ6Pxzx2dzI7\n+9mF7Dvf73znOzpFURSEEEKIGqb3dgFCCCGuDxI4QgghPEICRwghhEdI4AghhPAIo7cLqC6LxcL+\n/fuJiorCYDB4uxwhhLiuORwOsrKyaNmyJWaz+byfXfWBs3//fh555BFvlyGEEKKS5cuX06FDh/PW\nXfWBExUVBahvLjY21svVCCHE9S0jI4NHHnmk/Lu5sqs+cFzdaLGxsdSrV8/L1QghhAB+8xCHDBoQ\nQgjhERI4QgghPEICRwghhEdI4AghhPAICRwhhBAeIYEjhBDCIyRwhBBCeIQEjhBCCI+oNYGzbt06\nevfuTdu2benbty9ff/21t0sSQgjhRrVipoFjx44xadIk3nvvPdq1a8eWLVsYNWoUmzdvJjw83Nvl\nCSGEcINaEThxcXGkpKQQEBCA3W4nOzubgIAAfHx8vF2aEEIIN6kVgQMQEBBAWloavXr1wul0Mnny\nZAIDA71dlhBCCDepNYEDUKdOHfbs2UNqaiqjR4+mYcOGJCQkeLssIYQQblBrBg0AGI1GTCYTCQkJ\n9OzZk2+++cbbJQkhhHCTWhE4mzZtYvjw4eets9lsBAUFeacgIYQQblcrAqdFixbs37+ftWvX4nQ6\n2bRpE5s2beKee+7xdmlCCCHcpFYETlRUFAsWLGDp0qV06NCB2bNnM3fuXBo3buzt0oQQQrhJrRk0\n0KFDBz7++GNvlyGEEKKG1IoWjhBCiGufBI4QQgiPkMARQgjhERI4QgghPEICRwghhEdI4AghhPAI\nCRwhhBAeIYEjhBDCIyRwhBBCeIQEjhBCCI+QwBFCCOEREjhCCCE8QgJHCCGER0jgCCGE8AgJHCGE\nEB4hgSOEEMIjJHCEEEJ4hASOEEIIj5DAEUII4RESOEIIITxCAkcIIYRH1JrASU1NZeDAgbRv354e\nPXqwcuVKb5ckhBDCjYzeLgCgoKCA0aNH8/LLL9O3b19+/vlnkpKSaNCgAV27dvV2eUIIIdygVrRw\nTp8+ze23306/fv3Q6/XEx8fTuXNndu7c6e3ShBBCuEmtCJzmzZvz2muvlT8uKCggNTWVZs2aebEq\nIYQQ7lQrAqeywsJCkpOTiY+P54477vB2OUIIIdykVgVOWloaDz30ECEhIcyZMwe9vlaVJ4QQohpq\nzTf6gQMHGDRoEN26dWPevHmYzWZvlySEEMKNasUotezsbEaMGEFSUhKjRo3ydjlCCCFqQK1o4axe\nvZrc3Fzmz59P27Zty5d//OMf3i5NCCGEm9SKFk5ycjLJycneLkMIIUQNqhUtHCGEENc+CRwhhBAe\nIYEjhBDCIyRwhBBCeIQEjhBCCI+QwBFCCOEREjhCCCE8QgJHCCGER0jgCCGE8IjLmmng119/5X//\n+x/79u0jJycHvV5PZGQkLVu25Pbbb6dhw4Y1XacQQoir3CVbOKmpqQwfPpzExET++9//EhAQQKtW\nrWjevDlGo5G1a9fSt29fkpKS2L59u6dqFkIIcRWqsoUzceJEjh07xpAhQ3j77bcJCgr6ze2KiopY\nt24dM2bM4KabbmLmzJk1VqwQQoirV5WB06NHD+68887f3UFgYCCDBg1i0KBBbNiwwa3FCSGEuHZU\n2aV2OWFzobvuuqtaxQghhLh2VdnCefPNN//wznQ6Hc8991y1ChJCCHFtqjJw3nnnHdq0aYPJZLqs\nHdlsNvbs2SOBI4QQ4jddclj03LlziYiIuKwdZWVlcdttt7mlKCGEENeeKo/hvP/++4SEhFz2jsLC\nwnj//ffdUpQQQohrT5UtnK1bt7J169Y/vMOEhIRqFSSEEOLa5PZjOM8//7zbihNCCHHtkGM4Qggh\nPMJtx3BCQ0Pdcgxn7969dOvWrdr7EUIIUbtU2cJJSEjg1KlT/Pvf/2bXrl1kZmZitVoxm83ExMTQ\ntm1bBg0axA033ACAyWSq1vEbRVFYs2YNM2fOxGAwXPF+hBBC1E5VBk5KSgpjxoyhVatWdOrUicjI\nSHx8fLBarWRlZZGamsrSpUuZN28eXbp0qXYhCxYs4MsvvyQ5OZmFCxdWe39CCCFqlyoDZ+bMmYwa\nNYrRo0dX+eR58+Yxffp0Pvvss2oX8sADD5CcnMy2bduqvS8hhBC1T5XHcNLS0rj77rsv+eRevXpx\n4sQJtxQSHR2NTqdzy76EEELUPlUGTnx8PMuXL8fpdP7mzxVFYcmSJTRr1qzGihNCCHHtqLJLbcqU\nKYwYMYINGzbQrl07oqOj8fX1LT+Gs2vXLhwOB++8844n6xVCCHGVqjJwmjRpwvr16/nyyy/ZtWsX\nJ0+exGKxlI9Se+aZZ+jZsyf+/v6erFcIIcRV6pInfvr6+nLfffdx3333eaoeIYQQ16gqj+FcjrKy\nMtauXeuuWgDo3LkzP/74o1v3KYQQwvsu2cL5PYWFhUyYMEFaQEL8DkVRKCwoID87m7ysLPIuuM3P\nzqakqIiQ8HDCoqIIjYwkXLsNi4oiTLs1+/l5+60IccWqFTiRkZEcPHjQXbUIcc04l5fHlytXsm7F\nCtKOHCE/Oxu73V7t/Zr9/QmPiqJlp04kDh9OQs+eGI3V+jUWwmMu63+qw+EgOzu7fGqbsLAw+U8u\nxAXsdjtbN2zgs8WL2bh2LTartcptjUZjRetFa8H4BQRQkJtLfnY2uVqrpyA397znWUpKOH3iBKdP\nnOC/q1YRGRvLPUOHkjh8OI1btKjptyhEtVwyNdatW8fSpUvZv38/DoejfL3JZKJly5YMHz6cnj17\n1niRQtRmR3/+mU8XL+bzZcvIOnOmfL2v2UyPBx6g45//THh0dHm3WGhkJEEhIeh0Ohx2O5nHjpF+\n8CCF2dlEx8VRr1kzQmNi0Ol02O12CnJzy7vdcrOyyExP5+s1a9j1/fdkZ2Sw+LXXWPzaa7Ts2JHE\npCR6P/QQwWFhXvxEhPhtVQbO4sWLeeutt3jsscd47rnnfnMutQkTJnD27FmGDBniyZqF8LpzeXl8\n9e9/8+nixey7YJBLm65dSUxKoufAgQSFhKAoCsX5+Zw6dIgjKSmcOniQU4cOkX7wIBlHjmC32S7a\nv39wMHWbNaPuzTeX39Zr1oxWnTvjYzYz9NlnOfHLL3y2ZAmfLVlCZno6+7dvZ//27fz92We54777\nuHf4cBLuukt6I0StoVMURfmtH9x+++289NJLl2zBrF+/nhkzZvDdd9/VVH2/Kz09nTvvvJNvvvmG\nevXqea0OcX3Iz8nhrUmT+HTx4vO6zGLq1aPfsGHc++ij1Klfn/3ffceOdevY/d//cvb4cexVdK/p\nUf/qM+n1+AcEUFhYiAOwA7/5iwn4BQXR8JZbaN+nD+379KHBLbeQ+t13fLp4MV+vWUOZxVK+bURM\nDE9OnswDI0fKLOzCIy71nVzlnz7FxcXExcVdcscNGjSgoKDAPVUKUYs5nU4+XrSI2RMmlB9X8TWb\nufP++0kcPpyGcXHsXr+eZc89x76NG7FqX/oBQCBgQP1l8zEaMfv4oFcUlLIyFNfUUU4n+sJCQiu9\nps5oROfjg11RKLVYsCsKdsBRWMjRLVs4uGULy//yF8Lq1KFd797069+fZ6dP53/r1/PZ4sXs3rKF\nnMxMpj75JJ8sWsRL8+bRsmNHz31oQlygyhbOCy+8wJkzZ5g6dSqNGjW66OcnTpxg/PjxxMTEMHv2\n7BovtCrSwhE17acdO5g6ejT7tZnM/QICGDlpEvHx8fy8eTM7160jvdJoTT0QAkSYTOi07jIdWktG\nu71wMWjbOAEHYENt5fzW4mLz8SHbaqWo0jqD0Ujzbt1o36cPsS1a8MkHH/DVypVqDTodA0aN4unp\n0wkJD3fnRyREuUt9J1cZOEVFRUyaNIkNGzYQFhZ20VxqOTk5dO/enVmzZhEcHOyRN/JbJHBETTmX\nl8fbf/kLH82fj+vXpEf//jSKjOT7Dz/EUlR03vbBgYE0jIrCeuoUTquVQCAIMOv16CpNgqtDDR4T\n4FPpvgE1UKyogWPT7lcOGQVw6vWUOp0UAGWAT2go9tBQfj15EvsFk+3GxMXR6cEH+eyTTzh26BAA\noRERPDtrFvclJaHXV+vcbyEuckWB43L69Gl2795NZmYmpaWlmM1mYmNjadOmTfnVPr1JAke4m9Pp\n5D9Ll/LmuHHkZWUB0LBpU+4bMIDvFy8mr9JItIYtW9L05puxHj/OmR070APBQIhOh0FRMAO+nB8u\nRtTQQVvnWgxUhEwZaksH1JC5MIRKUYPIAhQAxYDJ35+6XbtSGhDAvq1bKTh7trzOFn/6E9Ht27Pi\nX//CUlICQOuEBCbNnUvztm3d+OmJ6121Aqe2k8AR7nRozx6mjxnDrpQUAMx+fjw8ahTZu3dzYNMm\nAEy+vvR7+mmi/P3Z88EH5Pz6KybUbrQgnQ69ohAIhKKGDahh4goW30r3XV1tFwaOq2vNSkUAue4r\n2lIM5KOGjl2no0BRKNSe27RHDxr36cOWb75hxxdfqDUYjfz5scc4kpHBRu2iiXq9ngdHj2bM3/5G\ncGjlI0hCXBkJHCF+h6IovDVpEotfe638nLM/JyZyS+PGfPnWWzi0WQLa3n03beLj2f7OO5QVFuID\nhAOuOdNDgTAqAiYctUvN4HSWH8PxAUy6itDRuzZ2JY4N7M6KgLEpFS0cB2DX6yl2OslDDRcLkAuU\nAIoWPPnaz6JuuokWjz7KZwsXcvb4cQDCb7iBu555hhXvvsuJX35R10VH87fFi7mtd++a+HjFdUQC\nR4hLcDqdTBszhlULFgBQv3FjJrz1FsUnT7LgyScBiGrQgEf//neOrVnD3lWr0AERBgPBTicoCn5A\njNGI0W5Hjxo04UCYrqJLTV85cSo3d/y0WwMVzZlSKhKnPHnAoQVRiQJ5BgPZDgeF2vso1uvJcjrV\nIdUmE1k2G0WA3mik3z//ycnMTD6eNQu71Yp/SAgzUlJY/8knLJw2jTKLBZOPD2+sXk33fv1q9PMW\n17ZLfSfLEUNxXXM4HEweMaI8bO4ZMoSP9+8nKiyMd59+GoD2ffowfeNGdr7xBntXrcIfaOTnR7DD\ngUFRqBcQQF3AaLcTpNNxo68vDYD6BgiOAN+6oI8DGgNNgKbATdrSHGj2G0vzSts01Z7XCAwNwa8O\nRIRAA6eDhkCDoCB8gACnkxtNJiKMRnQ2G9FAQz8/9HY7n44dS1BxMW/u3ElIVBQlBQW88eCDDH32\nWdbs20fDpk2xWa08f//9bFizxiOfvbj+SOCI65bdbuelYcNY+/77ADwwciRTlyyhOC+PWQ88gN1m\n44amTXlo/Hj+1b07adu3EwrEAkppKeFGI3E+PpiLi/EBGgQG0kBRqG8tIyJUC5lG2tKY8wOnKXBz\npaU50IKKALqZiqCpvDSu2KdPfajjB3WLCrlRB9FmM3qbjTC7ncaBgfgBhtJSGphM+AKb33yTrydM\n4Jn330dvMHDywAHmPP449Rs35r1Nm2jUvDl2u51xDz7IFytWeOBfQFxvqjXnRVFREdOmTcPX15de\nvXqRkJDgrrqEqFE2m40JgwezYfVqAB4eO5bxs2fjsNt5beBA8s6cwRwQwMCxY3m/d2/sJSXEGAwE\nOBz4AjeYzRgsFvRAhNlMmLWMiOIiAv2ASG2JQj2g4xqqZtYWP+3WX1tc931QD8SUoh6YKdEWS6V1\nZdpSjDZKAQKzwT8bAi0WAs1mzlosFBUVURcoNpnItNmoq3W3/fz55+SdOMGDEybw4bRppHz0EU06\ndKD///0fi777jlE9evDLvn1MGjIEu9VK4vDhNfwvIa4n1Qoch8NBeno606dPZ9GiRXTq1EmmzxC1\nnrWsjBcHDeI7baTWsBde4IXXXkOn0/HuCy/wszZC7e777uPLZ59FryjUN5kw2mwEATF6PVgsBKK2\nKkItFsIMoI8EIoBo7TaSihEFlYPGFTBB2jp/1OkIXIFTXOm2iPMDx3W/CHX8tXayjz4QIrIh6JwF\nP+BcYCCZRUVgs9HQZCLdZiNKe4mMffsoysykc48e/Pj11yybMIFGbdvSukcPFn37LU/07MnPO3fy\nclISNquVAaNG1di/hbi+yKABcV2xlJby3P33k/LVVwCMfOklxv7tb+h0Or5dupTZjz4KQMdbbiF/\n3z58gLrajAERqA0WExBjNhNssRClA59gKlo0EdptOBCjPcEVOAFUhIwfamC4gsgUADozOM6B1VbR\nwilEDZhS4BwVLZ4iIAc4q91mA1na/SwoLIMcnY5cdOQ6nTgNBs44HOrTjUbO2O0YfH2xREdzLC2N\noIgIXk9NJebGGzmXn09yr17lMyuMnz2bR7TjWUL8niuaS82luLiYAwcOkJGRgdVqxc/Pj5iYGFq0\naIG/v//vPV2IWmXpm2+Wh82YV1/liZdfBqCspIR/jRkDQIumTcnftw+A+mYzisVCqI8PYdoEnHXN\nZnwsFuro1JYF0aiBEwrcgBos9VADJRrKh62Fop54469tbwQwgfEG0IeoBRrqgOEs+GZBiKJ2m+Wg\njokORw2WQm3fTvXpKFSc0KOtCzoJRoeCU1Ew+PlxtrSUGwwGjjkcmO12bggK4lRhIabTpwkOCuJc\nTg5Lxo1j3EcfERwayjsbNjCmTx92paQw65ln6NKjh1xvR1RblYFTVlbGjBkzWLNmDU6nk9DQ0PLL\nE+Tn52M0Ghk4cCDjxo3Dx8fHkzULccV+2LABgHuGDi0PG4Cfvv++fKqaOtHRnPzlF5olJGDduhWA\nevXqYTl6lDqtWmHYu5dgHeh1oEQa0BkdEKqDYEXts4pGDYJIHQQqEG0Ag0P9bYvyBb8yQAf6GDDE\ngM6AmlRBoDsLRj0osWA/Bv6F4KeDfD3kO9QRCyE6yFTUlpQDNZQUHRQq6muWKhAMfvngo0CQ00lB\nSAhlBQXcfOut/JSSQpBej19ICKUFBST06MH6Tz5h3zffYCsrw+TrS2BwMPO/+orecXHkZWezbeNG\nCRxRbVWOUps6dSrbtm3jvffeY8+ePaSkpPDtt9+SkpLCnj17WLhwIVu2bGHq1KmerFeIK2a329m/\nfTsACXfddd7Pvpo/H4Cm7dtzZtcuAIK1P6Rib74Zy9GjAARmZ6MDQnx9wYwaNuGAXoFwndpl5o8a\nOsEK1AGMDggzqOOk/cpAFwim1mCMBl0E0BI1SSKBBkBz0PmDKQ6MN4POCGEOaOCn7ttPUVtQQagt\nnUgdGBW1Dl9F3SZCPZYa4usDZWXUadIEAEU70dNaUECDZs0A8LHb0ev1FObmkrJqVfln4h8YSOuu\nXQHY+8MP1f78hagycL788ktmzpxJx44dL7qAk9FopFOnTsyYMYP169e7pZCffvqJAQMG0KZNGxIT\nE9m9e7db9iuEy5H9+8vnEWvVpUv5+rMnTrBdG0CQ0KcPNm2bc3v3AuooNIDI+vVxnj5NAKC3llUc\n0AlG7S4zaq0O18H8aNTjM/V91MDQ6dUAMTVSj9fQDGiIGjZtUYMnHjV4mgMNQO8PpnitNWRRN401\nqMd+olFDxkd73UDU0XDhAA4IhECnAz1gOHBAncjg7FluaNwYgADt8O3p1FTa33MPAF+8/fZ5n1lr\nbeTpHq2lJ0R1VBk4ZrOZEu0Xryrnzp1zy2yzZWVlJCcnc//997N9+3aGDh3Kk08+SXFxcbX3LYSL\n66/00IgIGmh/8YPaunE6nYRERZW3aqKio7Hm5akn///0EwAhZWUAhIcEq0Hjjxo6RtRurUgqvvCj\nUVsgdQCjDfTRYGqpBggNUMMmCjVgGmv3G2i3twA3oqbLLaALB2OM2irSBYK/Q+2BC9Ze1/Va/qiD\nFlwDFCL06BwOtZVjsRDbsKH6PrT3XazNHn3uzBn+1L8/AL9s28ZhbbAAVARz+tGj5FSaDFSIK1Fl\nWjz00EO8+OKLrFy5kl9//ZXCwkKsVitFRUUcPXqUVatWMWHCBAYNGlTtIn744Qf0ej2DBw/GZDIx\nYMAAIiMj2aRNliiEO7gCp1WXLuh06nzNZaWlbFi4EICeo0ZxXBsSHaFNZBlbpw6KzUaw2QxnzxIA\nGAvPqd/aJtQv+gjUlozryz8UrasL0AWA6RYwxoIuFDVMYlED5RbUgLkJ9WzOKNSzP+ujJkobbduG\nQDO1VWRqBIY4MBorRsYFVnptP+1xOGB0gh+E6HXoAXNGBnrApk02aisoINjPDwCTzUY9rYtt3Zw5\n5Z9ZfIcO5X9UXngpbSH+qCoDZ+zYsSQnJ/Puu+/St29fOnXqROvWrenYsSN9+vRhwYIFjBw5kuee\ne67aRRw7dozGWjPfJS4ujqNav7kQ7lA5cFx++PhjCnNz0RsM3DVyJMe//x4AZ0YGAAFaqyZCG5EZ\nHhaijggL1Vec1BnA+a0M1/k3Jn8wNtKuRdAQNWRMQCvUpk8QasAEadW4LsMWo633QW39xGkv0gwI\nAUMQmJqowRJIxTBsM2pN4agtLX8gXI++rIxg7VhORJD6WjHR0QBER0UBcGzzZvqMHQvA9//+N0X5\n+YB6HOemVq0A6VYT1XfJYdFDhgxhyJAhZGdnX3Q9nMjISLcVUVJSgp/2l5aL2WzGUuna7EJUV1lp\nKQDmSsP5LVq3rcnXF3NAAHYtYJzaEGi0WaJdV+7UO5xqJihONR9cf7LpqXjsuo8BdK4r35i0W12l\n+5WvjAMVV8Bx/UyvrXNd5KDyc7VwMigVOVW5BtcmRgPgxGgyQZkVg0l9vutWr9VnKy0lWAsfu9WK\n3fX+K31eVvl9FNV0WQdgIiMjiY+Pp0OHDrRs2dKtYQPg5+d3UbhYLBY5z0e4VXzHjsD5XUMJDzyA\nj9lMWUkJm5Yvp36nTgD4aCeslYWoRzwKtC/mvOJi9VyXEiCPihkAclCvhFaqrc8FHIVgP6UOWeYY\n6lmadmAf6lmc+cBR1PmfKysEDmrbngF+QQ2e40A2OO3qkGmLom6ape3OdZ2CfO2pRUCODUWvJ7+o\nGPR6snNzAcg8fRqAHK0lc+Ott5Z3pbW7+25CtRaQzWrlpx07AGipfTZCXKlqHfHPzc3lzjvvrHYR\njRo14tixY+etO3bsGE0qHdgVorp+a8RVcEQEtw0eDKjHLuL+9CcA8rWWzpn0dAByz52D4GAKHU4c\n/v5qqLhmA8hB/eIvRv3yz0b9ss8FnDlgOwCOYiAN+Fnb4CfgCGoIHQBOoYbMUeCwtqN9qCGTod5X\n8sGeAfafwFGibpJLxawDOVRcAjQXcOqhCIrMftgBW4MG2AFjvXpYFAWD2UxuQQEA5jp1+Ol//wOg\nz1NPlX8+h/bswap9FpW7IoW4EtUKHLPZTH9tdEt1JCQkYLVaWbZsGTabjdWrV5OdnU23bt2qvW8h\nXFxfmBlpaWSeOlW+vq927OLMkSMoAQHq/bQ0jP7+WBUF3+bNUYCS6GgUIN9Spn6xW/XqF3sZ6pf+\nWdQWTgFqRri++BUFHIfBdhSUMtRgOQ5kAruB06gtmUJtJ7+ihlCWtu1JNbBs+8F5FuxGdf+FlE9l\nQ0ml17OhtnhynQBq60anI0MLz1LtOE5QixYogDkkhG3aCbExjRrR7u67yz8b13GviJgYbtBGuQlx\npaoVOP7+/ozVflmrw8fHh4ULF/LFF1/QqVMnPvjgA+bPny9dasKtmrdrV35OWeVutUZt29L81lsB\n2LF5M3qDAQUIad0agELteEfGkSPoQkMptDtQjEb1C92C2rLJBaw6NQDyqGh15ANpDij1A6UIbHvA\nnglKHmoLJgO1u203sEO7zQBOAAfVgLIdVQNLcUKeAdLsasBkUtGNd5aKWnJQZy84B6VmP2wArVph\ntdsxBASQpg2HLtPOL6rfuTObP/wQgD5jxpx3qoOrNdg6IaF8ZJ8QV+qSgwYURSElJYXdu3eXz6Vm\nNpuJiYmhXbt2br0cQbNmzVi5cqXb9ifEhcx+ftzcpg0HUlPZnZJCj/vvL/9Z7zFj+DklhV0bNnB7\nu3ac2rEDe2AgAGn799M8NhZrRga2m27CuG0bhU6F4GJAMUKuXR1Elq+of8IF6yBL0S7ziTpy7Eyp\nOmosyhfIBGeuOoJNfxLIBqUe6PJRxzWna62is+DMBBQ1sLJK1WMzDj1kOtVQy9MWK2rQOPRQ6FSn\nwXFCfkkp6HRkZmUB4Nu+Pc7NmzH4+pLu6sYODsZaWoqPnx93JiWVfyZOp7M8cKQ7TbhDlS2ctLQ0\n+vXrx9NPP82PP/5IcXExOp2OwsJCtm7dypgxY0hMTORUpa4JIWq7drfdBsCKt9/mm08+KV/ftlcv\ntWWjKBQ71a6o3Rs2oDOZUJxOcrVBLWnbtqH4+lJg10aUZdsrutTyUbuzMhV1XT7q4ZqTVFzb5qRF\nDQjFBvZDWjdbHugOozZTTqhhZDsAzgywK2qD50ypuu884LizogvPitq6KkR9zWynOplnAVgVbUxD\naChF2iCBQ1u2AOAIDqbgzBn1fWrdZs27dSMwLEz9ucPBlFGjOH38+HmfmxDVUWUL55VXXqFRo0as\nWrXqoiHLoA5lnjBhAq+88gqLFi2q0SKFcJfHxo/nf+vWcfzQIV4cOJAZy5dz94MPEhQezv3jx7N6\n+nR27NpFu0aNKDh6lAybjRi9nsz8fAwGA34OB2fKyoj19SWrrIyIAtC7WjJ61G4up3brQA2EENQw\nCNIW1wiyYMCvCJTDoI9QR5/pHOAsrBiQcE57bilqgLmuh1OoLRnavrKpGKGWpV7hIFMBi48PGXl5\noG1SZrfjNJlId7V4mjThlyNHMJpMDH71VUCdc+6Vxx7j82XLAOj/2GPlAy6EqI4qA2fXrl2sWbPm\nN8MG1OM3Tz/9NAMHDqyx4oRwt4iYGN777jtG9ujBrwcOMGHwYGxWK/2GDuXhV1/l1x072LV+PXvT\n07mjZ08/WatrAAAYvElEQVR+/e9/OeV0coPJxCmbDdcJAcfLyojw9aXEZiU8WyGoEDUgIrTbItRA\nCEKdecCXiguw+VFxkqg/EKCol+0sQ/2NdO3rUhdhc4VMNuddF8eZA3kOyNcuSVBktaIYDGQ4HBQD\nZUYjZ2w2dCYTDXv35mttDrkRb73FzV26YLPZmDRkCOs/+giAgcnJvDR3rhy/EW5RZZdanTp12Llz\n5yWf/OOPPxIREeH2ooSoSZGxsSz69ltubt0ap9PJXx59lI8XLcJgMPD8ihXExMVhtVpJ/eknbvu/\n/6MMOGmzYffxIRu18aIYjWSXlXHMqXAqIJDTFrCmox7//1W7PY46yvlXbTlSaTmoLT8B+1EHpR1B\nHUdwQFvv2ubIBcvRSvs/pj0+DkVZkKboSDObOVZaShFg9/EhTQubczodp+x2zOHh3DZlSnnY3PnY\nY/R64glsVivjHnywPGweeeYZ/jJvnlvmSxQCLtHCef7553nxxRfZsmULHTt2JDo6Gl9fX6xWK1lZ\nWaSmprJ+/XpmzZrlyXqFcIvwqCje3biR5F69OJCayuQRI7CWlfHQ6NGM//hjJnTtSnZ6Otu2b+fh\nDz5g1eOPk1ZWRqzRCHY7JXY7sQEBUFxMWlERgUCpr5nQAgthRaAvRO0Oc81z5oPayqm8BHD+Jad9\nUVs0rtaMaym7YLFy3jk/1nOQrUBhUBAZhYXqjAAmEzlOJ3lWK+j1ZDmdFCoKUTffzD1z5jB1wAAA\nmnbsyBNz52ItK+OFAQPY/MUXACSNG8ezM2dKy0a4VZWBc9ddd/Hhhx/ywQcfsGzZMjIzM7FYLPj6\n+hIbG0ubNm348MMPadmypSfrFcJtQsLDeefrrxnduzd7tm5l+pgxlJWW8ugLLzBm4UL+MWQI+7/7\njkZt2/LEt9+ypH9/zmRmEqbTEabTcaq4GD/ghsBAioqKKLFYiDCbKSqzEJEFgYWoQRKKOmLNl/OD\nxxU2rsdG1DBxBUtVYWMDiit1n5nNnLVYKCosBMASEMCZ4mJ1ohxfX06XlWEBmvbowYBFi5jcpw8l\nBQWEREUxbs0a7HY7z91/P1u1c3GeeOUVRk+eLGEj3E6nKNpFMS5w5MgRGjVqdNnNaYfD4ZXZAS51\n/WwhLkdxYSFj77mHHZs3A3B7v36Mnz2br2bP5vPZswFo07MnD06cyJfPPMOZvXsxAXX8/DBq87OF\nAhEGAzqHQ73op48PoVYrYTo1S3RG1LC5cPGjIohcgeMKndJKj12LDZwO9fBONpCn05PrdOIEHD4+\nZFitlALodBQZjWTZbChAlyeeoM6dd/L+iy+SdfIkeoOBKRs2kF9ayoynniJdmyj3qWnTGDlpUs19\n2OKad6nv5CoDp3nz5qSkpBAeHn5ZL5Kdnc1tt93Gzz//XP2K/wAJHOEOpSUlPHf//WzRLijoazbz\n2Pjx5O7cyfb//AcAo48P/Z56CuOZMxxYuxZbSQmBQIROh0FRMKKOGQhEnUvTNSjND7WBY0LNFZOu\n4r6+chAZUFsvWrDYHZUeKhX3HahZlK09dup05CkKea73gjqOwAoYfX25bdw4tm/fzq6vvgLAYDQy\naMoUtm7fzsa1a9V1BgPPzJzJ8BdfrIFPV1xPLvWdXGWXmqIovPLKK/j6+la1yXnKtPmWhLga+fn7\nM//LL/li+XLeePFFcjIzmT9lCg0aN6b/pElsWbKEnFOn+OSNN4hq0IAHXn2V/L172b1yJSesVgJQ\nWzl21NHJIaijowsNBgwOR3kjxgf1Ap0+qKFktIGPDXxLwKGo61w9Z05tcT22VrqvoAZQPnBOUXCi\nHtYp0LYJrlOHbiNGcLa4mPdmzSqf/bll9+6Et27NG1OnYtFaZ227deOluXPLL0MgRE2psoUzceLE\nK9rhjBkzqlXQHyUtHOFuhQUFzH3lFVbOmYNTOwn0z/fey0116rBx0SIc2iULQqKjad29OyGKQvrm\nzRRlZuKLGjYBqA0WP+2+CfDRWkIurgByhZGrgVM5YOzatgpg1+mwKgo21FZMMWrrpkBROIfa8olt\n1YrQ1q1Jy8jgwObN2FxXKb3hBroNH84nq1Zx4pdfAAiLiuL5117j3mHD5HiNcJsr6lI7cuQIcXFx\nGAyGy3oROYYjrjUHd+9m2ujR5dO7mP39GTxqFFm7dnHggqvR6vV6bmraFHNREcWnTmFEPa/T32jE\nx26v2I5KXWsX3Hdd/cY1LsB161oq/6K6WjdFAAYDgY0bc7akhHRtgk4Xg9HIn5OSOJqVxdda95le\nr2fQk08ydupUgrUrmwrhLlfUpdavX78/dAwnLy+Pfv36efwYjhA1pVmbNiz5/ns+W7KEf4wbR152\nNu/985/ceNNNjFy4EHteHju//JKf/vc/HHY7B7VJMc1AtJ8fdouFXLsdI1xycQ3L0VERKg7U1k1V\niwPQmUzkKwo5djuOw4fL6w4MC6Pt3XfTpmdPfjl2jPffeINS7UJzt3TuzEvz5tGiXbua+tiEqJLb\njuHI1TnFtUiv13NfUhJ/Tkzk7b/8hVULFnD88GFeGjmSxi1akJiUxKh//Yv0ffvYsW4dO9atI+/M\nGU6Wlpa3ckyov2g+RiNGnQ60q4e6GKgIHlegKOcXAUYjVrsdm9OJHfVUnUJtBBqoM16379OHdr17\nYzMY+PyDD/jr889zTpvWJjQigmdnzeK+pCQ5kVN4jRzDEeIPOJCayrTRo9m/fXv5OoPBwK133829\nw4dz+z33cOrgQTV8vviCwz/8UH4cqDIDFUHjazIRFhlJWWEhOl9fCnJzsSkKDtSWzMXPBr+gIFrf\ndRcd+val3d1349Tr+WL5cj5dvJgj+/eXb6fT6Xhg5Eienj6dUJkVRHjAFR3DuVpI4AhPc03b/5+l\nS/lq5UqKzp0r/1lwWBh9Bg8mcfhwWrRvT8m5c+SeOkXumTOcPnyYUwcPcurQIdIPHiTrxInz9qvT\n6aj862j08eGGm26iXrNm1L35Zuo2a0Zs48aEREUR1bAhiqLw3X/+w6eLF5Py5Zc4HI7y5zZs2pR7\nhw+nz+DB1L3xxhr/TIRwkcARooZYSkvZuHYtn77/Pj98/fV5gdGkZUvuGTKE1l27UjcujrDISHy1\ni54BlJWUcPqXX8pDKCAsDKfdzg0330y9Zs2IatiwfNCOoigUFxaSl5VF5qlTfL16NetWrCA/J6d8\nfwFBQdz90EMkDh8uF0wTXiOBI4QHZKSl8Z9ly/hs8eLyoccX8g8MJCwqirDISMKiogjVbsMiI/EP\nDCQ/J4f87GzysrLI027zs7PJy87Gpp1LU5lOp6PznXeSOHw4d/Tvj59cJVd42RWNUhNC/DGx9esz\nctIkRkycyJ6tW/l08WJ+PXCA3dpFzwBKioooKSrilOtqm1eoQZMm3Dt8OP2GDqVOgwbVLV0Ij5DA\nEcLNdDodbbp2pU3XrjgcjotaK7muVssF60uKiggJD7+o5XNhiyg8Koqg0FDpMhNXnWoFjtPpJCMj\nA19fX7kujhC/wWAwEBkbS2RsrLdLEcLrqhU4ubm53HHHHSQmJpKXl8eCBQtkjL8QQojfVK3ACQkJ\nYenSpXTq1ImTJ0+6JWymTp2KyWRi/Pjx1d6XEEKI2qPKhLicwWsmk4m6desC0KCaBy7z8vKYMGEC\ny5Ytq9Z+hBBC1E5VBs7DDz9MWlraJZ+8fPly+vXr55ZCBg8ejMFgoFevXm7ZnxBCiNqlysCJjIwk\nMTGRjz766KKfnTp1ikcffZSpU6fSv3//y3ohu93OuXPnLlqKiooAWLx4MdOmTcNfziMQQohrUpXH\ncObMmcNHH33EjBkz2LhxI9OnTyc8PJwVK1bw+uuvExUVxbJly+jQocNlvdC2bdtISkq6aH3dunXZ\nuHEjMTExV/4uhBBC1HqXHDQwaNAgOnXqxEsvvUTfvn2Ji4tj7969DB06lGefffayZ5IG6Nq1K4e0\n6duFEEJcf353WNmNN95Inz59KCwsZOfOnbRt25bHH3/8D4WNEEIIccnAOX78OIMHD2bmzJmMHj2a\nTz75hMLCQvr27cvnn3/uqRqFEEJcA6oMnHfffZf77ruPsrIy1qxZw+jRo2nevDmrVq1i0KBBjB8/\nnqeeeoqcSrPVCiGEEFWpcrboVq1aMWbMGEaMGFE+RXplO3fuZPz48RQVFbFVu+a7N8hs0UIIUXtc\n0WzRn3zyCY0bN65yp+3atePTTz9l1qxZ7qtUCCHENavKLrVzla5iWBV/f3+mTJlS/njnzp3uqUoI\nIcQ1p8oWzrx58zCZTAwbNoxOnTpVOU+a0+lk8+bNLF++HKfTyaJFi2qsWCGEEFevKgNn4cKFrFu3\njqlTp5KVlUXnzp1p0qQJYWFhOJ1O8vLyOHjwILt37yYqKork5GT69u3rydqFEEJcRS554mefPn3o\n06cPW7duZfPmzWzfvp3c3Fz0ej2RkZHEx8czYsSIy55tQAghxPXrsi5PkJCQQEJCwm/+zOFwuLUg\nIYQQ16bLuoDNb03gCbBr167LnrxTCCHE9e2yAmf69OksWbKk/PG5c+d4+eWXGTx4cLWvgyOEEOL6\ncFldau+++y6jR4+mpKSEOnXqMGvWLIKDg/nXv/7Fn/70p5quUQghxDXgsgKnQ4cOLF26lJEjR5KT\nk8Ozzz5LUlISJpOppusTQghxjagycL7//vuL1o0ePZoZM2aQnp7Otm3byi9D3a1bt5qrUAghxDWh\nysAZMWJElU/66KOPygcS6HQ6fv75Z/dXJoQQ4ppSZeAcPHjQk3UIIYS4xl3WKDUhhBCiuiRwhBBC\neIQEjhBCCI+QwBFCCOEREjhCCCE8QgJHCCGER0jgCCGE8AgJHCGEEB4hgSOEEMIjak3gzJs3j+7d\nu9OhQweGDh3K4cOHvV2SEEIIN6oVgfPxxx/z6aefsmzZMn744QcSEhJ44okncDqd3i5NCCGEm9SK\nwMnLyyM5OZn69etjNBoZNmwYp0+fJiMjw9ulCSGEcJPLuh6OO9jtdkpKSi5ar9frefzxx89bt3Hj\nRkJDQ4mNjfVUeUIIIWqYxwJn27ZtJCUlXbS+bt26bNy48bzt/vrXv/Lqq6+i19eKBpgQQgg38Fjg\ndO3alUOHDl1ym7Vr1zJlyhRefvll+vXr56HKhBBCeILHAuf3zJ07l6VLlzJv3jwSEhK8XY4QQgg3\nqxWBs2bNGpYsWcKHH35I48aNvV2OEEKIGlArAuedd96huLiYAQMGnLd+9erVEkBCCHGNqBWBs379\nem+XIIQQoobJMDAhhBAeIYEjhBDCIyRwhBBCeIQEjhBCCI+QwBFCCOEREjhCCCE8QgJHCCGER0jg\nCCGE8AgJHCGEEB4hgSOEEMIjJHCEEEJ4hASOEEIIj5DAEUII4RESOEIIITxCAkcIIYRHSOAIIYTw\nCAkcIYQQHiGBI4QQwiMkcIQQQniEBI4QQgiPkMARQgjhEbUicKxWK5MnT6ZLly60b9+eJ598kszM\nTG+XJYQQwo1qReDMnTuXX3/9la+++oqtW7cSGhrK3/72N2+XJYQQwo2M3i4A4Omnn8Zms2E2m8nL\ny6O4uJiwsDBvlyWEEMKNPBY4drudkpKSi9br9XoCAwMxGAzMmTOHOXPmEB0dzfLlyz1VmhBCCA/w\nWJfatm3b6Nix40XLvffeW77NyJEj2b17Nz179uTxxx/HZrN5qjwhhBA1zGMtnK5du3Lo0KFLbuPr\n6wvAuHHjWLlyJYcPHyY+Pt4T5QkhhKhhtWLQwMSJE1mxYkX5Y4fDgdPpJDg42ItVCSGEcKdaETit\nWrXivffeIz09ndLSUqZNm0b79u2pX7++t0sTQgjhJrVilNpDDz1ETk4ODz/8MDabjVtvvZXZs2d7\nuywhhBBuVCsCR6fTMXbsWMaOHevtUoQQQtSQWtGlJoQQ4tongSOEEMIjJHCEEEJ4hASOEEIIj5DA\nEUII4RESOEIIITxCAkcIIYRH1IrzcKrD4XAAkJGR4eVKhBBCuL6LXd/NlV31gZOVlQXAI4884uVK\nhBBCuGRlZdGwYcPz1ukURVG8VI9bWCwW9u/fT1RUFAaDwdvlCCHEdc3hcJCVlUXLli0xm83n/eyq\nDxwhhBBXBxk0IIQQwiMkcIQQQniEBI4QQgiPkMARQgjhERI4QgghPEICRwghhEdI4AghhPAICZzL\nYLVamTx5Ml26dKF9+/Y8+eSTZGZmerusKzZv3jy6d+9Ohw4dGDp0KIcPH/Z2SW4xdepUZs2a5e0y\n/rCffvqJAQMG0KZNGxITE9m9e7e3S3KLvXv30q1bN2+XUS2pqakMHDiQ9u3b06NHD1auXOntkqpl\n3bp19O7dm7Zt29K3b1++/vprzxagiN/15ptvKkOGDFHy8vKUsrIyZcKECcqYMWO8XdYVWbNmjdKz\nZ0/l5MmTis1mU+bOnat0795dcTgc3i7tiuXm5irjx49XbrrpJmXmzJneLucPsVgsym233aYsX75c\nsVqtyqpVq5QuXbooRUVF3i7tijmdTmXVqlVK+/btlU6dOnm7nCuWn5+vdOzYUfnss88Uh8Oh7N+/\nX+nYsaOSkpLi7dKuyNGjR5XWrVsrO3bsUBRFUVJSUpT4+HglJyfHYzVIC+cyPP300yxcuJDQ0FCK\ni4spLi4mLCzM22Vdkby8PJKTk6lfvz5Go5Fhw4Zx+vTpq3ry08GDB2MwGOjVq5e3S/nDfvjhB/R6\nPYMHD8ZkMjFgwAAiIyPZtGmTt0u7YgsWLGDp0qUkJyd7u5RqOX36NLfffjv9+vVDr9cTHx9P586d\n2blzp7dLuyJxcXGkpKTQrl077HY72dnZBAQE4OPj47EarvrJO93FbrdTUlJy0Xq9Xk9gYCAGg4E5\nc+YwZ84coqOjWb58uReqvDyXei+PP/74ees2btxIaGgosbGxnirvD/u9f5vFixcTExPDhAkTvFBd\n9Rw7dozGjRufty4uLo6jR496qaLqe+CBB0hOTmbbtm3eLqVamjdvzmuvvVb+uKCggNTUVBITE71Y\nVfUEBASQlpZGr169cDqdTJ48mcDAQI+9vgSOZtu2bSQlJV20vm7dumzcuBGAkSNHMmLECF5//XUe\nf/xxvvjiC0wmk6dL/V2X815c2/31r3/l1VdfRa+vvY3d33s/MTExXqjKPUpKSvDz8ztvndlsxmKx\neKmi6ouOjvZ2CW5XWFhIcnIy8fHx3HHHHd4up1rq1KnDnj17SE1NZfTo0TRs2JCEhASPvLYEjqZr\n164cOnToktv4+voCMG7cOFauXMnhw4eJj4/3RHl/yOW8l7Vr1zJlyhRefvll+vXr56HKrszlvJ+r\nlZ+f30XhYrFY8Pf391JF4kJpaWnl3dD//Oc/a/UfZ5fDaFS/9hMSEujZsyfffPONxwLn6v7kPGTi\nxImsWLGi/LHD4cDpdBIcHOzFqq7c3LlzmTFjBvPmzeP+++/3djnXtUaNGnHs2LHz1h07dowmTZp4\nqSJR2YEDBxg0aBDdunVj3rx5F023fzXZtGkTw4cPP2+dzWYjKCjIYzVI4FyGVq1a8d5775Genk5p\naSnTpk2jffv21K9f39ul/WFr1qxhyZIlrFixwmN/1YiqJSQkYLVaWbZsGTabjdWrV5OdnX3VDye+\nFmRnZzNixAiSkpKYOHHiVd+yadGiBfv372ft2rU4nU42bdrEpk2buOeeezxWg3SpXYaHHnqInJwc\nHn74YWw2G7feeiuzZ8/2dllX5J133qG4uJgBAwact3716tUXHbwWNc/Hx4eFCxcyefJk3nzzTRo2\nbMj8+fOlS60WWL16Nbm5ucyfP5/58+eXrx82bBjPPfecFyu7MlFRUSxYsIDp06fz6quvcuONNzJ3\n7lyP/t7LBdiEEEJ4xNXdRhRCCHHVkMARQgjhERI4QgghPEICRwghhEdI4AghhPAICRwhhBAeIYEj\nhJccOXKEW265hXfeeeein40dO5YHH3wQh8PhhcqEqBkSOEJ4SZMmTRgzZgxz5sw5b3bojz76iJSU\nFP7+979jMBi8WKEQ7iUnfgrhRXa7nUGDBuHj48OKFSs4efIk/fv3Z9y4cTz88MPeLk8It5LAEcLL\nDh06xAMPPMDEiRP54osvCAwM/M1uNiGudhI4QtQCc+bMYf78+QQGBvL5558TFRXl7ZKEcDs5hiNE\nLXDHHXdgt9uJi4sjMjLS2+UIUSMkcITwMqvVysSJE+nQoQN79+4979pLQlxLJHCE8LK3336bM2fO\n8I9//INhw4bx+uuvk5aW5u2yhHA7OYYjhBft3LmTIUOGMGPGDBITE7FYLNx7773ExMSwdOlSdDqd\nt0sUwm2khSOEl5SUlDB+/Hi6d+9OYmIiAGazmWnTprF9+3bpWhPXHAkcIbxk5syZnDt3jilTppy3\nvmPHjgwePFi61sQ1R7rUhBBCeIS0cIQQQniEBI4QQgiPkMARQgjhERI4QgghPEICRwghhEdI4Agh\nhPAICRwhhBAeIYEjhBDCI/4fJJSF9IvZbm4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a1cb79dd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from ipywidgets import interactive\n",
    "def plot_kernel(variance=1., ell1=0.3, ell2=0.3):\n",
    "    k = GPy.kern.RBF(2, ARD=True, variance=variance,\n",
    "                     lengthscale=[ell1, ell2])  # Notice that I just changed the dimension here\n",
    "    k.plot()\n",
    "interactive(plot_kernel, variance=(1e-3, 10., 0.01), ell1=(1e-3, 10., 0.01), ell2=(1e-3, 10., 0.01))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 2: Properties of the covariance matrix\n",
    "Let $\\mathbf{x}_{1:n}$ be an arbitrary set of input points. The covariance matrix $\\mathbf{K}\\in\\mathbb{R}^{n\\times n}$ defined by:\n",
    "$$\n",
    "\\mathbf{K}\\equiv\\mathbf{K}(\\mathbf{x}_{1:n}, \\mathbf{x}_{1:n}) = \\left(\n",
    "\\begin{array}{ccc}\n",
    "k(\\mathbf{x}_1,\\mathbf{x}_1) & \\dots & k(\\mathbf{x}_1, \\mathbf{x}_n)\\\\\n",
    "\\vdots & \\ddots & \\vdots\\\\\n",
    "k(\\mathbf{x}_n, \\mathbf{x}_1) & \\dots & k(\\mathbf{x}_n, \\mathbf{x}_n)\n",
    "\\end{array}\n",
    "\\right),\n",
    "$$\n",
    "must be [positive definite](https://en.wikipedia.org/wiki/Positive-definite_matrix). Mathematically this can be expressed in two equivalent ways:\n",
    "\n",
    "+ For all vectors $\\mathbf{v}\\in\\mathbb{R}^T$, we have:\n",
    "$$\n",
    "\\mathbf{v}^t\\mathbf{K}\\mathbf{v} > 0,\n",
    "$$\n",
    "+ All the eigenvalues of $\\mathbf{K}$ are positive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the code provided:\n",
    "+ Verify that the the sum of two covariance functions is a valid covariance function.\n",
    "+ Verify that the product of two covariance functions is a valid covariance function.\n",
    "+ Is the following function a covariance function:\n",
    "$$\n",
    "k(x, x') = k_1(x, x')k_2(x, x') + k_3(x, x') + k_4(x, x'),\n",
    "$$\n",
    "where all $k_i(x, x')$'s are covariance functions.\n",
    "+ What about:\n",
    "$$\n",
    "k(x, x') = k_1(x, x') / k_2(x, x')?\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> plotting eigenvalues of K\n",
      "> they must all be positive\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x1a1cb96910>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEQCAYAAACwSgOGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAE+lJREFUeJzt3X9QVWXix/EPoHRBS2yHtEZgiCQIMhTKnxMu6KqpOeZa\nGtPuspneaKcZrRz7YbnrzqTjoGWskGYW0jeaqM1GG2eAiGbSnGEYZfkhbUJp41oSI1MqiJf7/cMT\nu1Rmeu/lOZfzfs3wxz1yuR+Zy/nc5znnPCfE6/V6BQBwvFDTAQAA9kAhAAAkUQgAAAuFAACQJA0y\nHeBKdHZ2qr6+XtHR0QoLCzMdBwCCgsfj0cmTJ5WamiqXy/WTfw/KQqivr1dOTo7pGAAQlN544w1l\nZGT8ZHtQFkJ0dLSkC/+pkSNHGk4DAMHhxIkTysnJ6d2H/lhQFsIP00QjR47UqFGjDKcBgOBysal2\nDioDACRRCAAAC4UAAJBEIQAALBQCAEBSkJ5lBABO1O3p0cbyz3Tw6CmlxUZpxfREDQ7z3+d6CgEA\ngsTG8s9U+NERSdL+lm8VImnlzCS//XymjAAgSBw8eqrv42OnLvKdV4ZCAIAgkRYb1fdxTNRFvvPK\nMGUEAEFixfREhejCyCAtJkrLpyf69ecbLYT3339fzz33XJ9tZ8+e1cKFC7V27VpDqQDAngaHhfr1\nmMGPGS2Eu+++W3fffXfv43379mnlypV65JFHDKYCAGeyzTGE06dPa9WqVVqzZg0rmAKAAbYphFde\neUWJiYmaNm2a6SgA4Ei2OKh8+vRplZSUaNu2baajAIBj2WKEUFFRoRtuuEFpaWmmowCAY9miEKqq\nqjRr1izTMQDA0WxRCIcOHWJ0AACGGS8Ej8ej//znPxe9xycAoH8YP6gcFhamw4cPm44BAI5nfIQA\nALAHCgEAIIlCAABYKAQAgCQKAQBgoRAAAJIoBACAhUIAAEiiEAAAFgoBACCJQgAAWCgEAIAkCgEA\nYKEQAACSKAQAgIVCAABIohAAABajhXDixAktW7ZM48aN05133qni4mKTcQDA0YwVgtfrVV5enm68\n8UYdOHBA27dvV0FBgWpra01FAgBHM3ZP5UOHDumbb77R448/rrCwMI0ePVqlpaUaPny4qUgA4GjG\nRggNDQ0aPXq0NmzYoMmTJ2vGjBk6dOgQhQAAhhgbIXR0dOjAgQOaMGGCqqqqVF9fryVLligmJkYZ\nGRmmYgGAYxkbIYSHh2vYsGFatmyZwsPDNW7cOM2YMUOVlZWmIgGAoxkrhPj4eHk8Hnk8nt5tHo9H\nXq/XVCQAcDRjhTB58mS5XC4VFBTo/Pnzqq2tVXl5uWbOnGkqEgA4mrFjCC6XSzt37tTf/vY3TZo0\nSUOHDtUzzzyjtLQ0U5EAwNGMFYIkxcXFafv27SYjAAAsLF0BAJBEIQAALBQCAEAShQAAsFAIAABJ\nFAIAwEIhAAAkUQgAAAuFAACQRCEAACwUAgBAEoUAALBQCAAASRQCAMBCIQAAJFEIAAALhQAAkEQh\nAAAsRgth+/btSk1N1dixY3u/ampqTEYCAMcyek/lxsZGLV++XA8++KDJGAAAGR4hNDU1KTk52WQE\nAIDFWCGcPXtWra2tKi4u1uTJkzVr1iyVlZWZigMAjmdsyqitrU3p6elavHixNm/erLq6OrndbkVH\nRyszM9NULABwLGOFEBMTo5KSkt7HGRkZmjdvniorKykEADDA2JRRQ0ODtm7d2mdbV1eXwsPDDSUC\nAGczVgiRkZEqKCjQ3r171dPTo/3792vPnj2aP3++qUgA4GjGpozi4+P1wgsvaNOmTVq1apVGjBih\n559/XikpKaYiAYCjGb0OISsrS1lZWSYjAAAsLF0BAJBEIQAALBQCAEAShQAAsFAIAABJFAIAwEIh\nAAAkUQgAAAuFAACQRCEAACwUAgBAEoUAALBQCAAASRQCAMBCIQAAJFEIAAALhQAAkGSTQmhra9PE\niRNVVVVlOgoAOJYtCuHpp5/WqVOnTMcAAEczXghvvvmmIiIidP3115uOAgCOZrQQWltbtWPHDq1Z\ns8ZkDACApEGmXvj8+fNauXKlnn76aUVFRZmKAQD9ptvTo43ln+ng0VNKi43SiumJGhxmfKKml7FC\n2LJli5KTk5WZmWkqAgD0q43ln6nwoyOSpP0t3ypE0sqZSWZD/Q9j1fTBBx9oz549ysjIUEZGho4f\nP64VK1Zo69atpiIBQEAdPNr35JmDx+x1Mo2xEcLevXv7PM7KytLq1av129/+1lAiAAistNgo7W/5\n9r+PY+w1XW6sEADAaVZMT1SILowM0mKitHx6oulIfdimED788EPTEQAgoAaHhdrqmMGPXfExhAce\neECS9NJLL6mqqkrffPON30IBAPrfFY8QCgsLJUler1elpaVqaGhQSEiIUlJSlJqaqr/85S9+CwkA\nCLwrLoShQ4dKkh599NHebV9//bXq6+vV0NDgezIAQL/y6zGEESNGaMSIEcrOzvbnjwUA9AP7XCIH\nADDKpxGCx+NRS0uLmpubdfjwYTU3N2vbtm3+ygYA6Ee/uhDa29vV3NzcZ+d/5MgRdXd3y+v1Kjw8\nXImJ9jqnFgD8ye5rEfnqkoXwzjvvaPPmzX1OK42IiNCYMWOUk5OjpKQkJScnKyEhQWFhYQENCwAm\n2X0tIl9dshDy8/N1zTXX6LHHHlN8fLx27NihgwcP6vbbb5fb7aYEADiG3dci8tUlC6G9vV35+fma\nOHGiJCk7O1tvvPGG8vPzVVlZqXXr1jFVBMAR7L4Wka8uOflVUVGh2267rc+2nJwcvf/++7rmmmu0\nYMECFRYWqqenJ2AhAcAOVkxPVN7UBE1K+I3ypibYbi0iX11yhDBq1KiLbn/ttdf01ltvacOGDaqo\nqNC6des0evRov4cEADuw+1pEvvL58Ph9992n3bt3a/jw4brnnnv8kQkAYIBfrlQeOXKkXnnlFf3z\nn//0x48DABjg1xNo58+f788fBwDoRwPnigoAgE8oBACAJMOF8MEHH2jWrFkaO3asZs+erYqKCpNx\nAMDRjN1Cs7W1VU899ZReffVVjRs3Tvv27dPSpUv18ccf69prrzUVCwAcy1ghxMfH65NPPtGQIUN0\n/vx5tbW1aciQIQoPDzcVCQAczVghSNKQIUN07NgxzZgxQz09PVqzZk3vndgAAP3LaCFI0vXXX69D\nhw6ppqZGeXl5iouL6103CQDQf4yfZTRo0CANHjxYEydO1O9+9ztVVlaajgQAjmSsEKqrq/WnP/2p\nz7bu7m5dffXVZgIBgMMZK4RbbrlF9fX1eu+999TT06Pq6mpVV1drzpw5piIBgKMZK4To6GgVFRWp\nuLhYGRkZevHFF/WPf/xDCQkJpiIBgKMZPaickZGhd99912QEAA4y0O+J7CvjZxkBQH8Z6PdE9hXV\nCMAxBvo9kX1FIQBwjLTYvvdAHmj3RPYVU0YAHGPF9ESF6MLIIC0masDdE9lXFAIAxxjo90T2FVNG\nAABJFAIAwEIhAAAkUQgAAAuFAACQRCEAACwUAgBAEoUAALBQCAAASRQCAMBCIQAAJFEIAAALhQAA\nkGS4EGpqarRw4UKlp6dr2rRpKi0tNRkHABzN2PLXHR0dysvL0+rVqzV79mw1NTUpNzdXsbGxmjRp\nkqlYAOBYxkYIx48fV2ZmpubOnavQ0FClpKRo/Pjxqq2tNRUJABzNWCEkJydrw4YNvY87OjpUU1Oj\npCRuXgEAJtjioPJ3330nt9utlJQUZWVlmY4DAI5kvBCOHTumRYsWadiwYSooKFBoqPFIAOBIRve+\nDQ0NuvfeezVlyhRt2bJFLpfLZBwAcDRjZxm1tbVpyZIlys3N1dKlS03FAABYjI0QysrK1N7ersLC\nQo0dO7b3a9OmTaYiAYCjGRshuN1uud1uUy8PAPgRjuACACRRCAAAC4UAAJBEIQAALBQCAEAShQAA\nsFAIAABJFAIAwEIhAAAkUQgAAIuxpSsA4HJ1e3q0sfwzHTx6SmmxUVoxPVGDw/hc6y8UAoCgsbH8\nMxV+dESStL/lW4VIWjmTuyz6C9UKIGgcPHqq7+Njpy7ynbgSFAKAoJEWG9X3cUzURb4TV4IpIwBB\nY8X0RIXowsggLSZKy6cnmo40oFAIAILG4LBQjhkEEFNGAABJNimEuro6TZkyxXQMAHA0o4Xg9XpV\nVlamP//5z+ru7jYZBQAcz2ghFBUVqbi4mHsrA4ANGC2EBQsWaNeuXbr11ltNxgAAyPBZRtddd53J\nlwcA/A9bHFQGAJhHIQAAJFEIAAALVyoD6DcsX21vtiiE8ePH68CBA6ZjAAgwlq+2N6oZQL9h+Wp7\noxAA9BuWr7Y3W0wZAXAGlq+2NwoBQL9h+Wp7Y8oIACCJQgAAWCgEAIAkCgEAYKEQAACSKAQAgIXT\nTgH8aqxFNLBRCAB+NdYiGtiodgC/GmsRDWwUAuAg3Z4erd97WIu3fqr1ew+r29NzWc9nLaKBjSkj\nwEF8nfJhLaKBjUIAHMTXKR/WIhrYmDICHIQpH/wSoyOExsZGPfvss/r8888VFxenv/71r0pLSzMZ\nCbA1X0/7ZMoHv8RYIXR1dcntdsvtdmvhwoXatWuXHn74YVVUVGjIkCGmYgEB5esO3ddjAEz54JcY\nK4RPP/1UoaGhuv/++yVJv//97/X666+rurpad911V0Be09c/Rn9clGM6A883+3xfd+ic9olAMlYI\nra2tSkhI6LMtPj5eLS0tAXtNX/8Y/XFRjukMPN/s833doafFRml/y7f/fcwxAPiRsUI4c+aMIiIi\n+mxzuVzq7OwM2Gv6+sfoj09npjPwfLPP93WHzjEABJKxs4wiIiJ+svPv7OxUZGRkwF7T1zMs/HGG\nhukMPN/s81dMT1Te1ARNSviN8qYmXPYO/YdjAP/30AStnJnEOkLwK2MjhBtvvFElJSV9trW2tmrO\nnDkBe01fP13549OZ6Qw83+zzOagLOwvxer1eEy987tw5ZWdna+nSpVq0aJF27dql/Px8VVZWXnKU\n8NVXXyk7O1uVlZUaNWpUPyUGgOB2qX2nsfFmeHi4tm3bpj179uiOO+5QSUmJCgsLAzplBAC4OKMX\npiUlJam0tNRkBACAhSNSAABJFAIAwEIhAAAkBeny1x6PR5J04sQJw0kAIHj8sM/8YR/6Y0FZCCdP\nnpQk5eTkGE4CAMHn5MmTiouL+8l2Y9ch+KKzs1P19fWKjo5WWFiY6TgAEBQ8Ho9Onjyp1NRUuVyu\nn/x7UBYCAMD/OKgMAJBEIQAALBQCAEAShQAAsFAIAABJFAIAwEIhAAAkUQiOsn37dqWmpmrs2LG9\nXzU1NaZjBYW6ujpNmTKl93FHR4ceeeQRpaena+rUqXr77bcNprO/H//+/vWvfyk5ObnPe7GoqMhg\nQnuqqanRwoULlZ6ermnTpvXeLiBQ77+gXLoCV6axsVHLly/Xgw8+aDpK0PB6vXrnnXe0bt26PlfF\nr169WpGRkdq3b5+am5v10EMPafTo0UpLSzOY1n4u9vtramrSnXfeqZdfftlgOnvr6OhQXl6eVq9e\nrdmzZ6upqUm5ubmKjY1VaWlpQN5/jBAcpKmpScnJyaZjBJWioiIVFxfL7Xb3bjt9+rQqKir06KOP\n6qqrrtKYMWM0Z84cvffeewaT2tPP/f6kCx9OkpK4t/QvOX78uDIzMzV37lyFhoYqJSVF48ePV21t\nbcDefxSCQ5w9e1atra0qLi7W5MmTNWvWLJWVlZmOZXsLFizQrl27dOutt/Zu+/LLLzVo0CDFxMT0\nbouPj1dLS4uJiLb2c78/6cKHk9raWmVlZWnq1Klav369zp07ZyilPSUnJ2vDhg29jzs6OnqneAP1\n/qMQHKKtrU3p6elavHixqqqqtHbtWq1bt07V1dWmo9naddddp5CQkD7bzpw585OFwVwulzo7O/sz\nWlD4ud+fJA0fPlxZWVnavXu3du7cqQMHDmjz5s0GEgaH7777Tm63u3eUEKj3H4XgEDExMSopKVFm\nZqbCw8OVkZGhefPmqbKy0nS0oBMREaGurq4+2zo7OxUZGWkoUfApKipSbm6uIiMjFRMTo2XLlqm8\nvNx0LFs6duyYFi1apGHDhqmgoECRkZEBe/9RCA7R0NCgrVu39tnW1dWl8PBwQ4mCV1xcnLq7u3X8\n+PHeba2trbrpppsMpgoeHR0dWr9+vb7//vvebV1dXbrqqqsMprKnhoYG3XvvvZoyZYq2bNkil8sV\n0PcfheAQkZGRKigo0N69e9XT06P9+/drz549mj9/vuloQWfo0KHKzs5Wfn6+zp49q7q6Ou3evVtz\n5841HS0oXH311SovL1dBQYG6u7v15ZdfqqioSPfcc4/paLbS1tamJUuWKDc3V08++aRCQy/srgP5\n/uO0U4eIj4/XCy+8oE2bNmnVqlUaMWKEnn/+eaWkpJiOFpTWrl2r5557TpmZmYqMjNQTTzyh2267\nzXSsoBAaGqqioiL9/e9/14QJE+RyuXTffffpj3/8o+lotlJWVqb29nYVFhaqsLCwd/sf/vCHgL3/\nuEEOAEASU0YAAAuFAACQRCEAACwUAgBAEoUAALBQCAAASRQCAMBCIQB+VldXp5tvvlm1tbWmowCX\nhQvTAD9rb2/X0aNHNWbMmN7lBoBgQCEAACQxZQT43bx58/Tkk0+ajgFcNgoB8KNz587pyJEj3KoU\nQYlCAPzo3//+t7q7u3XLLbeYjgJcNgoB8KPGxkaFhIRwA3kEJQoB8KOmpibFxsZq6NChpqMAl41C\nAPyosbGR4wcIWhQC4Cc9PT1qbm7m+AGCFoUA+MkXX3yhM2fOMEJA0KIQAD9pamqSJAoBQYsrlQEA\nkhghAAAsFAIAQBKFAACwUAgAAEkUAgDAQiEAACRRCAAAC4UAAJAk/T/L2sfj2oy6RQAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a1cd8f5d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Number of dimensions\n",
    "dim = 1\n",
    "\n",
    "# Number of input points\n",
    "n = 20\n",
    "\n",
    "# The lengthscale\n",
    "ell = .1\n",
    "\n",
    "# The variance \n",
    "variance = 1.\n",
    "\n",
    "# The covariance function\n",
    "k1 = GPy.kern.RBF(dim, lengthscale=ell, variance=variance)\n",
    "\n",
    "# Draw a random set of inputs points in [0, 1]^dim\n",
    "X = np.random.rand(n, dim)\n",
    "\n",
    "# Evaluate the covariance matrix on these points\n",
    "K = k1.K(X)\n",
    "\n",
    "# Compute the eigenvalues of this matrix\n",
    "eig_val, eig_vec = np.linalg.eigh(K)\n",
    "\n",
    "# Plot the eigenvalues (they should all be positive)\n",
    "print '> plotting eigenvalues of K'\n",
    "print '> they must all be positive'\n",
    "fig, ax = plt.subplots()\n",
    "ax.plot(np.arange(1, n+1), eig_val, '.')\n",
    "ax.set_xlabel('$i$', fontsize=16)\n",
    "ax.set_ylabel('$\\lambda_i$', fontsize=16)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEPCAYAAABcA4N7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xlc1HX+B/DX3BenHII34hGCKYYgSVbitvszr9RcU7Pa\nLJCy7bf1sDU2b1fLX3ZIWNG6m2aHkZulVu6mq2V5kJYLiKWMgiHKORzDDMzM9/fHzPcLA3MBM98B\n5v18PFyd73yZecvmvHl/3p9DwDAMA0IIIcQOobcDIIQQ0rNRoiCEEOIQJQpCCCEOUaIghBDiECUK\nQgghDom9HYA76XQ65OfnIywsDCKRyNvhEEJIr2A0GlFRUYG4uDjI5fIOz/OaKA4dOoTt27ejvLwc\nAwYMwP/+7/9i2rRpHe47cOAAXnnlFVRVVSEpKQmbNm1CaGio09fPz8/H4sWLPRE6IYT0eXv27EFC\nQkKH6wK+1lGo1Wrcd9992LlzJyZMmIDvvvsOjz/+OI4fP45+/fpx9xUVFWHx4sXYuXMnRo8ejQ0b\nNuDmzZvIyclx+h5Xr17FPffcgz179iAiIsKTfx1CCOkzysvLsXjxYhw+fBhDhw7t8DxvFUVUVBRO\nnDgBlUoFg8GAyspKqFQqSKVSq/s+//xzpKamYty4cQCAZ599FsnJyaisrHRaVbDDTRERERg0aJBn\n/iKEENJH2Ruy53XoSaVSobS0FL/97W9hMpmwdu1a+Pn5Wd1TXFyM+Ph47nFwcDACAwOhVqtdGn4i\nhBDiXrw3syMjI/HTTz8hLy8PGRkZGDp0KJKTk7nnm5qaOjRTFAoFmpqa+A6VEEIIvDA9ViwWQyKR\nIDk5Gffccw++/vprq+flcjl0Op3VtaamJiiVSj7DJIQQYsFbojh27Bgefvhhq2stLS3w9/e3uhYd\nHQ21Ws09rq6uhkajQXR0NB9hEkIIaYe3RDFmzBjk5+fj008/hclkwrFjx3Ds2DHMmDHD6r4ZM2bg\n8OHDyMvLg16vx7Zt2zBlyhQEBwfzFSohhJA2eEsUYWFhePPNN7Fr1y4kJCTgtddewxtvvIHo6Gis\nXr0aq1evBgDExMRgw4YNyMzMRHJyMm7evInNmzfzFSYhhJB2eFtHwYdr164hNTUVX3/9NU2PJcSO\nnAP/xeVrtfjr4ykQCgXeDof0AM4+O/vUFh6EEMcMRhPe/uw8AKDkZj2GRQR4OSLSG9CmgIT4kKs3\n6rg/G4wmL0ZCehNKFIT4kItXq7k/65sNXoyE9CaUKAjxIRdLa7g/N+kpURDXUKIgxIcUlbRWFE1U\nURAXUaIgxEeYTAwulrRWFLpmoxejIb0JJQpCfERZZQMadS3cYxp6Iq6iREGIj2g77AQAOkoUnMuX\nL2P06NH49ttvvR1Kj0SJghAfcbPWegdmHfUoOPn5+QCAuLg4L0fSM1GiIMRHtJ8O20Q9Ck5BQQEG\nDRqEoKAgb4fSI1GiIMRHsBWESi62ekyAwsJCq2qioaEBTz31FCZPnozTp097MbKegRIFIT6CneUU\n5Gc+GIya2WYMw1gliosXL2LevHm4ceMG9u3bh8TERC9H6H201xMhPoKtIIL9Zfi1ssFtFcXTrx/F\nifwyt7xWV02OG4BXn7q7S1+rVqvR2NiIuLg4fPrpp1i7di3mzJmD559/HlKp1M2R9k6UKAjxEWwF\nEeRvrih0eupRAOZhJwD46KOPcPToUaxbtw5z5syxuueNN97AZ599hqtXryIrKwvTpk3zRqheQ4mC\nEB/BDj0F+8kAuG9ldld/ku8pCgoKEBAQgMOHD2PJkiUdkgQATJ48GbNmzcLzzz/vhQi9jxIFIT6C\nHWoKYhMF9SgAmKfGpqSkICEhARs3bkRCQgLuueceq3vGjx/vpeh6BmpmE+IjuIqCHXqiWU9gGAYX\nLlxATEwMFi9ejMWLF2PlypU4f/68t0PrUShREOIjuIrCX2b12JeVlJSgvr4et9xyCwBg1apVSEpK\nQkZGBsrKvNug70koURDiI1p7FDQ9llVQUAAAiImJAQCIRCJs27YNISEhSEtLQ0NDgzfD6zGoR0GI\nj2g7Pdb8mGY9TZ8+HdOnT7e6plKpsH//fi9F1DNRRUGIj2gdeqKKorO2b9+OKVOm4Ny5c8jMzMSU\nKVNQXl7u7bB4QxUFIT6CrSACVeaKQt9ihMnEQCgUeDOsXmHFihVYsWKFt8PwGqooCPEBDMNwFYVC\nJoJMIgJgThaEOEOJghAf0GwwgWEAmUQEkVAIudQ8mEDDT8QVvA495eXl4cUXX0RxcTGCg4OxbNky\nLFy4sMN9aWlp+P777yESibhr586d4zNUQvoUNiHIpeZ/UwqZCJpGmiJLXMNbotBoNMjIyMALL7yA\ne++9FxcuXMAjjzyCIUOG4Pbbb7e6t7CwEHv27MHYsWP5Co+QPo1NCDJLJaGQWSoKShTEBbwNPZWV\nleHOO+/EzJkzIRQKERsbi6SkJJw9e9bqvqqqKlRXV2PUqFF8hUZIn8c2stmKgh16ouNQiSt4SxQx\nMTHYunUr91ij0SAvL49bEckqLCyESqVCWloaJk2ahIULF9KwEyHdxJ5uxyYILlHQWgriAq80s+vr\n65Geno7Y2FhMnTrV6jm9Xo/x48cjMzMTx48fx6xZs/DYY4+hoqLCG6ES0idwM57aDz1RRUFcwHui\nKC0txcKFCxEYGIisrCwIhdYhTJs2DW+//TZGjhwJqVSKRYsWITIyEqdOneI7VEL6jCZ9+6En8+/U\noyCu4DVRFBQUYMGCBUhJSUF2djbkcnmHe7788kscOnTI6pper4dMJuMrTEL6HF27oSe2oqBZT8QV\nvM16qqysxLJly/DII4/g8ccft3ufVqvFyy+/jFGjRmHo0KF49913odPpMHnyZL5CJaTP4RKFzLqZ\nTUNPxBW8JYrc3FxUV1djx44d2LFjB3d96dKlqKmpAQCsX78ec+fORUVFBZYtW4ba2lqMGTMGOTk5\nUCqVfIVKSJ/TOuupXTObEgVxAW+JIj09Henp6S7dm5aWhrS0NA9HRIjvsD/0RLOeiHO0hQchPqCp\nud3KbGpmW7l8+TJGjx6Nb7/91tuh9EiUKAjxAfaGnqhHYZafnw8AiIuL83IkPRMlCkJ8QPuhJzkN\nPVkpKCjAoEGDEBQU5O1QeiRKFIT4AH27LTy4HgVVFADMO0K0rSYaGhrw1FNPYfLkyTh9+rQXI+sZ\nKFEQ4gM6VBSWhEHrKMxndbRNFBcvXsS8efNw48YN7Nu3D4mJiV6O0PvohDtCfADbi2ArCXYrD3c0\ns59+/ShO5Jd1+3W6Y3LcALz61N1d+lq1Wo3GxkbExcXh008/xdq1azFnzhw8//zzkEqlbo60d6JE\nQYgPaL97LO311KqwsBAA8NFHH+Ho0aNYt24d5syZwz2v0WiwcuVKXLlyBTKZDKGhoVizZg2GDh3q\nrZB5R4mCEB/QfuhJ5sbdY7v6k3xPUVBQgICAABw+fBhLliyxShIAIBAI8NBDD3Hn5uzatQt/+ctf\nsHv3bm+E6xXUoyDEB9irKKhHYZ4am5KSgszMTOzevRuHDx+2ej4gIMDqcLX4+Hj8+uuvfIfpVZQo\nCPEB7U+443aP9fGhJ4ZhcOHCBcTExGDx4sVYvHgxVq5cifPnz9v9mnfffbfD8Qh9HSUKQnyAzk4z\n29cripKSEtTX13MHqK1atQpJSUnIyMhAWVnHBn1WVhauXbuGZ555hu9QvYoSBSE+wO5RqM1GmEyM\n1+LytoKCAgDmEzgBQCQSYdu2bQgJCUFaWhoaGhq4e7Ozs3Hs2DHk5ORAoVB4JV5voWY2IT6gfTNb\nKBRAJhFB32KEvsXIVRq+Zvr06Zg+fbrVNZVKhf3791tdy8rKwrFjx7Bz5074+/vzGWKP4Jv/dRDi\nQxiG4SoKmUTEXZdLxdC3GKFrNvhsonDFL7/8gu3bt2PIkCFYsmQJAHPlsW/fPi9Hxh/6r4OQPq7F\nYIKJYSARCyEWtY42K2QiaBrNDe1g3/sh2WUjR47ExYsXvR2GV1GPgpA+jh12YhvYLLkbV2eTvo0S\nBSF9XFO7RjaLDi8irqJEQUgf176RzaLjUImrKFEQ0se1X2zHoh1kiasoURDSx7VfQ8GijQGJqyhR\nENLHsUNL7YeeuERBFQVxghIFIX0cN+tJZqdHQc1s4gQlCkL6OHtDT9TMJq6iREFIH2dv1hP1KIir\neE0UeXl5uP/++3Hbbbdh2rRp+PDDD23ed+DAAaSmpmL8+PFIS0tDZWUln2ES0qfYryho1hNxDW+J\nQqPRICMjA0uXLsWZM2fw2muvYdu2bfjuu++s7isqKsKaNWuwbds2nDx5EqGhoVi1ahVfYRLS59it\nKGhlNnERb4mirKwMd955J2bOnAmhUIjY2FgkJSXh7NmzVvd9/vnnSE1Nxbhx4yCXy/Hss8/im2++\noaqCkC5ih5bkdprZTXpqZhPHeEsUMTEx2Lp1K/dYo9EgLy+POzCEVVxcjBEjRnCPg4ODERgYCLVa\nzVeohPQpdoeeZObHeqoocPnyZYwePRrffvutt0MB0PPi8Uozu76+Hunp6YiNje1wpGBTUxPkcrnV\nNYVCgaamJj5DJKTPoGa2c/n5+QCAuLg4L0di1tPi4T1RlJaWYuHChQgMDERWVhaEQusQ5HI5dDqd\n1bWmpiYolUo+wySkz3A6PZbWUaCgoACDBg1CUFCQ21976tSp2L59e4+Jpyt4TRQFBQVYsGABUlJS\nkJ2d3aFyAIDo6GirYabq6mpoNBpER0fzGSohfYaemtlOFRYWWv303tDQgKeeegqTJ0/G6dOnfT4e\n3g4uqqysxLJly/DII4/g8ccft3vfjBkzsGTJEsybNw9jx47Ftm3bMGXKFAQHB/MVKiF9Cju01OE8\nCjcNPT39+lGcyC/r1mt01+S4AXj1qbu79LUMw6CwsBDLly8HAFy8eBFPPfUUgoKCsG/fPvTv39+d\nofa6eIBOJgq1Wo2qqioIhUKEhYVh8ODBLn9tbm4uqqursWPHDuzYsYO7vnTpUtTU1AAA1q9fj5iY\nGGzYsAGZmZmoqKhAQkICNm/e3JkwCSFtOFtH4evNbLVajcbGRsTFxeHTTz/F2rVrMWfOHDz//POQ\nSqWdei2GYWA0dhzKM5lMMBhav88CgQAikajDfe6Ox12cJooffvgB7733Hk6cOIG6ujruukAgQGBg\nIO644w488MADmDBhgsPXSU9PR3p6uktB2TrwnBDSNU6b2d1MFF39Sb6nKCwsBAB89NFHOHr0KNat\nW4c5c+ZY3fPGG2/gs88+w9WrV5GVlYVp06bZfK3Tp09j6dKlHa5nZ2cjOzube5yYmIjdu3d3KR6N\nRoOVK1fiypUrkMlkCA0NxZo1azB06NDO/cU7wW6iuHr1KlavXo3y8nKkpqbitddeQ3R0NIKCgmAy\nmVBTU4OioiLk5eXhmWeewaBBg7B+/XpERUV5LFhCSOe1Jop224xLadYTYO6dBgQE4PDhw1iyZEmH\nJAEAkydPxqxZs/D88887fK3Y2Fjk5uZaXVu+fDnuvvtuLFiwgLumUqm6HI9AIMBDDz2E22+/HQCw\na9cu/OUvf7GbeNzBbqJ47rnn8MQTT+COO+6w+XxkZCQiIyNx991349lnn8WRI0fw5z//GR999JHH\ngiWkt9r9VSH+W1wJlUKC5bPHITyYv1l8rUNP1v/cZdwWHkYwDAOBQMBbTD1Jfn4+UlJSkJCQgI0b\nNyIhIQH33HOP1T3jx4936bX8/PwwduxYq2tSqRTh4eEdrnc1noCAAC5JAEB8fDz+8Y9/uPTaXWU3\nUdjbh8kWgUCA1NRUpKamuiUoQvoSTaMer39yjns8LCIAD/0ulrf3Z4eW2q/MFgmFkElE0LcYoW82\ndnjeFzAMgwsXLuDxxx/H4sWLoVarsXLlSkRERODWW2/tFfG8++67HdajuZvd6bHNzc1d+kUIsVbX\naP3vQtOg5/X97U2PbXvNV6fIlpSUoL6+ntshYtWqVUhKSkJGRgbKyvifydXZeLKysnDt2jU888wz\nHo3L7o8Q48aN6/SLCQQCrhFDCDFraGpu97iFt/dmGKZ16EnScZaNXCqCptF3d5AtKCgAYN5iCABE\nIhG2bduGRYsWIS0tDR988AH8/Px6ZDzZ2dk4duwYdu7cCYVC4dG47CYKhmGwfft2BAYGuvRCtbW1\n+OMf/+i2wAjpK9onBj4ThcFogtHEQCwSQizuOIDQuo2Hb67OtjXDUqVSYf/+/W57jyNHjrg9nqys\nLC5J+Pv7uyVOR+wmigkTJiApKQkBAQEuvZBGo0F8fLzbAiOkr2ATg0gogNHEdKgwPMneGgoWnZvt\nmu3bt+Pjjz9GdXU1fv75Z6xfvx579+5FREQE77H88ssv2L59O4YMGYIlS5YAMFce+/bt89h72k0U\nbBfd1b6DQqHweOedkN6ITQwR/VT4tbKB14qC22LcRn/CfN0y88nHp8g6s2LFCqxYscLbYQAARo4c\niYsXL/L6ntSjIMTD6rXmxBAZwn+iYHsPCjszmlo3BqREQeyjHgUhHtbIVhQh5kVWPXHoiXaQJY5Q\nj4IQD2MriIh+KqvHfLC3fQdLTquziQvsJor3338fANDY2IiCggKUl5ejubkZCoUC/fv3x5gxY6zO\niAgMDOS+hhDSik0MYUEKCATmD2WD0QSxyPO7/LOVgoya2aQb7CYKvV6PzZs345NPPoHJZEJQUBCk\nUimam5tRW1sLsViM+++/HytXrvTajoaE9AbsUFOAUgqVXIKGphY06loQqJJ5/L2dNbNbt/GgREHs\ns5soNm7ciB9++AE7d+5EfHw8xOLWWw0GA86ePYu1a9di48aNWL9+PS/BEtIbsc1sP4UUfgpzomho\n4idRsKuy259FwWKv06wn4ojd2veLL77Ali1bMHHiRKskAQBisRiJiYnYvHkzvvrqK48HSUhvxlYU\nfgoJ/JVSq2ue5vI6CkoUxAG7iUIul0Or1Tr84rq6ug5nXhNCrDU2sRWFBH4KCQCgQctPQ9vVZjbN\neiKO2B16WrhwIZ599lk8+eSTmDhxIsLDwyGTydDc3IybN2/ihx9+wKuvvmq1xzohpKOGptahJ5WC\n74rCnCjsNrN9fFNA4hq7ieLJJ59EUFAQ3nnnHaxdu9Zqr3qGYTBw4EA89thjePjhh/mIk5Beixt6\nUrapKHiaIqu1DCkp5RKbz8tltDKbOOdwA/olS5ZgyZIlqKysxI0bN9DU1AS5XI6IiAiEhobyFSMh\nvZa+xYhmg3kqrEwi4j1RsL0HJa3MJt3g0kkloaGhlBgI6YK2jWyBQAA/noee2IrC3hYevr57LHFN\ntzrR1dXVdKodIQ40tGlkt/2dt6Ennfl97FUU1KMgruhWopDL5bjvvvvcFQshfU7bRjYA+HMVRQ/p\nUdCCO+KCbiUKpVKJJ5980l2xENLnNGhbG9lAm4pCy8/QU5POSY9CRgvuiHO0CIIQD2pfUah4n/Vk\nfh+FnYqCdo8lrvBKojh//jxSUlLsPp+WloZbb70V8fHx3C9CeqO2zWwA8ON5ZbbWWUVBPQriAruz\nnv70pz9ZrZ1w5OWXX3bpPoZh8Mknn2DLli0QiWwvAAKAwsJC7NmzB2PHjnXpdQnpqbzdzG6yVBSu\n9CgYhnH53zzxLXYriujoaHzxxRe4cOECpFKpw1+uevPNN7Fr1y6kp6fbvaeqqgrV1dUYNWpU5/4m\nhPRA7Yee/LzVzLZTUYiE5vUdDGNe80GILXYriieeeAKRkZHYsGEDtm/fjujo6G6/2bx585Ceno7T\np0/bvaewsBAqlQppaWkoKirCsGHD8Nxzz9HwE+mV2CEmf0sl4c9VFJ4femIYhht6srd7LGCuKvQt\nRuj0Brt7QhHf5rBHMXfuXPz2t7912zbi4eHhTktbvV6P8ePHIzMzE8ePH8esWbPw2GOPoaKiwi0x\nEMKnem7Wk7mSUMjEEAkF0DUb0ezhn+D1LUaYGAZSsRBisf1/6tSnIM44/fFh3bp1qKqq4iMWAMC0\nadMwbdo07vGiRYvwwQcf4NSpU5gxYwZvcRDiDpoGPQBwZ08IBAIEqmSortehrrEZoUEKj713k5M1\nFCzaQZY4Y/fHjEuXLsFkMkEmk2HAgAFOX8hoNOLSpUvdDujLL7/EoUOHrK7p9XrIZJ4/5IUQd9M0\nmiuKIL/WXl6g5c+1jXqPvnejk1XZLDqTgjhjN1HMnDkTtbW1Lr9QTU0NZs6c2e2AtFotNm3ahEuX\nLqGlpQXvvPMOdDodJk+e3O3XJoRv7SuKtn9mn/OUJif7PLEUlh1kKVEQe+z+F8QwDFavXu3yT/J6\nfdf/o1+9ejUAYP369Zg7dy4qKiqwbNky1NbWYsyYMcjJyYFSqezy6xPiLRpL1RDoZyNReLiicLZ9\nB0tGO8gSJ+wmis7u4SSVSjFnzhyX7k1KSsKpU6e4x+2b5WlpaUhLS+vU+xPS0xhNJtRZmtnsEahA\na9Jgh6U8xdn2HSwF9SiIE3b/C3r00UcRFRXlcGFcW0ajEWq12m2BEdLb1WtbwDDmJCEWtY7yBqrM\nScPTQ0/szrH2tu9gUY+COOOwR6HRaFx+IXf1KAjpK7hhJ5X1otTWioKnoScnFQXtIEuccVuPQqfT\nuS0oQvoCW43sto89XlFw23c4SRQyWkdBHPNKj4IQX2Crkd32scd7FFxF4WToiXoUxAm7iWLz5s18\nxkFIn1PbYGfoyfK4tsGzVTjXo3A69ERnUhDH6DwKQjxE02CuGOxWFA2erShc7VFQM5s4Q4mCEA9p\nbWbb6VF4uJnt6hYeClpHQZygREGIh3DN7PYVhWXoqa6xGSYT47H3d3ULD3bWEzWziT2UKAjxEHvT\nYyViEVRyMUwM49HtxtkFdwons57Y59n7CWmvW4nCZDKhrKyM191lCektuB6FquMU89bhJ88lCq2L\ns574PkyJ9D7dShTV1dWYOnUqXnrpJTz++OMwmUzuiouQXo+tKIL8bCQKP8+vpWjtUTiuKPx4PEyJ\n9E7dOs4qMDAQu3btQmJiIkpKSiAU0kgWISx7zey21zzZ0G6dHuu4ovC3VBT1VFEQO+x+sjOM8yab\nRCLBwIEDAQBDhgxxX1SE9HIMw9htZpuveX6/J1enx7Kn7zVoqaIgttlNFA888ABKS0sdfvGePXto\nfydCbNA1G9FsMEEqFnKzitpiK4paD/YomrgtPBxXFEqZGAKBObEYjDR8TDqymyhCQ0Mxe/Zs7N27\nt8Nzv/76Kx566CFs3Lix01t9EOIL2lYTts6JD/LwxoAMw3AVBXswkT1CoYBraLNTaglpy26iyMrK\nwp///Gds3rwZ6enpqK6uBgC8//77mDlzJsrLy7F792688MILvAVLSG9RXW/enqOfv9zm88GW6zV1\nntnGo7HJvMW5UiaGyIXeob+loV1Pw0/EBoeDlwsWLEBiYiIyMzNx7733IioqCufPn8eDDz6Ip59+\nms6xJsSOqromAEBIoMLm8+z1Kg8lClsHJjnip5QCVY1opIY2scHpjxrDhg3D9OnTUV9fj7NnzyI+\nPh6PPvooJQlCHKjSmBNASIDtioK9XqVp8sj7s5VBgMrFREEVBXHAYaK4cuUKFi1ahC1btiAjIwP/\n/Oc/UV9fj3vvvRcHDhzgK0ZCeh3XKwrPJIrOVhQ0RZY4YjdRvPPOO5gzZw70ej0++eQTZGRkICYm\nBh9//DEWLFiA5557DitWrKBV2YTY4LSisPQoqup0Lk1F76x6y2yqgM4MPYEW3RHb7CaK119/HcuX\nL8fevXsxatQo7rpEIsEzzzyD3bt3o6ioCDNmzOAlUEJ6E2cVhVwmhkouQYvB5JHhnk73KLihJ6oo\nSEd2m9n//Oc/ER0dbfcLJ0yYgP379+PFF1/0SGCE9GZsRWFv1hNgrjYadS2oqtMhwMbq7e5gk4+/\niz0Kf1p0RxywW1HU1dU5/WKlUol169Zxj8+ePeueqAjp5djpsfYqCvNz5iRS7YGZT2xF4erQEzc9\nloaeiA12K4rs7GxIJBIsXboUiYmJdvdxMplMOH78OPbs2QOTyYS//e1vHguWkN6Cnc0UaqdHAQAh\nAQqre92pvtNDT7SDLLHPbqLIycnBoUOHsHHjRlRUVCApKQkjRoxAcHAwTCYTampqUFRUhB9//BFh\nYWFIT0/Hvffe69Kbnj9/HhkZGfj2229tPn/gwAG88sorqKqqQlJSEjZt2oTQ0NCu/Q0J4ZlW1wKt\n3gCZRASVwv72GWy1UemJiqLTzWzLDrI09ERscLjgbvr06Zg+fTq+//57HD9+HGfOnEF1dTWEQiFC\nQ0MRGxuLZcuWISEhwaU3YxgGn3zyCbZs2QKRyPa2AkVFRVizZg127tyJ0aNHY8OGDVi1ahVycnI6\n/7cjxAvYoaSQALnN7TtY3FoKD0yR7WxFwd5HzWxii0vbjCcnJyM5Odnmc0aj0eU3e/PNN/HFF18g\nPT3d7gf/559/jtTUVIwbNw4A8OyzzyI5ORmVlZVUVZBewdmMJ1br0JMHexQuNsnpTAriiEsHSNja\nGBAAzp0716lNAefNm4f9+/dj7Nixdu8pLi7GiBEjuMfBwcEIDAyEWq12+X0I8SZnayhYbDO7R1QU\ntOCOOOBSRfHXv/4VTU1NeOihhwCYZ0Rt3boVubm5SE1NdfnNwsPDnd7T1NQEudz6H5hCoUBTk2dW\nsBLiblXshoBtEkVeUTkultZAKZfgfxKHQS4Tc897ZNZTI3sMaycX3FGPgtjgUqJ45513kJGRAa1W\ni8jISLz44osICAjAW2+9hSlTprg1ILlcDp3O+h9OU1MTlEqlW9+HEE9hZzGxQ0sN2maseO0od9aD\nvtmAham3eGzWE8Mw3BASWyk40zr01AKGYRz2VojvcSlRJCQkYNeuXXjsscdQVVWFp59+Go888ggk\nEscHonRFdHS01TBTdXU1NBqNw8V/hPQk7I6w7NDStYoGqwOBrpSb1yixi/Fq6vUwmkwubQfuikad\nAUYTA6VMDLHYtdcUi4RQyMRo0hug1RugcnLYEfEtdhOFramrGRkZ2Lx5M65du4bTp09ze9SkpKS4\nLaAZM2aPDcJLAAAdk0lEQVRgyZIlmDdvHsaOHYtt27ZhypQpCA4Odtt7EOJJFTVaAECopZldVtVg\n9XxZpfmxWCxEP385qut1qNLoEB7snqq5znIYkqv9CZa/QoImvQH12mZKFMSK3USxbNkyu1+0d+9e\nrsEtEAhw4cKFbgWxevVqAMD69esRExODDRs2IDMzExUVFUhISMDmzZu79fqE8Km8uhEAENFPBQC4\nXmV+PGFUOM7+fJN7DAD9+ylRXa/DjRqt2xJFZ7cYZ/kppbhZ20SL7kgHdhNFUVGRx940KSkJp06d\n4h6vX7/e6nl2/QYhvVF5tbmi6N8hUfTnEgXbB+jfT4ULV6tRXt2IscPdM/27sxsCsth+Rp2Hjmcl\nvZd7BkUJIQDMjetGXQvkUhE344hNFCMHBiFQJYW+xYiaevOHcUQ/cxXBViHu0NktxlmenIVFejdK\nFIS4UdthJ3bm0HVLjyIyVIXIED+ra+zw1A1LFeIOXa0oKFEQeyhREOJG7LATmwAYhkFZpTl5RIb4\nISLEfL2syrqPUV7lxoqik1uMszy5pQjp3ShREOJGN2rMH/j9LUNK9ZahKKVMjECVFAMsieJ6JVtR\nKC1f5/6KorNDT9y6DqooSDuUKAhxo/YVBdufiAwxD0VFtqso2Ia3O3sU7AK+fgGO95pqr19A6/Gs\nhLRFiYIQN2o/NZZNCJGh5t5Ea4/CfL2fvxwSsRC1DXro9Aa3xFBpSRRhTjYlbI/dxJB6FKQ9ShSE\nuNGNauuhJ3aIKdKSOAaEspWG+bpQKEB4kGXmk5uGn9hEEdrJRMFVFB44SIn0bpQoCHGj9kNPZW2G\nnsy/myuKsspGbmeD1plP7hl+qqi1JIqgTlYUli1Fqut1XGyEAJQoCHEbo8mEm5aqgF1lzc4gCrN8\naPspJFDIxNC3GNGoMw81RYS4by1Fi8GI2gY9REIBgv1dO4uCJZeJoZKL0WIwcTOnCAEoURDiNpUa\nHYwmBv0C5JBJzCc41loW1gX7t245zv651rIdeWtDu/tDT23PwujKJoNsA5z6FKQtShSEuElZu34E\nYB7GAdolCj+Z1XPs/b9W1Hc7hgqNa6fr2RNCM5+IDZQoCHGTkhvm7cOH9PfnrtWwhxi1TRQBcqvn\n2PtLb7ohUdSaq5KwoK5tMNiPFt0RGyhREOImJZYP+sH9AwCYexaaBvNYf5Bfa7+ArSjY/Z6GhJvv\nd0ei4KbGdrKRzQqhoSdiAyUKQtyk5Ib5g35ouLlCqGtsholhEKCUWh0g1L6iCAmUQykTQ9PYjNqG\n7u3cWlnbtamxLFp0R2yhREGIm5Rahp4GW4aSamw0sgEg2E9u9bxAIMDgcPcMP1V0cQ0FixbdEVso\nURDiBiYTg2sV5mb2YMtQUg3XyLaeptrP37qZDQBDLMNVbJ+jq9iKoqtDT2wvhRbdkbYoURDiBjdr\ntNC3GNEvQA4/hfkYUVsznoCOQ09AaxVS0s2KgluV3cVEER5s/jp37j1Fej9KFIS4wVVLJTDU1oyn\nAMdDT0BrX6O7FUVFN3sUg8LMcVyraKDV2YRDiYIQN2B7C+ywE9CaCNrOeALsVRSWmU83ul5RNLcY\noWm0rMr2kzv/Ahv8lVIEqmTQtxi56oQQShSEuAGbKJytoQDaTo9t3VOp7VqKrv4k/6tlwV//fioI\nhYIuvQYADA4370fF9lwIoURBiBtcLbfMeArvmCiC2w09SSUiqOQSGE0Mt6dSoEqGQJUMWr2BGz7q\nagxth7+6YpCbZmCRvoMSBSFu8MuvtQCA6IFB3DUuUfh13JwvmJv51NqnGD4gEABwyfJancX1SSIC\nnNzpGNenoERBLChRENJNmkY9btZoIZeKMCjMj7vOraMI6NgvaL8xIACMGhwMAPjlWk2X4mitKLqX\nKAaH0dATsUaJgpBu+qXU/ME+YmCQ1Y6t9noUba+1XUsxcpC5GulyonBXRRHOznyiioKY8ZooCgsL\nMX/+fIwfPx6zZ8/Gjz/+aPO+tLQ03HrrrYiPj+d+EdJT/XzNPFQ0clAwd81gNEHT2AyBAAhQSTt8\nTZBl6Km2zdAT+/U/l3Zx6KncsoVINysKduipO4110rfwlij0ej3S09Mxd+5cnDlzBg8++CCWL1+O\nxsaOC3sKCwuxZ88enDt3jvtFSE/FVgAjB7cmCnbPpiA/mc1zIbgV0G0qiuEDAiEUCFByow76FmOn\nYqht0EPTqIdSJu7yqmxWsL8MKrkYDU0t0DTSAUaEx0Rx8uRJCIVCLFq0CBKJBPPnz0doaCiOHTtm\ndV9VVRWqq6sxatQovkIjpFvYoadRg1ob2exeSe1XZbPYRXg1bfZUkkvFGBrhD6OJQXFZ56qKq9wW\n5wEQCLo+NRYw7z3VtqoghLdEoVarER0dbXUtKioKxcXFVtcKCwuhUqmQlpaGSZMmYeHChVRRkB7L\nYDCh+LoGADCizdATe54Du213e/bOfWgdfupcn4JrZEd0b2osa1ikefjqchdnYJG+hbdEodVqoVBY\n/6ORy+XQ6ax3qdTr9Rg/fjwyMzNx/PhxzJo1C4899hgqKir4CpUQl10p16DFYMLAUD+o5BLuOnck\naaDtioJNIOx9LDZR/HKtixVFePf6E6zRQ/oBAIpKqt3yeqR34y1RKBSKDklBp9NBqbQ+iWvatGl4\n++23MXLkSEilUixatAiRkZE4deoUX6ES4rJ8dRUA4BbLByur2klFwW3nXW/9byJmqPl18osrOxXH\nzyXmCqTtOo7uGDM0BABw4SolCsJjohg+fDjUarXVNbVajREjRlhd+/LLL3Ho0CGra3q9HjJZx0VL\nhHjbT5fNle6tI8KsrrMH/9irKEIDbG/nHRsVCqFAgKKSauj0BpdiMJkYFFwxJ6y44aGuB+/A6DZr\nOloMnWusk76Ht0SRnJyM5uZm7N69Gy0tLcjNzUVlZSVSUlKs7tNqtdi0aRMuXbqElpYWvPPOO9Dp\ndJg8eTJfoRLisvOXzIliXIdEYako7DSzVQoJpGIhtHoDtLoW7rqfQoLogUEwtvnwd6bkRh3qtc0I\nC1Kgf3DXzspuz08pxZBwf7QYTCgu07jlNUnvxVuikEqlyMnJwcGDB5GYmIj33nsPO3bsgFKpxOrV\nq7F69WoAwNy5c7F06VIsW7YMEydOxJEjR5CTk9NhiIoQb6up16HkZj3kUhFGt2lkA217FLaHngQC\ngd3T5MaNMFcF5y+71pdjh79io9xTTbBihpmHwQpp+Mnnifl8s1tuuQUffvhhh+vr16+3epyWloa0\ntDS+wiKkS9gP8thhoVZnYgNtZz3Z3+47JECO61WNqKpr4lZDA8C46DDk/ucX/HTJ1URh7meMjQrp\nVPzO3DIkBF+dvoqiq1XAHSOcfwHps2gLD0K66Cdu2KnjT/LVdY4rCqDNzKcOFYV5GOt8cSVMJucr\no9nGt9srCraxrnZtCIz0XZQoCOmiHy2J4tZo6/6E+QChZoiEAgSq7E/CYBvd7RNFRD8VwoOVqNc2\n47KThXc6vQGXfq2FUCDgPtjdJXZYCCRiIX65VmO1eSHxPZQoCOmC2nod8tWVEIuEiB8ZbvVc27Oy\nHR0g1I9bS2E980kgECAxJgIA8H1+mcM4zv5yE0YTg1GDg6Fss47DHeQyMcaPCAfDAKeLyt362qR3\noURBSBecLLwOhgEmjArv8AHNTY110J9o+3z7igIAJscNAACccJIovrM8f/vYAa4F3klJY8wJ6/QF\nShS+jBIFIV3AfoCzH+htsRWCo/5E2+fbVxQAkDQmEiKhAD9dqkCD1v7GfFyisBGHO0waEwkAOFlw\nnXaS9WGUKAjpJKPJhO/zrwMAbh87sMPz1Z2sKNpPjwUAf6UUt0aHwWhicMrOT/MlN+pQerMegSop\n4tw844k1clAwgv1luFGjxdUbtEGgr6JEQUgn5auroGnUY2Con83zqbmpsa5WFHW2z8iebBlO+ub8\nrzafZ6uJSbGRNrcydwehUIAkS1Vx/MdSj7wH6fkoURDSSYdPXwEATBk3yOaW3uxiu37OKgr/1h6F\nrWGdO8cNAgD851ypzfMpjp4zf3DfHtexqnGnabcNAQB8deaqR9+H9FyUKAjpBIPBhMOWD8zpk6Js\n3lNRqwUAhDqpKOQyMVRyCVoMJu6go7aGRQYiZmg/NOpa8M1P16yeK6tswNmfb0ImEWGKJaF4SnLs\nAPgrpfi5tAbq67Sdhy+iREFIJ3xfWIbaBj2iIgMwekiwzXvYw34Ghfk5fT32nmsVDTaf/12SORl9\nceqK1fVDJ80bbN49YTD8FO6dFtueVCLC1AmDAQBfnb7i+GbSJ1GiIKQT2A/o/5kUZXPYyWRiuA99\n9pQ4RwaHOz5J7reJQyESCnDiv7+ixrI+g2EYLo57Jw3v/F+iC+6ZOAyAOWEZTSZe3pP0HJQoCHHR\njRotjp4thVAgwP8k2h52qtQ0Qd9iRJCfDP5KqdPXZPd4umYnUYQEKJAcGwmjicE/j18CAORdvIHS\nm/UIDVRgYkz/Lv5tOue20eEYGOqHssoGfPOT7eY66bsoURDioo++LoLRxCD1tiGICFHZvIetDAaH\nu3YkKVdRVNifevrAtBjz+x+9CH2LEbu+LAQAzL9rpMdmO7UnEgrx+9TRAID3/13Ey3uSnoMSBSEu\naGhqwT7LT/RL7omxe9+1Ctf7E0BrorBXUQDAxFv6Y9TgYFTX6fD3Q/k4WXgdCpkY8+8a5Wr4bjFr\ncjRUcgnO/XIThS6elUH6BkoUhLjgg39fQKOuBfEjwzFmmP3FbZ2uKMLYHoXtZjZg3vuJTU57/nUB\nADAnZYTDDQc9QSWXYI5lu/Edn/7E63sT76JEQYgTN2q0eNcy3JM++1aH916zfOAPcjFRhATKIZeK\noGnUo66x4xRZ1m8ShiIyRAVdsxFikQCLHVQ1nvTw78bATyHBycLr+L7A8T5UpO+gREGIE1n7zkHf\nYsTUCYMxYZTj5jHbaxjswownwFwtcMNPdqbItr2X/V1kY8YVH4L85fjD9DgAwKsfn0WzjYWApO+h\nREGIA0d+KMGXp65AKhZixbx4h/cyDMP1GlytKIDWabSO+hR/P5SPssoGyCQitBhMWP/u9y4dauQJ\nC6aOxuBwfxSXafDmfhqC8gWUKAixo7y6EZt2nwIAPDV/gtN1EdX1Omj1BgQopQjyc71/4Gzm09mf\nb+CdA/kQCIDVD09CoEqK7wuu48MjF11+D3eSSURY94fbIRQI8N6/LuBU4XWvxEH4Q4mCEBvqtc14\n+vWjqNM2Y3LcACy42/kMowtXqgEAQyMCOvVeUZGBAIBCy9e3dfVGHf781jcwMQwe+l0s7pk4DJlL\nJwEAXs89i2/OX+vwNXwYOzwUf7g3DgwD/Pmtb1BcRlt79GWUKAhpp17bjD9l/QeXyzQYFhGAdY/e\nbnMVdntsc5fdbdVVEy2n2eUVlaPF0Drmf62iHitePYKaej0mxUYibaa5kX53/GD84d44GE0MVr31\nrdcOFVo2Iw53xw9GQ1MLVrx2BFdoH6g+ixIFIW2UVTbgsZcO48dLFQgPUuD1P97t8jTUkwXmIZhJ\nsZ1LFP2DlYiKDIRWb8D5y5UAgPOXK/CHLV/helUjxg4PxUvpUyAWt/5zTZ91K2anREPfYsQfXz+K\ng98Xd+o93UEkFGL9o7dj/Igw3KzRYtlL/8LZn2/wHgfxPEoUhMDciD74fTEWrT+Ey2UaREUG4J3n\n7kFkiGsL565V1KPkZj38FBLEOlhnYU9ynDm5nMj/FTkH/ovHt/7LXEmMicTrf5wKhUxsdb9AIMDz\nS5Kw6De3wGA0Ye3fv8fqv52wuQutJ8mlYmz/41SkjB0ATaMey1/+Gm/u/8nmtuik9xI7v4WQvoth\nGJy+UI63PjuP/xabf5q/a/wg/OWhSZ1a0MZWE0kxkRCLOv/zV+LoCLz/ryJ8+PVFtBjMm+49kHoL\nnpoXb1VJtCUUCvC/99+GYRGBePnDPHxx6gpO/LcMD/9PLO6bMtLju8qy5DIxtmbcibc/O49/fFmA\nvx3Mxxcn1Xhs5q34beJQSMQiXuIgnsNroigsLMTq1atx6dIlDB06FOvWrcP48eM73HfgwAG88sor\nqKqqQlJSEjZt2oTQ0FA+QyV9GMMwuFJeh29+uoaDJ9VcI7ZfgBxPzh2PGcnDXepJsIwmEzf005lh\nJ6PJhIslNThytgRfWHaDbTGYMCBEhcylk5Bo6V04c98dIxA/MhxbPziD0xfK8fon5/C3g//FneMH\nY+qEIZgUGwmZxLMf1mKREBn3jcek2Ehs/SAPl36txbp/fI/t+84hdcIQpN42BONHhvG2NxVxLwHD\n04nper0ev/nNb5Ceno77778f+/fvx8svv4x///vfUKlaN1grKirC4sWLsXPnTowePRobNmzAzZs3\nkZOT4/Q9rl27htTUVHz99dcYNMizh7mQ3sFkYnCzVgv1dQ2KyzS4cLUaeUXlqGpzTnVIgBy/nzoa\nC6aOhkre+Z/C//5FPrL/+RNCAxX4aO29CLBRiRgMJlyrbMCV6xqoyzU4f7kS536+iUZdC3dPgFKK\nOm0zhkX4Y/dfpkMu7dzPcQzD4PuC69j1ZSF+aNMrUMjEiB0WgjGWX0P6+2NAqF+X/q6uMBhN+PLU\nFez6qtDqoKNAlQxxUSGIjQrBLUP7YXCYPyJD/TyexIhzzj47eUsUx44dw5o1a/Cf//yHuzZz5kws\nX74c06dP565t3boVFRUVeOmllwAANTU1SE5Oxrfffuu0quhuovjilBo3ahpbL1i+M22/RR2+We0u\ntP9uMmj3tYyte+y8qOU3kwv/F3W4x/olLLExtm5p87z1RcZGZNaXGDhb82UzLFfisHqOgclk/gnc\naGJgNFp+b/NnfYsR+hYjmpuN0LUYoNMbUd/UDK2uxWaMSrkYQ8L9MWpwPwyLDGj3k67tvxTDAEYT\nA4PRhBaDEZqGZhSVViO/uBIMA8xIHo5gfxkamlosv5pRU69HRa0W1fW6Dt8LABgY6oekMRG4Z+Iw\nxAzph4e3fAn19TqMGhyMeVNGYlC4P+RSEYQCAQQCc29CKBBAIAQEsF/1lFU24ExROU5fKMeV8jqb\n9/grpQhUSeGnkMJfKYGfQgqVXAKpRAipWASJRASpWAiJ2Py7+X0FEAoAoUAAoVDAxcX+WShsXT3O\nMAzKqhpRoK5EoboKNTb6JwIAfkoJlHIJVDIJlHIxlHIJZBIRJGIhJCIhxG1+FwuF5u+BsPV7IeS+\nL4BAKLT6XrHvYY+zwrHD99jxQ9fv8ZDZKdEI9HN8BK8tzj47eRt6UqvViI6OtroWFRWF4mLr2RrF\nxcWIj29dARscHIzAwECo1WqPDj/9KesovjlPe9f4Cq3OgKKSGhSV1LjtNQ84mHkkEAAR/ZQYFhmI\nqIhAjB4SjNtG9e+wXfnGZSlY8doR/Fxag817TrstNlvqtc2o1zZ79D2cYQDUa1tQr21xei9xLvvT\nn3DyzUVuf13eEoVWq4VCYX2GsFwuh06ns7rW1NQEudw6IyoUCjQ1NXk0vkljBuDEf8ts/tTnVt7Z\noscm/kLp3ju1/6nP+rHlZz6B+V0EAgH3Z/YnS+6CGyIzv7z5dWVSMQJUUowYGIRgPzmkEiH8FFL4\nKSSWX1IEB8gRFqhAvwC5S03uUYODsf+vs/HlqSvIu3gDN2u0MBhNMJkYrmps++euYhi2ImNaKzUT\nA5OJAcMwYBhzlcow5nsZsJUew1XGbX6zrro7ERr7dVy1bf4f60q47RvBcTXqVt7ZIaVbhnVysaer\neEsUCoWiQ1LQ6XRQKpVW1+wlj/b3udsCyxg1Id4ml4ox544R3JbehHgbb1MQhg8fDrVabXVNrVZj\nxAjrfwzR0dFW91VXV0Oj0XQYtiKEEMIP3hJFcnIympubsXv3brS0tCA3NxeVlZVISUmxum/GjBk4\nfPgw8vLyoNfrsW3bNkyZMgXBwcF8hUoIIaQN3hKFVCpFTk4ODh48iMTERLz33nvYsWMHlEolVq9e\njdWrVwMAYmJisGHDBmRmZiI5ORk3b97E5s2b+QqTEEJIO7xNj+UDraMghJDOc/bZScskCSGEOESJ\nghBCiEOUKAghhDjUp3aPNRrNWxuXl3vnIBdCCOmN2M9M9jO0vT6VKCoqKgAAixcv9nIkhBDS+1RU\nVGDo0KEdrvepWU86nQ75+fkICwuDSEQ7UhJCiCuMRiMqKioQFxfXYQsloI8lCkIIIe5HzWxCCCEO\nUaIghBDiECUKQgghDlGiIIQQ4hAlCkIIIQ5RoiCEEOIQJQpCCCEOUaLwsNzcXCQlJXk7DK/Kzs7G\nXXfdhYSEBDz44IP4+eefvR0S7woLCzF//nyMHz8es2fPxo8//ujtkLwmLy8P999/P2677TZMmzYN\nH374obdD8rrKykokJyfj6NGj3g7FJkoUHlRaWootW7Z4Owyv2rdvH/bv34/du3fj5MmTSE5ORlpa\nGkwmk7dD441er0d6ejrmzp2LM2fO4MEHH8Ty5cvR2Njo7dB4p9FokJGRgaVLl+LMmTN47bXXsG3b\nNnz33XfeDs2rMjMzUVtb6+0w7KJE4SFGoxErV67E73//e2+H4lU1NTVIT0/H4MGDIRaLsXTpUpSV\nlfnUxo0nT56EUCjEokWLIJFIMH/+fISGhuLYsWPeDo13ZWVluPPOOzFz5kwIhULExsYiKSkJZ8+e\n9XZoXvPBBx9AoVAgMjLS26HY1ac2BeSTwWCAVqvtcF0oFMLPzw9vv/02Ro4ciSlTpiA3N9cLEfLH\n0ffi0Ucftbp25MgRBAUFISIigq/wvE6tViM6OtrqWlRUFIqLi70UkffExMRg69at3GONRoO8vDzM\nnj3bi1F5j1qtxt///nfs3bsXc+fO9XY4dlGi6KLTp0/jkUce6XB94MCBeP311/HZZ58hNzcX+fn5\nXoiOX46+F0eOHLG6b82aNVi/fj2EQt8pZrVaLRQKhdU1uVwOnU7npYh6hvr6eqSnpyM2NhZTp071\ndji8MxgMWLlyJTIzMxEUFOTtcByiRNFFt99+Oy5evNjhuk6nw/z587Fx40aoVCovRMY/e9+Ltj79\n9FOsW7cOL7zwAmbOnMlTZD2DQqHokBR0Oh2USqWXIvK+0tJSbkjy1Vdf9akfHFjZ2dmIiYnBnXfe\n6e1QnPK9/3c8LD8/H6WlpUhLS0NCQgLS09Oh0WiQkJCAsrIyb4fnFW+88QY2b96M7OzsHl1ee8rw\n4cOhVqutrqnVaowYMcJLEXlXQUEBFixYgJSUFGRnZ9vc1toXHDp0CAcPHkRCQgL3+fCnP/0Jb7/9\ntrdD64ghHnXy5EkmMTHR22F4TW5uLjNx4kTm0qVL3g7Fa/R6PZOSksLs2rWLaW5uZj7++GNm0qRJ\nTGNjo7dD411FRQUzadIk5q233vJ2KD3O3XffzRw5csTbYdhEFQXxqLfffhuNjY2YP38+4uPjuV+X\nL1/2dmi8kUqlyMnJwcGDB5GYmIj33nsPO3bs8Mmhp9zcXFRXV2PHjh1W/z288sor3g6NOEAHFxFC\nCHGIKgpCCCEOUaIghBDiECUKQgghDlGiIIQQ4hAlCkIIIQ5RoiCEEOIQJQpCPOTSpUsYO3aszZW2\nTz75JH7/+9/DaDR6ITJCOocSBSEeMmLECDzxxBPIysqy2il27969OHHiBF566SWIRCIvRkiIa2jB\nHSEeZDAYsGDBAkilUrz//vsoKSnBfffdh5UrV+KBBx7wdniEuIQSBSEedvHiRcybNw+rVq3CwYMH\nufNKCOktKFEQwoOsrCzs2LEDfn5+OHDgAMLCwrwdEiEuox4FITyYOnUqDAYDoqKiEBoa6u1wCOkU\nShSEeFhzczNWrVqFhIQEnD9/Hu+//763QyKkUyhREOJh27dvx/Xr1/HKK69g6dKl+L//+z+UlpZ6\nOyxCXEY9CkI86OzZs1iyZAk2b96M2bNnQ6fTYdasWejfvz927doFgUDg7RAJcYoqCkI8RKvV4rnn\nnsNdd92F2bNnAwDkcjk2bdqEM2fO0BAU6TUoURDiIVu2bEFdXR3WrVtndX3ixIlYtGgRDUGRXoOG\nngghhDhEFQUhhBCHKFEQQghxiBIFIYQQhyhREEIIcYgSBSGEEIcoURBCCHGIEgUhhBCHKFEQQghx\n6P8B3vjsThqzlXUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c9840d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Now create another (arbitrary) covariance function\n",
    "k2 = GPy.kern.Exponential(dim, lengthscale=0.2, variance=2.1)\n",
    "\n",
    "# Create a new covariance function that is the sum of these two:\n",
    "k_new = k1 + k2\n",
    "\n",
    "# Let's plot the new covariance\n",
    "fig, ax = plt.subplots()\n",
    "k1.plot(ax=ax, label='$k_1$')\n",
    "k2.plot(ax=ax, label='$k_2$')\n",
    "k_new.plot(ax=ax, label='$k_1 + k_2$')\n",
    "plt.legend(fontsize=16);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> plotting eigenvalues of K\n",
      "> they must all be positive\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEQCAYAAABBQVgLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFP5JREFUeJzt3W9MlefBx/EfoHA4w4ov/POiB8KqDAZFDG7YlAgBTeqU\nGHX+ack0pBrPdCHRxSc1GWsWmxRisC0jnmO7Zh3ahEXMdMHEBEzbN1oWQgwKR9yEoQ3pJiWeaAVk\nwPPCp+fZ0dKryp/rhvP9JL64L+7D+UlO7t99Xfd9zokaGxsbEwAA3yHadgAAgPNRFgAAI8oCAGBE\nWQAAjObYDjAVBgcHde3aNS1cuFAxMTG24wCA442MjOjOnTvKzMyUy+V64uezsiyuXbumkpIS2zEA\nYMb5+OOPtXLlyifGZ2VZLFy4UNKj//SSJUsspwEA5/vyyy9VUlISOn4+blaWxTdLT0uWLNHzzz9v\nOQ0AzBzjLd1zgRsAYGS1LFpaWrR161bl5ORozZo1qqurkyRdvXpV6enpWrFiReif3++3GRUAIpq1\nZahgMKh9+/apvLxc69evVyAQUGlpqZKSkvTFF19o9erVOnHihK14AID/Yq0sent7lZ+fr+LiYklS\nRkaGcnNz1draqr6+PqWlpdmKBgB4jLVlqPT0dB09ejS0HQwG1dLSorS0NAUCAbW2tqqwsFAFBQWq\nrKzUw4cPbUUFgIjniAvc9+7dk9frVUZGhgoLC7VgwQIVFhaqoaFBJ0+eVHNzs6qrq23HBADHGh4Z\nVeWF63r1/c9VeeG6hkdGJ/X3Wy+L27dva8eOHZo/f75qamoUHR0tv9+v0tJSud1ueTwe7d27V42N\njbajAoBjHWu8Id+nN3W56yv5Pr2pdxpvTOrvt1oW7e3t2rZtm/Ly8nT8+HG5XC4Fg0FVVlbq/v37\nof2GhoYUFxdnMSkAONuVW3fDt2/fHWfPZ2OtLPr6+rR7926Vlpbq8OHDio5+FGXevHlqbGxUTU2N\nhoeH1dPTI7/fr82bN9uKCgCOl52UGL7tSRxnz2dj7W6o+vp69ff3y+fzyefzhcZ37twpv9+vt956\nS6tWrZLL5dL27du1a9cuW1EBwPEOrk1VlB7NKLI9iTqwNnVSf3/UbPxa1S+++EJFRUW6ePEiH/cB\nAN+D6bhp/QI3AMD5KAsAgBFlAQAwoiwAAEaUBQDAiLIAABhRFgAAI8oCAGBEWQAAjCgLAIARZQEA\nMKIsAABGlAUAwIiyAAAYURYAACPKAgBgRFkAAIwoCwCAEWUBADCiLAAARpQFAMCIsgAAGFEWAAAj\nygIAYERZAACMKAsAgBFlAQAwoiwAAEaUBQDAiLIAABhZLYuWlhZt3bpVOTk5WrNmjerq6iRJwWBQ\n+/fvV05OjgoKCnT69GmbMQEg4s2x9cTBYFD79u1TeXm51q9fr0AgoNLSUiUlJamurk5ut1uXLl1S\nZ2en9uzZo2XLlik7O9tWXACIaNZmFr29vcrPz1dxcbGio6OVkZGh3Nxctba2qqmpSWVlZYqLi1NW\nVpY2bNigs2fP2ooKABHPWlmkp6fr6NGjoe1gMKiWlhZJ0pw5c+TxeEI/S0lJUVdX17RnBAA84ogL\n3Pfu3ZPX6w3NLlwuV9jPXS6XBgcHLaUDAFgvi9u3b2vHjh2aP3++ampq5Ha7NTQ0FLbP4OCg3G63\npYQAAKtl0d7erm3btikvL0/Hjx+Xy+VScnKyhoeH1dvbG9qvu7tbS5cutZgUACKbtbLo6+vT7t27\nVVpaqsOHDys6+lGUhIQEFRUVqaqqSgMDA2pra1NDQ4OKi4ttRQWAiGft1tn6+nr19/fL5/PJ5/OF\nxnfu3KkjR47ozTffVH5+vtxutw4dOqTly5fbigoAEc9aWXi9Xnm93nF//t57701jGgDAd7F+gRsA\n4HyUBQDAiLIAABhRFgAAI8oCAGBEWQAAjCgLAIARZQEAMKIsAABGlAUAwIiyAAAYURYAACPKAgBg\nRFkAAIwoCwCAEWUBADCiLAAARpQFAMCIsgAAGFEWAAAjygIAYERZAACMKAsAgBFlAQAwoiwAAEaU\nBQDAiLIAABhRFgAAI8oCAGBEWQAAjBxRFm1tbcrLywttX716Venp6VqxYkXon9/vt5gQACLbHJtP\nPjY2pjNnzqiiokIxMTGh8UAgoNWrV+vEiRMW0wEAvmF1ZuH3+1VbWyuv1xs23tHRobS0NEupAACP\nszqz2LJli7xer/72t7+FjQcCAcXGxqqwsFCjo6Nat26dDhw4oNjYWEtJASCyWZ1ZLFq0SFFRUU+M\nL1iwQIWFhWpoaNDJkyfV3Nys6upqCwkBAJJDLnA/zu/3q7S0VG63Wx6PR3v37lVjY6PtWAAQsRxX\nFsFgUJWVlbp//35obGhoSHFxcRZTAUBkc1xZzJs3T42NjaqpqdHw8LB6enrk9/u1efNm29EAIGI5\nriyio6Pl9/t1/fp1rVq1Sq+99ppeeeUV7dq1y3Y0AIhYVu+G+kZubq6am5tD20uXLtVHH31kLxAA\nIIzjZhYAAOehLAAARpQFAMCIsgAAGFEWAAAjygIAYERZAACMKAsAgBFlAQAwoiwAAEaUBQDA6JnL\n4he/+IUk6fe//70++eQT/fvf/560UAAAZ3nmDxL0+XySpLGxMdXV1am9vV1RUVHKyMhQZmamfvWr\nX01aSACAXc9cFgkJCZKksrKy0Ni//vUvXbt2Te3t7RNPBgBwjEn9iPLFixdr8eLFKioqmsxfCwCw\njAvcAACjCc0sRkZG1NXVpc7OTl2/fl2dnZ364IMPJisbAMAhvndZ9Pf3q7OzM6wYbt68qeHhYY2N\njSk2NlapqalTmRUAYImxLM6cOaPq6uqwW2Pj4+OVlZWlkpISpaWlKT09XS+88IJiYmKmNCwAzFbD\nI6M61nhDV27dVXZSog6uTdXcGOdcKTCWRVVVlZ577jn9+te/VkpKiv74xz/qypUr+slPfiKv10tB\nAMAkONZ4Q75Pb0qSLnd9pShJ//NKmt1Q/8VYFv39/aqqqtJLL70kSSoqKtLHH3+sqqoqXbx4URUV\nFSw/AcAEXbl1N3z79t1x9rTDOMdpamrS8uXLw8ZKSkr017/+Vc8995y2bNkin8+n0dHRKQsJALNd\ndlJi+LYncZw97TDOLJ5//vlxxz/66CP9+c9/1tGjR9XU1KSKigotW7Zs0kMCwGx3cG2qovRoRpHt\nSdSBtc5asZnwm/K2b9+u/Px8/eY3v9HmzZt19erVycgFABFlbky0o65RPG5S3sG9ZMkS/eEPf9Bf\n/vKXyfh1AACHmdT7sjZt2jSZvw4A4BDOuYkXAOBYlAUAwIiyAAAYURYAACNHlEVbW5vy8vJC28Fg\nUPv371dOTo4KCgp0+vRpi+kAAJP65UdPa2xsTGfOnFFFRUXYZ0yVl5fL7Xbr0qVL6uzs1J49e7Rs\n2TJlZ2dbTAsAkcvqzMLv96u2tlZerzc09vXXX6upqUllZWWKi4tTVlaWNmzYoLNnz1pMCgCRzWpZ\nbNmyRefOndOLL74YGuvp6dGcOXPk8XhCYykpKerq6rIREQAgy8tQixYtemLswYMHcrlcYWMul0uD\ng4PTFQsA8BhHXOD+b/Hx8RoaGgobGxwclNvttpQIAOC4skhOTtbw8LB6e3tDY93d3Vq6dKnFVAAQ\n2RxXFgkJCSoqKlJVVZUGBgbU1tamhoYGFRcX244GABHLcWUhSUeOHNF//vMf5efnq6ysTIcOHXri\nC5gAANPH6gXub+Tm5qq5uTm0nZiYqPfee89iIgB4OsMjozrWeENXbt1VdlKiDq5N1dwYR56PPxNH\nlAUAzHTHGm/I9+lNSdLlrq8UJTn6y4ye1uypPQCw6Mqtu+Hbt++Os+fMRFkAwCTITkoM3/YkjrPn\nzMQyFABMgoNrUxWlRzOKbE+iDqxNtR1pUlEWADAJ5sZEz6prFI9jGQoAYERZAACMKAsAgBFlAQAw\noiwAAEaUBQDAiLIAABhRFgAAI8oCAGBEWQAAjCgLAIARZQEAMKIsAABGlAUAwIiyAAAYURYAACO+\n/AgAJA2PjOpY4w1duXVX2UmJOrg2VXNjOJ/+BmUBAJKONd6Q79ObkqTLXV8pSprV33z3tKhNAJB0\n5dbd8O3bd8fZMzJRFgAgKTspMXzbkzjOnpGJZSgAkHRwbaqi9GhGke1J1IG1qbYjOQplAQCS5sZE\nc43iO7AMBQAwoiwAAEaUBQDAyNFl8eGHHyozM1MrVqwI/WtpabEdCwAijqMvcHd0dOjAgQN6/fXX\nbUcBgIjm6JlFIBBQenq67RgAEPEcWxYDAwPq7u5WbW2tXn75Za1bt0719fW2YwFARHLsMlRfX59y\ncnL06quvqrq6Wm1tbfJ6vVq4cKHy8/NtxwOAiOLYsvB4PDp16lRoe+XKldq4caMuXrxIWQDANHPs\nMlR7e7vef//9sLGhoSHFxsZaSgQAkcuxZeF2u1VTU6MLFy5odHRUly9f1vnz57Vp0ybb0QAg4jh2\nGSolJUXvvvuu3nnnHb3xxhtavHix3n77bWVkZNiOBgARx7FlIUmFhYUqLCy0HQMAIp5jl6EAAM5B\nWQAAjCgLAIARZQEAMKIsAABGlAUAwIiyAAAYOfp9FgDwfQ2PjOpY4w1duXVX2UmJOrg2VXNjOB+e\nLJQFgFnhWOMN+T69KUm63PWVoiT9zytpdkPNItQugFnhyq274du3746zJ54FZQFgVshOSgzf9iSO\nsyeeBctQAGaFg2tTFaVHM4psT6IOrE21HWlWoSwAzApzY6K5RjGFWIYCABhRFgAAI8oCAGDENQsA\njsCb6pyNsgDgCLypztmobQCOwJvqnI2yAOAIvKnO2ViGAuAIvKnO2SgLAI7Am+qcjbIAMCm4m2l2\noywATAruZprdKAsAkiY+M+BuptmNsgAgaeIzg+ykRF3u+ur/t7mbaVahLABImvjMgLuZZjfKAnCI\niS4DTfTxE50ZcDfT7EZZAA4x0WWgiT6emQG+C2UB/B/bZ/YTXQaa6OOZGeC7OLosOjo69Nvf/lb/\n+Mc/lJycrN/97nfKzs62HQvjsH2wnejjbZ/ZT3QZiAvMmEqOLYuhoSF5vV55vV5t3bpV586d0y9/\n+Us1NTXpBz/4ge14U8L2wXKmH2wn+njbZ/YTXQZiGQlTybFl8fnnnys6OlqvvfaaJOnnP/+5/vSn\nP+mzzz7Tz372syl5Tg62M/tgO9HH2z6zn+gyEMtImEqOLYvu7m698MILYWMpKSnq6uqasufkYDuz\nD7YTfTxn9sD4HFsWDx48UHx8fNiYy+XS4ODglD0nB9uZfbCd6OM5swfG59iyiI+Pf6IYBgcH5Xa7\np+w5OdjO7IMtB2tg6ji2LH74wx/q1KlTYWPd3d3asGHDlD0nB1sOtgC+nWPL4qWXXtLDhw918uRJ\n7dixQ+fOnVNfX5/y8vKm7Dk52ALAt3Psh83Hxsbqgw8+0Pnz5/XTn/5Up06dks/nm9JlKADAt3Ps\nzEKS0tLSVFdXZzsGAEQ8x84sAADOQVkAAIwoCwCAkaOvWTyrkZERSdKXX35pOQkAzAzfHC+/OX4+\nblaWxZ07dyRJJSUllpMAwMxy584dJScnPzEeNTY2NmYhz5QaHBzUtWvXtHDhQsXExNiOAwCONzIy\nojt37igzM1Mul+uJn8/KsgAATC4ucAMAjCgLAIARZQEAMKIsAABGlAUAwIiyAAAYURYAACPKAiEf\nfvihMjMztWLFitC/lpYW27Ecr62tLexLuYLBoPbv36+cnBwVFBTo9OnTFtM53+N/v6tXryo9PT3s\ndej3+y0mdKaWlhZt3bpVOTk5WrNmTejrHKbq9TcrP+4Dz6ajo0MHDhzQ66+/bjvKjDA2NqYzZ86o\noqIi7JMCysvL5Xa7denSJXV2dmrPnj1atmyZsrOzLaZ1nvH+foFAQKtXr9aJEycspnO2YDCoffv2\nqby8XOvXr1cgEFBpaamSkpJUV1c3Ja8/ZhYICQQCSk9Ptx1jxvD7/aqtrZXX6w2Nff3112pqalJZ\nWZni4uKUlZWlDRs26OzZsxaTOtO3/f2kRyctaWl8PfF36e3tVX5+voqLixUdHa2MjAzl5uaqtbV1\nyl5/lAUkSQMDA+ru7lZtba1efvllrVu3TvX19bZjOdqWLVt07tw5vfjii6Gxnp4ezZkzRx6PJzSW\nkpKirq4uGxEd7dv+ftKjk5bW1lYVFhaqoKBAlZWVevjwoaWUzpSenq6jR4+GtoPBYGjJeKpef5QF\nJEl9fX3KycnRq6++qk8++URHjhxRRUWFPvvsM9vRHGvRokWKiooKG3vw4METH8Lmcrk0ODg4ndFm\nhG/7+0nSggULVFhYqIaGBp08eVLNzc2qrq62kHBmuHfvnrxeb2h2MVWvP8oCkiSPx6NTp04pPz9f\nsbGxWrlypTZu3KiLFy/ajjajxMfHa2hoKGxscHBQbrfbUqKZx+/3q7S0VG63Wx6PR3v37lVjY6Pt\nWI50+/Zt7dixQ/Pnz1dNTY3cbveUvf4oC0iS2tvb9f7774eNDQ0NKTY21lKimSk5OVnDw8Pq7e0N\njXV3d2vp0qUWU80cwWBQlZWVun//fmhsaGhIcXFxFlM5U3t7u7Zt26a8vDwdP35cLpdrSl9/lAUk\nSW63WzU1Nbpw4YJGR0d1+fJlnT9/Xps2bbIdbUZJSEhQUVGRqqqqNDAwoLa2NjU0NKi4uNh2tBlh\n3rx5amxsVE1NjYaHh9XT0yO/36/NmzfbjuYofX192r17t0pLS3X48GFFRz86lE/l649bZyHp0UWw\nd999V++8847eeOMNLV68WG+//bYyMjJsR5txjhw5ojfffFP5+flyu906dOiQli9fbjvWjBAdHS2/\n36+33npLq1atksvl0vbt27Vr1y7b0Rylvr5e/f398vl88vl8ofGdO3dO2euPLz8CABixDAUAMKIs\nAABGlAUAwIiyAAAYURYAACPKAgBgRFkAAIwoC2CatbW16Uc/+pFaW1ttRwG+N96UB0yz/v5+3bp1\nS1lZWaGPaQCcjrIAABhxWgNMs40bN+rw4cO2YwBPhbIAptHDhw918+ZNvr4WMw5lAUyjv//97xoe\nHtaPf/xj21GAp0JZANOoo6NDUVFRSktLsx0FeCqUBTCNAoGAkpKSlJCQYDsK8FQoC2AadXR0cL0C\nMxJlAUyT0dFRdXZ2cr0CMxJlAUyTf/7zn3rw4AEzC8xIlAUwTQKBgCRRFpiReAc3AMCImQUAwIiy\nAAAYURYAACPKAgBgRFkAAIwoCwCAEWUBADCiLAAARv8L4TpXhcG7a58AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10cc3be10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# If this is a valid covariance function, then it must \n",
    "# be positive definite\n",
    "# Compute the covariance matrix:\n",
    "K_new = k_new.K(X)\n",
    "\n",
    "# and its eigenvalues\n",
    "eig_val_new, eig_vec_new = np.linalg.eigh(K_new)\n",
    "\n",
    "# Plot the eigenvalues (they should all be positive)\n",
    "print '> plotting eigenvalues of K'\n",
    "print '> they must all be positive'\n",
    "fig, ax = plt.subplots()\n",
    "ax.plot(np.arange(1, n+1), eig_val_new, '.')\n",
    "ax.set_xlabel('$i$', fontsize=16)\n",
    "ax.set_ylabel('$\\lambda_i$', fontsize=16);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 3: Sampling from a Gaussian Process. \n",
    "\n",
    "Samples from a Gaussian process are functions. But, functions are infinite dimensional objects?\n",
    "We cannot sample directly from a GP....\n",
    "However, if we are interested in the values of $f(\\cdot)$ at any given set of test points $\\mathbf{x}_{1:n} = \\{\\mathbf{x}_1,\\dots,\\mathbf{x}_b\\}$, then we have that:\n",
    "$$\n",
    "\\mathbf{f} | \\mathbf{x}_{1:n}, m(\\cdot), k(\\cdot, \\cdot) \\sim \\mathcal{N}\\left(\\mathbf{f} | \\mathbf{m}(\\mathbf{x}_{1:n}), \\mathbf{K}(\\mathbf{x}_{1:n}, \\mathbf{x}_{1:n}) \\right),\n",
    "$$\n",
    "where all the quantities have been introduced above.\n",
    "This is\n",
    "What we are going to do is pick a dense set of points $\\mathbf{x}_{1:n}\\in\\mathbb{R}^{n\\times d}$\n",
    "sample the value of the GP, $\\mathbf{f} = (f(\\mathbf{x}_1),\\dots,f(\\mathbf{x}_n))$ on these points.\n",
    "We saw above that the probability density of $\\mathbf{f}$ is just a multivariate normal\n",
    "with a mean vector that is specified from the mean function and a covariance matrix\n",
    "that is specified by the covariance function.\n",
    "Therefore, all we need to know is how to sample from the multivariate normal.\n",
    "This is how we do it:\n",
    "+ Compute the Cholesky of $\\mathbf{L}$:\n",
    "$$\n",
    "\\mathbf{K} = \\mathbf{L}\\mathbf{L}^T.\n",
    "$$\n",
    "+ Draw $n$ random samples $\\mathbf{z} = (z_1,\\dots,z_n)$ independently from a standard normal.\n",
    "+ Get one sample by:\n",
    "$$\n",
    "\\mathbf{f} = \\mathbf{m} + \\mathbf{L}\\mathbf{z}.\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEPCAYAAAC3NDh4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XdcVfX/B/AXQwTN3IoTJ2pishQHTtzixtTMzMyRlvWt\nftk3c9a3bVMtzZErTVFzp+XeGw0HLlQUSZAhMgTuPb8/3iCiHr3AvfecC6/n48GjvOK9nwP3ntc5\nn/H+2CmKooCIiOgx7LVuABER6RdDgoiIVDEkiIhIFUOCiIhUOWrdAHNJTU1FaGgoypcvDwcHB62b\nQ0RkEwwGA6Kjo+Hh4QFnZ+dH/r7AhERoaCgGDx6sdTOIiGzS0qVL4evr+8jjBSYkypcvD0AO1NXV\nVePWEBHZhqioKAwePPj+OfRhBSYksrqYXF1dUbVqVY1bQ0RkW9S66TlwTUREqhgSRESkiiFBRESq\nGBJERKSKIUFERKoYEkREpIohQUREqhgSRESkiiFBRESqGBJERKSKIUFERKoYEkREpIohQUREqhgS\nRESkiiFBRESqdBsSMTExaN68OXbs2KF1U4iICi3dhsSECRMQHx+vdTOIiAo1XYbEsmXL4OLigkqV\nKmndFCKiQk13IREeHo4FCxZgypQpWjeFiKjQ01VIZGRk4P3338eECRNQqlQprZtDRFTo6SokZs2a\nhQYNGqBNmzZaN4WIiKCzkNi0aRM2btwIX19f+Pr6IjIyEu+88w7mzJmjddOIiAolR60b8KA///wz\nx5/bt2+PiRMnol27dhq1iIiocNPVnQQREemLru4kHrZ9+3atm0BEVKjxToKIiFQxJIiISBVDgoiI\nVDEkiIhIFUOCiIhUMSSIiEgVQ4KIiFQxJIiISBVDgoiIVDEkiIhIFUOCiIhUMSSIiEgVQ4KIiFQx\nJIiISBVDgoiIVDEkiIhIFUOCiIhUMSSIiEgVQ4KIiFQxJIiISBVDgoiIVDEkiIhIFUOCiIhUMSSI\niEgVQ4KIiFQxJIiISBVDgoiIVDEkiIhIFUOCiIhUMSSIiEgVQ4KIiFQxJIiISBVDgoiIVDEkiIhI\nle5C4ujRo+jfvz98fHzQoUMHLF++XOsmEREVWo5aN+BBCQkJGDNmDCZOnIju3bvj7NmzGDZsGKpX\nr44WLVpo3TwiokJHV3cSkZGRaNOmDXr06AF7e3s0bNgQfn5+OH78uNZNIyIqlHQVEg0aNMBXX311\n/88JCQk4evQo6tevr2GriIgKL12FxIMSExMxevRoNGzYEO3bt9e6OUREhZIuQyIiIgIDBw5EyZIl\nMWPGDNjb67KZREQFnu7OvqdPn8YLL7wAf39/zJo1C87Ozlo3iYio0NLV7KaYmBi89tprGDZsGEaO\nHKl1c4iICj1d3UkEBwcjNjYWP/30E7y8vO5/ffvtt1o3jYioUNLVncTo0aMxevRorZtBRESZdHUn\nQURE+sKQICIiVQwJIiJSxZAgIiJVDAkiIlLFkCAiIlUMCSIiUsWQICIiVQwJIiJSxZAgIiJVDAki\nIlKlq9pNpLHEf4Eza4HTq4FnKwOB3wLOJbVuFRFpiHcShV1yLHDsV2BhD2BmE+DGUaDl24BLGWBO\nWyDqH61bSEQa4p1EYZSaAJzbBISuAiIOAXUCgCYjgLodgSIu8j31ugCnVgKLegEdpwFeL2nbZiLS\nBEOisEhLAsI2A6fXAOG7gRqtgMYDgf6/AkWfefy/eb4/4NoIWDEEuHYQ6PZVdogQUaHAkCjI0lOB\ni38BoauBi38D1ZoCDfsCvWYCLqVMe44K9YER24F144B5HYEXFgFlalm23USkGwyJgiYjDbi8U7qS\nzm8GXJ8HPPoB3b4GipfN23MWLQEEzQcO/wLM7Qj0/AGo392szSYifWJIFARGA3BljwTD2Q1AuboS\nDB2nAiVczfMadnaA30igshcQPEzGMtpPAhz4FiIqyPgJt1VGo5yoQ1fJtNVnK0swjNoNlKpmudet\n1gQYuQtY/ZoMagfNB0pUtNzrEZGmGBK2RFGAyOMyxnB6jaxh8OgLvPonULa29dpRvCwwOBjY9SUw\npw3Qby5Qw996r09EVsOQ0DtFAf49LXcMp1cDdg5yx/DSKqBCA+3aZe8AtPuv3FmsHAY0Hwu0fEu6\npSj/FAW4cwO4dQ6IPiv/TY0Hun/DOzeyKoaEXkWfl1AIXSWzlDz6yMwi1+f1dSKu00FmP618BYg4\nDPSeZfrMKcoMg8jsIIjO+gqT6cbl68vFQFVf4PZFYGk/4JVNgPOzWrecCgmGhJ6kxANH50t3UlI0\n0LAP0GuWnCD0FAwPK1UNGLYZ2DpBVmm/sAio9LzWrdIXRQESox4Ig6z/hgGOTtlhUNkL8HxR/lys\nzKPPsek9YPmL0t1XxFmbY6FChSGhF9cOAqtGADVaAl0/B6o3ly4dW+HoJIvt/gkGFvcGOkwBvF/W\nulXWpyjA3VuPCYOzgL0jUL6BrD1xfR54foD82dSpyXZ2QNcvZXbZmpFA0ALbeo+QTWJIaM1oAPZM\nlzUIPX8A6nXVukX50yhIVmn/PgS4dgjo/nXBXaV9N/rxYQBkh0GFhjKGVL4B8Ez5/L+mvQPQ9xdg\nST9g8/uy/kXPd5lk8xgSWkq4AaweKR/yUbtkGmtBUL6ejFOsf0sW372w0Lqzr8wtJU4mD9w6K+MF\nWWFgzMgOg/INgOd6ZYZBBcueuB2LAgN/A37tJjPM2o633GtRTooCJFwHbp4Eok4BN09JJYOWbxXY\nuzqGBAAY0oG1bwBVfAD3TkDpGpZ/zbMbgA1vA81el6qrBe0NVvQZmRp7ZC4wrxPQ43ugQaDWrcqd\njHvA/h/lq5x7dhjU7y7/LeFq+av4lDjg+jHg+hEgJgyo8BxQzU/GqQavAuZ3kjsU31ct247CyGgE\nYi8DN0MyA+GkhIK9I1CpsXw93x84PBcI3wX0mVMgZ54xJADAoQjwXE/g3EZg95dSJtu9E1C3M1C9\nmfy9uaSnAFs/Ai78BQxcJlNICyo7O6DpCBmMXfkKEHEQCJhiG6u0L+2QQeKydeUuzxoXDkaj3Klc\nPyyhEHFEpsFW9gKqNpH3463TwPZPgH9D5Y6tegvg76kyNdpnqOXbWFAZ0mUSwf0wOAlEhQLFSsv4\nUSVPoNlYmZDxcBWDBr2AXZ8Ds1sDfWcDtdpqcQQWY6coiqJ1I8zh+vXrCAgIwLZt21C1atW8P5HR\nCNw8AZzfClzYCsRekl963c5SSvuZCnl/7ltngeBXZeZKj+8K14Y+ybHA6hFAWjLQf4H5yoWY251I\nYMsE2VejyxdA/W6We62UOOD6UZk6fP0IcOO4DGJXbSoXD1WbyJjG40I1PRWIPAFcOyDv02sH5Y6i\nZluguh9QrZnMlipod6jmkJ4C/HsGiDqZfXcQfQ4oWTUzEBpLGLg+/+gMsye5tB1Y87pM2Ggz3jYu\nhvD0cydD4mnu3pKr/gtbpHBemVoSGO6dgEpegL0J+zYpCnBsgVwBdpgqezMUxsFGoxHY/ZX8LPr+\nAtRspXWLshkygMOzgd1fS9dNq3cBp2Lme36jQU5EWYFw/Qhw5yZQ2VP6tKs2lS6k4uXy9vyXdgLB\nrwBNRgJ3rktoJMXIc1Zvlt1F5VTcfMdkC1LvyMZZD3YXxV4GytbJ7jKq9DxQ0UO9ZH5uJP4rF0PG\nDOlutYFxRoaEORnS5cN3YYvcaaTEAnU6SmDUbv/4O4PkWGD9OCDuCtBvPlDe3TJtsyUXtwFrRgPN\nxwAt3jItaC3p6gFg47tyJd7taymQmF/JsXKXcP2wBEPkCaB4+cxAaCL/Ld/AvFeboauBLR9KmZbS\nNWT2VcQh6ea7dki6qMq5Z4dG9WY2cRIzWdLtB+4OMgMh8aaM4zwYCBWek8F/SzEagD3fAIfnSFl+\n906Wey0zYEhYUtwVucs4v0XCo7KndEnV7Sz9xdcOyOyl+oFSkdWSb0xbk3BdximKl89cpV3a+m24\nGw38PVnGHzp/Intt5OUOz2iQrsTrh2Uc4fphuaKs4p0dCFWb5K7rIq8O/wIc/Al4dcujU26zuqiy\nQiPiIOBUQsLC1rqoUuLkM/dgINy7k7O7qFJjGVPSqtvnyj65q/DoBwRMMu/YphkxJKwlLVl2fLuw\nVUIjLVH2dmj5FuD/dsFdK5AfGWnAXxOB839mrtJubJ3XNRqky2vHZ7I7X9sPZM8MUyXdljGLiMMS\nCDdOyBhL1SaZYwlNtT3Zbv+fvA9f2fDk4zIagdsX5GQbcUj/XVSJ/wJhG4Gz6yWMq/rKoH5WIJSq\nof1d6cOSYuSuOTVeKiaXqq51ix5hcyFx5swZTJo0CRcvXoSbmxumTp0KT0/Pp/47zUMiS8J1uXrI\nuAfUagNcPSh9om4tMmdMddLlG0VToauATf8HBEyWQT9LjtfcOA5sfAdwdAa6TwcqNnz6v7l9Scaj\nrh+RYEiKzr5LyBpLsMZdgqkURdaoxF8FXlyRuztYvXVRxV+T6eJn18vMrjodgQY9pGaYOcYQrMFo\nBA78COz7QZdTwfMdEhs2bEBgoHUO6t69e+jYsSNGjx6N/v37Y+3atZg+fTr+/vtvFC/+5KsZXYTE\n2fXAhv8AzcbkXFyTEiczH85vle1Ei1fInmJbralub0OtKvq87KVdxUfGBcw5aAzI72Dbx8C5DVIy\npPEg08Io6h/ZN8O9q4RBtaYyO03vXTKGDGDlUMDBCeg3L+9X2I90UR0CnqkIdJgMuHexTKDHXJA9\nUs6uBxIipApBg15y0WXLXbYRh4Hg4TJjruM03RxLvkPCw8MD3t7emDx5MmrXtuyq2V27dmHy5MnY\nuXPn/cd69OiB119/Hd26PXkqYn5CIi3DiJsJKXlpMgDALiMFpfdOg/PVHYjpPBNprj7q32w0wOlW\nCFyubIPLlW1wvBOB1GqtkFKjA1JqtIfRJY9bjBYAdmlJKLvjfRS5HYbobnOQUcoMe2krCoqfW4lS\n+z9FSu2uiG82HkZn06rU2qfEwnVFN8Q3/wDJ7r3z3xZry0hFxbUvIq3cc4hr/bF5TuiKAuerO1B6\n3zQYilVAnP9kpJc34W7sKc9ZJCYUxS5tRrFLm2B/7w6Sa3dFcu1uuFfZTxavFRD2qfEou+0dOCTe\nQEyXn5FRqqbZnrtSSRc4Oeb+YuBp586n/vRXrVqFqVOnolevXhgyZAjeeOONp17V51V4ePgjQVSz\nZk1cvnzZIq8HSEC0n74T1+PyFhLudhH4sciPCFGqYUL6JCQuTgSw04R/2RRAU1RAHNreDUG7sCVo\naf9fHDLWxzvpY5AIM19J24wgvOTwN95e1A3T0l/FFmPTPD9TPbtr+LjIAjghHcPSx+GfI7WAIyEm\n/Vt7GPFrkS+wWWmET9eWgmm/U/15FsPx+/WPsf7YXcwymCvo7OGAiRjksB1vRQRht8EL0zP6Ixqm\nTz6wgxFedhfRxeEIutgfRjrssNzYFFsMgxGi1IZy2x44nA5gr5narCdDMNRhK8Yt6oJJ6cOw0djM\nLM9atbQLtr/bNk9B8SRPDYl69erht99+w5o1a/D1119jw4YNGD9+vEW6oJKTk+HiknOA19nZGamp\nqWZ/rfxT8JLD3/iPYzA+y3gRwYbWAHJ/pXYLpbHC0A4rDO1QBBmY4LgEK52mYlja+7iJwnhXYYcl\nho44ZayFWU7fw89wDp9nDEIaTO+SK44UvO24Cn0c9uLbjCAsM7SHEbn74LznuAIOMOKLjIG5PQBd\nuYPiGJo2HqucpuA2SuJ3QzuzPK8BDlhi6Ii1hpYY6/gHthYdj3kZXfGLoTvuwemx/8YRGfCzP4su\n9kfQ2eEoYpUS+NPYBCPT38U5pRry8vmxTXZYaOiMY8a6mFHkR7Qwnsa0jCGqPzet5WrgOjExEd98\n8w1+//13+Pj4YNKkSahb1wxzyjMtWLAA+/btw9y5c+8/Nm7cONSvXx9jxox54r+1ZneTfWocym57\nDw6JEYjpPAsZpevk6vWeSFFQImQOng2Zg1uBC5Fe3sN8z21j5Of8buat+U9P735SFBS7uB6l90xF\navXWiGv5UZ6674pdWI/S+z7GzQGbC0z3n2PcJVRc3Q+x7b5ASq3O5n/+hKsotf9/KBp1HPEtPkSS\ne2/Azh7ISIVLxG4Uu7QJLuF/IaNkDelKqtUVGaVtuOijmdjdu4Oy2/8PReIuIrrLz8gok/fzqaW6\nm6DkwenTp5XAwEClYcOGymeffaYkJibm5WkesXPnTqV9+/Y5HgsMDFS2bNny1H8bERGhuLu7KxER\nEWZpi6rwvYoy/TlF2fxfRUlPtdzrhK5RlC9qKcr5vyz3GrbAaFSUQ3MU5YuainLyd/Xviz6vKAt7\nKsrM5opyZX/eXy/qtLzWjRN5fw69un5U3lNX9lnuNa7sV5SfWinKd40V5ddARfm0mqLM76YoB39W\nlHgLfzZtldGoKEfmy/vuxG9Wf/mnnTtNGhFKT0/H2bNnERISgpMnTyIkJAQ3btwAACxduhQbN27E\nlClTEBAQkOsUe1Dz5s2RlpaGxYsXY+DAgVi7di1iYmLg7++fr+c1C0NGdkmJnjMsv4qyYW+gRCXg\n95eA9hMAn1cs+3p6lVUksJqfbLZzeadsbpQ1dz8tWfbjODofaP0e0HRU3hdPpcQDvw8GOn8qCyML\nmio+QL9fgBUvAy+vNW36r6mSY4GwzTIjKfYyUNoNuJk59bvLZ0AZ8w3QFjh2doDvMJlSvfIV4Mqe\nnO9xjT21u2nAgAE4e/Ys0tPTYW9vj3r16sHb2xs+Pj7w9vZG8eLFMWPGDCxZsgQTJkzAoEGD8tWg\nc+fOYcqUKQgLC4ObmxumTJmi/TqJ+AhZ++DgBPSdY93idLcvAUuDZCvT9hMLZ82nLPfuynqK60ek\nSGB8BPDnePlwdfof8GylvD+30QgsGwCUrgl0+9J8bdajf4KBvyZJ+Y78rNlJjJIpxWfWyTTZWm2A\nBj0B985SoiYtGTgwEzg4E/AaIiFemIpa5sW9u1J9+MYxoP+v5g1yFfmeAjt8+HB4eXnBx8cHjRs3\nRrFij59188svv2Dp0qU5pq9ak8VC4sxaYMM7QIs3LFdnKC1ZSniE75JVyK3ezVlSISkGWDZQ6vH0\nmqmb+dWa2T8T2DYZcC4F9J0L1G6b/+fc8SkQvgcYuq5wrFs5+BNwZJ6U7zBl+9SMNCkcGH9NSmCc\n2yCltd07y+K22gHqa1sSo6S45fktskGS9ys2UyFVMyG/STXiDpMB76EWvTi02orrkJAQDBw4EOfO\nnTPH0+Wa2UMiLVmKpV3eIYuRqvrm/zmzGNLlSuHyLinlEXlCSgvUbCP1Z079DrT+P6DJa9knrPQU\nqQOVHAsMXKJNrSOtZdwD9v8AHJgli+HCM/d56DUjfz+PcxuBTe8DI3fkrxS8rfl7qvwMX14nFx53\nbkgIxF8D4q5m/3/8NSDpltxBl3KTFdj1ugE1W8ve5qa6eQrYOkEqK3f6ROqckbroMOl+qtAACPwO\ncH7WIi9jtZBITU3F/v370b59e3M8Xa6ZNST+PS37PlT0AAK/zf8vx2iUkgKXd8mH8tpB6bOt2Ub2\nqqjePGeJgegwYPN4uQLr+nn2JiZGo9Q6urAVGLzSOhvh6MWl7dLVVK6e9HGXdpPQ+GuynOT7zZUi\ndbkVfR5Y0FXKV1R9wiJIW2fIABIjc574Y6/Iz/VegtSzKuEq3U+lqksYZP1/aTegRGXzXP0ritTq\n2vqRvEanT4CKz+X/eQuq9BQ5F1zZAwQtsMhYmc3Vbsors4SEosh2mzs/kzevqaUbHvc8sZclEC7v\nkl+wcynps63ZBqjR6um3+IoiJ78tH0rxsk6fyIcVAA7NllLEg5ZJDaGC7E6k/AxuHJfBPPfHTN88\nt0nKsfuNBvz/Y3rJjNQ7wNwAoMWbUjPKlhkNclERf/Whu4HMPyfeBIqVk/fQg0HwbBVg33dSjbfv\nL9YrkGdIl+6u3V9Jd1W7CY9WraVs/wQDm98H2nwgEznM2P3EkDBVcqzsc50QIYldLpdrHxKjsu8U\nwnfLpiM128gtea02sutVXqSnyB7LB38C/EZJTagiLhIg696UMYp6XfP23HqWmiCBvX+GdLu1eufJ\nlXQTbgCrMrvn+v7y9L2GjUapFfVMBblbtAV3bwGx4Zkn/qs5u4Xu3JBtd7Ou/O8HQWYYlKyqPpaV\nlgws7iOznzr/z7qTI5JjZaOnk8uAluMAv9eBIs7We31bcvuSdD+VdpMZli6mlZd5GoaEKTLSgB99\nZJ/rgEmmDQynxEm9+Ky7hbv/AjX8pWuoZhvZuMacH7b4COlqunFMZvI06CFX18sHyfhF0xHmey0t\nJUZJIB5fKBU/234AlDVx0dX9acq/Ar1nSqVQNbu/koKLr2zMXb+6tSmKvMcOzJTiemXr5uwGuh8C\n1fJ3ck2JA+Z3BTwHyYWItd2+JDOuok5JAca87u1R0GXcA7ZOBM5vlotZM4yVMiRMFR8BlKqm/vdp\nyVIJM+tuIeaCVASt2UbuFFyft05l0PDd0kdZvDzQ9Qspeb20P1CvC9Bhmv7q6Zvq9iVg3/cym+z5\nF4Dmb2R3r+VW+B5gzSigUZBMG354ttL5rdI9NWJH/qbNWlLGPeliODhLumaaj5WfiyX3JbkTCczr\nDLT7L+D5ouVe50nC90j3oqOzjD2Zc8JIQXJmnVSc9n8baDY2X597hkReGdLlSj2r++jGccDVIzsU\nqjbRbiqqIQM4Og/Y9QXw/ABZQPbHaCnh3Ge2bd2u3zgG7P0OuLpPupWajsz7Ps8PSroN/PE6kHwb\nCJqXPch/+xIwvzMwYInsjaA3SbdlYeCRX2SOfLOxQJ0A611VR58Hfu0uM8YeN/5jDUajdD9t/xhw\naynTQLkHy6PirsoC02LlgN4/mTaV+TEYEqbKmoEUvlvuFq4dkNv4rMFmt+a5273MGpJi5IMUthlo\nM16uwhIjgYHL8vyGsQpFkVk1+74Dbl+WNSheQ8y/iYyiyJX4nm+A7l9L99W8jkCT4RJIehIdJm09\nvUa6EpuN1W7Wz/WjwG8DZGJEtbxX4c23tCTZqOfwbMD3VZmUoLfPoNYy0oDt02R/8xE7nj4W9xgM\nCVMY0oHvPaVvOmuwuWZr81zRWkPkCZnnb8yQsZDrR4GXgoEyZtiPwZwMGcCZP6RbyZAut8oe/Sy/\neO3GcZnSrCiAWzOg98/66O9+cLwh8gTgO1wCTA9rNS78LXdiQ9cDFepr25aEG3IxdGk70O5DuaDQ\n+6ZP1hZ5AqjQME/ja/neT6JQcCgCjD1ou1cplb1k5ew/K4C/p8hMlrkdgUHLZc9lraWnACeWAAdm\nSD2qdhNkG1drjZ9U8Za9rA/9LB+m6HOyQEkrD443GDNkJ8MXFulrH/S6HYBOHwNL+gHDt+R9dp45\nlKwC9PlZwn7rR8ChOTILq7Z5yp4XCJW9LPbUDIksthoQWezt5URYvzuw60s5ES7qCfScCTTqq02b\nUuKAw3OBw3NkALLPnLwteMuvS9uBowuAUXvkyv3X7tbZT/thD483dJwq5Sz0cFfzOI0Hyn7ei/tK\nnSet9/Gu4i2z0c5tkEHbZysDjfpLvSg9d6/aOIZEQVO0hFwBer8MrB0DrHkNuLIb6PGd9dqQcEOu\nkk8skfINWnZZxF0BVo+SgoClqgFeL2VW28ysKNvjO8sXncsx3tATGPKH7awybvGmrM/4bYBUjjX3\n3uO5ZWcnYzZ1O0vlgdBVMnW2WlPpuqzfnUUEzcxG50vSU5WrC7y6VWq+nFgCfN9YZvZYUnQY8McY\n4KcW0t/++j6gz0/aBURaspRab/WOrGHJUr4eMGKbXBnPbg1cP2b+11YU4NIOmZ78a3fgGVfgjaMy\na8hWAiJLh6myVmXlKzKWpAeOTkCDQAn/d8/JlN1zG4FvPYBlL0p3XlqS1q0sEDhwXRgkRgHzOsnM\np2ZvAG3eN+8V4bVDMlPp+hGZjttkuPZdE4oiBRHt7GRasFqXTtZ885ZvydqM/I6TZI03HJgJKAZZ\n39DoBdualvw4hnRg+YuZ0y1n6beLLDVByrSErpLFh3UC5A6jTkfb/x1YCGc3kci4BwQPB67ulYVK\nnT+VPSry+mFXFLnd3/utLMJq8aZ05ehl8PXALJlr/+qWpwdi/DX52Tg/KzOf8lJDKCkmc7xhrow3\nNB+r7/GGvEhLAhb1Aip5yniKTjbFUZUcC5xdJ4Fx8yTg3hXw6AvUaqfvVfZW9rRzJ7ubCgvHosCA\nxbI+QFFk/4SFPaTibW4Y0oGTy6VLadvH8nxvHpeyIHoJiPA9El4Dlph2x1SqOjBsk6yan91a1smY\nKjoMWP8W8KO3hM2QP4Aha6QkSEEKCEBC4cUVQHIM8IO3BKJeup8ep1gZ2dFx6Hpg7GEZ+N7zDTC9\nntQ9u7xTpmXTE3HgujCxswPafyQnxb+nyjTUhT3ldrzdf5+8J0NaEnB8kXSjlK4hg+N6vFKOjwBW\nDZcdBHNT1sOhiKzsrdlKuqm8h0jFzceVx1YUOcEcnAVEhkj32hvHCkcV02JlZMe0yBPAtmlSgLH9\nR1JrSc8lYUq4SoFMv1HyHjnzh0wXT7gBPNdL7jCqNdP3MWiE3U2F1cW/5WQYMBm4GSJ7E7ebILOi\nHlyolHRbprAemSv7Ffu/LdVC9Sg9FVjQRbrR8lOk7u4tqf2Uliz7VGTV9Cqo4w35cXmXnGyN6UDA\nFOuWEDGH25eA06uB0DVAary8dzz6ApW9n34cMRekGGVilITQs5Vk340SrjI9t0QlmWml858HxyRI\nXdQ/MrWx6UipXvvnB0B6MtD1K3nD758hu+Q91wtoMS735dOtSVGAtWOl/UEL8v/BNBozd8GbIeM3\ncVcyxxs8Mscb2uv+w281iiIXGdumSf2wDpO1LeeRV7fOSnmL0FVyEdCwrwRGRY/s37WiSJ2x/TNk\nokaT4fL3iVEyMeTOTdm7I/GmPGbMkNC4Hx4PB4mrhImGWxJzxTWpc20EDP9LpmnGXwVeXi+34Stf\nATJS5K5i7CF5I+vdkbnS9fPaX+Y5edvby12TW0tg4zuy8ZMtrW+wJjs7KbNfrxtw8jd5/1TyBAIm\naruyPbe3KtW4AAARUUlEQVQqNADaT5DSH1GnJCyWvSh3is/1Boo+C4QGA2l35UKh/4Knj8PdS8wM\nkJuZARIpn7VrB7Ifv/uvrG8qUUm+nq30wP9Xzg6ZYmU16Q7jnQTJtMEVQwEHJyBovnQ3GQ3mL7hn\nKVcPyAZCw7fqr15VYZSeIqG99zupJNv2A9ut4pqaAGz/FAhZIt2NJSoDPkOBRv3Mt32w0SjVih++\nE7kTmR0kiTclcJ6pmBkgD3RplagkZVOqN89TiPBOgp7OuaTsmb3+beDXbjKDxRbuHgD5IAUPk1LJ\nDAh9KOKSvSXsvh9kxljjQUCrd22naGbCdan1dWKJdC0OXS93k9cOyB3GLwEyMcKjnxxbftYF2dvL\npIdnystrqMm4l92NdScyOzyi/pEZZ71mWqTGFkOChEMRWQ28+yspDtjnJ6B6C33P9si4B6x4Wabh\n1u2odWvoYc4lpcup6Uh5X83wle1Jm4/Rb620yBAZh7rwF+A5GBi1O+ddUA1/+er6ldQB+2el3DF1\n+kQ2hbLkOJVjUbl7MdcdjKkva9VXI32zs5PV2KVrAhvfA1JipZ+5QSBQo7X+FiBtHi+3363e1bol\n9CQlKsp+Hs3HyPqcH7zld+Y7TNMB2/uMRuDiX7KXfOxlwG800H36k2tAOTjKTK46AbJx1rq3ZJJH\n4DdWP4lbGkOCHvV8f/m6fUkqbu78Aoh5VUobNAiUhWJaXwkeWwhc3S81mDjLyDaUqSVTim+ekv0h\nDs6UadeN+muzP0R6qpzYD8yQKgQt3pQpsLnd36SKDzByh4TMnHYSgH6jH7/GxgZx4JpMkxgFhG0C\nzm4AIg7LmokGgVLqwNqLyLJ2Tnv1TylkSLbpyj5ZY5F2FwiYBLh3sU7gJ92WgfUjc4HKnlKzq2Zr\n87z27UuyAv9eItDzR6DS8/l/TgvjwDWZRwlX2ULS91WZ8XHhL5kbv+UjqVVUv7uEhqVvtRP/lXGI\nnj8yIGxdjZYyIy1ss1QA2Psd0GGKbBVsCTEX5e4ldJWs/bFECfuyteV5TywBFveRemZtP9BPyZo8\nYEhQ7jmXBBoFyVd6qgzgndsAzP1OSmJnBcaDi5DMwZAuc/C9hgD1u5nveUk7dnbyu3TvDJxaIVUA\nKjSQOwtXj/w/v6LIjKT9P8odsO+rUrLdklvE2tlJWRf3zjJuNqs50ON7oFYby72mBTEkKH+KOMuH\nwb2zrK2IOCR1/ZcPlr+vHyiBUc0v//3OWyZIpdY24/PfbtIXewfAc5CscD46H1jcW6aetv0vUKZm\n7p/PkAGcXSsro1MTZPFbv3nW3TTpmQqy4C7sT9lnpVZbqXmmdRn9XGJIkPnYO8hYhVsLmRL472m5\nw9j8viwSqtdVdhWr2Sb39Y5CfpN6UyO263taLuWPY1Gg2evSTXNgJvBLOxnYbv1/pl3930sEji+W\nmkolqwKt35NxMy3fM/W6SNfatmlyV9HlUyn5YSMTLjhwTdYRd0XuMM5tBKJCZRP7+oGAe6enbzcZ\neQJY0g94ZZN2u9yRNu5GA3umA6eWA01GAC3eePz7JeFG9uK3Wm2A5m8CVXVYiDLiMLBunKy96D49\nu3ikhrifBOlD6Rpyyz9sEzDuuEyjDQ0GvmkILO4LHJknM6gelhQD/P4yEPgtA6IweqY80PVzYOQu\nWQX9o490IaWnyt/fPCXjGD+1kGJ6I3dKKXM9BgQghQ9H7Qaq+spK9EOzpZtWx9jdRNZXvJwM7HkP\nke6Bi9ukW2rbNKCce+bAdw+glJuU3GgUJLNRqPAq7SZVAP49k7nG4icZq7h9SfaI6Pol4FJK61aa\nxtFJFq0+1xtYP05Wbff4QbfFIxkSpK2iJYCGveUrIw24skcCY0Hm7KWKDWVTGyJATqSDlkm3TcJ1\n6bLUWyUAU5V3ly7U478CCwMB3+GyEE9n+5Porrtp1qxZaNu2LXx9fTFkyBCcP39e6yaRtTg6SZmD\nwG+Bd84Cg1fIlqtarMYlfavWVGZC2WpAZLG3l2m5o/cCt84AP/tLJQEd0VVIrF69GmvXrsXixYtx\n8OBBNG/eHKNGjYLRaNS6aWRt9vZSEdOpuNYtIbK8ZysDA5fK+pDgV6Uic2qC1q0CoLOQiIuLw+jR\no1GtWjU4Ojri5ZdfRmRkJKKiHjOgSURU0DzXExhzUP5/ZjPgzDpt2wMNxiQyMjKQnJz8yOP29vYY\nPnx4jse2b9+OUqVKwdXVRvY2ICLKL5dSQI/vpLbV+szqst2+lh3rNGD1kDh8+DCGDRv2yONVqlTB\n9u3bc3zf5MmTMW3aNNhz8RQRFTY1WspYxZ6vgZ9bSsVcn2FWXxho9ZBo0aIFwsLCnvg9f/zxB6ZO\nnYqJEyeiR48eVmoZEZHOFHGW2X0N++acLlve3WpN0N0l+syZM/HZZ59h1qxZ6Nu3r9bNISLSXsXn\ngFe3SFjM7wzs+lKmjFuBrkJi1apVWLhwIX777Tc0b26hcsFERLbI3gHwGwmM3iO74c1uLetFLExX\ni+nmzJmDpKQkBAUF5Xg8ODgYtWvX1qhVREQ6UrIqMGg5cHo18PtLUo0gYJLFdovUVUhs2bJF6yYQ\nEemfnR3g0Q+o1Q74a6JMlx2xTTYHMzNdhQQREeVCsTJAr5lSWblYWYu8BEOCiMjWmWMXPxW6Grgm\nIiJ9YUgQEZEqhgQREaliSBARkSqGBBERqWJIEBGRKoYEERGpYkgQEZEqhgQREaliSBARkSqGBBER\nqWJIEBGRKoYEERGpYkgQEZEqhgQREaliSBARkSqGBBERqWJIEBGRKoYEERGpYkgQEZEqhgQREali\nSBARkSqGBBERqWJIEBGRKoYEERGpYkgQEZEqhgQREaliSBARkSqGBBERqWJIEBGRKoYEERGp0m1I\nBAcHw8/PT+tmEBEVaroMiYiICHz++edaN4OIqNDTXUgYDAa8//77GDBggNZNISIq9KweEhkZGbhz\n584jX3fv3gUAzJkzB3Xr1kXr1q2t3TQiInqIo7Vf8PDhwxg2bNgjj1epUgU//PAD1q1bh+DgYISG\nhlq7aURE9BCrh0SLFi0QFhb2yOOpqakICgrCJ598guLFi1u7WURE9BhWDwk1oaGhiIiIwKhRowDI\n2ERKSgp8fX2xbt06VK5cWeMWEhEVProJCV9fX5w8efL+nw8dOoRx48bh0KFDGraKiKhw093sJiIi\n0g/dhoSfnx/vIoiINKbbkCAiIu0xJIiISBVDgoiIVDEkiIhIFUOCiIhUMSSIiEgVQ4KIiFQxJIiI\nSBVDgoiIVDEkiIhIFUOCiIhUMSSIiEgVQ4KIiFQxJIiISBVDgoiIVOlmZ7r8MhgMAICoqCiNW0JE\nZDuyzplZ59CHFZiQiI6OBgAMHjxY45YQEdme6OhouLm5PfK4naIoigbtMbvU1FSEhoaifPnycHBw\n0Lo5REQ2wWAwIDo6Gh4eHnB2dn7k7wtMSBARkflx4JqIiFQxJIiISBVDgoiIVDEkiIhIFUOCiIhU\nMSSIiEgVQ4KIiFQVqpA4c+YMgoKC4OnpiV69eiEkJOSx37dhwwYEBATA09MTo0aNQkxMjJVbaj6m\nHvOKFSvQqVMneHt7o1+/fjh69KiVW2o+ph5zlgMHDqB+/fpISkqyUgvNz9RjPnr0KPr06QMvLy/0\n6NEDBw4csHJLzcPU4125ciUCAgLg4+ODgQMHIjQ01MotNb9Tp07B399f9e/Nfv5SConU1FSlVatW\nytKlS5W0tDRl5cqVSrNmzZS7d+/m+L6zZ88q3t7eSkhIiJKSkqJ8+OGHymuvvaZRq/PH1GM+cOCA\n4ufnp5w5c0YxGAzK6tWrFR8fHyU2NlajluedqcecJT4+Xmnbtq3i7u6u+j16Z+oxR0VFKb6+vsqf\nf/6pGI1GZf369YqPj4+SkpKiUcvzJjef5aZNmyqXL19WDAaDMnv2bKV9+/YatTr/jEajsnLlSsXH\nx0dp2rTpY7/HEuevQnMncfDgQdjb2+PFF19EkSJFEBQUhHLlymHXrl05vm/9+vUICAhA48aN4ezs\njPfeew979uyxybsJU485KioKw4cPR4MGDWBvb48+ffrAwcEBFy9e1KjleWfqMWeZMmUKunXrZuVW\nmpepx7x27Vq0aNECnTt3hp2dHQIDA7Fw4ULY29vWacDU47169SqMRiMMBgMURYG9vf1jy07Yip9/\n/hmLFi3C6NGjVb/HEucv23p35EN4eDhq166d47GaNWvi8uXLOR67fPky6tSpc//PpUuXRsmSJREe\nHm6VdpqTqcfcu3dvjBgx4v6fjx07hqSkpEf+rS0w9ZgBYN26dbhz5w4GDRpkreZZhKnHfPr0aVSs\nWBFjx46Fn58fBgwYAIPBACcnJ2s2N99MPV5/f3/UqFED3bt3R6NGjTB79mx8/fXX1myqWfXr1w9r\n165Fo0aNVL/HEuevQhMSycnJcHFxyfGYs7MzUlNTczyWkpLyyNWGi4sLUlJSLN5GczP1mB908eJF\njBs3DuPGjUOZMmUs3USzM/WYIyMj8f333+PTTz+1ZvMswtRjTkhIwMqVKzFo0CDs3bsXPXv2xMiR\nI5GQkGDN5uabqcd779491KlTB8HBwThx4gSGDh2KN95444nvfz2rUKEC7Ozsnvg9ljh/FZqQcHFx\neeTNkZqaimLFiuV4TC04Hv4+W2DqMWfZu3cvBg0ahMGDB2PkyJHWaKLZmXLMRqMR48ePx3/+8x9U\nrFjR2k00O1N/z05OTmjdujX8/f1RpEgRDB48GMWKFcPx48et2dx8M/V4Z8yYAVdXVzRq1AhFixbF\n2LFjkZ6ejv3791uzuVZlifNXoQmJWrVqPXLLFR4enuPWDABq166d4/tiY2ORkJBgk10vph4zAKxa\ntQrjxo3D5MmTMWbMGGs10exMOeaoqCicPHkSU6ZMga+vL3r27AkAaNOmjU3O6jL191yzZk2kpaXl\neMxoNEKxsULQph5vZGRkjuO1s7ODg4NDgd5KwCLnr3wNe9uQe/fuKf7+/sqiRYtyzIhISkrK8X1n\nzpxRvL29lSNHjiipqanKhAkTlBEjRmjU6vwx9Zj379+vNGrUSDly5IhGLTUfU4/5QRERETY9u8nU\nYz59+rTi4eGh7NixQzEYDMqiRYueOPNLr0w93mXLlilNmzZVQkNDlfT0dGX+/PlKq1atlMTERI1a\nbh4HDx5Und1kifNXoQkJRZHpYQMGDFA8PT2VXr16KSdOnFAURVEmTpyoTJw48f73bdy4UenUqZPi\n5eWljBgxQomJidGqyflmyjEPGzZMqV+/vuLp6Znja9euXVo2Pc9M/T1nsfWQUBTTj3nPnj1Kr169\nFE9PT6VPnz5KSEiIVk3OF1OO12g0KrNnz1batWun+Pj4KC+99JISFhamZbPN4uGQsPT5i5sOERGR\nqkIzJkFERLnHkCAiIlUMCSIiUsWQICIiVQwJIiJSxZAgIiJVDAkiIlLFkCAiIlUMCSIiUsWQILKQ\nq1evomHDhvj+++9zPD558mR4eXnhn3/+0ahlRKZjSBBZiJubG4KCgrBw4ULExcUBkPLVq1atwsyZ\nM5+4eQyRXrB2E5EF3bp1C506dcKgQYNQq1YtTJo0CdOnT7f5LVOp8HDUugFEBVmFChUwdOhQzJ8/\nHwaDAR999BEDgmwKu5uILMzNzQ1paWnw9PTE4MGDtW4OUa4wJIgs6MCBA5g0aRK8vLxw/PhxnDt3\nTusmEeUKQ4LIQk6fPo2xY8eif//+WLRoESpXroxvvvlG62YR5QpDgsgCrl69ihEjRsDf3x8TJ06E\nk5MTxo4di127duHIkSNaN4/IZJzdRGRm0dHRGDhwICpXrox58+bByckJAGAwGBAYGIiSJUti+fLl\nGreSyDQMCSIiUsXuJiIiUsWQICIiVQwJIiJSxZAgIiJVDAkiIlLFkCAiIlUMCSIiUsWQICIiVf8P\nUvdKOfvP04QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a1d7f2f90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# To gaurantee reproducibility\n",
    "np.random.seed(123456)\n",
    "\n",
    "# Number of test points\n",
    "num_test = 10\n",
    "\n",
    "# Pick a covariance function\n",
    "k = GPy.kern.RBF(input_dim=1, variance=1., lengthscale=.1)\n",
    "\n",
    "# Pick a mean function\n",
    "mean_func = lambda(x): np.zeros(x.shape)\n",
    "\n",
    "# Pick a bunch of points over which you want to sample the GP\n",
    "X = np.linspace(0, 1, num_test)[:, None]\n",
    "\n",
    "# Evaluate the mean function at X\n",
    "m = mean_func(X)\n",
    "\n",
    "# Compute the covariance function at these points\n",
    "nugget = 1e-6 # This is a small number required for stability\n",
    "C = k.K(X) + nugget * np.eye(X.shape[0])\n",
    "\n",
    "# Compute the Cholesky of the covariance\n",
    "# Notice that we need to do this only once\n",
    "L = np.linalg.cholesky(C)\n",
    "\n",
    "# Number of samples to take\n",
    "num_samples = 3\n",
    "\n",
    "# Take 3 samples from the GP and plot them:\n",
    "fig, ax = plt.subplots()\n",
    "# Plot the mean function\n",
    "ax.plot(X, m)\n",
    "for i in xrange(num_samples):\n",
    "    z = np.random.randn(X.shape[0], 1)    # Draw from standard normal\n",
    "    f = m + np.dot(L, z)                  # f = m + L * z\n",
    "    ax.plot(X, f, color=sns.color_palette()[1], linewidth=1)\n",
    "#ax.set_ylim(-6., 6.)\n",
    "ax.set_xlabel('$x$', fontsize=16)\n",
    "ax.set_ylabel('$y$', fontsize=16)\n",
    "ax.set_ylim(-5, 5);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The solid line is the mean function and the dashed lines are 3 samples of   f . These don’t look like functions yet. This is because we have used only 10 test points to represent the GP."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Questions\n",
    "\n",
    "1. Edit the code above changing the number of test points ``num_test`` to 20, 50, 100. Rerun the example. How do your samples of   f look like now? Do they look more like functions to you? Imagine that the true nature of the GP appears when these test points become infinitely dense.\n",
    "\n",
    "2. Edit the code above and change the random seed to an arbitrary integer (just make up one). Rerun the example and notice how the sampled functions change.\n",
    "\n",
    "3. Edit the code above and change the variance first to 0.1 and then to 5 each time rerunning the example. Notice the values on the vertical axis of the plot. What happens to the sampled functions as you do this? What does the variance parameter of the SE control?\n",
    "\n",
    "4. Edit the code above and now change the length-scale parameter first to 0.05 and then to 1. What happens to the sampled functions as you do this? What does the length- scale parameter of the SE control?\n",
    "\n",
    "5. Now set the variance and the length-scale back to their original values (1. and 0.1, respectively). Edit the code and change the mean function to:\n",
    "```\n",
    "     mean_fun = lambda(x): 5 * x\n",
    "```\n",
    "Re-run the example. What do you observe? Try a couple more. For example, try:\n",
    "```\n",
    "     mean_fun = lambda(x): np.sin(5 * np.pi * x)\n",
    "```\n",
    "\n",
    "6. So far, all the samples we have seen are smooth. There is this theorem that says that the samples of the GP will be as smooth as the covariance function we use. Since the SE covariance is infinitely smooth, the samples we draw are infinitely smooth. The [Matern 3-2 covariance function](https://en.wikipedia.org/wiki/Matérn_covariance_function) is twice differentiable. Edit the code and\n",
    "change ``RBF`` to ``Matern32``. Rerun the example. How smooth are the samples now?\n",
    "\n",
    "7. The exponential covariance function is continuous but not differentiable. Edit the code and change ``RBF`` to ``Exponential``. Rerun the example. How smooth are the samples now?\n",
    "\n",
    "8. The covariance function can also be used to model invariances. The periodic exponential covariance function is... a periodic covariance function. Edit line 29 and change ``RBF`` to \n",
    "```\n",
    "k = GPy.kern.PeriodicMatern32(input_dim=1, variance=500., lengthscale=0.01, period=0.1)\n",
    "```\n",
    "Rerun the example. Do you notice the periodic pattern?\n",
    "\n",
    "9. How can you encode the information that there are two lengthscales in $f(\\cdot)$. There are many ways to do this.\n",
    "Try summing or multiplying covariance functions."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  },
  "latex_envs": {
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 0
  },
  "widgets": {
   "state": {
    "9d75ca2df1164d908aba9f67229246d2": {
     "views": [
      {
       "cell_index": 14
      }
     ]
    },
    "abdf17dbd32e43eea86365f2f2a8bd34": {
     "views": [
      {
       "cell_index": 13
      }
     ]
    }
   },
   "version": "1.2.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}