lab1/Part1_tensorflow_solution.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Part1_tensorflow_solution.ipynb",
      "version": "0.3.2",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 2",
      "language": "python",
      "name": "python2"
    }
  },
  "cells": [
    {
      "metadata": {
        "id": "57knM8jrYZ2t",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<table align=\"center\">\n",
        "  <td align=\"center\"><a target=\"_blank\" href=\"http://introtodeeplearning.com\">\n",
        "        <img src=\"http://introtodeeplearning.com/images/colab/mit.png\" style=\"padding-bottom:5px;\" />\n",
        "      Visit MIT Deep Learning</a></td>\n",
        "  <td align=\"center\"><a target=\"_blank\" href=\"https://colab.research.google.com/github/aamini/introtodeeplearning_labs/blob/master/lab1/Part1_tensorflow_solution.ipynb\">\n",
        "        <img src=\"http://introtodeeplearning.com/images/colab/colab.png?v2.0\"  style=\"padding-bottom:5px;\" />Run in Google Colab</a></td>\n",
        "  <td align=\"center\"><a target=\"_blank\" href=\"https://github.com/aamini/introtodeeplearning_labs/blob/master/lab1/Part1_tensorflow_solution.ipynb\">\n",
        "        <img src=\"http://introtodeeplearning.com/images/colab/github.png\"  height=\"70px\" style=\"padding-bottom:5px;\"  />View Source on GitHub</a></td>\n",
        "</table>\n",
        "\n",
        "# Lab 1: Intro to TensorFlow and Music Generation with RNNs\n",
        "# Part 1: Intro to TensorFlow"
      ]
    },
    {
      "metadata": {
        "id": "OhuYRQfjYZ2v",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## 0.1 Install TensorFlow\n",
        "\n",
        "TensorFlow is a software library extensively used in machine learning. Here we'll learn how computations are represented and how to define a simple neural network in TensorFlow.\n",
        "\n",
        "Let's install TensorFlow and a couple of dependencies: \n"
      ]
    },
    {
      "metadata": {
        "id": "LkaimNJfYZ2w",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import tensorflow as tf\n",
        "tf.enable_eager_execution()\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "3oWpEMtmYZ3I",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We'll then check to make sure things installed properly:"
      ]
    },
    {
      "metadata": {
        "id": "zLLaY8hvdbvQ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "is_correct_tf_version = '1.13.' in tf.__version__\n",
        "assert is_correct_tf_version, \"Wrong tensorflow version {} installed\".format(tf.__version__)\n",
        "\n",
        "is_eager_enabled = tf.executing_eagerly()\n",
        "assert is_eager_enabled,      \"Tensorflow eager mode is not enabled\""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "vDJGsR2NoYtu",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "TensorFlow is set to release the next major version of TensorFlow, [TensorFlow 2.0](https://www.tensorflow.org/community/roadmap#tensorflow_20_is_coming), very soon. In this set of labs we'll be working in TensorFlow 1.12.0. The 6.S191 team is **Eager** to show you this version, as it features a (relatively) new imperative programming style called Eager execution. Under Eager execution, TensorFlow operations execute immediately as they're called from Python (which wasn't always the case!). This allows for fast debugging and a more intuitive way to get started with TensorFlow.\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "iD3VO-LZYZ2z",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## 1.1 The computation graph\n",
        "\n",
        "TensorFlow is called TensorFlow because it handles the flow (node/mathematical operation) of Tensors (data), which you can think of as multidimensional arrays. In TensorFlow, computations can be thought of as graphs. First, we'll explore defining a computational graph with Tensors and mathematical operations before diving in to how we can build deep learning models in TensorFlow. \n",
        "\n",
        "Let's look at a simple example, and define this computation using TensorFlow:\n",
        "\n",
        "![alt text](img/add-graph.png \"Computation Graph\")\n",
        "\n",
        "<!-- Keras is a high-level API to build and train deep learning models. It's used for fast prototyping, advanced research, and production, with three key advantages:\n",
        "\n",
        "User friendly\n",
        "Keras has a simple, consistent interface optimized for common use cases. It provides clear and actionable feedback for user errors.\n",
        "Modular and composable\n",
        "Keras models are made by connecting configurable building blocks together, with few restrictions.\n",
        "Easy to extend\n",
        "Write custom building blocks to express new ideas for research. Create new layers, loss functions, and develop state-of-the-art models. -->\n",
        "<!-- \n",
        "TensorFlow programs are usually structured into a phase that assembles a graph, and a phase that uses a session to execute operations in the graph. In TensorFlow we define the computational graph with Tensors and mathematical operations to create a system for machine learning and deep learning.\n",
        "\n",
        "We can think of a computation graph as a series of math operations that occur in some order.  -->\n"
      ]
    },
    {
      "metadata": {
        "id": "X_YJrZsxYZ2z",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "3465850f-20ad-4a90-aec1-2c3e382bc0c0"
      },
      "cell_type": "code",
      "source": [
        "# Create the nodes in the graph, and initialize values\n",
        "a = tf.constant(15, name=\"a\")\n",
        "b = tf.constant(61, name=\"b\")\n",
        "\n",
        "# Add them!\n",
        "c = tf.add(a,b, name=\"c\")\n",
        "print(c)"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tf.Tensor(76, shape=(), dtype=int32)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "mjYCF0EdYZ22",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Notice how we've created a computation graph consisting of TensorFlow operations, and how  the output is a Tensor with value 76 -- we've just created a computation graph consisting of operations, and it's executed them and given us back the result. That's because of Eager!"
      ]
    },
    {
      "metadata": {
        "id": "Mbfv_QOiYZ23",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Building a computation graph\n",
        "\n",
        "Now let's consider a slightly more complicated computation graph:\n",
        "![alt text](img/computation-graph.png \"Computation Graph\")\n",
        "\n",
        "This graph takes two inputs, `a, b`, and computes an output `e`. Each node in the graph is an operation that takes some input, does some computation, and passes its output to another node.\n",
        "\n",
        "Let's define a simple function in TensorFlow to construct this computation graph:"
      ]
    },
    {
      "metadata": {
        "scrolled": true,
        "id": "PJnfzpWyYZ23",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Construct a simple computation graph\n",
        "def graph(a,b):\n",
        "  '''TODO: Define the operation for c, d, e (use tf.add, tf.subtract, tf.multiply).'''\n",
        "  c = tf.add(a, b)\n",
        "  d = tf.subtract(b, 1)\n",
        "  e = tf.multiply(c, d)\n",
        "  return e"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "AwrRfDMS2-oy",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now, we can call this function to execute the computation graph given some inputs `a,b`:"
      ]
    },
    {
      "metadata": {
        "id": "pnwsf8w2uF7p",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "23191b02-eef8-4d89-8b6f-f11d02a95a98"
      },
      "cell_type": "code",
      "source": [
        "# Consider example values for a,b\n",
        "a, b = 1.5, 2.5\n",
        "# Execute the computation\n",
        "e_out = graph(a,b)\n",
        "print e_out"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tf.Tensor(6.0, shape=(), dtype=float32)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "6HqgUIUhYZ29",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Again, notice how our output is a Tensor with value defined by the output of the computation (thanks to Eager!)."
      ]
    },
    {
      "metadata": {
        "id": "1h4o9Bb0YZ29",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## 1.2 Neural networks in TensorFlow\n",
        "We can also define neural networks in TensorFlow, and it's often helpful to think about this using the idea of computation graphs. TensorFlow uses a high-level API called [Keras](https://www.tensorflow.org/guide/keras) that provides a powerful, intuitive framework for building and training deep learning models. In the 6.S191 labs we'll be using the Keras API to build and train our models.\n",
        "\n",
        "Let's consider this example of a very simple neural network of just one dense layer:\n",
        "\n",
        "![alt text](img/computation-graph-2.png \"Computation Graph\")\n",
        "\n",
        "This graph takes an input `x` and computes an output `out`. It does so how we learned in lecture today: `out = sigmoid(W*x+b)`.\n",
        "\n",
        "First, let's define this computation graph in TensorFlow via a simple function, as we did before:"
      ]
    },
    {
      "metadata": {
        "id": "ToJIeFqNcLAR",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# n_in: number of inputs\n",
        "# n_out: number of outputs\n",
        "def our_dense_layer(x, n_in, n_out):\n",
        "  # Define and initialize parameters, a weight matrix W and biases b\n",
        "  W = tf.Variable(tf.ones((n_in, n_out)))\n",
        "  b = tf.Variable(tf.zeros((1, n_out)))\n",
        "  \n",
        "  '''TODO: define the operation for z (hint: use tf.matmul)'''\n",
        "  z = tf.matmul(x,W) + b\n",
        "  \n",
        "  '''TODO: define the operation for out (hint: use tf.sigmoid)'''\n",
        "  out = tf.sigmoid(z)\n",
        "  return out"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "OgSBEuEtwb2e",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "As before, we can define an example input, feed it into `our_dense_layer` function, and immediately execute:"
      ]
    },
    {
      "metadata": {
        "id": "PSI3I0CFcxnv",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 105
        },
        "outputId": "13722bb9-083b-4289-b30c-76118951c20b"
      },
      "cell_type": "code",
      "source": [
        "'''TODO: define an example input x_input'''\n",
        "x_input = tf.constant([[1,2.]], shape=(1,2))\n",
        "'''TODO: call `our_dense_layer` to get the output of the network!'''\n",
        "print our_dense_layer(x_input, n_in=2, n_out=3)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/resource_variable_ops.py:642: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Colocations handled automatically by placer.\n",
            "tf.Tensor([[0.95257413 0.95257413 0.95257413]], shape=(1, 3), dtype=float32)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "Jt1FgM7qYZ3D",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now, instead of explicitly defining a simple function, we'll use the Keras API to define our neural network. This will be especially important as we move on to more complicated network architectures. \n",
        "\n",
        "Specifically, for this network we'll use the Keras [`Sequential`](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/Sequential) model from the `tf.keras` API to define our network. The `tf.keras.Sequential` model lets us conveniently define a linear stack of network layers. We'll use [`tf.keras.layers.Dense` ](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/layers/Dense) to define our single fully connected network layer. "
      ]
    },
    {
      "metadata": {
        "id": "7WXTpmoL6TDz",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Import relevant packages\n",
        "from tensorflow.keras import Sequential\n",
        "from tensorflow.keras.layers import Dense\n",
        "\n",
        "# Define the number of inputs and outputs\n",
        "n_input_nodes = 2\n",
        "n_output_nodes = 3\n",
        "\n",
        "# First define the model \n",
        "model = Sequential()\n",
        "\n",
        "\n",
        "'''TODO: Define a dense (fully connected) layer to compute z'''\n",
        "# Remember: dense layers are defined by the parameters W and b!\n",
        "# You can read more about the initialization of W and b in the TF documentation :) \n",
        "dense_layer = Dense(n_output_nodes, input_shape=(n_input_nodes,),activation='sigmoid')\n",
        "\n",
        "# Add the dense layer to the model\n",
        "model.add(dense_layer)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "HDGcwYfUyR-U",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "That's it! We've defined our model. Now, we can test it out using an example input:"
      ]
    },
    {
      "metadata": {
        "id": "sg23OczByRDb",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "f40aacac-fe8a-453c-dd40-deba8777a2ef"
      },
      "cell_type": "code",
      "source": [
        "# Test model with example input\n",
        "x_input = tf.constant([[1,2.]], shape=(1,2))\n",
        "\n",
        "'''TODO: feed input into the model and predict the output!'''\n",
        "print model(x_input)"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tf.Tensor([[0.31025296 0.48313126 0.7821198 ]], shape=(1, 3), dtype=float32)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "dQwDhKn8kbO2",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## 1.3 Automatic differentiation\n",
        "\n",
        "[Automatic differentiation](https://en.wikipedia.org/wiki/Automatic_differentiation)\n",
        "is one of the most important parts of TensorFlow and is the backbone of training with \n",
        "[backpropagation](https://en.wikipedia.org/wiki/Backpropagation). During Eager execution, use `tf.GradientTape` to trace operations for computing gradients later. \n",
        "\n",
        "All forward-pass operations get recorded to a \"tape\"; then, to compute the gradient, the tape is played backwards and then discarded. A particular `tf.GradientTape` can only\n",
        "compute one gradient; subsequent calls throw a runtime error.\n",
        "\n",
        "Let's take a look at a simple example! We can use automatic differentiation and stochastic gradient descent (SGD) to find the minimum of $y=(x-1)^2$. While we can clearly solve this problem analytically ($x_{min}=1$), solving this simple example sets us up nicely for future labs where we use gradient descent to optimize entire neural network losses. "
      ]
    },
    {
      "metadata": {
        "attributes": {
          "classes": [
            "py"
          ],
          "id": ""
        },
        "colab_type": "code",
        "id": "7g1yWiSXqEf-",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 395
        },
        "outputId": "a1f9e6db-84e7-459a-bc4c-a8486f298f52"
      },
      "cell_type": "code",
      "source": [
        "x = tf.Variable([tf.random.normal([1])])\n",
        "print \"Initializing x={}\".format(x.numpy())\n",
        "learning_rate = 1e-2\n",
        "history = []\n",
        "\n",
        "for i in range(500):\n",
        "  with tf.GradientTape() as tape:\n",
        "    y = (x - 1)**2 # record the forward pass on the tape\n",
        "\n",
        "  grad = tape.gradient(y, x) # compute the gradient of y with respect to x\n",
        "  new_x = x - learning_rate*grad # sgd update\n",
        "  x.assign(new_x) # update the value of x\n",
        "  history.append(x.numpy()[0])\n",
        "\n",
        "plt.plot(history)\n",
        "plt.plot([0, 500],[1,1])\n",
        "plt.legend(('Predicted', 'True'))\n",
        "plt.xlabel('Iteration')\n",
        "plt.ylabel('x value')"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Initializing x=[[1.5673906]]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0,0.5,'x value')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe8AAAFYCAYAAAB6RnQAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8E3X+P/DXJJM0TZuWHklPWmop\n5SiHRUSUQ8oh6724ggco64GKiPjVVUQQvoIHXj9XXB8qivoFFFlglQUFl11ZEAGBIrRcpVDo3ab3\nkd6Z3x895OidpJNJXs/Ho4ZkJpN33yqvzHxmPiNIkiSBiIiIFEMldwFERETUNQxvIiIihWF4ExER\nKQzDm4iISGEY3kRERArD8CYiIlIYUe4COstsLrfr9vz89Cgutth1m+6IfbQde2g79tB27KHtHNFD\no9HQ6utuu+ctimq5S3AJ7KPt2EPbsYe2Yw9t15M9dNvwJiIiUiqGNxERkcIwvImIiBSG4U1ERKQw\nDG8iIiKFYXgTEREpDMObiIhIYRjeRETkUnJysjFp0ljMnTsbc+fOxuzZs/Df//7U5e1s2vQNPvvs\nY5w5cxqfffZxm+v9/PN/UVdX16ltnjuXirlzZ3e5lsspZoY1IiKizoqIiMQHH3wCACgrK8Wf/3w/\nrrtuFDw8dF3eVkxMLGJiYttcvn79OsTHj+h2rd3B8CYiIpfm4+OLgIBAvPXW69BotCgrK8Err7yB\nN998FdnZWaivr8cjjzyO4cNH4NChX/H+++/A3z8AAQGBCA0NQ2LiIWzevAHLl7+J7du3YePGbyAI\nAu65537U1dXhxIlkPPfcPKxbtwabNm3Azp3bIQgqjBlzI+69dwby8/OwePECaDQa9O3bzy6/k1uG\nd01dA/5zKB39w3yg4ZSAREQOseE/qTh4Kt+u2xzR34RpCX279J6cnGyUlZXCarXCx8cHL7zwErZv\n34aAgEC8+OLLKCkpwdNPP44vv1yPjz/+AIsXL0NMTD8899w8hIaGtWzHYqnEF198ii+//Bq1tXV4\n9dUleOONd/Hppx/h7bffR15eHnbt+jc+/PAzAMATTzyM8eMnYvPmbzBhwmRMm3Yv1q79AqmpKTb3\nwS3DO+lsIT78NhkP3TwAo4eEyF0OERHZWXr6hZaxZa1Wi0WL/hfffbcZAwcOAgAkJx/D0aNHcOzY\nbwCAmpoa1NXVIScnBzExjXvHw4bFo6ampmWb58+nISKiDzw8dPDw0OGNN9695DOTkpKQmZmBp556\nDEBj2OfmZuP8+TSMHz8RAHD11ddg//5fbP793DK8fb21AIDsgkqZKyEicl3TEvp2eS/ZXi4e8272\n3XebIYoaAIAoavDAAw9h0qQpl6yjUv1+HrckSZctU0OSrG1+pkajwahRN+D551+65PV1676EIKia\nttn2+7vCLc82D/bXAwByi3j7OyIidzRwYBx+/vm/AIDi4iJ8/PHfAACBgUakp5+HJEk4cuTwJe+J\njOyD9PQLsFgsqKmpwfz5cyBJEgRBhYaGBgwaNAiJiYdRXV0NSZLw3ntvo6amGhERkTh16gQAIDHx\nkF3qd8s9b4NeC4Nei5xC7nkTEbmjhISJSEw8iMcffwgNDQ146KHGQ+yzZ8/BokUvIDg4BCZT0CXv\n8fT0xMMPP4758+cAAKZPvw+CIODqq+MxZ87D+OqrdZg27V48+eSjUKlUGDv2Rnh46HD33fdi8eIF\n2L37J0RHx9ilfkG6/LiAkzKby+26vbfW/4bTF4rx0XPjIKrd8gCEXRiNBrv/u3E37KHt2EPbsYe2\nc0QPjUZDq6+7bWqFm7xhlSTkF1fJXQoREVGXuG14hxm9AXDcm4iIlMd9w9vE8CYiImVy2/AObwpv\nnrRGRERK47bhHRzgBbVK4J43EREpjtuGt6hWwdjLE7mFlisuxCciInJmbnmdd7Ngfz1yiywor6qD\nj14rdzlERGSjlSv/H06fPomiokJUV1cjNDQMPj6+eO21t+Quza7cOrxDAvT4LRXILbQwvImIXMBT\nTz0DAPj++3/i3LmzmDt3vswVOYbbHjYHOE0qEZE7SEw8hOefn4+5c2fj1KmTuOWWCS3LFi16HomJ\nh2CxVGLRoufx9NNPYO7c2UhNPSNjxR1z6z3v4ICm8C5keBMR2dvm1K04kp9k121ebRqMqX1v7fL7\nzp5Nxddfb4ZW2/pR1g0bvsbIkdfjttvuRFraOfz1r2/jvfc+tLVch3Hr8A4J8ALAy8WIiFxd374x\nbQY3ACQlHUNJSTF27PgeAFBTU91TpXWLW4e3t6cG3p4aHjYnInKAqX1v7dZesiNoNJpWX6+vr29a\nLuKZZ/6CuLghPVlWt7n1mDfQeOjcXFKN+gb73GOViIicmyAIqK6uRnV1NVJSTgNovEXo7t27AABp\naeewfv1aGSvsmFvveQONJ62lZpYiv7gKoYFecpdDREQOduedf8Ls2Q+iT5+rEBs7AADwpz9Nx6uv\nLsWcOY/AarVi/vznZK6yfW57S9DmW7f9cOAC/v7TWcydOhjx/Yx2/Qx3wNsI2o49tB17aDv20Ha8\nJWgP4uViRESkNG4f3i1nnBfwjHMiIlIGtw9vYy8dRLWAbF4uRkRECuH24a1WqRDsr0d2gQVWZQz/\nExGRm3NoeKekpGDixIlYu/bKU+4TEhJw3333YebMmZg5cyby8vIcWUq7QgO9UFPXgKJS574on4iI\nCHDgpWIWiwXLli3DqFGj2lxn1apV8PKS//KssKZLxLILKxHYy1PmaoiIiNrnsD1vrVaLVatWwWQy\nOeoj7Kb5+u4snrRGREQK4LA9b1EUIYrtb37JkiXIysrC8OHD8eyzz0IQBEeV067m8M42M7yJiMj5\nyTbD2rx58zBmzBj4+vriySefxI4dOzBlypQ21/fz00MU1Xatofnid39/L4hqFfJLq9u8IJ7axp7Z\njj20HXtoO/bQdj3VQ9nC+84772z589ixY5GSktJueBcX23cSlctnwgn290R6bjny8sugkukIgBJx\nVibbsYe2Yw9txx7azuVnWCsvL8fDDz+M2tpaAMDBgwcRExMjRykteMY5EREphcP2vJOTk7FixQpk\nZWVBFEXs2LEDCQkJCA8Px6RJkzB27FhMnz4dHh4eGDhwYLt73T0h7KKT1njGOREROTOHhXdcXBzW\nrFnT5vIHH3wQDz74oKM+vstCL7pcbGjfQJmrISIiapvbz7DWjGecExGRUjC8m5j8PCGqBV7rTURE\nTo/h3aR5jvOcQs5xTkREzo3hfRGecU5ERErA8L5IGKdJJSIiBWB4X6TlpDWGNxEROTGG90UY3kRE\npAQM74vwjHMiIlIChvdF1CoVQgK8kF1QCauVZ5wTEZFzYnhfJtzojdp6K/JLquQuhYiIqFUM78v0\nNnkDADLzK2SuhIiIqHUM78s0h3c6w5uIiJwUw/sy4dzzJiIiJ8fwvoyvlxY+eg0yzQxvIiJyTgzv\nVvQ2eaOgtBqW6nq5SyEiIroCw7sVLYfOufdNREROiOHdinAjw5uIiJwXw7sVvFyMiIicGcO7FSEB\nXlCrBGQwvImIyAkxvFuhEVUIDtAj01wJq8RpUomIyLkwvNvQ2+SNmroGFHCaVCIicjIM7zb0bjpp\nLSOfdxgjIiLnwvBuQ/PlYhn55TJXQkREdCmGdxtazjg3c8+biIicC8O7Db5eWnh7ani5GBEROR2G\ndxsEQUBvkzfyS6pQVcNpUomIyHkwvNvRu2Xcm3vfRETkPBje7YgMNgAALuTxpDUiInIeDO929GkK\n7/RchjcRETkPhnc7gvz08NCouedNREROheHdDpVKQO8gb2QXWFBb1yB3OURERAAY3h2KDDLAKknI\n4O1BiYjISTC8OxAZxHFvIiJyLgzvDvThGedERORkGN4dCAnUQyOqcJ573kRE5CQY3h1Qq1QIN3oj\ny1yJunqr3OUQERExvDsjMtiABquE7ALepISIiOTH8O6EyKDGaVI57k1ERM6A4d0JfYJ9AAAXOO5N\nREROgOHdCaGBXlCrBJ60RkRETsGh4Z2SkoKJEydi7dq1ba7zzjvvYObMmY4sw2YaUYUwoxcy8itQ\n38CT1oiISF4OC2+LxYJly5Zh1KhRba6TmpqKgwcPOqoEu4oMMqC+wYrcQovcpRARkZtzWHhrtVqs\nWrUKJpOpzXXeeOMNPPPMM44qwa6aJ2tJyy2TuRIiInJ3DgtvURSh0+naXL5582Zce+21CAsLc1QJ\ndtUnpPGktbQcjnsTEZG8RDk+tKSkBJs3b8bnn3+OvLy8Tr3Hz08PUVTbtQ6j0dDpdXv5eUEjqpBh\nrujS+9wB+2E79tB27KHt2EPb9VQPZQnv/fv3o6ioCPfffz9qa2uRnp6O1157DQsXLmzzPcXF9h1r\nNhoNMJu7thcdYfLG+ewyZGWXQKux7xcJpepOH+lS7KHt2EPbsYe2c0QP2/oyIEt4T5kyBVOmTAEA\nZGZm4sUXX2w3uJ1FVKgPzmaXIT2vAn3DfeUuh4iI3JTDwjs5ORkrVqxAVlYWRFHEjh07kJCQgPDw\ncEyaNMlRH+tQVzWNe5/LKWN4ExGRbBwW3nFxcVizZk2H64WHh3dqPWcQFdoU3tmlAHrLWwwREbkt\nzrDWBaZenvDSiUjL4eViREQkH4Z3FwiCgKhQH5hLqlFuqZW7HCIiclMM7y66quV6b+59ExGRPBje\nXXRVy7g3w5uIiOTB8O6iqIvOOCciIpIDw7uLDHotjL10SMsugyRJcpdDRERuiOHdDVEhPqisroe5\npEruUoiIyA0xvLvhqtDGCVo47k1ERHJgeHdDy0xrDG8iIpIBw7sbIoO9IaoFpGaVyl0KERG5IYZ3\nN2hENSKDDUjPq0BNbYPc5RARkZtheHdTTFgvWCWJl4wREVGPY3h3U/NdxVIzS2SuhIiI3A3Du5v6\nhjWG9xmOexMRUQ9jeHeTj5cWQX6eOJtVBisnayEioh7E8LZB33BfVNXUI9tcKXcpRETkRhjeNogJ\n7wWAh86JiKhnMbxt0DzuzZPWiIioJzG8bRAcoIeXTsSZTO55ExFRz2F420AlCOgb5ouC0mqUVNTI\nXQ4REbkJhreNfr/em3vfRETUMxjeNmo5aY3hTUREPYThbaM+wQaoVQLO8KQ1IiLqIQxvG2k1akSF\n+CA9rwJVNfVyl0NERG6A4W0HsRGNNynhoXMiIuoJDG876B/hBwA4nV4scyVEROQOGN520DfMF2qV\ngFPpHPcmIiLHY3jbgYe2cdz7Qm45x72JiMjhGN52wnFvIiLqKQxvO+G4NxER9RSGt500j3ufzuC4\nNxERORbD206ax73P53Dcm4iIHIvhbUfN496pvL83ERE5EMPbjprHvU9x3JuIiByI4W1HLePevN6b\niIgciOFtRxz3JiKinsDwtrP+kX6wShL3vomIyGEY3nY2qE/juPfxtCKZKyEiIlfVqfBOSUnBzp07\nAQBlZWUOLUjposN84aFV4/h5hjcRETmG2NEKX3zxBbZu3Yra2lpMnDgRH374IXx8fDBnzpyeqE9x\nRLUK/Xv3wtGzhSgsrUaAr07ukoiIyMV0uOe9detWbNiwAb6+vgCA559/Hrt27erUxlNSUjBx4kSs\nXbv2imUbNmzAtGnTcM8992Dp0qWQJKlrlTuxgVH+AMC9byIicogOw9vLywsq1e+rqVSqS563xWKx\nYNmyZRg1atQVy6qqqrBt2zasW7cO69evx7lz53DkyJEulu684prC+wTDm4iIHKDDFI6IiMAHH3yA\nsrIy/Pjjj5g/fz6io6M73LBWq8WqVatgMpmuWObp6Ykvv/wSGo0GVVVVqKiogNFo7N5v4ISC/fXw\nM3jgxPliWF3oiAIRETmHDsP75ZdfhqenJ4KCgrBlyxYMHToUS5Ys6XDDoihCp2t/vPeTTz7BpEmT\nMGXKFPTu3bvzVTs5QRAwqI8/KqrqkJ5XLnc5RETkYgSpg8Fmq9Xa6uudOXQOACtXroSfnx9mzJjR\n6vLq6mo8+uijmD9/PoYPH97mdurrGyCK6k59pjPYfSQTb609jAduHoC7J/STuxwiInIhHZ5tPnDg\nQAiC0PJcEAQYDAYcOHCg2x9aUlKCM2fOYMSIEdDpdBg7diwSExPbDe/iYku3P681RqMBZrPj9orD\n/D0BAAeP5+LGISEO+xy5ObqP7oA9tB17aDv20HaO6KHRaGj19Q7D+9SpUy1/rq2txb59+3D69Gmb\niqmvr8eCBQuwZcsWeHl5ISkpCbfffrtN23Q2PnotIoK8cSazBDV1DfDQKOeoARERObcOw/tiWq0W\n48aNw+rVqzF79ux2101OTsaKFSuQlZUFURSxY8cOJCQkIDw8HJMmTcKTTz6JBx54AKIoIjY2FhMm\nTLDpF3FGg6L8kZ5XgdPpJRgSHSB3OURE5CI6DO+NGzde8jw3Nxd5eXkdbjguLg5r1qxpc/nUqVMx\nderUTpSoXIOjAvDD/nQknS1keBMRkd10GN6HDx++5Lm3tzfee+89hxXkSvqG+8LTQ8TRswW4T4q5\n5NwBIiKi7uowvF9//fWeqMMliWoVBkX549CpfOQUWhAa6CV3SURE5ALaDO9x48a1u6fY2SlS3d3Q\n6AAcOpWPo2cLGN5ERGQXbYb3V1991eabeGexzht8VQAEAMdSC/GHkZFyl0NERC6gzZlWwsLCWn6q\nqqqQnZ2N7OxsnD9/Hv/zP//TkzUqmo+XFlGhPjiTWQpLdZ3c5RARkQvocMx7+fLl2Lt3LwoKChAR\nEYGMjAw89NBDPVGbyxgSHYBz2WVITivCtQOC5C6HiIgUrsM5TpOSkvDDDz+gf//+2LRpE1avXo2q\nqqqeqM1lDI0OBAAcO1socyVEROQKOgxvrVYLAKirq4MkSYiLi0NiYqLDC3MlEUHe8PXW4tjZQlit\nvMsYERHZpsPD5lFRUVi3bh2uueYa/PnPf0ZUVBTKyzn/bVcIgoCh0QHYfTQHaTlliA7zlbskIiJS\nsA7D+5VXXkFJSQl8fHywdetWFBUV4bHHHuuJ2lzKkOhA7D6ag6NnCxjeRERkkw4Pm0+bNg3btm1D\nWVkZbr/9dsyaNQvBwcE9UZtLGdjHDxpRhSMpBXKXQkRECtdheL/wwgtIS0vDH//4RzzxxBPYvn07\namtre6I2l6LTihjUxx9ZBZXILbLv7U2JiMi9dBjew4cPx6JFi/Cf//wHs2bNwp49ezB27NieqM3l\nxPczAgASU8wyV0JERErWqVuClpWVYefOndi+fTsyMjIwffp0R9flkobFBEIlCEhMMePm6zjbGhER\ndU+H4f3www/jzJkzmDRpEh5//HHEx8f3RF0uydtTg9iIXjh5oRjF5TXwM3jIXRIRESlQh+H9wAMP\nYPTo0VCr1T1Rj8uL72fEyQvFSEwxY8LwcLnLISIiBepwzHvcuHEMbjviuDcREdmqw/Am+/IzeOCq\nUB+cTi9BRRVvVEJERF3XYXhbLFde1pSXl+eQYtxFfD8jrJKEo6m85puIiLquw/C+6667cOjQoZbn\n3333HWbMmOHQolxd86Hzw6d56JyIiLquwxPW/va3v+GVV15BbGwscnJyoNFosH79+p6ozWUF++sR\nZvRCclohLNX10Os6dcUeERERgE7seV911VWYN28efvjhB5w5cwbz5s1DQEBAT9Tm0q4dEIT6Bokn\nrhERUZd1uMu3ePFinD9/HmvXrkVJSQmeeeYZTJo0CU888URP1OeyRg4w4R+7z+HXk3kYPSRE7nKI\niEhBOtzzjo6Oxv/93/8hIiICQ4YMwddff42KioqeqM2lmfz0iAox4MT5YpRZOFc8ERF1XofhPWvW\nLAiC0PLcw8MDf/nLXxxalLsYOSAIVknC4VP5cpdCREQKwuu8ZTRiQBAEAAdO8NI7IiLqPIa3jPwM\nHojp3QspmaUoKquWuxwiIlIIhrfMRg4MAgAc5KFzIiLqJIa3zIbHGqESBB46JyKiTmN4y8xHr8XA\nPn44n1uOnMJKucshIiIFYHg7gevjggEAvyTnylwJEREpAcPbCVzdzwhPDzV+Sc6F1SrJXQ4RETk5\nhrcT8NCoMaJ/EIrLa3DyQrHc5RARkZNjeDuJ0YMbp0jdm5QjcyVEROTsGN5OIjrMB0F+njicYoal\nul7ucoiIyIkxvJ2EIAi4fnAI6uqtOHiKl40REVHbGN5O5PpBwRAA7OVZ50RE1A6GtxMJ8NVhQB8/\npGaWIrfIInc5RETkpBjeTqb5xLU9R7NlroSIiJwVw9vJDI81wttTgz3HclBXb5W7HCIickIODe+U\nlBRMnDgRa9euvWLZ/v37MW3aNNxzzz148cUXYbUyqABAI6oxenAIKqrqcDiFNyshIqIrOSy8LRYL\nli1bhlGjRrW6/OWXX8b777+P9evXo7KyEnv27HFUKYozblgoAGDXER46JyKiKzksvLVaLVatWgWT\nydTq8s2bNyM4uHFOb39/fxQXc2axZkH+egzs44eUjBJkFfBmJUREdCnRYRsWRYhi25v39vYGAOTn\n52Pv3r14+umn292en58eoqi2a41Go8Gu27On28f1xYnzB/HraTNmDwiWu5x2OXMflYI9tB17aDv2\n0HY91UOHhXdnFBYW4vHHH8eSJUvg5+fX7rrFxfa9dMpoNMBsLrfrNu3pKpMXfL202PlrOm6+tjc8\nNPb94mIvzt5HJWAPbcce2o49tJ0jetjWlwHZzjavqKjAo48+ivnz52P06NFyleG0RLUKY4aGoqqm\nHr+e4IxrRET0O9nC+4033sCDDz6IsWPHylWC0xs3NBSCAPz7cCYkibcKJSKiRg47bJ6cnIwVK1Yg\nKysLoihix44dSEhIQHh4OEaPHo1vv/0WFy5cwMaNGwEAt956K6ZPn+6ochQpwFeH4f2MOHTajJSM\nEsRGtD+0QERE7sFh4R0XF4c1a9a0uTw5OdlRH+1SJo+IwKHTZvx4MIPhTUREADjDmtOLDvNBVIgB\nv50pQL6dT9ojIiJlYng7OUEQMGlEb0gAdh7KlLscIiJyAgxvBbgm1gQ/gwf2JOXAUl0vdzlERCQz\nhrcCiGoVEuLDUFPbgD3HOGUqEZG7Y3grxLhhYdBqVPjXoQzUN/AmLkRE7ozhrRDenhqMHRKKorIa\n7D/OSVuIiNwZw1tBpoyMgFol4Pv9F2C1ctIWIiJ3xfBWEH8fHa6PC0ZukQWJKWa5yyEiIpkwvBXm\n5usiIQjA1n3nOWUqEZGbYngrTJC/HiP6m5CeV4HktCK5yyEiIhkwvBXo5usiAQBbfzkvbyFERCQL\nhrcCRQQZMCQ6AGcyS3HyPPe+iYjcDcNboe4cEwUA2Lz7HMe+iYjcDMNbofoE+yC+nxFns8tw7Gyh\n3OUQEVEPYngr2J1joiAA+Meec7By75uIyG0wvBUs3OiNkQODkJ5XgcTTvO6biMhdMLwV7o7RUVAJ\nQuPeN2ddIyJyCwxvhQvy1+OGwcHIKbTgl+RcucshIqIewPB2AXeMjoJGVGHz7rOoqW2QuxwiInIw\nhrcL8PfR4aZrI1BSUYvtv6bLXQ4RETkYw9tF/GFkBHy8tPjhwAUUl9fIXQ4RETkQw9tFeHqI+OOY\nKNTWWfGP3efkLoeIiByI4e1CxgwJRbjRC3uTcpCeVy53OURE5CAMbxeiUgmYnhADCcC6f6Vw2lQi\nIhfF8HYxg6L8Ed/PiDOZpbx0jIjIRTG8XdC9E2Kg1aiw4adUVFbXyV0OERHZGcPbBQX46nD7DVEo\nt9Rh83958hoRkatheLuoySN6IyRAj11HspCWUyZ3OUREZEcMbxclqlWYOTkWEoAvt59CfYNV7pKI\niMhOGN4urH+kH26IC0Z6XgV2cOY1IiKXwfB2cdMnxMDXS4vvfk5DVkGl3OUQEZEdMLxdnLenBg/c\nFIv6Bgmff3+Stw0lInIBDG83cHU/I0YODMK57DL8eDBD7nKIiMhGDG83cd/EGBj0GvxjzzlkmSvk\nLoeIiGzA8HYTBr0WD07pj7p6Kz7ecgJ19bzvNxGRUjG83Uh8PyPGDQtFprkCG3dx8hYiIqVieLuZ\nexJiEOyvx78OZSD5XKHc5RARUTcwvN2Mh1aNx24fBLVKwKfbTqKsslbukoiIqIsY3m4oMtiAu8ZF\no6yyFh9vOc7Lx4iIFIbh7aYmX9sbw/oG4uSFYvxjD8e/iYiUxKHhnZKSgokTJ2Lt2rVXLKupqcEL\nL7yAqVOnOrIEaoNKEPDIrQNg6uWJbfsu4LczBXKXREREneSw8LZYLFi2bBlGjRrV6vI333wTAwYM\ncNTHUyfodRrM+WMctKIKq7aeQH6xRe6SiIioExwW3lqtFqtWrYLJZGp1+TPPPIOJEyc66uOpkyKC\nDJh5Uyyqaurx/qYkWKrr5S6JiIg6IDpsw6IIUWx7897e3igpKen09vz89BBFtT1Ka2E0Guy6PaW6\nM8EAc3kNtuw+h9XbT+Hlh0ZCre789zr20Xbsoe3YQ9uxh7brqR46LLztrdjOh3SNRgPM5nK7blPJ\nbr8uEuezSpF4Kh8ffHME903q16n3sY+2Yw9txx7ajj20nSN62NaXAZ5tTgAAlUrAY7cPQligF3Ye\nzsRPiZlyl0RERG1geFMLTw8R8/40BAa9Bmv/lYLDp/PlLomIiFqhXrp06VJHbDg5ORnPPvssfv31\nVyQlJeHHH39EaWkpCgoKEB0djXnz5mHTpk04e/Ys9u7dC41Gg9jY2Da3Z7HYdyYwLy8Pu2/TFXjp\nNOgf4Yf9x/Nw8JQZ/Xr7ItDXs+312UebsYe2Yw9txx7azhE99PLyaPV1QZIkRUyv5YhxBI7vtO14\nWhHe+/tRaDUqvHBfPCKCWh93YR9txx7ajj20HXtoO455k+wGRfnj0dsGorqmAe9uOIqcwkq5SyIi\noiYMb2rTtQOCMGNyP5RV1uLNr48gt4iTuBAROQOGN7VrfHw47p0Qg9KKWrz5VSLyOAsbEZHsGN7U\noUkjemN6Ql+UVNTiza+OII974EREsmJ4U6fcdG0E7h4fjeLyGry+9jDS83hiCxGRXBje1Gl/GBmJ\nmZP7odxShxVfJSIlo/PT2xIRkf0oZnpUcg7j48Oh12nw6dYTeOeb36DRaRBl9JK7LCIit8I9b+qy\nkQOD8NRdQyAAePXzX/Hf37ICTLJMAAARSklEQVTkLomIyK0wvKlbhkQH4Nl7hsFLJ+LL7afx9c4z\nsFoVMd8PEZHiMbyp22LCe+Gdp8chJECPfx3KwF83HkNVDe8HTkTkaAxvsklIoBdemnkN4qL8kXSu\nEK+uOcxrwYmIHIzhTTbT60Q8ffcQTBwejuyCSvzv5wdx6BTvSEZE5CgMb7ILtUqF+yb1w6O3DoRV\nkvDht8lY968U1NVb5S6NiMjlMLzJrkbFBWPxgyMQGuiFfx/OxBvrDiOfh9GJiOyK4U12FxbohcUP\nXIPr44KRllOOJasP4qcjWVDI3WeJiJwew5scwkOrxsO3DMDs2wZCrRKwZsdp/L8NR1FcXiN3aURE\nisfwJocRBAHXDQrGskdGIi7KH8lpRVj86QHsOZbNvXAiIhswvMnh/AweeGbaUDxwUywarBI+//4U\nVqxLRJa5Qu7SiIgUieFNPUIQBNx4dRhefXQk4vsZkZJZiqWfH8TGXWdRU9sgd3lERIrC8KYe5e+j\nw9ypgzHvriHo5a3F9/sv4MVP9uHnYzmw8lA6EVGnMLxJFsNiArH8ketwy6hIVFbXY/X3J/HK5wdx\n8nyR3KURETk93hKUZOOhVeOucdG4cVgYNu8+i33H8/DW+t8QF+WPO0ZHITrMV+4SiYicEsObZBfg\nq8Ojtw3CxGt64+8/pSI5rQjJaUWIu8ofd9zAECciuhzDm5xGVIgPnr8vHqfTi/Hdz2lIPleE5HON\nIX7LdZHo17sXBEGQu0wiItkxvMnpxEb44fn7/K4I8cggAyaP6I0RA0wQ1Txdg4jcF8ObnFZziKdm\nluLHg+k4nGLGqq0n8PddqUiID8eYoaHw9dLKXSYRUY9jeJPT6xvui77hg2EuqcK/D2di99FsbN59\nDt/9nIZhMYEYOzQUg/r4Q6XiIXUicg8Mb1IMYy9P3DMhBneMjsIvybn472/ZOHzajMOnzfD38cDo\nwSEYNSgYQf56uUslInIohjcpjqeHiAnDw5EQH4bzueXYfTQb+0/kYcve89iy9zwigw0YOSAI1w4w\nwd9HJ3e5RER2x/AmxRIEAVEhPogK8cH0hL5ITDHjwIl8HE8rwoXccvz9p1T0DffF1TFGXB0TyD1y\nInIZDG9yCTqtiOvjQnB9XAjKLbU4dNqMAyfycCajBGcyS7Hhp1SEBOgxrG8ghsUEIjrUl2PkRKRY\nDG9yOQa9FuOvDsP4q8NQVlmLo6kF+C21AMfTivDDgXT8cCAdXjoR/SP9MDDSDwP7+MPk58lryIlI\nMRje5NJ8vLQYMzQUY4aGorauAScuFOO3M41B3nyyGwAE+HhgQKQ/+vXuhb7hvghimBORE2N4k9vQ\natSNh837BkKSJJhLqnDifDFOnC/CyQvF+DkpBz8n5QAAvD016Bvmi+gwH/QN80WfEB94aNQy/wZE\nRI0Y3uSWBEGAyU8Pk58eN14dBqskISOvAqlZpTibVYrUrFL81nS4HQBUgoCQAD0iggyIDPJGZLAB\nvU0G6HX8X4iIeh7/5iFCYzhHBhsQGWzAhOHhAIDi8pqWIE/LKUN6fgWyCiqx7/jv7zP20iEs0Buh\ngV4IDdQjJMALIQF66LT8X4uIHId/wxC1wc/ggWv6m3BNfxMAwCpJyC+uQnpeOS7kljc+5lVcsofe\nLMBHh9BALwT5e8LYyxOmXo2Pxl46aEQefici2zC8iTpJJQgI9tcj2F+PawcEtbxeZqlFTkElsgsq\nkV1oaXqsRNK5QiSdu3I7fgaPliAP9PVERKgvNJDgZ/CAn0EHTw81T5YjonYxvIls5KPXwidCi9gI\nv0ter6yuQ35xFcwlv/80Pq/GmcwSpGS0vj0PjbopyD3gb/CAr7cHfPQaGLy0jZ/lpYWPXgNvvQZq\nFe+uRuSOGN5EDuKl0yAqRIOoEJ8rltU3WFFYWo3CsmrUQ8CF7FIUl9eguKwaxeU1KCqvQW6RpcPP\n8PbUwKDXwEevhZenBnqdCC+dCL1O0/QowkunueRR7yHylqpECufQ8E5JScGcOXMwa9YszJgx45Jl\nv/zyC959912o1WqMHTsWTz75pCNLIXIqolqFIH89gvz1MBoNMJvLr1intq4BxRU1KKusRVllHcot\ntSiz1KK8sq7x0VKLMksdyiprkVtogdSFz9dqVNBpRei0aug0aui0ang0PffQNj5vWa5Vw0PT+KPV\nqKAR1dCIKmhFVdOjGhrN7895NIDI8RwW3haLBcuWLcOoUaNaXb58+XJ89tlnCAoKwowZM3DTTTeh\nb9++jiqHSHG0GjWC/PQI8ut4TvYGqxVVNQ2orK6Dpbr+osd6WKrrUFlVf8myqtoGVNc2oLq2HmWV\ntaipbehS+LdHrRIuCvemwFeroFarIKoFiGoV1GoBoqrpUa2CqBKgVguN66ga17t8HbXq9/eqVQJU\nggBV02Ov3ApUVFQ3vQaom5YJQtO6l6yPxsfm15pfVzVvt/FSQpVKgIDGPwsCmn5+f41ITg4Lb61W\ni1WrVmHVqlVXLMvIyICvry9CQkIAAOPGjcO+fft6LLw3p27Fsf3JaLDa668r96VWCeyjjXqkh7qm\nn4uIALybfgBAkiRIUtMjcNmfG5e1rAcAElqWoXn9phclAPUSUA8Jlqbnzf9o9zdtaPqps/k3djjh\non+2trCz8X7l9wChnWedWdDJz22tEKmL/x12t3YFa+t7mwBgXPS1uDPqDz1Sh8PCWxRFiGLrmzeb\nzfD392957u/vj4yMNs7eaeLnp4dop0ts9FlaAI1/aZLt2EfbOUcPe6aG5niQmsNcuuj1y74ANK4n\nXbR+83rSJdu6eH1c9v6Lt930iUCr7/u9pivqbSPTrlhTam/5pS+2vkkJF/eknY9uZ0ErizqVyZev\n1EofqE0CBFiq62A0Gnrk8xRzwlpxcccn73TWlLDJmDnsrlbHGalr2hqvpc5jD23HHtquqz2UWvkC\n1fhw6Zet39e/dMXLv7S1996u6Pi9ba/Q0Vvb27ZGrUJ4WC+7/3fY1pcBWcLbZDKhoOD3SS3y8vJg\nMpnkKIWIiLqhedy/5XjNlX8gB5LltNDw8HBUVFQgMzMT9fX1+Omnn3DDDTfIUQoREZHiOGzPOzk5\nGStWrEBWVhZEUcSOHTuQkJCA8PBwTJo0CUuXLsWzzz4LALj55psRFRXlqFKIiIhciiBJtowu9BxH\njCNwjMx27KPt2EPbsYe2Yw9t54getjXmzdkUiIiIFIbhTUREpDAMbyIiIoVheBMRESkMw5uIiEhh\nGN5EREQKw/AmIiJSGIY3ERGRwihmkhYiIiJqxD1vIiIihWF4ExERKQzDm4iISGEY3kRERArD8CYi\nIlIYhjcREZHCiHIXIIfXXnsNR48ehSAIWLhwIYYMGSJ3SU4tJSUFc+bMwaxZszBjxgzk5OTg+eef\nR0NDA4xGI9566y1otVps2bIFX375JVQqFaZNm4a7775b7tKdxptvvonDhw+jvr4ejz32GAYPHswe\ndkFVVRUWLFiAwsJC1NTUYM6cOejfvz972A3V1dW49dZbMWfOHIwaNYo97IIDBw7g6aefRkxMDACg\nX79+eOSRR+TpoeRmDhw4IM2ePVuSJElKTU2Vpk2bJnNFzq2yslKaMWOGtGjRImnNmjWSJEnSggUL\npO+//16SJEl65513pHXr1kmVlZXS5MmTpbKyMqmqqkq65ZZbpOLiYjlLdxr79u2THnnkEUmSJKmo\nqEgaN24ce9hF27Ztkz755BNJkiQpMzNTmjx5MnvYTe+++640depUadOmTexhF+3fv1966qmnLnlN\nrh663WHzffv2YeLEiQCA6OholJaWoqKiQuaqnJdWq8WqVatgMplaXjtw4AAmTJgAABg/fjz27duH\no0ePYvDgwTAYDNDpdIiPj0diYqJcZTuVESNG4K9//SsAwMfHB1VVVexhF91888149NFHAQA5OTkI\nCgpiD7vh7NmzSE1NxY033giA/y/bg1w9dLvwLigogJ+fX8tzf39/mM1mGStybqIoQqfTXfJaVVUV\ntFotACAgIABmsxkFBQXw9/dvWYd9/Z1arYZerwcAbNy4EWPHjmUPu+mee+7Bc889h4ULF7KH3bBi\nxQosWLCg5Tl72HWpqal4/PHHce+992Lv3r2y9dAtx7wvJnF2WJu01T/29Uo7d+7Exo0bsXr1akye\nPLnldfaw89avX4+TJ0/iL3/5yyX9YQ879u2332LYsGHo3bt3q8vZw4716dMHc+fOxR/+8AdkZGTg\ngQceQENDQ8vynuyh24W3yWRCQUFBy/P8/HwYjUYZK1IevV6P6upq6HQ65OXlwWQytdrXYcOGyVil\nc9mzZw8++ugjfPrppzAYDOxhFyUnJyMgIAAhISEYMGAAGhoa4OXlxR52wa5du5CRkYFdu3YhNzcX\nWq2W/x12UVBQEG6++WYAQEREBAIDA5GUlCRLD93usPkNN9yAHTt2AACOHz8Ok8kEb29vmatSluuv\nv76lhz/++CPGjBmDoUOHIikpCWVlZaisrERiYiKuueYamSt1DuXl5XjzzTfx8ccfo1evXgDYw646\ndOgQVq9eDaBx6MtisbCHXfTee+9h06ZN2LBhA+6++27MmTOHPeyiLVu24LPPPgMAmM1mFBYWYurU\nqbL00C3vKvb222/j0KFDEAQBS5YsQf/+/eUuyWklJydjxYoVyMrKgiiKCAoKwttvv40FCxagpqYG\noaGheP3116HRaLB9+3Z89tlnEAQBM2bMwO233y53+U7hm2++wcqVKxEVFdXy2htvvIFFixaxh51U\nXV2Nl156CTk5OaiursbcuXMRFxeHF154gT3shpUrVyIsLAyjR49mD7ugoqICzz33HMrKylBXV4e5\nc+diwIABsvTQLcObiIhIydzusDkREZHSMbyJiIgUhuFNRESkMAxvIiIihWF4ExERKQzDm8gFxcbG\nor6+HgDw3Xff2W27//znP2G1WgEAM2fOvGR2KSLqOQxvIhfW0NCADz/80G7bW7lyZUt4r1mzBmq1\n2m7bJqLOc7vpUYncycKFC5GVlYWHHnoIq1evxvfff4+1a9dCkiT4+/tj+fLl8PPzQ3x8PP70pz/B\narVi4cKFWLJkCc6dO4fa2loMHToUixYtwvvvv48LFy5g1qxZ+OCDDzBy5EgcP34ctbW1WLx4MXJz\nc1FfX4877rgD9913HzZv3oxffvkFVqsVaWlpCAsLw8qVKyEIgtxtIVI+u95glIicQr9+/aS6ujop\nIyNDGjNmjCRJkpSdnS3ddtttUk1NjSRJkvTFF19Ir7/+uiRJkhQbGyv9/PPPkiQ13nO8+d7tkiRJ\nN910k3T69OlLtnvxnz/66CNp6dKlkiRJUlVVlTR+/HgpPT1d2rRpk5SQkCBVVVVJVqtVmjBhgnT8\n+PGeaQCRi+OeN5GbOHLkCMxmMx5++GEAQG1tLcLDwwE03vUoPj4eQOM9x3NycjB9+nRotVqYzWYU\nFxe3ud2jR49i6tSpAACdToe4uDgcP34cADBkyJCWW8qGhISgtLTUYb8fkTtheBO5Ca1WiyFDhuDj\njz9udblGowEAbNu2DUlJSVi3bh1EUWwJ5rZcfhhckqSW1y4fE5c4GzORXfCENSIXplKpWs46Hzx4\nMI4dOwaz2QwA+OGHH7Bz584r3lNYWIioqCiIoojk5GSkp6ejtrYWQGNQN2+v2dChQ7Fnzx4AgMVi\nwfHjxzFo0CBH/lpEbo/hTeTCTCYTAgMDMXXqVBgMBrz00kt47LHHcP/992Pjxo2t3mN4ypQp+O23\n3zBjxgz8+OOPeOihh7B8+XKUlpZizJgxuOuuu5Cent6y/syZM1FZWYn7778fDz74IObMmdNyOJ6I\nHIN3FSMiIlIY7nkTEREpDMObiIhIYRjeRERECsPwJiIiUhiGNxERkcIwvImIiBSG4U1ERKQwDG8i\nIiKF+f/svkvl+TFlsgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 576x396 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "collapsed": true,
        "id": "UxBEH9__YZ3G",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## 1.4 Control flow\n",
        "\n",
        "As you've seen, TensorFlow now an imperative programming style, and that's all because of Eager. \n",
        "\n",
        "As another example of the power of Eager, let's take a look at how we can build a dynamic model that uses Python flow control. Here's an example of the [Collatz conjecture](https://en.wikipedia.org/wiki/Collatz_conjecture) using TensorFlow’s arithmetic operations. Such dynamic behavior is not possible in past versions of TensorFlow (up to v1.4)!"
      ]
    },
    {
      "metadata": {
        "id": "LCfX4kfRYZ3W",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        },
        "outputId": "45cece74-ed18-44e5-e046-6449fcca38ca"
      },
      "cell_type": "code",
      "source": [
        "a = tf.constant(12)\n",
        "counter = 0\n",
        "while not tf.equal(a, 1):\n",
        "  if tf.equal(a % 2, 0):\n",
        "    a = a / 2\n",
        "  else:\n",
        "    a = 3 * a + 1\n",
        "  print(a)"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tf.Tensor(6, shape=(), dtype=int32)\n",
            "tf.Tensor(3, shape=(), dtype=int32)\n",
            "tf.Tensor(10, shape=(), dtype=int32)\n",
            "tf.Tensor(5, shape=(), dtype=int32)\n",
            "tf.Tensor(16, shape=(), dtype=int32)\n",
            "tf.Tensor(8, shape=(), dtype=int32)\n",
            "tf.Tensor(4, shape=(), dtype=int32)\n",
            "tf.Tensor(2, shape=(), dtype=int32)\n",
            "tf.Tensor(1, shape=(), dtype=int32)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "P-dYmvJPbtWa",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}