pretraining authors and objectives (#212)

dl/Hyperparameter_tuning.ipynb

@@ -34,9 +34,9 @@
     "```{admonition} Audience & Objectives\n",
     "This chapter builds on {doc}`layers` and {doc}`../ml/classification`. After completing this chapter, you should be able to \n",
     "\n",
-    "  * Distinguish between training and model design-related hyperparamters \n",
+    "  * Distinguish between training and model design-related hyperparameters \n",
     "  * Understand the importance of validation data in hyperparameter tuning  \n",
-    "  * Understand how each hyperparameter can affect model's performance\n",
+    "  * Understand how each hyperparameter can affect a model's performance\n",
     "```\n",
     "\n",
     "Hyperparameters can be categorized into two groups: those used for training and those related to model structure and design."
@@ -583,7 +583,7 @@
   "celltoolbar": "Tags",
   "hide_input": false,
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3.10.7 64-bit",
    "language": "python",
    "name": "python3"
   },
@@ -597,7 +597,12 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.12"
+   "version": "3.10.7"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e"
+   }
   }
  },
  "nbformat": 4,

dl/pretraining.ipynb

@@ -4,7 +4,21 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Pretraining"
+    "# Pretraining \n",
+    "\n",
+    "```{admonition} Authors:\n",
+    "Heta Gandhi (https://github.com/@gandhi_heta) & Sam Cox (https://github.com/SamCox822)\n",
+    "```\n",
+    "\n",
+    "Up until this point, we have been building deep learning models from scratch and mostly training on labelled data to complete a task. A lot of times, especially in chemistry, labelled data is not readily accessible or abundant. In this scenerio, it is helpful to use a pretrained model and leverage the pretrained weights and architecture to learn a new task. In this chapter, we will look into pretraining, how it works, and some applications. \n",
+    "\n",
+    "```{admonition} Audience & Objectives\n",
+    "This chapter builds on {doc}`layers` and {doc}`gnn. After completing this chapter, you should be able to \n",
+    "\n",
+    "  * Understand why pretraining is useful, and in which situations it is appropriate\n",
+    "  * Understand transfer learning and fine-tuning\n",
+    "  * Be able to use a pretrained model for a simple downstream task\n",
+    "``"
    ]
   },
   {
@@ -80,7 +94,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Fine Tuning"
+    "### Fine-Tuning"
    ]
   },
   {
@@ -289,7 +303,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The model performs quite well on our test set. We have finetuned the pretrained model for a task that it was not trained for. This shows that even though the original model was trained on the ZINC dataset, the input representations can be used to make predictions on another dataset, with a different task. Using pre-trained models saves time and effort spent in training the model. To further improve performance on this silubility prediction task, you can change some other parameters like the learning rate or add additional layers before the output layer.   "
+    "The model performs quite well on our test set. We have fine-tuned the pretrained model for a task that it was not trained for. This shows that even though the original model was trained on the ZINC dataset, the input representations can be used to make predictions on another dataset, with a different task. Using pre-trained models saves time and effort spent in training the model. To further improve performance on this silubility prediction task, you can change some other parameters like the learning rate or add additional layers before the output layer.   "
    ]
   },
   {
@@ -321,7 +335,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.13"
+   "version": "3.9.6"
   },
   "vscode": {
    "interpreter": {