bad-libs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bad-Libs\n",
    "\n",
    "Automatically converts any book into a Mad-Libs style game of silliness.\n",
    "\n",
    "This is a tutorial notebook to teach the basics of spaCy from scratch.\n",
    "\n",
    "Requires spaCy: https://spacy.io\n",
    "\n",
    "How it works:\n",
    " 1. First it scans text for adjectives, nouns, verbs, people, and locations. \n",
    " 2. Then it creates placeholders that the \"player\" (that's you!) can provide their own answers for.\n",
    " 3. It then fills-in-the blanks to replace the missing spots with a word of the appropriate part of speech.\n",
    "\n",
    "Finally we read the finished bad-lib result while hilarity ensues."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import io\n",
    "import re\n",
    "import spacy\n",
    "import datetime\n",
    "import math\n",
    "import random\n",
    "from operator import itemgetter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Initialize with the large model, it has the most accuracy.\n",
    "nlp = spacy.load('en_core_web_lg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the content\n",
    "\n",
    "Always understand the content structure first.  We'll be splitting up by chapters and paragraphs of those chapters.  We really need to know what the text looks like before we start processing it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "filename = \"great_expectations.txt\"\n",
    "with io.open(filename, mode=\"r\", encoding=\"utf-8\") as f:\n",
    "    content = f.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1013443\n"
     ]
    }
   ],
   "source": [
    "print(len(content))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Project Gutenberg EBook of Great Expectations, by Charles Dickens\n",
      "\n",
      "This eBook is for the use of anyone anywhere at no cost and with\n",
      "almost no restrictions whatsoever.  You may copy it, give it away or\n",
      "re-use it under the terms of the Project Gutenberg License included\n",
      "with this eBook or online at www.gutenberg.org\n",
      "\n",
      "\n",
      "Title: Great Expectations\n",
      "\n",
      "Author: Charles Dickens\n",
      "\n",
      "Posting Date: August 20, 2008 [EBook #1400]\n",
      "Release Date: July, 1998\n",
      "Last Updated: March 4, 2018\n",
      "\n",
      "Language: English\n",
      "\n",
      "Character set encoding: UTF-8\n",
      "\n",
      "*** START OF THIS PROJECT GUTENBERG EBOOK GREAT EXPECTATIONS ***\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "Produced by An Anonymous Volunteer\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "GREAT EXPECTATIONS\n",
      "\n",
      "[1867 Edition]\n",
      "\n",
      "by Charles Dickens\n",
      "\n",
      "\n",
      "[Project Gutenberg Editor's Note: There is also another version of\n",
      "this work etext98/grexp10.txt scanned from a different edition]\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "Chapter I\n",
      "\n",
      "My father's family name being Pirrip, and my Christian name Philip, my\n",
      "infant tongue could make of both names nothing longer or more explicit\n",
      "than Pip. So, I called myself Pip, and came to be called Pip.\n",
      "\n",
      "I give Pirrip as my father's family name, on the authority of his\n",
      "tombstone and my sister,--Mrs. Joe Gargery, who married the blacksmith.\n",
      "As I never saw my father or my mother, and never saw any likeness\n",
      "of either of them (for their days were long before the days of\n",
      "photographs), my first fancies regarding what they were like were\n",
      "unreasonably derived from their tombstones. The shape of the letters on\n",
      "my father's, gave me an odd idea that he was a square, stout, dark man,\n",
      "with curly black hair. From the character and turn of the inscription,\n",
      "“Also Georgiana Wife of the Above,” I drew a childish conclusion that\n",
      "my mother was freckled and sickly. To five little stone lozenges, each\n",
      "about a foot and a half long, which were arranged in a neat row beside\n",
      "their grave, and were sacred to the memory of five little brothers of\n",
      "mine,--who gave up trying to get a living, exceedingly early in\n",
      "that universal struggle,--I am indebted for a belief I religiously\n",
      "entertained that they had all been born on their backs with their hands\n",
      "in their trousers-pockets, and had never taken them out in this state of\n",
      "existence.\n",
      "\n",
      "Ours was the marsh country, down by the river, within, as the river\n",
      "wound, twenty miles of the sea. My first most vivid and broad impression\n",
      "of the identity of things seems to me to have been gained on a memorable\n",
      "raw afternoon towards evening. At such a time I found out for certain\n",
      "that this bleak place overgrown with nettles was the churchyard; and\n",
      "that Philip Pirrip, late of this parish, and also Georgiana wife of the\n",
      "above, were dead and buried; and that Alexander, Bartholomew, Abraham,\n",
      "Tobias, and Roger, infant children of the aforesaid, were also dead\n",
      "and buried; and that the dark flat wilderness beyond the churchyard,\n",
      "intersected with dikes and mounds and gates, with scattered cattle\n",
      "feeding on it, was the marshes; and that the low leaden line beyond\n",
      "was the river; and that the distant savage lair from which the wind was\n",
      "rushing was the sea; and that the small bundle of shivers growing afraid\n",
      "of it all and beginning to cry, was Pip.\n",
      "\n",
      "“Hold your noise!” cried a terrible voice, as a man started up from\n",
      "among the graves at the side of the church porch. “Keep still, you\n",
      "little devil, or I'll cut your throat!”\n",
      "\n",
      "A fearful man, all in coarse gray, with a great iron on his leg. A man\n",
      "with no hat, and with broken shoes, and with an old rag tied round his\n",
      "head. A man who had been soaked in water, and smothered in mud, and\n",
      "lamed by stones, and cut by flints, and stung by nettles, and torn by\n",
      "briars; who limped, and shivered, and glared, and growled; and whose\n",
      "teeth chattered in his head as he seized me by the chin.\n",
      "\n",
      "“Oh! Don't cut my throat, sir,” I pleaded in terror. “Pray don't do it,\n",
      "sir.”\n",
      "\n",
      "“Tell us your name!” said the man. “Quick!”\n",
      "\n",
      "“Pip, sir.”\n",
      "\n",
      "“Once more,” said the man, staring at me. “Give it mouth!”\n",
      "\n",
      "“Pip. Pip, sir.”\n",
      "\n",
      "“Show us where you live,” said the man. “Pint out the place!”\n",
      "\n",
      "I pointed to where our village lay, on the flat in-shore among the\n",
      "alder-trees and pollards, a mile or more from the church.\n",
      "\n",
      "The man, after looking at me for a moment, turned me upside down, and\n",
      "emptied my pockets. There was nothing in them but a piece of bread. When\n",
      "the church came to itself,--for he was so sudden and strong that he\n",
      "made it go head over heels before me, and I saw the steeple under my\n",
      "feet,--when the church came to itself, I say, I was seated on a high\n",
      "tombstone, trembling while he ate the bread ravenously.\n",
      "\n",
      "“You young dog,” said the man, licking his lips, “what fat cheeks you\n",
      "ha' got.”\n",
      "\n",
      "I believe they were fat, though I was at that time undersized for my\n",
      "years, and not strong.\n",
      "\n",
      "“Darn me if I couldn't eat em,” said the man, with a threatening shake\n",
      "of his head, “and if I han't half a mind to't!”\n",
      "\n",
      "I earnestly expressed my hope that he wouldn't, and held tighter to\n",
      "the tombstone on which he had put me; partly, to keep myself upon it;\n",
      "partly, to keep myself from crying.\n",
      "\n",
      "“Now lookee here!” said the man. “Where's your mother?”\n",
      "\n",
      "“There, sir!” said I.\n",
      "\n",
      "He started, made a short run, and stopped and looked over his shoulder.\n",
      "\n",
      "“There, sir!” I timidly explained. “Also Georgiana. That's my mother.”\n",
      "\n",
      "“Oh!” said he, coming back. “And is that your father alonger your\n",
      "mother?”\n",
      "\n",
      "“Yes, sir,” said I; “him too; late of this parish.”\n",
      "\n",
      "“Ha!” he muttered then, considering. “Who d'ye live with,--supposin'\n",
      "you're kindly let to live, which I han't made up my mind about?”\n",
      "\n",
      "“My sister, sir,--Mrs. Joe Gargery,--wife of Joe Gargery, the\n",
      "blacksmith, sir.”\n",
      "\n",
      "“Blacksmith, eh?” said he. And looked down at his leg.\n",
      "\n",
      "After darkly looking at his leg and me several times, he came closer\n",
      "to my tombstone, took me by both arms, and tilted me back as far as he\n",
      "could hold me; so that his eyes looked most powerfully down into mine,\n",
      "and mine looked most helplessly up into his.\n",
      "\n",
      "“Now lookee here,” he said, “the question being whether you're to be let\n",
      "to live. You know what a file is?”\n",
      "\n",
      "“Yes, sir.”\n",
      "\n",
      "“And you know what wittles is?”\n",
      "\n",
      "“Yes, sir.”\n",
      "\n",
      "After each question he tilted me over a little more, so as to give me a\n",
      "greater sense of helplessness and danger.\n",
      "\n",
      "“You get me a file.” He tilted me again. “And you get me wittles.” He\n",
      "tilted me again. “You bring 'em both to me.” He tilted me again. “Or\n",
      "I'll have your heart and liver out.” He tilted me again.\n",
      "\n",
      "I was dreadfully frightened, and so giddy that I clung to him with both\n",
      "hands, and said, “If you would kindly please to let me keep upright,\n",
      "sir, perhaps I shouldn't be sick, and perhaps I could attend more.”\n",
      "\n",
      "He gave me a most tremendous dip and roll, so that the church jumped\n",
      "over its own weathercock. Then, he held me by the arms, in an upright\n",
      "position on the top of the stone, and went on in these fearful terms:--\n",
      "\n",
      "“You bring me, to-morrow morning early, that file and them wittles. You\n",
      "bring the lot to me, at that old Battery over yonder. You do it, and you\n",
      "never dare to say a word or dare to make a sign concerning your having\n",
      "seen such a person as me, or any person sumever, and you shall be let to\n",
      "live. You fail, or you go from my words in any partickler, no matter how\n",
      "small it is, and your heart and your liver shall be tore out, roasted,\n",
      "and ate. Now, I ain't alone, as you may think I am. There's a young man\n",
      "hid with me, in comparison with which young man I am a Angel. That young\n",
      "man hears the words I speak. That young man has a secret way pecooliar\n",
      "to himself, of getting at a boy, and at his heart, and at his liver. It\n",
      "is in wain for a boy to attempt to hide himself from that young man. A\n",
      "boy may lock his door, may be warm in bed, may tuck himself up, may draw\n",
      "the clothes over his head, may think himself comfortable and safe, but\n",
      "that young man will softly creep and creep his way to him and tear him\n",
      "open. I am a keeping that young man from harming of you at the present\n",
      "moment, with great difficulty. I find it wery hard to hold that young\n",
      "man off of your inside. Now, what do you say?”\n",
      "\n",
      "I said that I would get him the file, and I would get him what broken\n",
      "bits of food I could, and I would come to him at the Battery, early in\n",
      "the morning.\n",
      "\n",
      "“Say Lord strike you dead if you don't!” said the man.\n",
      "\n",
      "I said so, and he took me down.\n",
      "\n",
      "“Now,” he pursued, “you remember what you've undertook, and you remember\n",
      "that young man, and you get home!”\n",
      "\n",
      "“Goo-good night, sir,” I faltered.\n",
      "\n",
      "“Much of that!” said he, glancing about him over the cold wet flat. “I\n",
      "wish I was a frog. Or a eel!”\n",
      "\n",
      "At the same time, he hugged his shuddering body in both his\n",
      "arms,--clasping himself, as if to hold himself together,--and limped\n",
      "towards the low church wall. As I saw him go, picking his way among the\n",
      "nettles, and among the brambles that bound the green mounds, he looked\n",
      "in my young eyes as if he were eluding the hands of the dead people,\n",
      "stretching up cautiously out of their graves, to get a twist upon his\n",
      "ankle and pull him in.\n",
      "\n",
      "When he came to the low church wall, he got over it, like a man whose\n",
      "legs were numbed and stiff, and then turned round to look for me. When I\n",
      "saw him turning, I set my face towards home, and made the best use of\n",
      "my legs. But presently I looked over my shoulder, and saw him going on\n",
      "again towards the river, still hugging himself in both arms, and picking\n",
      "his way with his sore feet among the great stones dropped into the\n",
      "marshes here and there, for stepping-places when the rains were heavy or\n",
      "the tide was in.\n",
      "\n",
      "The marshes were just a long black horizontal line then, as I stopped\n",
      "to look after him; and the river was just another horizontal line, not\n",
      "nearly so broad nor yet so black; and the sky was just a row of long\n",
      "angry red lines and dense black lines intermixed. On the edge of the\n",
      "river I could faintly make out the only two black things in all the\n",
      "prospect that seemed to be standing upright; one of these was the beacon\n",
      "by which the sailors steered,--like an unhooped cask upon a pole,--an\n",
      "ugly thing when you were near it; the other, a gibbet, with some chains\n",
      "hanging to it which had once held a pirate\n"
     ]
    }
   ],
   "source": [
    "#Inspect the text to see what it looks like.  Note the breaks with '\\n\\nChapter X\\n\\n'\n",
    "print(content[0:10000])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Breaking up the chapters and paragraphs\n",
    "\n",
    "spaCy (and many other NLP libraries), don't break up text by paragraph.  So we need to do some string nerding on this thing.  A more professional name is \"text preprocessing\".  We will split on chapter using a breaker.  We also do some text cleanup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Chapter 1 :  45\n",
      "Chapter 2 :  64\n",
      "Chapter 3 :  46\n",
      "Chapter 4 :  55\n",
      "Chapter 5 :  73\n",
      "Chapter 6 :  5\n",
      "Chapter 7 :  92\n",
      "Chapter 8 :  105\n",
      "Chapter 9 :  73\n",
      "Chapter 10 :  56\n",
      "Chapter 11 :  128\n",
      "Chapter 12 :  26\n",
      "Chapter 13 :  68\n",
      "Chapter 14 :  8\n",
      "Chapter 15 :  93\n",
      "Chapter 16 :  17\n",
      "Chapter 17 :  75\n",
      "Chapter 18 :  133\n",
      "Chapter 19 :  110\n",
      "Chapter 20 :  73\n",
      "Chapter 21 :  43\n",
      "Chapter 22 :  109\n",
      "Chapter 23 :  49\n",
      "Chapter 24 :  51\n",
      "Chapter 25 :  54\n",
      "Chapter 26 :  54\n",
      "Chapter 27 :  65\n",
      "Chapter 28 :  40\n",
      "Chapter 29 :  119\n",
      "Chapter 30 :  70\n",
      "Chapter 31 :  35\n",
      "Chapter 32 :  46\n",
      "Chapter 33 :  64\n",
      "Chapter 34 :  30\n",
      "Chapter 35 :  61\n",
      "Chapter 36 :  75\n",
      "Chapter 37 :  36\n",
      "Chapter 38 :  110\n",
      "Chapter 39 :  104\n",
      "Chapter 40 :  117\n",
      "Chapter 41 :  53\n",
      "Chapter 42 :  46\n",
      "Chapter 43 :  58\n",
      "Chapter 44 :  77\n",
      "Chapter 45 :  55\n",
      "Chapter 46 :  44\n",
      "Chapter 47 :  40\n",
      "Chapter 48 :  64\n",
      "Chapter 49 :  82\n",
      "Chapter 50 :  48\n",
      "Chapter 51 :  66\n",
      "Chapter 52 :  42\n",
      "Chapter 53 :  84\n",
      "Chapter 54 :  82\n",
      "Chapter 55 :  64\n",
      "Chapter 56 :  35\n",
      "Chapter 57 :  107\n",
      "Chapter 58 :  65\n",
      "Chapter 59 :  103\n"
     ]
    }
   ],
   "source": [
    "chapters = []\n",
    "content = content.replace('--',' ')\n",
    "ch_split = '\\n\\nChapter '\n",
    "pa_split = '\\n\\n'\n",
    "for chapter in content.split(ch_split)[1:]:\n",
    "    paragraphs = chapter.split(pa_split)[2:]\n",
    "    paragraphs = [para.replace('\\n',' ').strip() for para in paragraphs]\n",
    "    chapters.append(paragraphs)\n",
    "    print('Chapter',len(chapters),': ',len(paragraphs))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parsing the paragraphs with spaCy nlp\n",
    "\n",
    "This will take the text and make a spaCy \"Doc\" for each chapter's paragraphs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I give Pirrip as my father's family name, on the authority of his tombstone and my sister, Mrs. Joe Gargery, who married the blacksmith. As I never saw my father or my mother, and never saw any likeness of either of them (for their days were long before the days of photographs), my first fancies regarding what they were like were unreasonably derived from their tombstones. The shape of the letters on my father's, gave me an odd idea that he was a square, stout, dark man, with curly black hair. From the character and turn of the inscription, “Also Georgiana Wife of the Above,” I drew a childish conclusion that my mother was freckled and sickly. To five little stone lozenges, each about a foot and a half long, which were arranged in a neat row beside their grave, and were sacred to the memory of five little brothers of mine, who gave up trying to get a living, exceedingly early in that universal struggle, I am indebted for a belief I religiously entertained that they had all been born on their backs with their hands in their trousers-pockets, and had never taken them out in this state of existence.\n",
      "\n",
      "Ours was the marsh country, down by the river, within, as the river wound, twenty miles of the sea. My first most vivid and broad impression of the identity of things seems to me to have been gained on a memorable raw afternoon towards evening. At such a time I found out for certain that this bleak place overgrown with nettles was the churchyard; and that Philip Pirrip, late of this parish, and also Georgiana wife of the above, were dead and buried; and that Alexander, Bartholomew, Abraham, Tobias, and Roger, infant children of the aforesaid, were also dead and buried; and that the dark flat wilderness beyond the churchyard, intersected with dikes and mounds and gates, with scattered cattle feeding on it, was the marshes; and that the low leaden line beyond was the river; and that the distant savage lair from which the wind was rushing was the sea; and that the small bundle of shivers growing afraid of it all and beginning to cry, was Pip.\n",
      "\n",
      "---------\n",
      "She was not a good-looking woman, my sister; and I had a general impression that she must have made Joe Gargery marry her by hand. Joe was a fair man, with curls of flaxen hair on each side of his smooth face, and with eyes of such a very undecided blue that they seemed to have somehow got mixed with their own whites. He was a mild, good-natured, sweet-tempered, easy-going, foolish, dear fellow, a sort of Hercules in strength, and also in weakness.\n",
      "\n",
      "My sister, Mrs. Joe, with black hair and eyes, had such a prevailing redness of skin that I sometimes used to wonder whether it was possible she washed herself with a nutmeg-grater instead of soap. She was tall and bony, and almost always wore a coarse apron, fastened over her figure behind with two loops, and having a square impregnable bib in front, that was stuck full of pins and needles. She made it a powerful merit in herself, and a strong reproach against Joe, that she wore this apron so much. Though I really see no reason why she should have worn it at all; or why, if she did wear it at all, she should not have taken it off, every day of her life.\n",
      "\n",
      "---------\n",
      "The mist was heavier yet when I got out upon the marshes, so that instead of my running at everything, everything seemed to run at me. This was very disagreeable to a guilty mind. The gates and dikes and banks came bursting at me through the mist, as if they cried as plainly as could be, “A boy with somebody else's pork pie! Stop him!” The cattle came upon me with like suddenness, staring out of their eyes, and steaming out of their nostrils, “Halloa, young thief!” One black ox, with a white cravat on, who even had to my awakened conscience something of a clerical air, fixed me so obstinately with his eyes, and moved his blunt head round in such an accusatory manner as I moved round, that I blubbered out to him, “I couldn't help it, sir! It wasn't for myself I took it!” Upon which he put down his head, blew a cloud of smoke out of his nose, and vanished with a kick-up of his hind-legs and a flourish of his tail.\n",
      "\n",
      "All this time, I was getting on towards the river; but however fast I went, I couldn't warm my feet, to which the damp cold seemed riveted, as the iron was riveted to the leg of the man I was running to meet. I knew my way to the Battery, pretty straight, for I had been down there on a Sunday with Joe, and Joe, sitting on an old gun, had told me that when I was 'prentice to him, regularly bound, we would have such Larks there! However, in the confusion of the mist, I found myself at last too far to the right, and consequently had to try back along the river-side, on the bank of loose stones above the mud and the stakes that staked the tide out. Making my way along here with all despatch, I had just crossed a ditch which I knew to be very near the Battery, and had just scrambled up the mound beyond the ditch, when I saw the man sitting before me. His back was towards me, and he had his arms folded, and was nodding forward, heavy with sleep.\n",
      "\n",
      "---------\n",
      "“And where the deuce ha' you been?” was Mrs. Joe's Christmas salutation, when I and my conscience showed ourselves.\n",
      "\n",
      "I said I had been down to hear the Carols. “Ah! well!” observed Mrs. Joe. “You might ha' done worse.” Not a doubt of that I thought.\n",
      "\n",
      "---------\n",
      "The sergeant and I were in the kitchen when Mrs. Joe stood staring; at which crisis I partially recovered the use of my senses. It was the sergeant who had spoken to me, and he was now looking round at the company, with his handcuffs invitingly extended towards them in his right hand, and his left on my shoulder.\n",
      "\n",
      "“Excuse me, ladies and gentleman,” said the sergeant, “but as I have mentioned at the door to this smart young shaver,” (which he hadn't), “I am on a chase in the name of the king, and I want the blacksmith.”\n",
      "\n",
      "---------\n",
      "I do not recall that I felt any tenderness of conscience in reference to Mrs. Joe, when the fear of being found out was lifted off me. But I loved Joe, perhaps for no better reason in those early days than because the dear fellow let me love him, and, as to him, my inner self was not so easily composed. It was much upon my mind (particularly when I first saw him looking about for his file) that I ought to tell Joe the whole truth. Yet I did not, and for the reason that I mistrusted that if I did, he would think me worse than I was. The fear of losing Joe's confidence, and of thenceforth sitting in the chimney corner at night staring drearily at my forever lost companion and friend, tied up my tongue. I morbidly represented to myself that if Joe knew it, I never afterwards could see him at the fireside feeling his fair whisker, without thinking that he was meditating on it. That, if Joe knew it, I never afterwards could see him glance, however casually, at yesterday's meat or pudding when it came on to-day's table, without thinking that he was debating whether I had been in the pantry. That, if Joe knew it, and at any subsequent period of our joint domestic life remarked that his beer was flat or thick, the conviction that he suspected tar in it, would bring a rush of blood to my face. In a word, I was too cowardly to do what I knew to be right, as I had been too cowardly to avoid doing what I knew to be wrong. I had had no intercourse with the world at that time, and I imitated none of its many inhabitants who act in this manner. Quite an untaught genius, I made the discovery of the line of action for myself.\n",
      "\n",
      "As I was sleepy before we were far away from the prison-ship, Joe took me on his back again and carried me home. He must have had a tiresome journey of it, for Mr. Wopsle, being knocked up, was in such a very bad temper that if the Church had been thrown open, he would probably have excommunicated the whole expedition, beginning with Joe and myself. In his lay capacity, he persisted in sitting down in the damp to such an insane extent, that when his coat was taken off to be dried at the kitchen fire, the circumstantial evidence on his trousers would have hanged him, if it had been a capital offence.\n",
      "\n",
      "---------\n",
      "When I was old enough, I was to be apprenticed to Joe, and until I could assume that dignity I was not to be what Mrs. Joe called “Pompeyed,” or (as I render it) pampered. Therefore, I was not only odd-boy about the forge, but if any neighbor happened to want an extra boy to frighten birds, or pick up stones, or do any such job, I was favored with the employment. In order, however, that our superior position might not be compromised thereby, a money-box was kept on the kitchen mantel-shelf, into which it was publicly made known that all my earnings were dropped. I have an impression that they were to be contributed eventually towards the liquidation of the National Debt, but I know I had no hope of any personal participation in the treasure.\n",
      "\n",
      "Mr. Wopsle's great-aunt kept an evening school in the village; that is to say, she was a ridiculous old woman of limited means and unlimited infirmity, who used to go to sleep from six to seven every evening, in the society of youth who paid two pence per week each, for the improving opportunity of seeing her do it. She rented a small cottage, and Mr. Wopsle had the room upstairs, where we students used to overhear him reading aloud in a most dignified and terrific manner, and occasionally bumping on the ceiling. There was a fiction that Mr. Wopsle “examined”  the scholars once a quarter. What he did on those occasions was to turn up his cuffs, stick up his hair, and give us Mark Antony's oration over the body of Caesar. This was always followed by Collins's Ode on the Passions, wherein I particularly venerated Mr. Wopsle as Revenge throwing his blood-stained sword in thunder down, and taking the War-denouncing trumpet with a withering look. It was not with me then, as it was in later life, when I fell into the society of the Passions, and compared them with Collins and Wopsle, rather to the disadvantage of both gentlemen.\n",
      "\n",
      "---------\n",
      "It was in the early morning after my arrival that I entertained this speculation. On the previous night, I had been sent straight to bed in an attic with a sloping roof, which was so low in the corner where the bedstead was, that I calculated the tiles as being within a foot of my eyebrows. In the same early morning, I discovered a singular affinity between seeds and corduroys. Mr. Pumblechook wore corduroys, and so did his shopman; and somehow, there was a general air and flavor about the corduroys, so much in the nature of seeds, and a general air and flavor about the seeds, so much in the nature of corduroys, that I hardly knew which was which. The same opportunity served me for noticing that Mr. Pumblechook appeared to conduct his business by looking across the street at the saddler, who appeared to transact his business by keeping his eye on the coachmaker, who appeared to get on in life by putting his hands in his pockets and contemplating the baker, who in his turn folded his arms and stared at the grocer, who stood at his door and yawned at the chemist. The watchmaker, always poring over a little desk with a magnifying-glass at his eye, and always inspected by a group of smock-frocks poring over him through the glass of his shop-window, seemed to be about the only person in the High Street whose trade engaged his attention.\n",
      "\n",
      "Mr. Pumblechook and I breakfasted at eight o'clock in the parlor behind the shop, while the shopman took his mug of tea and hunch of bread and butter on a sack of peas in the front premises. I considered Mr. Pumblechook wretched company. Besides being possessed by my sister's idea that a mortifying and penitential character ought to be imparted to my diet, besides giving me as much crumb as possible in combination with as little butter, and putting such a quantity of warm water into my milk that it would have been more candid to have left the milk out altogether, his conversation consisted of nothing but arithmetic. On my politely bidding him Good morning, he said, pompously, “Seven times nine, boy?” And how should I be able to answer, dodged in that way, in a strange place, on an empty stomach! I was hungry, but before I had swallowed a morsel, he began a running sum that lasted all through the breakfast. “Seven?” “And four?” “And eight?” “And six?” “And two?” “And ten?” And so on. And after each figure was disposed of, it was as much as I could do to get a bite or a sup, before the next came; while he sat at his ease guessing nothing, and eating bacon and hot roll, in (if I may be allowed the expression) a gorging and gormandizing manner.\n",
      "\n",
      "---------\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "If a dread of not being understood be hidden in the breasts of other young people to anything like the extent to which it used to be hidden in mine, which I consider probable, as I have no particular reason to suspect myself of having been a monstrosity, it is the key to many reservations. I felt convinced that if I described Miss Havisham's as my eyes had seen it, I should not be understood. Not only that, but I felt convinced that Miss Havisham too would not be understood; and although she was perfectly incomprehensible to me, I entertained an impression that there would be something coarse and treacherous in my dragging her as she really was (to say nothing of Miss Estella) before the contemplation of Mrs. Joe. Consequently, I said as little as I could, and had my face shoved against the kitchen wall.\n",
      "\n",
      "The worst of it was that that bullying old Pumblechook, preyed upon by a devouring curiosity to be informed of all I had seen and heard, came gaping over in his chaise-cart at tea-time, to have the details divulged to him. And the mere sight of the torment, with his fishy eyes and mouth open, his sandy hair inquisitively on end, and his waistcoat heaving with windy arithmetic, made me vicious in my reticence.\n",
      "\n",
      "---------\n",
      "The Educational scheme or Course established by Mr. Wopsle's great-aunt may be resolved into the following synopsis. The pupils ate apples and put straws down one another's backs, until Mr. Wopsle's great-aunt collected her energies, and made an indiscriminate totter at them with a birch-rod. After receiving the charge with every mark of derision, the pupils formed in line and buzzingly passed a ragged book from hand to hand. The book had an alphabet in it, some figures and tables, and a little spelling, that is to say, it had had once. As soon as this volume began to circulate, Mr. Wopsle's great-aunt fell into a state of coma, arising either from sleep or a rheumatic paroxysm. The pupils then entered among themselves upon a competitive examination on the subject of Boots, with the view of ascertaining who could tread the hardest upon whose toes. This mental exercise lasted until Biddy made a rush at them and distributed three defaced Bibles (shaped as if they had been unskilfully cut off the chump end of something), more illegibly printed at the best than any curiosities of literature I have since met with, speckled all over with ironmould, and having various specimens of the insect world smashed between their leaves. This part of the Course was usually lightened by several single combats between Biddy and refractory students. When the fights were over, Biddy gave out the number of a page, and then we all read aloud what we could, or what we couldn't in a frightful chorus; Biddy leading with a high, shrill, monotonous voice, and none of us having the least notion of, or reverence for, what we were reading about. When this horrible din had lasted a certain time, it mechanically awoke Mr. Wopsle's great-aunt, who staggered at a boy fortuitously, and pulled his ears. This was understood to terminate the Course for the evening, and we emerged into the air with shrieks of intellectual victory. It is fair to remark that there was no prohibition against any pupil's entertaining himself with a slate or even with the ink (when there was any), but that it was not easy to pursue that branch of study in the winter season, on account of the little general shop in which the classes were holden and which was also Mr. Wopsle's great-aunt's sitting-room and bedchamber being but faintly illuminated through the agency of one low-spirited dip-candle and no snuffers.\n",
      "\n",
      "It appeared to me that it would take time to become uncommon, under these circumstances: nevertheless, I resolved to try it, and that very evening Biddy entered on our special agreement, by imparting some information from her little catalogue of Prices, under the head of moist sugar, and lending me, to copy at home, a large old English D which she had imitated from the heading of some newspaper, and which I supposed, until she told me what it was, to be a design for a buckle.\n",
      "\n",
      "---------\n",
      "The passage was a long one, and seemed to pervade the whole square basement of the Manor House. We traversed but one side of the square, however, and at the end of it she stopped, and put her candle down and opened a door. Here, the daylight reappeared, and I found myself in a small paved courtyard, the opposite side of which was formed by a detached dwelling-house, that looked as if it had once belonged to the manager or head clerk of the extinct brewery. There was a clock in the outer wall of this house. Like the clock in Miss Havisham's room, and like Miss Havisham's watch, it had stopped at twenty minutes to nine.\n",
      "\n",
      "We went in at the door, which stood open, and into a gloomy room with a low ceiling, on the ground-floor at the back. There was some company in the room, and Estella said to me as she joined it, “You are to go and stand there boy, till you are wanted.” “There”, being the window, I crossed to it, and stood “there,” in a very uncomfortable state of mind, looking out.\n",
      "\n",
      "---------\n",
      "When the day came round for my return to the scene of the deed of violence, my terrors reached their height. Whether myrmidons of Justice, specially sent down from London, would be lying in ambush behind the gate; whether Miss Havisham, preferring to take personal vengeance for an outrage done to her house, might rise in those grave-clothes of hers, draw a pistol, and shoot me dead: whether suborned boys a numerous band of mercenaries might be engaged to fall upon me in the brewery, and cuff me until I was no more; it was high testimony to my confidence in the spirit of the pale young gentleman, that I never imagined him accessory to these retaliations; they always came into my mind as the acts of injudicious relatives of his, goaded on by the state of his visage and an indignant sympathy with the family features.\n",
      "\n",
      "However, go to Miss Havisham's I must, and go I did. And behold! nothing came of the late struggle. It was not alluded to in any way, and no pale young gentleman was to be discovered on the premises. I found the same gate open, and I explored the garden, and even looked in at the windows of the detached house; but my view was suddenly stopped by the closed shutters within, and all was lifeless. Only in the corner where the combat had taken place could I detect any evidence of the young gentleman's existence. There were traces of his gore in that spot, and I covered them with garden-mould from the eye of man.\n",
      "\n",
      "---------\n",
      "At breakfast-time my sister declared her intention of going to town with us, and being left at Uncle Pumblechook's and called for “when we had done with our fine ladies” a way of putting the case, from which Joe appeared inclined to augur the worst. The forge was shut up for the day, and Joe inscribed in chalk upon the door (as it was his custom to do on the very rare occasions when he was not at work) the monosyllable HOUT, accompanied by a sketch of an arrow supposed to be flying in the direction he had taken.\n",
      "\n",
      "We walked to town, my sister leading the way in a very large beaver bonnet, and carrying a basket like the Great Seal of England in plaited Straw, a pair of pattens, a spare shawl, and an umbrella, though it was a fine bright day. I am not quite clear whether these articles were carried penitentially or ostentatiously; but I rather think they were displayed as articles of property, much as Cleopatra or any other sovereign lady on the Rampage might exhibit her wealth in a pageant or procession.\n",
      "\n",
      "---------\n",
      "Home had never been a very pleasant place to me, because of my sister's temper. But, Joe had sanctified it, and I had believed in it. I had believed in the best parlor as a most elegant saloon; I had believed in the front door, as a mysterious portal of the Temple of State whose solemn opening was attended with a sacrifice of roast fowls; I had believed in the kitchen as a chaste though not magnificent apartment; I had believed in the forge as the glowing road to manhood and independence. Within a single year all this was changed. Now it was all coarse and common, and I would not have had Miss Havisham and Estella see it on any account.\n",
      "\n",
      "How much of my ungracious condition of mind may have been my own fault, how much Miss Havisham's, how much my sister's, is now of no moment to me or to any one. The change was made in me; the thing was done. Well or ill done, excusably or inexcusably, it was done.\n",
      "\n",
      "---------\n",
      "When I went to Lunnon town sirs,      Too rul loo rul      Too rul loo rul      Wasn't I done very brown sirs?      Too rul loo rul      Too rul loo rul\n",
      "\n",
      "still, in my desire to be wiser, I got this composition by heart with the utmost gravity; nor do I recollect that I questioned its merit, except that I thought (as I still do) the amount of Too rul somewhat in excess of the poetry. In my hunger for information, I made proposals to Mr. Wopsle to bestow some intellectual crumbs upon me, with which he kindly complied. As it turned out, however, that he only wanted me for a dramatic lay-figure, to be contradicted and embraced and wept over and bullied and clutched and stabbed and knocked about in a variety of ways, I soon declined that course of instruction; though not until Mr. Wopsle in his poetic fury had severely mauled me.\n",
      "\n",
      "---------\n",
      "Joe had been at the Three Jolly Bargemen, smoking his pipe, from a quarter after eight o'clock to a quarter before ten. While he was there, my sister had been seen standing at the kitchen door, and had exchanged Good Night with a farm-laborer going home. The man could not be more particular as to the time at which he saw her (he got into dense confusion when he tried to be), than that it must have been before nine. When Joe went home at five minutes before ten, he found her struck down on the floor, and promptly called in assistance. The fire had not then burnt unusually low, nor was the snuff of the candle very long; the candle, however, had been blown out.\n",
      "\n",
      "Nothing had been taken away from any part of the house. Neither, beyond the blowing out of the candle, which stood on a table between the door and my sister, and was behind her when she stood facing the fire and was struck, was there any disarrangement of the kitchen, excepting such as she herself had made, in falling and bleeding. But, there was one remarkable piece of evidence on the spot. She had been struck with something blunt and heavy, on the head and spine; after the blows were dealt, something heavy had been thrown down at her with considerable violence, as she lay on her face. And on the ground beside her, when Joe picked her up, was a convict's leg-iron which had been filed asunder.\n",
      "\n",
      "---------\n",
      "So unchanging was the dull old house, the yellow light in the darkened room, the faded spectre in the chair by the dressing-table glass, that I felt as if the stopping of the clocks had stopped Time in that mysterious place, and, while I and everything else outside it grew older, it stood still. Daylight never entered the house as to my thoughts and remembrances of it, any more than as to the actual fact. It bewildered me, and under its influence I continued at heart to hate my trade and to be ashamed of home.\n",
      "\n",
      "Imperceptibly I became conscious of a change in Biddy, however. Her shoes came up at the heel, her hair grew bright and neat, her hands were always clean. She was not beautiful, she was common, and could not be like Estella, but she was pleasant and wholesome and sweet-tempered. She had not been with us more than a year (I remember her being newly out of mourning at the time it struck me), when I observed to myself one evening that she had curiously thoughtful and attentive eyes; eyes that were very pretty and very good.\n",
      "\n",
      "---------\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A highly popular murder had been committed, and Mr. Wopsle was imbrued in blood to the eyebrows. He gloated over every abhorrent adjective in the description, and identified himself with every witness at the Inquest. He faintly moaned, “I am done for,” as the victim, and he barbarously bellowed, “I'll serve you out,” as the murderer. He gave the medical testimony, in pointed imitation of our local practitioner; and he piped and shook, as the aged turnpike-keeper who had heard blows, to an extent so very paralytic as to suggest a doubt regarding the mental competency of that witness. The coroner, in Mr. Wopsle's hands, became Timon of Athens; the beadle, Coriolanus. He enjoyed himself thoroughly, and we all enjoyed ourselves, and were delightfully comfortable. In this cosey state of mind we came to the verdict Wilful Murder.\n",
      "\n",
      "Then, and not sooner, I became aware of a strange gentleman leaning over the back of the settle opposite me, looking on. There was an expression of contempt on his face, and he bit the side of a great forefinger as he watched the group of faces.\n",
      "\n",
      "---------\n",
      "Joe and Biddy were very sympathetic and pleasant when I spoke of our approaching separation; but they only referred to it when I did. After breakfast, Joe brought out my indentures from the press in the best parlor, and we put them in the fire, and I felt that I was free. With all the novelty of my emancipation on me, I went to church with Joe, and thought perhaps the clergyman wouldn't have read that about the rich man and the kingdom of Heaven, if he had known all.\n",
      "\n",
      "After our early dinner, I strolled out alone, purposing to finish off the marshes at once, and get them done with. As I passed the church, I felt (as I had felt during service in the morning) a sublime compassion for the poor creatures who were destined to go there, Sunday after Sunday, all their lives through, and to lie obscurely at last among the low green mounds. I promised myself that I would do something for them one of these days, and formed a plan in outline for bestowing a dinner of roast-beef and plum-pudding, a pint of ale, and a gallon of condescension, upon everybody in the village.\n",
      "\n",
      "---------\n",
      "We Britons had at that time particularly settled that it was treasonable to doubt our having and our being the best of everything: otherwise, while I was scared by the immensity of London, I think I might have had some faint doubts whether it was not rather ugly, crooked, narrow, and dirty.\n",
      "\n",
      "Mr. Jaggers had duly sent me his address; it was, Little Britain, and he had written after it on his card, “just out of Smithfield, and close by the coach-office.” Nevertheless, a hackney-coachman, who seemed to have as many capes to his greasy great-coat as he was years old, packed me up in his coach and hemmed me in with a folding and jingling barrier of steps, as if he were going to take me fifty miles. His getting on his box, which I remember to have been decorated with an old weather-stained pea-green hammercloth moth-eaten into rags, was quite a work of time. It was a wonderful equipage, with six great coronets outside, and ragged things behind for I don't know how many footmen to hold on by, and a harrow below them, to prevent amateur footmen from yielding to the temptation.\n",
      "\n",
      "---------\n",
      "“So you were never in London before?” said Mr. Wemmick to me.\n",
      "\n",
      "“No,” said I.\n",
      "\n",
      "---------\n",
      "I derived from this speech that Mr. Herbert Pocket (for Herbert was the pale young gentleman's name) still rather confounded his intention with his execution. But I made a modest reply, and we shook hands warmly.\n",
      "\n",
      "“You hadn't come into your good fortune at that time?” said Herbert Pocket.\n",
      "\n",
      "---------\n",
      "I found out within a few hours, and may mention at once, that Mrs. Pocket was the only daughter of a certain quite accidental deceased Knight, who had invented for himself a conviction that his deceased father would have been made a Baronet but for somebody's determined opposition arising out of entirely personal motives, I forget whose, if I ever knew, the Sovereign's, the Prime Minister's, the Lord Chancellor's, the Archbishop of Canterbury's, anybody's, and had tacked himself on to the nobles of the earth in right of this quite supposititious fact. I believe he had been knighted himself for storming the English grammar at the point of the pen, in a desperate address engrossed on vellum, on the occasion of the laying of the first stone of some building or other, and for handing some Royal Personage either the trowel or the mortar. Be that as it may, he had directed Mrs. Pocket to be brought up from her cradle as one who in the nature of things must marry a title, and who was to be guarded from the acquisition of plebeian domestic knowledge.\n",
      "\n",
      "So successful a watch and ward had been established over the young lady by this judicious parent, that she had grown up highly ornamental, but perfectly helpless and useless. With her character thus happily formed, in the first bloom of her youth she had encountered Mr. Pocket: who was also in the first bloom of youth, and not quite decided whether to mount to the Woolsack, or to roof himself in with a mitre. As his doing the one or the other was a mere question of time, he and Mrs. Pocket had taken Time by the forelock (when, to judge from its length, it would seem to have wanted cutting), and had married without the knowledge of the judicious parent. The judicious parent, having nothing to bestow or withhold but his blessing, had handsomely settled that dower upon them after a short struggle, and had informed Mr. Pocket that his wife was “a treasure for a Prince.” Mr. Pocket had invested the Prince's treasure in the ways of the world ever since, and it was supposed to have brought him in but indifferent interest. Still, Mrs. Pocket was in general the object of a queer sort of respectful pity, because she had not married a title; while Mr. Pocket was the object of a queer sort of forgiving reproach, because he had never got one.\n",
      "\n",
      "---------\n",
      "He advised my attending certain places in London, for the acquisition of such mere rudiments as I wanted, and my investing him with the functions of explainer and director of all my studies. He hoped that with intelligent assistance I should meet with little to discourage me, and should soon be able to dispense with any aid but his. Through his way of saying this, and much more to similar purpose, he placed himself on confidential terms with me in an admirable manner; and I may state at once that he was always so zealous and honorable in fulfilling his compact with me, that he made me zealous and honorable in fulfilling mine with him. If he had shown indifference as a master, I have no doubt I should have returned the compliment as a pupil; he gave me no such excuse, and each of us did the other justice. Nor did I ever regard him as having anything ludicrous about him or anything but what was serious, honest, and good in his tutor communication with me.\n",
      "\n",
      "When these points were settled, and so far carried out as that I had begun to work in earnest, it occurred to me that if I could retain my bedroom in Barnard's Inn, my life would be agreeably varied, while my manners would be none the worse for Herbert's society. Mr. Pocket did not object to this arrangement, but urged that before any step could possibly be taken in it, it must be submitted to my guardian. I felt that this delicacy arose out of the consideration that the plan would save Herbert some expense, so I went off to Little Britain and imparted my wish to Mr. Jaggers.\n",
      "\n",
      "---------\n",
      "Startop had been spoilt by a weak mother and kept at home when he ought to have been at school, but he was devotedly attached to her, and admired her beyond measure. He had a woman's delicacy of feature, and was “as you may see, though you never saw her,” said Herbert to me “exactly like his mother.” It was but natural that I should take to him much more kindly than to Drummle, and that, even in the earliest evenings of our boating, he and I should pull homeward abreast of one another, conversing from boat to boat, while Bentley Drummle came up in our wake alone, under the overhanging banks and among the rushes. He would always creep in-shore like some uncomfortable amphibious creature, even when the tide would have sent him fast upon his way; and I always think of him as coming after us in the dark or by the back-water, when our own two boats were breaking the sunset or the moonlight in mid-stream.\n",
      "\n",
      "Herbert was my intimate companion and friend. I presented him with a half-share in my boat, which was the occasion of his often coming down to Hammersmith; and my possession of a half-share in his chambers often took me up to London. We used to walk between the two places at all hours. I have an affection for the road yet (though it is not so pleasant a road as it was then), formed in the impressibility of untried youth and hope.\n",
      "\n",
      "---------\n",
      "There were some people slinking about as usual when we passed out into the street, who were evidently anxious to speak with him; but there was something so conclusive in the halo of scented soap which encircled his presence, that they gave it up for that day. As we walked along westward, he was recognized ever and again by some face in the crowd of the streets, and whenever that happened he talked louder to me; but he never otherwise recognized anybody, or took notice that anybody recognized him.\n",
      "\n",
      "He conducted us to Gerrard Street, Soho, to a house on the south side of that street. Rather a stately house of its kind, but dolefully in want of painting, and with dirty windows. He took out his key and opened the door, and we all went into a stone hall, bare, gloomy, and little used. So, up a dark brown staircase into a series of three dark brown rooms on the first floor. There were carved garlands on the panelled walls, and as he stood among them giving us welcome, I know what kind of loops I thought they looked like.\n",
      "\n",
      "---------\n",
      "“I write this by request of Mr. Gargery, for to let you know that he is going to London in company with Mr. Wopsle and would be glad if agreeable to be allowed to see you. He would call at Barnard's Hotel Tuesday morning at nine o'clock, when if not agreeable please leave word. Your poor sister is much the same as when you left. We talk of you in the kitchen every night, and wonder what you are saying and doing. If now considered in the light of a liberty, excuse it for the love of poor old days. No more, dear Mr. Pip, from your ever obliged, and affectionate servant,\n",
      "\n",
      "“BIDDY.”\n",
      "\n",
      "---------\n",
      "Having settled that I must go to the Blue Boar, my mind was much disturbed by indecision whether or not to take the Avenger. It was tempting to think of that expensive Mercenary publicly airing his boots in the archway of the Blue Boar's posting-yard; it was almost solemn to imagine him casually produced in the tailor's shop, and confounding the disrespectful senses of Trabb's boy. On the other hand, Trabb's boy might worm himself into his intimacy and tell him things; or, reckless and desperate wretch as I knew he could be, might hoot him in the High Street. My patroness, too, might hear of him, and not approve. On the whole, I resolved to leave the Avenger behind.\n",
      "\n",
      "It was the afternoon coach by which I had taken my place, and, as winter had now come round, I should not arrive at my destination until two or three hours after dark. Our time of starting from the Cross Keys was two o'clock. I arrived on the ground with a quarter of an hour to spare, attended by the Avenger, if I may connect that expression with one who never attended on me if he could possibly help it.\n",
      "\n",
      "---------\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "She had adopted Estella, she had as good as adopted me, and it could not fail to be her intention to bring us together. She reserved it for me to restore the desolate house, admit the sunshine into the dark rooms, set the clocks a-going and the cold hearths a-blazing, tear down the cobwebs, destroy the vermin, in short, do all the shining deeds of the young Knight of romance, and marry the Princess. I had stopped to look at the house as I passed; and its seared red brick walls, blocked windows, and strong green ivy clasping even the stacks of chimneys with its twigs and tendons, as if with sinewy old arms, had made up a rich attractive mystery, of which I was the hero. Estella was the inspiration of it, and the heart of it, of course. But, though she had taken such strong possession of me, though my fancy and my hope were so set upon her, though her influence on my boyish life and character had been all-powerful, I did not, even that romantic morning, invest her with any attributes save those she possessed. I mention this in this place, of a fixed purpose, because it is the clew by which I am to be followed into my poor labyrinth. According to my experience, the conventional notion of a lover cannot be always true. The unqualified truth is, that when I loved Estella with the love of a man, I loved her simply because I found her irresistible. Once for all; I knew to my sorrow, often and often, if not always, that I loved her against reason, against promise, against peace, against hope, against happiness, against all discouragement that could be. Once for all; I loved her none the less because I knew it, and it had no more influence in restraining me than if I had devoutly believed her to be human perfection.\n",
      "\n",
      "I so shaped out my walk as to arrive at the gate at my old time. When I had rung at the bell with an unsteady hand, I turned my back upon the gate, while I tried to get my breath and keep the beating of my heart moderately quiet. I heard the side-door open, and steps come across the courtyard; but I pretended not to hear, even when the gate swung on its rusty hinges.\n",
      "\n",
      "---------\n",
      "As we were going back together to London by the midday coach, and as I breakfasted under such terrors of Pumblechook that I could scarcely hold my cup, this gave me an opportunity of saying that I wanted a walk, and that I would go on along the London road while Mr. Jaggers was occupied, if he would let the coachman know that I would get into my place when overtaken. I was thus enabled to fly from the Blue Boar immediately after breakfast. By then making a loop of about a couple of miles into the open country at the back of Pumblechook's premises, I got round into the High Street again, a little beyond that pitfall, and felt myself in comparative security.\n",
      "\n",
      "It was interesting to be in the quiet old town once more, and it was not disagreeable to be here and there suddenly recognized and stared after. One or two of the tradespeople even darted out of their shops and went a little way down the street before me, that they might turn, as if they had forgotten something, and pass me face to face, on which occasions I don't know whether they or I made the worse pretence; they of not doing it, or I of not seeing it. Still my position was a distinguished one, and I was not at all dissatisfied with it, until Fate threw me in the way of that unlimited miscreant, Trabb's boy.\n",
      "\n",
      "---------\n",
      "Several curious little circumstances transpired as the action proceeded. The late king of the country not only appeared to have been troubled with a cough at the time of his decease, but to have taken it with him to the tomb, and to have brought it back. The royal phantom also carried a ghostly manuscript round its truncheon, to which it had the appearance of occasionally referring, and that too, with an air of anxiety and a tendency to lose the place of reference which were suggestive of a state of mortality. It was this, I conceive, which led to the Shade's being advised by the gallery to “turn over!” a recommendation which it took extremely ill. It was likewise to be noted of this majestic spirit, that whereas it always appeared with an air of having been out a long time and walked an immense distance, it perceptibly came from a closely contiguous wall. This occasioned its terrors to be received derisively. The Queen of Denmark, a very buxom lady, though no doubt historically brazen, was considered by the public to have too much brass about her; her chin being attached to her diadem by a broad band of that metal (as if she had a gorgeous toothache), her waist being encircled by another, and each of her arms by another, so that she was openly mentioned as “the kettle-drum.” The noble boy in the ancestral boots was inconsistent, representing himself, as it were in one breath, as an able seaman, a strolling actor, a grave-digger, a clergyman, and a person of the utmost importance at a Court fencing-match, on the authority of whose practised eye and nice discrimination the finest strokes were judged. This gradually led to a want of toleration for him, and even on his being detected in holy orders, and declining to perform the funeral service to the general indignation taking the form of nuts. Lastly, Ophelia was a prey to such slow musical madness, that when, in course of time, she had taken off her white muslin scarf, folded it up, and buried it, a sulky man who had been long cooling his impatient nose against an iron bar in the front row of the gallery, growled, “Now the baby's put to bed let's have supper!” Which, to say the least of it, was out of keeping.\n",
      "\n",
      "Upon my unfortunate townsman all these incidents accumulated with playful effect. Whenever that undecided Prince had to ask a question or state a doubt, the public helped him out with it. As for example; on the question whether 'twas nobler in the mind to suffer, some roared yes, and some no, and some inclining to both opinions said “Toss up for it;” and quite a Debating Society arose. When he asked what should such fellows as he do crawling between earth and heaven, he was encouraged with loud cries of “Hear, hear!” When he appeared with his stocking disordered (its disorder expressed, according to usage, by one very neat fold in the top, which I suppose to be always got up with a flat iron), a conversation took place in the gallery respecting the paleness of his leg, and whether it was occasioned by the turn the ghost had given him. On his taking the recorders, very like a little black flute that had just been played in the orchestra and handed out at the door, he was called upon unanimously for Rule Britannia. When he recommended the player not to saw the air thus, the sulky man said, “And don't you do it, neither; you're a deal worse than him!” And I grieve to add that peals of laughter greeted Mr. Wopsle on every one of these occasions.\n",
      "\n",
      "---------\n",
      "“I am to come to London the day after to-morrow by the midday coach. I believe it was settled you should meet me? At all events Miss Havisham has that impression, and I write in obedience to it. She sends you her regard.\n",
      "\n",
      "“Yours, ESTELLA.”\n",
      "\n",
      "---------\n",
      "We stood in the Inn Yard while she pointed out her luggage to me, and when it was all collected I remembered having forgotten everything but herself in the meanwhile that I knew nothing of her destination.\n",
      "\n",
      "“I am going to Richmond,” she told me. “Our lesson is, that there are two Richmonds, one in Surrey and one in Yorkshire, and that mine is the Surrey Richmond. The distance is ten miles. I am to have a carriage, and you are to take me. This is my purse, and you are to pay my charges out of it. O, you must take the purse! We have no choice, you and I, but to obey our instructions. We are not free to follow our own devices, you and I.”\n",
      "\n",
      "---------\n",
      "Yet Estella was so inseparable from all my restlessness and disquiet of mind, that I really fell into confusion as to the limits of my own part in its production. That is to say, supposing I had had no expectations, and yet had had Estella to think of, I could not make out to my satisfaction that I should have done much better. Now, concerning the influence of my position on others, I was in no such difficulty, and so I perceived though dimly enough perhaps that it was not beneficial to anybody, and, above all, that it was not beneficial to Herbert. My lavish habits led his easy nature into expenses that he could not afford, corrupted the simplicity of his life, and disturbed his peace with anxieties and regrets. I was not at all remorseful for having unwittingly set those other branches of the Pocket family to the poor arts they practised; because such littlenesses were their natural bent, and would have been evoked by anybody else, if I had left them slumbering. But Herbert's was a very different case, and it often caused me a twinge to think that I had done him evil service in crowding his sparely furnished chambers with incongruous upholstery work, and placing the Canary-breasted Avenger at his disposal.\n",
      "\n",
      "So now, as an infallible way of making little ease great ease, I began to contract a quantity of debt. I could hardly begin but Herbert must begin too, so he soon followed. At Startop's suggestion, we put ourselves down for election into a club called The Finches of the Grove: the object of which institution I have never divined, if it were not that the members should dine expensively once a fortnight, to quarrel among themselves as much as possible after dinner, and to cause six waiters to get drunk on the stairs. I know that these gratifying social ends were so invariably accomplished, that Herbert and I understood nothing else to be referred to in the first standing toast of the society: which ran “Gentlemen, may the present promotion of good feeling ever reign predominant among the Finches of the Grove.”\n",
      "\n",
      "---------\n",
      "Whatever my fortunes might have been, I could scarcely have recalled my sister with much tenderness. But I suppose there is a shock of regret which may exist without much tenderness. Under its influence (and perhaps to make up for the want of the softer feeling) I was seized with a violent indignation against the assailant from whom she had suffered so much; and I felt that on sufficient proof I could have revengefully pursued Orlick, or any one else, to the last extremity.\n",
      "\n",
      "Having written to Joe, to offer him consolation, and to assure him that I would come to the funeral, I passed the intermediate days in the curious state of mind I have glanced at. I went down early in the morning, and alighted at the Blue Boar in good time to walk over to the forge.\n",
      "\n",
      "---------\n",
      "Herbert himself had come of age eight months before me. As he had nothing else than his majority to come into, the event did not make a profound sensation in Barnard's Inn. But we had looked forward to my one-and-twentieth birthday, with a crowd of speculations and anticipations, for we had both considered that my guardian could hardly help saying something definite on that occasion.\n",
      "\n",
      "I had taken care to have it well understood in Little Britain when my birthday was. On the day before it, I received an official note from Wemmick, informing me that Mr. Jaggers would be glad if I would call upon him at five in the afternoon of the auspicious day. This convinced us that something great was to happen, and threw me into an unusual flutter when I repaired to my guardian's office, a model of punctuality.\n",
      "\n",
      "---------\n",
      "“My son, sir,” said the old man, after securing the drawbridge, “rather had it in his mind that you might happen to drop in, and he left word that he would soon be home from his afternoon's walk. He is very regular in his walks, is my son. Very regular in everything, is my son.”\n",
      "\n",
      "I nodded at the old gentleman as Wemmick himself might have nodded, and we went in and sat down by the fireside.\n",
      "\n",
      "---------\n",
      "The lady with whom Estella was placed, Mrs. Brandley by name, was a widow, with one daughter several years older than Estella. The mother looked young, and the daughter looked old; the mother's complexion was pink, and the daughter's was yellow; the mother set up for frivolity, and the daughter for theology. They were in what is called a good position, and visited, and were visited by, numbers of people. Little, if any, community of feeling subsisted between them and Estella, but the understanding was established that they were necessary to her, and that she was necessary to them. Mrs. Brandley had been a friend of Miss Havisham's before the time of her seclusion.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "In Mrs. Brandley's house and out of Mrs. Brandley's house, I suffered every kind and degree of torture that Estella could cause me. The nature of my relations with her, which placed me on terms of familiarity without placing me on terms of favor, conduced to my distraction. She made use of me to tease other admirers, and she turned the very familiarity between herself and me to the account of putting a constant slight on my devotion to her. If I had been her secretary, steward, half-brother, poor relation, if I had been a younger brother of her appointed husband, I could not have seemed to myself further from my hopes when I was nearest to her. The privilege of calling her by her name and hearing her call me by mine became, under the circumstances an aggravation of my trials; and while I think it likely that it almost maddened her other lovers, I know too certainly that it almost maddened me.\n",
      "\n",
      "---------\n",
      "Mr. Pocket and I had for some time parted company as to our original relations, though we continued on the best terms. Notwithstanding my inability to settle to anything, which I hope arose out of the restless and incomplete tenure on which I held my means, I had a taste for reading, and read regularly so many hours a day. That matter of Herbert's was still progressing, and everything with me was as I have brought it down to the close of the last preceding chapter.\n",
      "\n",
      "Business had taken Herbert on a journey to Marseilles. I was alone, and had a dull sense of being alone. Dispirited and anxious, long hoping that to-morrow or next week would clear my way, and long disappointed, I sadly missed the cheerful face and ready response of my friend.\n",
      "\n",
      "---------\n",
      "The impossibility of keeping him concealed in the chambers was self-evident. It could not be done, and the attempt to do it would inevitably engender suspicion. True, I had no Avenger in my service now, but I was looked after by an inflammatory old female, assisted by an animated rag-bag whom she called her niece, and to keep a room secret from them would be to invite curiosity and exaggeration. They both had weak eyes, which I had long attributed to their chronically looking in at keyholes, and they were always at hand when not wanted; indeed that was their only reliable quality besides larceny. Not to get up a mystery with these people, I resolved to announce in the morning that my uncle had unexpectedly come from the country.\n",
      "\n",
      "This course I decided on while I was yet groping about in the darkness for the means of getting a light. Not stumbling on the means after all, I was fain to go out to the adjacent Lodge and get the watchman there to come with his lantern. Now, in groping my way down the black staircase I fell over something, and that something was a man crouching in a corner.\n",
      "\n",
      "---------\n",
      "What would alone have set a division between that man and us, if there had been no other dividing circumstance, was his triumph in my story. Saving his troublesome sense of having been “low” on one occasion since his return, on which point he began to hold forth to Herbert, the moment my revelation was finished, he had no perception of the possibility of my finding any fault with my good fortune. His boast that he had made me a gentleman, and that he had come to see me support the character on his ample resources, was made for me quite as much as for himself. And that it was a highly agreeable boast to both of us, and that we must both be very proud of it, was a conclusion quite established in his own mind.\n",
      "\n",
      "“Though, look'ee here, Pip's comrade,” he said to Herbert, after having discoursed for some time, “I know very well that once since I come back for half a minute I've been low. I said to Pip, I knowed as I had been low. But don't you fret yourself on that score. I ain't made Pip a gentleman, and Pip ain't a going to make you a gentleman, not fur me not to know what's due to ye both. Dear boy, and Pip's comrade, you two may count upon me always having a gen-teel muzzle on. Muzzled I have been since that half a minute when I was betrayed into lowness, muzzled I am at the present time, muzzled I ever will be.”\n",
      "\n",
      "---------\n",
      "“I've been done everything to, pretty well except hanged. I've been locked up as much as a silver tea-kittle. I've been carted here and carted there, and put out of this town, and put out of that town, and stuck in the stocks, and whipped and worried and drove. I've no more notion where I was born than you have if so much. I first become aware of myself down in Essex, a thieving turnips for my living. Summun had run away from me a man a tinker and he'd took the fire with him, and left me wery cold.\n",
      "\n",
      "“I know'd my name to be Magwitch, chrisen'd Abel. How did I know it? Much as I know'd the birds' names in the hedges to be chaffinch, sparrer, thrush. I might have thought it was all lies together, only as the birds' names come out true, I supposed mine did.\n",
      "\n",
      "---------\n",
      "A new fear had been engendered in my mind by his narrative; or rather, his narrative had given form and purpose to the fear that was already there. If Compeyson were alive and should discover his return, I could hardly doubt the consequence. That Compeyson stood in mortal fear of him, neither of the two could know much better than I; and that any such man as that man had been described to be would hesitate to release himself for good from a dreaded enemy by the safe means of becoming an informer was scarcely to be imagined.\n",
      "\n",
      "Never had I breathed, and never would I breathe or so I resolved a word of Estella to Provis. But, I said to Herbert that, before I could go abroad, I must see both Estella and Miss Havisham. This was when we were left alone on the night of the day when Provis told us his story. I resolved to go out to Richmond next day, and I went.\n",
      "\n",
      "---------\n",
      "“And what wind,” said Miss Havisham, “blows you here, Pip?”\n",
      "\n",
      "Though she looked steadily at me, I saw that she was rather confused. Estella, pausing a moment in her knitting with her eyes upon me, and then going on, I fancied that I read in the action of her fingers, as plainly as if she had told me in the dumb alphabet, that she perceived I had discovered my real benefactor.\n",
      "\n",
      "---------\n",
      "As I had asked for a night-light, the chamberlain had brought me in, before he left me, the good old constitutional rushlight of those virtuous days an object like the ghost of a walking-cane, which instantly broke its back if it were touched, which nothing could ever be lighted at, and which was placed in solitary confinement at the bottom of a high tin tower, perforated with round holes that made a staringly wide-awake pattern on the walls. When I had got into bed, and lay there footsore, weary, and wretched, I found that I could no more close my own eyes than I could close the eyes of this foolish Argus. And thus, in the gloom and death of the night, we stared at one another.\n",
      "\n",
      "What a doleful night! How anxious, how dismal, how long! There was an inhospitable smell in the room, of cold soot and hot dust; and, as I looked up into the corners of the tester over my head, I thought what a number of blue-bottle flies from the butchers', and earwigs from the market, and grubs from the country, must be holding on up there, lying by for next summer. This led me to speculate whether any of them ever tumbled down, and then I fancied that I felt light falls on my face, a disagreeable turn of thought, suggesting other and more objectionable approaches up my back. When I had lain awake a little while, those extraordinary voices with which silence teems began to make themselves audible. The closet whispered, the fireplace sighed, the little washing-stand ticked, and one guitar-string played occasionally in the chest of drawers. At about the same time, the eyes on the wall acquired a new expression, and in every one of those staring rounds I saw written, DON'T GO HOME.\n",
      "\n",
      "---------\n",
      "It matters not what stranded ships repairing in dry docks I lost myself among, what old hulls of ships in course of being knocked to pieces, what ooze and slime and other dregs of tide, what yards of ship-builders and ship-breakers, what rusty anchors blindly biting into the ground, though for years off duty, what mountainous country of accumulated casks and timber, how many rope-walks that were not the Old Green Copper. After several times falling short of my destination and as often overshooting it, I came unexpectedly round a corner, upon Mill Pond Bank. It was a fresh kind of place, all circumstances considered, where the wind from the river had room to turn itself round; and there were two or three trees in it, and there was the stump of a ruined windmill, and there was the Old Green Copper Rope-walk, whose long and narrow vista I could trace in the moonlight, along a series of wooden frames set in the ground, that looked like superannuated haymaking-rakes which had grown old and lost most of their teeth.\n",
      "\n",
      "Selecting from the few queer houses upon Mill Pond Bank a house with a wooden front and three stories of bow-window (not bay-window, which is another thing), I looked at the plate upon the door, and read there, Mrs. Whimple. That being the name I wanted, I knocked, and an elderly woman of a pleasant and thriving appearance responded. She was immediately deposed, however, by Herbert, who silently led me into the parlor and shut the door. It was an odd sensation to see his very familiar face established quite at home in that very unfamiliar room and region; and I found myself looking at him, much as I looked at the corner-cupboard with the glass and china, the shells upon the chimney-piece, and the colored engravings on the wall, representing the death of Captain Cook, a ship-launch, and his Majesty King George the Third in a state coachman's wig, leather-breeches, and top-boots, on the terrace at Windsor.\n",
      "\n",
      "---------\n",
      "My worldly affairs began to wear a gloomy appearance, and I was pressed for money by more than one creditor. Even I myself began to know the want of money (I mean of ready money in my own pocket), and to relieve it by converting some easily spared articles of jewelery into cash. But I had quite determined that it would be a heartless fraud to take more money from my patron in the existing state of my uncertain thoughts and plans. Therefore, I had sent him the unopened pocket-book by Herbert, to hold in his own keeping, and I felt a kind of satisfaction whether it was a false kind or a true, I hardly know in not having profited by his generosity since his revelation of himself.\n",
      "\n",
      "As the time wore on, an impression settled heavily upon me that Estella was married. Fearful of having it confirmed, though it was all but a conviction, I avoided the newspapers, and begged Herbert (to whom I had confided the circumstances of our last interview) never to speak of her to me. Why I hoarded up this last wretched little rag of the robe of hope that was rent and given to the winds, how do I know? Why did you who read this, commit that not dissimilar inconsistency of your own last year, last month, last week?\n",
      "\n",
      "---------\n",
      "“As we are going in the same direction, Pip, we may walk together. Where are you bound for?”\n",
      "\n",
      "“For the Temple, I think,” said I.\n",
      "\n",
      "---------\n",
      "The best light of the day was gone when I passed along the quiet echoing courts behind the High Street. The nooks of ruin where the old monks had once had their refectories and gardens, and where the strong walls were now pressed into the service of humble sheds and stables, were almost as silent as the old monks in their graves. The cathedral chimes had at once a sadder and a more remote sound to me, as I hurried on avoiding observation, than they had ever had before; so, the swell of the old organ was borne to my ears like funeral music; and the rooks, as they hovered about the gray tower and swung in the bare high trees of the priory garden, seemed to call to me that the place was changed, and that Estella was gone out of it for ever.\n",
      "\n",
      "An elderly woman, whom I had seen before as one of the servants who lived in the supplementary house across the back courtyard, opened the gate. The lighted candle stood in the dark passage within, as of old, and I took it up and ascended the staircase alone. Miss Havisham was not in her own room, but was in the larger room across the landing. Looking in at the door, after knocking in vain, I saw her sitting on the hearth in a ragged chair, close before, and lost in the contemplation of, the ashy fire.\n",
      "\n",
      "---------\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "When Herbert had been down to Hammersmith and seen his father, he came back to me at our chambers, and devoted the day to attending on me. He was the kindest of nurses, and at stated times took off the bandages, and steeped them in the cooling liquid that was kept ready, and put them on again, with a patient tenderness that I was deeply grateful for.\n",
      "\n",
      "At first, as I lay quiet on the sofa, I found it painfully difficult, I might say impossible, to get rid of the impression of the glare of the flames, their hurry and noise, and the fierce burning smell. If I dozed for a minute, I was awakened by Miss Havisham's cries, and by her running at me with all that height of fire above her head. This pain of the mind was much harder to strive against than any bodily pain I suffered; and Herbert, seeing that, did his utmost to hold my attention engaged.\n",
      "\n",
      "---------\n",
      "But when Herbert and I had held our momentous conversation, I was seized with a feverish conviction that I ought to hunt the matter down, that I ought not to let it rest, but that I ought to see Mr. Jaggers, and come at the bare truth. I really do not know whether I felt that I did this for Estella's sake, or whether I was glad to transfer to the man in whose preservation I was so much concerned some rays of the romantic interest that had so long surrounded me. Perhaps the latter possibility may be the nearer to the truth.\n",
      "\n",
      "Any way, I could scarcely be withheld from going out to Gerrard Street that night. Herbert's representations that, if I did, I should probably be laid up and stricken useless, when our fugitive's safety would depend upon me, alone restrained my impatience. On the understanding, again and again reiterated, that, come what would, I was to go to Mr. Jaggers to-morrow, I at length submitted to keep quiet, and to have my hurts looked after, and to stay at home. Early next morning we went out together, and at the corner of Giltspur Street by Smithfield, I left Herbert to go his way into the City, and took my way to Little Britain.\n",
      "\n",
      "---------\n",
      "Clarriker informing me on that occasion that the affairs of the House were steadily progressing, that he would now be able to establish a small branch-house in the East which was much wanted for the extension of the business, and that Herbert in his new partnership capacity would go out and take charge of it, I found that I must have prepared for a separation from my friend, even though my own affairs had been more settled. And now, indeed, I felt as if my last anchor were loosening its hold, and I should soon be driving with the winds and waves.\n",
      "\n",
      "But there was recompense in the joy with which Herbert would come home of a night and tell me of these changes, little imagining that he told me no news, and would sketch airy pictures of himself conducting Clara Barley to the land of the Arabian Nights, and of me going out to join them (with a caravan of camels, I believe), and of our all going up the Nile and seeing wonders. Without being sanguine as to my own part in those bright plans, I felt that Herbert's way was clearing fast, and that old Bill Barley had but to stick to his pepper and rum, and his daughter would soon be happily provided for.\n",
      "\n",
      "---------\n",
      "There was a melancholy wind, and the marshes were very dismal. A stranger would have found them insupportable, and even to me they were so oppressive that I hesitated, half inclined to go back. But I knew them well, and could have found my way on a far darker night, and had no excuse for returning, being there. So, having come there against my inclination, I went on against it.\n",
      "\n",
      "The direction that I took was not that in which my old home lay, nor that in which we had pursued the convicts. My back was turned towards the distant Hulks as I walked on, and, though I could see the old lights away on the spits of sand, I saw them over my shoulder. I knew the limekiln as well as I knew the old Battery, but they were miles apart; so that, if a light had been burning at each point that night, there would have been a long strip of the blank horizon between the two bright specks.\n",
      "\n",
      "---------\n",
      "We loitered down to the Temple stairs, and stood loitering there, as if we were not quite decided to go upon the water at all. Of course, I had taken care that the boat should be ready and everything in order. After a little show of indecision, which there were none to see but the two or three amphibious creatures belonging to our Temple stairs, we went on board and cast off; Herbert in the bow, I steering. It was then about high-water, half-past eight.\n",
      "\n",
      "Our plan was this. The tide, beginning to run down at nine, and being with us until three, we intended still to creep on after it had turned, and row against it until dark. We should then be well in those long reaches below Gravesend, between Kent and Essex, where the river is broad and solitary, where the water-side inhabitants are very few, and where lone public-houses are scattered here and there, of which we could choose one for a resting-place. There, we meant to lie by all night. The steamer for Hamburg and the steamer for Rotterdam would start from London at about nine on Thursday morning. We should know at what time to expect them, according to where we were, and would hail the first; so that, if by any accident we were not taken abroad, we should have another chance. We knew the distinguishing marks of each vessel.\n",
      "\n",
      "---------\n",
      "I imparted to Mr. Jaggers my design of keeping him in ignorance of the fate of his wealth. Mr. Jaggers was querulous and angry with me for having “let it slip through my fingers,” and said we must memorialize by and by, and try at all events for some of it. But he did not conceal from me that, although there might be many cases in which the forfeiture would not be exacted, there were no circumstances in this case to make it one of them. I understood that very well. I was not related to the outlaw, or connected with him by any recognizable tie; he had put his hand to no writing or settlement in my favor before his apprehension, and to do so now would be idle. I had no claim, and I finally resolved, and ever afterwards abided by the resolution, that my heart should never be sickened with the hopeless task of attempting to establish one.\n",
      "\n",
      "There appeared to be reason for supposing that the drowned informer had hoped for a reward out of this forfeiture, and had obtained some accurate knowledge of Magwitch's affairs. When his body was found, many miles from the scene of his death, and so horribly disfigured that he was only recognizable by the contents of his pockets, notes were still legible, folded in a case he carried. Among these were the name of a banking-house in New South Wales, where a sum of money was, and the designation of certain lands of considerable value. Both these heads of information were in a list that Magwitch, while in prison, gave to Mr. Jaggers, of the possessions he supposed I should inherit. His ignorance, poor fellow, at last served him; he never mistrusted but that my inheritance was quite safe, with Mr. Jaggers's aid.\n",
      "\n",
      "---------\n",
      "Being far too ill to remain in the common prison, he was removed, after the first day or so, into the infirmary. This gave me opportunities of being with him that I could not otherwise have had. And but for his illness he would have been put in irons, for he was regarded as a determined prison-breaker, and I know not what else.\n",
      "\n",
      "Although I saw him every day, it was for only a short time; hence, the regularly recurring spaces of our separation were long enough to record on his face any slight changes that occurred in his physical state. I do not recollect that I once saw any change in it for the better; he wasted, and became slowly weaker and worse, day by day, from the day when the prison door closed upon him.\n",
      "\n",
      "---------\n",
      "For a day or two, I lay on the sofa, or on the floor, anywhere, according as I happened to sink down, with a heavy head and aching limbs, and no purpose, and no power. Then there came, one night which appeared of great duration, and which teemed with anxiety and horror; and when in the morning I tried to sit up in my bed and think of it, I found I could not do so.\n",
      "\n",
      "Whether I really had been down in Garden Court in the dead of the night, groping about for the boat that I supposed to be there; whether I had two or three times come to myself on the staircase with great terror, not knowing how I had got out of bed; whether I had found myself lighting the lamp, possessed by the idea that he was coming up the stairs, and that the lights were blown out; whether I had been inexpressibly harassed by the distracted talking, laughing, and groaning of some one, and had half suspected those sounds to be of my own making; whether there had been a closed iron furnace in a dark corner of the room, and a voice had called out, over and over again, that Miss Havisham was consuming within it, these were things that I tried to settle with myself and get into some order, as I lay that morning on my bed. But the vapor of a limekiln would come between me and them, disordering them all, and it was through the vapor at last that I saw two men looking at me.\n",
      "\n",
      "---------\n",
      "It was evening when I arrived, much fatigued by the journey I had so often made so easily. The Boar could not put me into my usual bedroom, which was engaged (probably by some one who had expectations), and could only assign me a very indifferent chamber among the pigeons and post-chaises up the yard. But I had as sound a sleep in that lodging as in the most superior accommodation the Boar could have given me, and the quality of my dreams was about the same as in the best bedroom.\n",
      "\n",
      "Early in the morning, while my breakfast was getting ready, I strolled round by Satis House. There were printed bills on the gate and on bits of carpet hanging out of the windows, announcing a sale by auction of the Household Furniture and Effects, next week. The House itself was to be sold as old building materials, and pulled down. LOT 1 was marked in whitewashed knock-knee letters on the brew house; LOT 2 on that part of the main building which had been so long shut up. Other lots were marked off on other parts of the structure, and the ivy had been torn down to make room for the inscriptions, and much of it trailed low in the dust and was withered already. Stepping in for a moment at the open gate, and looking around me with the uncomfortable air of a stranger who had no business there, I saw the auctioneer's clerk walking on the casks and telling them off for the information of a catalogue-compiler, pen in hand, who made a temporary desk of the wheeled chair I had so often pushed along to the tune of Old Clem.\n",
      "\n",
      "---------\n",
      "“We giv' him the name of Pip for your sake, dear old chap,” said Joe, delighted, when I took another stool by the child's side (but I did not rumple his hair), “and we hoped he might grow a little bit like you, and we think he do.”\n",
      "\n",
      "I thought so too, and I took him out for a walk next morning, and we talked immensely, understanding one another to perfection. And I took him down to the churchyard, and set him on a certain tombstone there, and he showed me from that elevation which stone was sacred to the memory of Philip Pirrip, late of this Parish, and Also Georgiana, Wife of the Above.\n",
      "\n",
      "---------\n"
     ]
    }
   ],
   "source": [
    "docs = []\n",
    "for paragraphs in chapters:\n",
    "    for text in paragraphs[0:2]:\n",
    "        docs.append(nlp(text))\n",
    "        print(text)\n",
    "        print('')\n",
    "    print('---------')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exploring the parsed structure\n",
    "\n",
    "We'll look at the sentences, tokens, nouns, and entities of the first parsed paragraph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "chapter_1 = docs[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I give Pirrip as my father's family name, on the authority of his tombstone and my sister, Mrs. Joe Gargery, who married the blacksmith.\n",
      "----------\n",
      "As I never saw my father or my mother, and never saw any likeness of either of them (for their days were long before the days of photographs), my first fancies regarding what they were like were unreasonably derived from their tombstones.\n",
      "----------\n",
      "The shape of the letters on my father's, gave me an odd idea that he was a square, stout, dark man, with curly black hair.\n",
      "----------\n",
      "From the character and turn of the inscription, “Also Georgiana Wife of the Above,” I drew a childish conclusion that my mother was freckled and sickly.\n",
      "----------\n",
      "To five little stone lozenges, each about a foot and a half long, which were arranged in a neat row beside their grave, and were sacred to the memory of five little brothers of mine, who gave up trying to get a living, exceedingly early in that universal struggle, I am indebted for a belief I religiously entertained that they had all been born on their backs with their hands in their trousers-pockets, and had never taken them out in this state of existence.\n",
      "----------\n",
      "5\n"
     ]
    }
   ],
   "source": [
    "sents = 0\n",
    "for sent in chapter_1.sents:\n",
    "    print(sent)\n",
    "    print('----------')\n",
    "    sents+=1\n",
    "print(sents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PRON  |  I\n",
      "VERB  |  give\n",
      "PROPN  |  Pirrip\n",
      "ADP  |  as\n",
      "ADJ  |  my\n",
      "NOUN  |  father\n",
      "PART  |  's\n",
      "NOUN  |  family\n",
      "NOUN  |  name\n",
      "PUNCT  |  ,\n",
      "ADP  |  on\n",
      "DET  |  the\n",
      "NOUN  |  authority\n",
      "ADP  |  of\n",
      "ADJ  |  his\n",
      "NOUN  |  tombstone\n",
      "CCONJ  |  and\n",
      "ADJ  |  my\n",
      "NOUN  |  sister\n",
      "PUNCT  |  ,\n",
      "PROPN  |  Mrs.\n",
      "PROPN  |  Joe\n",
      "PROPN  |  Gargery\n",
      "PUNCT  |  ,\n",
      "NOUN  |  who\n",
      "VERB  |  married\n",
      "DET  |  the\n",
      "NOUN  |  blacksmith\n",
      "PUNCT  |  .\n",
      "ADP  |  As\n",
      "PRON  |  I\n",
      "ADV  |  never\n",
      "VERB  |  saw\n",
      "ADJ  |  my\n",
      "NOUN  |  father\n",
      "CCONJ  |  or\n",
      "ADJ  |  my\n",
      "NOUN  |  mother\n",
      "PUNCT  |  ,\n",
      "CCONJ  |  and\n",
      "ADV  |  never\n",
      "VERB  |  saw\n",
      "DET  |  any\n",
      "NOUN  |  likeness\n",
      "ADP  |  of\n",
      "DET  |  either\n",
      "ADP  |  of\n",
      "PRON  |  them\n",
      "PUNCT  |  (\n",
      "ADP  |  for\n",
      "ADJ  |  their\n",
      "NOUN  |  days\n",
      "VERB  |  were\n",
      "ADJ  |  long\n",
      "ADP  |  before\n",
      "DET  |  the\n",
      "NOUN  |  days\n",
      "ADP  |  of\n",
      "NOUN  |  photographs\n",
      "PUNCT  |  )\n",
      "PUNCT  |  ,\n",
      "ADJ  |  my\n",
      "ADJ  |  first\n",
      "NOUN  |  fancies\n",
      "VERB  |  regarding\n",
      "NOUN  |  what\n",
      "PRON  |  they\n",
      "VERB  |  were\n",
      "INTJ  |  like\n",
      "VERB  |  were\n",
      "ADV  |  unreasonably\n",
      "VERB  |  derived\n",
      "ADP  |  from\n",
      "ADJ  |  their\n",
      "NOUN  |  tombstones\n",
      "PUNCT  |  .\n",
      "DET  |  The\n",
      "NOUN  |  shape\n",
      "ADP  |  of\n",
      "DET  |  the\n",
      "NOUN  |  letters\n",
      "ADP  |  on\n",
      "ADJ  |  my\n",
      "NOUN  |  father\n",
      "PART  |  's\n",
      "PUNCT  |  ,\n",
      "VERB  |  gave\n",
      "PRON  |  me\n",
      "DET  |  an\n",
      "ADJ  |  odd\n",
      "NOUN  |  idea\n",
      "ADP  |  that\n",
      "PRON  |  he\n",
      "VERB  |  was\n",
      "DET  |  a\n",
      "ADJ  |  square\n",
      "PUNCT  |  ,\n",
      "ADJ  |  stout\n",
      "PUNCT  |  ,\n",
      "ADJ  |  dark\n",
      "NOUN  |  man\n",
      "PUNCT  |  ,\n",
      "ADP  |  with\n",
      "ADJ  |  curly\n",
      "ADJ  |  black\n",
      "NOUN  |  hair\n",
      "PUNCT  |  .\n",
      "ADP  |  From\n",
      "DET  |  the\n",
      "NOUN  |  character\n",
      "CCONJ  |  and\n",
      "NOUN  |  turn\n",
      "ADP  |  of\n",
      "DET  |  the\n",
      "NOUN  |  inscription\n",
      "PUNCT  |  ,\n",
      "ADV  |  “\n",
      "ADV  |  Also\n",
      "PROPN  |  Georgiana\n",
      "PROPN  |  Wife\n",
      "ADP  |  of\n",
      "DET  |  the\n",
      "PROPN  |  Above\n",
      "PUNCT  |  ,\n",
      "INTJ  |  ”\n",
      "PRON  |  I\n",
      "VERB  |  drew\n",
      "DET  |  a\n",
      "ADJ  |  childish\n",
      "NOUN  |  conclusion\n",
      "ADP  |  that\n",
      "ADJ  |  my\n",
      "NOUN  |  mother\n",
      "VERB  |  was\n",
      "ADJ  |  freckled\n",
      "CCONJ  |  and\n",
      "ADJ  |  sickly\n",
      "PUNCT  |  .\n",
      "ADP  |  To\n",
      "NUM  |  five\n",
      "ADJ  |  little\n",
      "NOUN  |  stone\n",
      "NOUN  |  lozenges\n",
      "PUNCT  |  ,\n",
      "DET  |  each\n",
      "ADP  |  about\n",
      "DET  |  a\n",
      "NOUN  |  foot\n",
      "CCONJ  |  and\n",
      "DET  |  a\n",
      "NOUN  |  half\n",
      "ADJ  |  long\n",
      "PUNCT  |  ,\n",
      "ADJ  |  which\n",
      "VERB  |  were\n",
      "VERB  |  arranged\n",
      "ADP  |  in\n",
      "DET  |  a\n",
      "ADJ  |  neat\n",
      "NOUN  |  row\n",
      "ADP  |  beside\n",
      "ADJ  |  their\n",
      "NOUN  |  grave\n",
      "PUNCT  |  ,\n",
      "CCONJ  |  and\n",
      "VERB  |  were\n",
      "ADJ  |  sacred\n",
      "ADP  |  to\n",
      "DET  |  the\n",
      "NOUN  |  memory\n",
      "ADP  |  of\n",
      "NUM  |  five\n",
      "ADJ  |  little\n",
      "NOUN  |  brothers\n",
      "ADP  |  of\n",
      "NOUN  |  mine\n",
      "PUNCT  |  ,\n",
      "NOUN  |  who\n",
      "VERB  |  gave\n",
      "PART  |  up\n",
      "VERB  |  trying\n",
      "PART  |  to\n",
      "VERB  |  get\n",
      "DET  |  a\n",
      "NOUN  |  living\n",
      "PUNCT  |  ,\n",
      "ADV  |  exceedingly\n",
      "ADV  |  early\n",
      "ADP  |  in\n",
      "DET  |  that\n",
      "ADJ  |  universal\n",
      "NOUN  |  struggle\n",
      "PUNCT  |  ,\n",
      "PRON  |  I\n",
      "VERB  |  am\n",
      "ADJ  |  indebted\n",
      "ADP  |  for\n",
      "DET  |  a\n",
      "NOUN  |  belief\n",
      "PRON  |  I\n",
      "ADV  |  religiously\n",
      "VERB  |  entertained\n",
      "ADP  |  that\n",
      "PRON  |  they\n",
      "VERB  |  had\n",
      "DET  |  all\n",
      "VERB  |  been\n",
      "VERB  |  born\n",
      "ADP  |  on\n",
      "ADJ  |  their\n",
      "NOUN  |  backs\n",
      "ADP  |  with\n",
      "ADJ  |  their\n",
      "NOUN  |  hands\n",
      "ADP  |  in\n",
      "ADJ  |  their\n",
      "NOUN  |  trousers\n",
      "PUNCT  |  -\n",
      "NOUN  |  pockets\n",
      "PUNCT  |  ,\n",
      "CCONJ  |  and\n",
      "VERB  |  had\n",
      "ADV  |  never\n",
      "VERB  |  taken\n",
      "PRON  |  them\n",
      "PART  |  out\n",
      "ADP  |  in\n",
      "DET  |  this\n",
      "NOUN  |  state\n",
      "ADP  |  of\n",
      "NOUN  |  existence\n",
      "PUNCT  |  .\n",
      "5 232\n"
     ]
    }
   ],
   "source": [
    "sents = 0\n",
    "toks = 0\n",
    "for sent in chapter_1.sents:\n",
    "    sents+=1\n",
    "    for tok in sent:\n",
    "        print(tok.pos_,' | ',tok.text)\n",
    "        toks+=1\n",
    "print(sents,toks)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'NUM', 'ADV', 'PROPN', 'ADJ', 'ADP', 'PART', 'PUNCT', 'CCONJ', 'INTJ', 'NOUN', 'DET', 'PRON', 'VERB'}\n"
     ]
    }
   ],
   "source": [
    "sents = 0\n",
    "toks = 0\n",
    "poss = set()\n",
    "for sent in chapter_1.sents:\n",
    "    sents+=1\n",
    "    for tok in sent:\n",
    "        toks+=1\n",
    "        poss.add(tok.pos_)\n",
    "print(poss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Here's a handy list of the PoS tags and their example words\n",
    "\n",
    "#### CC: conjunction, coordinating\n",
    "```\n",
    "& 'n and both but either et for less minus neither nor or plus so\n",
    "therefore times v. versus vs. whether yet\n",
    "```\n",
    "\n",
    "#### CD: numeral, cardinal\n",
    "```\n",
    "mid-1890 nine-thirty forty-two one-tenth ten million 0.5 one forty-\n",
    "seven 1987 twenty '79 zero two 78-degrees eighty-four IX '60s .025\n",
    "fifteen 271,124 dozen quintillion DM2,000 ...\n",
    "```\n",
    "\n",
    "#### DT: determiner\n",
    "```\n",
    "all an another any both del each either every half la many much nary\n",
    "neither no some such that the them these this those\n",
    "```\n",
    "\n",
    "#### EX: existential there\n",
    "```\n",
    "there\n",
    "```\n",
    "\n",
    "#### IN: preposition or conjunction, subordinating\n",
    "```\n",
    "astride among uppon whether out inside pro despite on by throughout\n",
    "below within for towards near behind atop around if like until below\n",
    "next into if beside ...\n",
    "```\n",
    "\n",
    "#### JJ: adjective or numeral, ordinal\n",
    "```\n",
    "third ill-mannered pre-war regrettable oiled calamitous first separable\n",
    "ectoplasmic battery-powered participatory fourth still-to-be-named\n",
    "multilingual multi-disciplinary ...\n",
    "```\n",
    "\n",
    "#### JJR: adjective, comparative\n",
    "```\n",
    "bleaker braver breezier briefer brighter brisker broader bumper busier\n",
    "calmer cheaper choosier cleaner clearer closer colder commoner costlier\n",
    "cozier creamier crunchier cuter ...\n",
    "```\n",
    "\n",
    "#### JJS: adjective, superlative\n",
    "```\n",
    "calmest cheapest choicest classiest cleanest clearest closest commonest\n",
    "corniest costliest crassest creepiest crudest cutest darkest deadliest\n",
    "dearest deepest densest dinkiest ...\n",
    "```\n",
    "\n",
    "#### LS: list item marker\n",
    "```\n",
    "A A. B B. C C. D E F First G H I J K One SP-44001 SP-44002 SP-44005\n",
    "SP-44007 Second Third Three Two * a b c d first five four one six three\n",
    "two\n",
    "```\n",
    "\n",
    "#### MD: modal auxiliary\n",
    "```\n",
    "can cannot could couldn't dare may might must need ought shall should\n",
    "shouldn't will would\n",
    "```\n",
    "\n",
    "#### NN: noun, common, singular or mass\n",
    "```\n",
    "common-carrier cabbage knuckle-duster Casino afghan shed thermostat\n",
    "investment slide humour falloff slick wind hyena override subhumanity\n",
    "machinist ...\n",
    "```\n",
    "\n",
    "#### NNP: noun, proper, singular\n",
    "```\n",
    "Motown Venneboerger Czestochwa Ranzer Conchita Trumplane Christos\n",
    "Oceanside Escobar Kreisler Sawyer Cougar Yvette Ervin ODI Darryl CTCA\n",
    "Shannon A.K.C. Meltex Liverpool ...\n",
    "```\n",
    "\n",
    "#### NNS: noun, common, plural\n",
    "```\n",
    "undergraduates scotches bric-a-brac products bodyguards facets coasts\n",
    "divestitures storehouses designs clubs fragrances averages\n",
    "subjectivists apprehensions muses factory-jobs ...\n",
    "```\n",
    "\n",
    "#### PDT: pre-determiner\n",
    "```\n",
    "all both half many quite such sure this\n",
    "```\n",
    "\n",
    "#### POS: genitive marker\n",
    "```\n",
    "' 's\n",
    "```\n",
    "\n",
    "#### PRP: pronoun, personal\n",
    "```\n",
    "hers herself him himself hisself it itself me myself one oneself ours\n",
    "ourselves ownself self she thee theirs them themselves they thou thy us\n",
    "```\n",
    "\n",
    "#### PRP$: pronoun, possessive\n",
    "```\n",
    "her his mine my our ours their thy your\n",
    "```\n",
    "\n",
    "#### RB: adverb\n",
    "```\n",
    "occasionally unabatingly maddeningly adventurously professedly\n",
    "stirringly prominently technologically magisterially predominately\n",
    "swiftly fiscally pitilessly ...\n",
    "```\n",
    "\n",
    "#### RBR: adverb, comparative\n",
    "```\n",
    "further gloomier grander graver greater grimmer harder harsher\n",
    "healthier heavier higher however larger later leaner lengthier less-\n",
    "perfectly lesser lonelier longer louder lower more ...\n",
    "```\n",
    "\n",
    "#### RBS: adverb, superlative\n",
    "```\n",
    "best biggest bluntest earliest farthest first furthest hardest\n",
    "heartiest highest largest least less most nearest second tightest worst\n",
    "```\n",
    "\n",
    "#### RP: particle\n",
    "```\n",
    "aboard about across along apart around aside at away back before behind\n",
    "by crop down ever fast for forth from go high i.e. in into just later\n",
    "low more off on open out over per pie raising start teeth that through\n",
    "under unto up up-pp upon whole with you\n",
    "```\n",
    "\n",
    "#### TO: \"to\" as preposition or infinitive marker\n",
    "```\n",
    "to\n",
    "```\n",
    "\n",
    "#### UH: interjection\n",
    "```\n",
    "Goodbye Goody Gosh Wow Jeepers Jee-sus Hubba Hey Kee-reist Oops amen\n",
    "huh howdy uh dammit whammo shucks heck anyways whodunnit honey golly\n",
    "man baby diddle hush sonuvabitch ...\n",
    "```\n",
    "\n",
    "#### VB: verb, base form\n",
    "```\n",
    "ask assemble assess assign assume atone attention avoid bake balkanize\n",
    "bank begin behold believe bend benefit bevel beware bless boil bomb\n",
    "boost brace break bring broil brush build ...\n",
    "```\n",
    "\n",
    "#### VBD: verb, past tense\n",
    "```\n",
    "dipped pleaded swiped regummed soaked tidied convened halted registered\n",
    "cushioned exacted snubbed strode aimed adopted belied figgered\n",
    "speculated wore appreciated contemplated ...\n",
    "```\n",
    "\n",
    "#### VBG: verb, present participle or gerund\n",
    "```\n",
    "telegraphing stirring focusing angering judging stalling lactating\n",
    "hankerin' alleging veering capping approaching traveling besieging\n",
    "encrypting interrupting erasing wincing ...\n",
    "```\n",
    "\n",
    "#### VBN: verb, past participle\n",
    "```\n",
    "multihulled dilapidated aerosolized chaired languished panelized used\n",
    "experimented flourished imitated reunifed factored condensed sheared\n",
    "unsettled primed dubbed desired ...\n",
    "```\n",
    "\n",
    "#### VBP: verb, present tense, not 3rd person singular\n",
    "```\n",
    "predominate wrap resort sue twist spill cure lengthen brush terminate\n",
    "appear tend stray glisten obtain comprise detest tease attract\n",
    "emphasize mold postpone sever return wag ...\n",
    "```\n",
    "\n",
    "#### VBZ: verb, present tense, 3rd person singular\n",
    "```\n",
    "bases reconstructs marks mixes displeases seals carps weaves snatches\n",
    "slumps stretches authorizes smolders pictures emerges stockpiles\n",
    "seduces fizzes uses bolsters slaps speaks pleads ...\n",
    "```\n",
    "\n",
    "#### WDT: WH-determiner\n",
    "```\n",
    "that what whatever which whichever\n",
    "```\n",
    "\n",
    "#### WP: WH-pronoun\n",
    "```\n",
    "that what whatever whatsoever which who whom whosoever\n",
    "```\n",
    "\n",
    "#### WRB: Wh-adverb\n",
    "```\n",
    "how however whence whenever where whereby whereever wherein whereof why\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5 232\n",
      "{belief, inscription, mother, father, tombstone, idea, shape, father, grave, struggle, family, memory, man, state, turn, blacksmith, row, foot, mother, stone, conclusion, likeness, existence, name, sister, hair, father, mine, authority, half, character, living}\n"
     ]
    }
   ],
   "source": [
    "# All the nouns in all the sentences\n",
    "sents = 0\n",
    "toks = 0\n",
    "nouns = set()\n",
    "tags = set()\n",
    "for sent in chapter_1.sents:\n",
    "    sents+=1\n",
    "    for tok in sent:\n",
    "        toks+=1\n",
    "        if tok.tag_ == 'NN':\n",
    "            nouns.add(tok)\n",
    "print(sents,toks)\n",
    "print(nouns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I\n",
      "Pirrip\n",
      "my father's family name\n",
      "the authority\n",
      "his tombstone\n",
      "my sister\n",
      "Mrs. Joe Gargery\n",
      "who\n",
      "the blacksmith\n",
      "I\n",
      "my father\n",
      "my mother\n",
      "any likeness\n",
      "them\n",
      "their days\n",
      "the days\n",
      "photographs\n",
      "my first fancies\n",
      "what\n",
      "they\n",
      "their tombstones\n",
      "The shape\n",
      "the letters\n",
      "my father\n",
      "me\n",
      "an odd idea\n",
      "he\n",
      "a square, stout, dark man\n",
      "curly black hair\n",
      "the character\n",
      "the inscription\n",
      "Also Georgiana Wife\n",
      "the Above\n",
      "I\n",
      "a childish conclusion\n",
      "my mother\n",
      "five little stone lozenges\n",
      "a foot\n",
      "a neat row\n",
      "their grave\n",
      "the memory\n",
      "five little brothers\n",
      "mine\n",
      "who\n",
      "a living\n",
      "that universal struggle\n",
      "I\n",
      "a belief\n",
      "I\n",
      "they\n",
      "their backs\n",
      "their hands\n",
      "their trousers-pockets\n",
      "them\n",
      "this state\n",
      "existence\n",
      "56\n"
     ]
    }
   ],
   "source": [
    "# All the noun chunks (chunked using the spaCy model and algorithm)\n",
    "chunks = 0\n",
    "for concept in chapter_1.noun_chunks:\n",
    "    print(concept)\n",
    "    chunks += 1\n",
    "print(chunks)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pirrip GPE 2 3\n",
      "Joe Gargery PERSON 21 23\n",
      "their days DATE 50 52\n",
      "long before the days DATE 53 57\n",
      "first ORDINAL 62 63\n",
      "Georgiana PERSON 118 119\n",
      "five CARDINAL 139 140\n",
      "half CARDINAL 150 151\n",
      "five CARDINAL 171 172\n"
     ]
    }
   ],
   "source": [
    "#Entities in each sentence\n",
    "for sent in chapter_1.sents:\n",
    "    for ent in sent.ents:\n",
    "        if(ent.text.strip()):\n",
    "            print(ent, ent.label_, ent.start, ent.end)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------\n",
      "I give Pirrip as my father's family name, on the authority of his tombstone and my sister, Mrs. Joe Gargery, who married the blacksmith.\n",
      "      Pirrip  |  GPE\n",
      "      Joe Gargery  |  PERSON\n",
      "--------\n",
      "As I never saw my father or my mother, and never saw any likeness of either of them (for their days were long before the days of photographs), my first fancies regarding what they were like were unreasonably derived from their tombstones.\n",
      "      their days  |  DATE\n",
      "      long before the days  |  DATE\n",
      "      first  |  ORDINAL\n",
      "--------\n",
      "From the character and turn of the inscription, “Also Georgiana Wife of the Above,” I drew a childish conclusion that my mother was freckled and sickly.\n",
      "      Georgiana  |  PERSON\n",
      "--------\n",
      "To five little stone lozenges, each about a foot and a half long, which were arranged in a neat row beside their grave, and were sacred to the memory of five little brothers of mine, who gave up trying to get a living, exceedingly early in that universal struggle, I am indebted for a belief I religiously entertained that they had all been born on their backs with their hands in their trousers-pockets, and had never taken them out in this state of existence.\n",
      "      five  |  CARDINAL\n",
      "      half  |  CARDINAL\n",
      "      five  |  CARDINAL\n"
     ]
    }
   ],
   "source": [
    "# All the entities in all the sentences\n",
    "for sent in chapter_1.sents:\n",
    "    if len(sent.ents)>0:\n",
    "        print('--------')\n",
    "        print(sent)\n",
    "    for ent in sent.ents:\n",
    "        if(ent.text.strip()):\n",
    "            print('     ',ent.text,' | ',ent.label_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Kinds of entities we have in our text\n",
    "\n",
    "Let's list the entity types that exist, and explore our interests for our Bad-Libs project!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'LANGUAGE', 'GPE', 'ORG', 'QUANTITY', 'EVENT', 'PRODUCT', 'NORP', 'ORDINAL', 'TIME', 'DATE', 'FAC', 'LOC', 'CARDINAL', 'MONEY', 'PERSON', 'WORK_OF_ART'}\n"
     ]
    }
   ],
   "source": [
    "labels = set()\n",
    "for doc in docs:\n",
    "    for ent in doc.ents:\n",
    "        labels.add(ent.label_)\n",
    "print(labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bad Libs will use People and Geo-Political Entities\n",
    "\n",
    "We want to ask the player for replacements for these names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[{'text': 'Philip Pirrip', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 358, 'end': 371, 'length': 13}, {'text': 'Georgiana', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 403, 'end': 412, 'length': 9}, {'text': 'Alexander', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 463, 'end': 472, 'length': 9}, {'text': 'Bartholomew', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 474, 'end': 485, 'length': 11}, {'text': 'Abraham', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 487, 'end': 494, 'length': 7}, {'text': 'Tobias', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 496, 'end': 502, 'length': 6}, {'text': 'Roger', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 508, 'end': 513, 'length': 5}, {'text': 'Pip', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 949, 'end': 952, 'length': 3}]\n"
     ]
    }
   ],
   "source": [
    "placeholders = []\n",
    "entity_filters = {'PERSON', 'GPE'}\n",
    "for doc in docs:\n",
    "    ph = []\n",
    "    for ent in doc.ents:\n",
    "        if len(ent.text)>0 and ent.label_ in entity_filters:\n",
    "            ph.append({\n",
    "                'text':  ent.text,\n",
    "                'kind':  ent.label_,\n",
    "                'tag':   ent.label_,\n",
    "                'start': ent.start_char,\n",
    "                'end':   ent.end_char,\n",
    "                'length':   len(ent.text)\n",
    "            })\n",
    "            #print(idx,ent.label_,ent.start,ent.end)\n",
    "    placeholders.append(ph)\n",
    "print(placeholders[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'abraham', 'alexander', 'roger', 'philip pirrip', 'georgiana', 'pip', 'tobias', 'bartholomew'}\n"
     ]
    }
   ],
   "source": [
    "# Normalize and group entities, (this is just an example and isnt used in the game, but could be)\n",
    "uniques = set()\n",
    "for ph in placeholders[1]:\n",
    "    uniques.add(ph['text'].lower())\n",
    "print(uniques)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting some Parts-of-Speech\n",
    "\n",
    "We're going to also get replacements for nouns, verbs, and adjectives."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "pos_filters = {'NOUN', 'ADJ', 'VERB'}\n",
    "tag_filters = {'JJS', 'JJR', 'NN', 'NNS', 'VB', 'VBD', 'VBG'}\n",
    "for idx,doc in enumerate(docs):\n",
    "    nouns = []\n",
    "    verbs = []\n",
    "    adjs  = []\n",
    "    for tok in doc:\n",
    "        if len(tok.text)>0 and tok.pos_ in pos_filters and tok.tag_ in tag_filters:\n",
    "            obj = {\n",
    "                'text':  tok.text,\n",
    "                'kind':  tok.pos_,\n",
    "                'tag':   tok.tag_,\n",
    "                'start': tok.idx,\n",
    "                'end':   tok.idx+len(tok.text),\n",
    "                'length':   len(tok.text)\n",
    "            }\n",
    "            if tok.pos_ == 'NOUN':\n",
    "                nouns.append(obj)\n",
    "            if tok.pos_ == 'VERB':\n",
    "                verbs.append(obj)\n",
    "            if tok.pos_ == 'ADJ':\n",
    "                adjs.append(obj)\n",
    "            #print(idx,tok.pos_,tok.idx,tok.idx+len(tok.text))\n",
    "    nouns.sort(key=itemgetter('length'), reverse=True)\n",
    "    verbs.sort(key=itemgetter('length'), reverse=True)\n",
    "    adjs.sort(key=itemgetter('length'), reverse=True)\n",
    "    placeholders[idx] += nouns[:8] + verbs[:8] + adjs[:8]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seeing what our placeholders look like\n",
    "\n",
    "These will be queried for replacements by the players (that's us!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'text': 'country', 'kind': 'NOUN', 'tag': 'NN', 'start': 19, 'end': 26, 'length': 7}\n",
      "{'text': 'impression', 'kind': 'NOUN', 'tag': 'NN', 'start': 130, 'end': 140, 'length': 10}\n",
      "{'text': 'identity', 'kind': 'NOUN', 'tag': 'NN', 'start': 148, 'end': 156, 'length': 8}\n",
      "{'text': 'have', 'kind': 'VERB', 'tag': 'VB', 'start': 182, 'end': 186, 'length': 4}\n",
      "{'text': 'afternoon', 'kind': 'NOUN', 'tag': 'NN', 'start': 218, 'end': 227, 'length': 9}\n",
      "{'text': 'found', 'kind': 'VERB', 'tag': 'VBD', 'start': 262, 'end': 267, 'length': 5}\n",
      "{'text': 'churchyard', 'kind': 'NOUN', 'tag': 'NN', 'start': 337, 'end': 347, 'length': 10}\n",
      "{'text': 'Philip Pirrip', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 358, 'end': 371, 'length': 13}\n",
      "{'text': 'Georgiana', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 403, 'end': 412, 'length': 9}\n",
      "{'text': 'were', 'kind': 'VERB', 'tag': 'VBD', 'start': 432, 'end': 436, 'length': 4}\n",
      "{'text': 'Alexander', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 463, 'end': 472, 'length': 9}\n",
      "{'text': 'Bartholomew', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 474, 'end': 485, 'length': 11}\n",
      "{'text': 'Abraham', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 487, 'end': 494, 'length': 7}\n",
      "{'text': 'Tobias', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 496, 'end': 502, 'length': 6}\n",
      "{'text': 'Roger', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 508, 'end': 513, 'length': 5}\n",
      "{'text': 'children', 'kind': 'NOUN', 'tag': 'NNS', 'start': 522, 'end': 530, 'length': 8}\n",
      "{'text': 'wilderness', 'kind': 'NOUN', 'tag': 'NN', 'start': 599, 'end': 609, 'length': 10}\n",
      "{'text': 'churchyard', 'kind': 'NOUN', 'tag': 'NN', 'start': 621, 'end': 631, 'length': 10}\n",
      "{'text': 'intersected', 'kind': 'VERB', 'tag': 'VBD', 'start': 633, 'end': 644, 'length': 11}\n",
      "{'text': 'feeding', 'kind': 'VERB', 'tag': 'VBG', 'start': 700, 'end': 707, 'length': 7}\n",
      "{'text': 'rushing', 'kind': 'VERB', 'tag': 'VBG', 'start': 840, 'end': 847, 'length': 7}\n",
      "{'text': 'growing', 'kind': 'VERB', 'tag': 'VBG', 'start': 898, 'end': 905, 'length': 7}\n",
      "{'text': 'beginning', 'kind': 'VERB', 'tag': 'VBG', 'start': 927, 'end': 936, 'length': 9}\n",
      "{'text': 'Pip', 'kind': 'PERSON', 'tag': 'PERSON', 'start': 949, 'end': 952, 'length': 3}\n"
     ]
    }
   ],
   "source": [
    "[placeholder.sort(key=itemgetter('start')) for placeholder in placeholders]\n",
    "for ph in placeholders[1]:\n",
    "    print(ph)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A sample document\n",
    "\n",
    "Now known as a \"lib\", we'll see what it looks like when we insert blank spaces into the text for our placeholders"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ours was the marsh country, down by the river, within, as the river wound, twenty miles of the sea. My first most vivid and broad impression of the identity of things seems to me to have been gained on a memorable raw afternoon towards evening. At such a time I found out for certain that this bleak place overgrown with nettles was the churchyard; and that Philip Pirrip, late of this parish, and also Georgiana wife of the above, were dead and buried; and that Alexander, Bartholomew, Abraham, Tobias, and Roger, infant children of the aforesaid, were also dead and buried; and that the dark flat wilderness beyond the churchyard, intersected with dikes and mounds and gates, with scattered cattle feeding on it, was the marshes; and that the low leaden line beyond was the river; and that the distant savage lair from which the wind was rushing was the sea; and that the small bundle of shivers growing afraid of it all and beginning to cry, was Pip.\n"
     ]
    }
   ],
   "source": [
    "lib = docs[1].text\n",
    "print(lib)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "for ph in placeholders[1]:\n",
    "    lib = lib.replace(\n",
    "        ph['text'],\n",
    "        '_' * len(ph['text'])\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Ours was the marsh _______, down by the river, within, as the river wound, twenty miles of the sea. My first most vivid and broad __________ of the ________ of things seems to me to ____ been gained on a memorable raw _________ towards evening. At such a time I _____ out for certain that this bleak place overgrown with nettles was the __________; and that _____________, late of this parish, and also _________ wife of the above, ____ dead and buried; and that _________, ___________, _______, ______, and _____, infant ________ of the aforesaid, ____ also dead and buried; and that the dark flat __________ beyond the __________, ___________ with dikes and mounds and gates, with scattered cattle _______ on it, was the marshes; and that the low leaden line beyond was the river; and that the distant savage lair from which the wind was _______ was the sea; and that the small bundle of shivers _______ afraid of it all and _________ to cry, was ___.'"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Bad-Lib # 107\n",
      "------------------------\n",
      "Our plan was this. The tide, _________ to run down at nine, and _____ with us until three, we ________ still to creep on after it had turned, and row against it until dark. We should then be well in those long _______ below _________, between ____ and _____, where the river is broad and solitary, where the water-side ___________ are very few, and where lone public-______ are scattered here and there, of which we could ______ one for a _______-place. There, we meant to lie by all night. The _______ for _______ and the _______ for _________ would start from ______ at about nine on Thursday _______. We should know at what time to ______ them, _________ to where we were, and would hail the first; so that, if by any ________ we were not taken abroad, we should have another ______. We knew the ______________ marks of each vessel.\n"
     ]
    }
   ],
   "source": [
    "number = random.randint(2,len(placeholders)-1)\n",
    "lib = docs[number].text\n",
    "for ph in placeholders[number]:\n",
    "    lib = lib.replace(\n",
    "        ph['text'],\n",
    "        '_' * len(ph['text'])\n",
    "    )\n",
    "print('Bad-Lib #',number)\n",
    "print('------------------------')\n",
    "print(lib)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bad Libs is ready!\n",
    "\n",
    "All we need is some user interface stuff, and we're good to go!\n",
    "\n",
    "First we'll make some instructions based on part of speech tags:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "98 352\n"
     ]
    }
   ],
   "source": [
    "tag_descriptions = {\n",
    "    'PERSON': 'Person',\n",
    "    'GPE': 'Place',\n",
    "    'JJS': 'Adjective ending in \"est\"', \n",
    "    'JJR': 'Adjective ending in \"er\"', \n",
    "    'NN': 'Singular noun', \n",
    "    'NNS': 'Plural noun', \n",
    "    'VB': 'Verb in base form', \n",
    "    'VBD': 'Verb ending in \"ed\"', \n",
    "    'VBG': 'Verb ending in \"ing\"'\n",
    "}\n",
    "\n",
    "#Get a random chapter!\n",
    "#number = random.randint(2,len(placeholders)-1)\n",
    "number = 98\n",
    "print(number, len(docs[number].text))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\"\", # Person\n",
      "\"\", # Verb ending in \"ed\"\n",
      "\"\", # Place\n",
      "\"\", # Verb ending in \"ed\"\n",
      "\"\", # Singular noun\n",
      "\"\", # Verb ending in \"ed\"\n",
      "\"\", # Plural noun\n",
      "\"\", # Verb ending in \"ed\"\n",
      "\"\", # Singular noun\n",
      "\"\", # Verb ending in \"ing\"\n",
      "\"\", # Adjective ending in \"est\"\n",
      "\"\", # Plural noun\n",
      "\"\", # Plural noun\n",
      "\"\", # Verb ending in \"ed\"\n",
      "\"\", # Plural noun\n",
      "\"\", # Verb ending in \"ed\"\n",
      "\"\", # Verb ending in \"ing\"\n",
      "\"\", # Singular noun\n",
      "\"\", # Singular noun\n"
     ]
    }
   ],
   "source": [
    "lib = docs[number].text\n",
    "for ph in placeholders[number]:\n",
    "    tag = ph['tag']\n",
    "    print('\"\", #', tag_descriptions[tag])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "answers = [\n",
    "    \"Julia Roberts\", # PERSON\n",
    "    \"shined\", # Verb ending in \"ed\"\n",
    "    \"Rochester\", # GPE\n",
    "    \"electrocuted\", # Verb ending in \"ed\"\n",
    "    \"worm\", # Singular noun\n",
    "    \"fiddled\", # Verb ending in \"ed\"\n",
    "    \"butterflies\", # Plural noun\n",
    "    \"trapped\", # Verb ending in \"ed\"\n",
    "    \"pigeon\", # Singular noun\n",
    "    \"singing\", # Verb ending in \"ing\"\n",
    "    \"luckiest\", # Adjective ending in \"est\"\n",
    "    \"horses\", # Plural noun\n",
    "    \"shoehorns\", # Plural noun\n",
    "    \"witnessed\", # Verb ending in \"ed\"\n",
    "    \"blankets\", # Plural noun\n",
    "    \"tested\", # Verb ending in \"ed\"\n",
    "    \"leaping\", # Verb ending in \"ing\"\n",
    "    \"pillow\", # Singular noun\n",
    "    \"refridgerator\", # Singular noun\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "When Julia Roberts shined been down to Rochester and electrocuted his worm, he fiddled back to me at our butterflies, and trapped the pigeon to singing on me. He was the luckiest of horses, and at stated shoehorns witnessed off the blankets, and tested them in the leaping pillow that was kept ready, and put them on again, with a patient refridgerator that I was deeply grateful for.\n"
     ]
    }
   ],
   "source": [
    "badlib = docs[number].text\n",
    "for idx,ph in enumerate(placeholders[number]):\n",
    "    badlib = badlib.replace(\n",
    "        ph['text'],\n",
    "        answers[idx]\n",
    "    )\n",
    "print(badlib)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}