MC Wiki Crawler

1. main.py
   • crawl: crawl data
   • split_content: split files whose content maybe too long as input to gpts
2. crawler_data
   • Block, Item, Mob: not split files data
   • _split suffix: split files data

TODO

• https://minecraft.wiki/w/Biome Binome name detail pages

• https://minecraft.wiki/w/Crafting Hidden recipe information

  • Formula ingredient quantity information

• https://minecraft.wiki/w/Smelting hunger info

  • the page uses an image to represent hunger, need to convert it to text hunger

• https://minecraft.wiki/w/Smithing

  • Template, material detail links
  • upgrading recipe: convert images to text info

• https://minecraft.wiki/w/Structure

  • detail links