This is a plugin for the Godot game engine that allows the user to load and run a local Large Language Model (LLM) in-engine using the LLamaSharp (v0.11.1) C# library.
Run Mind Game stand-alone:
- Install CUDA Toolkit 12.x if you haven't already (CPU-only support coming soon)
- Download the latest Mind Game release for your platform and run the executable
- Download a .gguf model of the Llama, Phi, or Mistral families
- Load your model and have fun!
Recommended model: Llama3-8B-Instruct
Next best: Mistral-7B-Instruct-v0.2
Smaller model for those with less VRAM: Phi-3
Run Mind Game as a Godot plug-in:
- Download and extract Godot 4.3 dev5 (.NET version)
- Download/install .NET8
- Clone/download this repo, open it with Godot 4.3 .NET, click Project > Project Settings > Plugins > Enabled (Mind Game) to make it appear in the bottom bar.
- Currently you have to manually add the LLamaSharp and Cuda12 backend or Cpu backend Nuget packages to the project solution, use Visual Studio Community for this or a visual NuGet Manager extension for VS Code.
- Load a .gguf file of the llama, mistral, mixtral, or phi families to get going!
The lower quantization (q), the smaller the model is to run but at the cost of accuracy. Llama-3-8B-Instruct.Q4_K_M is a great middle-ground for those with 8GB of VRAM. The absolute smallest model (phi-2.Q2_K.gguf) can run on less than 4GB of VRAM.
Future steps:
- Implement LLaVa support (including viewport analysis)
- Automatically include and reference LLamaSharp nuget packages
- Make Download Manager functional
- Make a singleton to be able to access currently loaded model in-game
- Transition to BatchedExecutor and add conversation forking/rewinding
- Add network graph generation
- Add project script crawling
- Expose LLamaSharp methods like quantization
- Integrate Kernel Memory for document ingestion