Thats it. Thanks for read.
Every choice transforms your body and shapes your adventure · By
Home · LostRuins/koboldcpp Wiki · GitHub This is the guide for Koboldcpp, which is the version that lets you run AI models. It also sets up an API you can point Formamorph at. For more specific info it's a bit more complicated since getting a model to actually run well on your own computer depends a ton on how much VRAM you have.
bartowski/gemma-2-9b-it-GGUF at main The 8_0.gguf or 8_0_L.gguf actually fit entirely in 12GB and are really good at following instructions. Kobold should set it to fully offload to GPU automatically, and you can crank the context size to 8192 in the Koboldcpp UI. Might still need to fiddle with the system prompt in the Formamorph settings. Also, by default the "Endpoint URL" for Formamorph should be "http://localhost:5001/v1/chat/completions".
You'll probably run out of context if you play for a while but that should be enough to get you going for starters.