The OpenAI API and a lot of the free tools that implement it support a thing called "structured output". See Structured Outputs - OpenAI API. If you use this to define what the response JSON should look like, you should be able to focus your prompts on other things rather than just on the data structure and you can also guarantee that the model returns valid JSON, which would fix a lot of the connection issues and let people use smaller models.

Just something to consider if you haven't already poked at it.

Thanks! I didn’t use it cus most self-hosted engines like koboldcpp don’t support it but after some looking most AI providers support it. Unfortunately having extra parameters may cause the endpoint to refuse the request hmm… For example if the game put response_format=‘json’ into the request and the endpoint doesn’t support it it may just outright refuse to do it. Maybe I can make this optional togglable in settings…

For the response_format you have to give it a whole json schema, which is a lot of work. It's just worth it for stuff like this, if you have the time for it. If you can't then that's probably not too bad either tbh, I've been using the Gemma 9B Instruct model with "Include no text aside from the JSON response. Do not wrap the JSON in quotes or markdown." added to the end of the prompt and it actually does it correctly 99% of the time.