A long time ago I did a crude PoC for a raycaster engine. It needed to run at very high speed, and the concept of PWM-ing the display to achieve distinct colors was visually quite painful:
https://github.com/JohnEarnest/Octo/blob/gh-pages/examples/demos/chipenstein.8o
Your progress so far looks rad, Timendus! Really excited to see how it turns out.