Ahh, you were SO close. The musicians I worked with wanted to control their own sounds and mixing, rather than whatever library I included in the app. Sorry, but I didn't consider the case of wanting the visualizer to ALSO be the audio source.
My typical setup is a full Midi keyboard that has its own audio output/speakers, and then using the keyboard's MIDI out to the PC to drive the visuals, which then can be projected to any external monitor or, better yet, projector from the PC's HDMI out.