The game is way too hard.
For one thing, you're immediately dropping the player into a level with a note density that is pretty much what you'd see in the harder difficulty levels of most other rhythm games.
You also start throwing those very fast notes at the player before they've had any chance to listen to the music for a few seconds and get a feel for the rhythm.
The lack of audio or visual feedback for hits and misses makes it even harder to establish yourself on the rhythm.
Bottom line: most players are going to lose your game within the first two seconds, and that means you're going to lose most players within the first two seconds. If you want to go for this level of difficulty, you *really* need to ease the player into it.