Just tested, and interacting with the sparkles to start the mini-game, at least in the normal story flow, works from both top and side. Are you sure you're on the most recent version, 1.0.5, as there was a bug like this earlier that was patched? Are you using mouse movement, or arrow keys? I recommend arrow, as mouse can introduce some strange issues occasionally.