I'm a fan of that idea! If it were a debuff, then it would force the player to rely on other movement options (namely the grappling plunger, which I think is indispensable even with flight) in order to get around during a fight, and whatever enemy cast it would become top priority to defeat. Maybe consider adding other movement items before doing this, though.
(Like, you've mentioned adding a dash. Maybe it could be tied to an item, like a grimoire or the grappling plunger that allowed you to dash in the direction you double-tap towards while it was held, and it had a cooldown that prevented you from spamming it. In design, it could be like one of those air-cannon toys that the juiceling can point behind them to give them a burst of speed, I dunno. In any event, debuff or not, I agree with OP on experimenting with situations where the player can't fly, it seems like a cool idea.)