I like the concept a lot, very interesting spin on an existing idea. The implementation works really well in my experience too, I didn't run into many bugs.
I did see the issue with the player rapidly teleporting back and forth between portals, one way to fix this might be to make it so that the player cannot reenter the portal they just exited for a short time (2-5 frames) so that they have time to move away from the portal before it re-teleports them,
Otherwise, I might just try and add a few more elements to the game.