It is a bit unintuitive. You'll need to type their ip into the box as "their ip:10567". (Edit: This is no longer necessary)
This is because the current version connects through websockets, so you'll need to specify a port if not playing with a remote player.