You need to press z while holding down to drop it :)
Wow ok I finished the game, I love it! I'm impressed by how many levels you guys made, it felt like a complete experience already