Interesting concept but very very hard. Couldn't manage to do much.
maybe consider a drag & drop of the object ? and you'll still have to balance them.
Still good work ! it looks great :)