If you spice up the Animations, the Crystals being stuck to the Handbone should already work out well enough, your Hands just don't really move a lot at the Moment.
>I already have the Mixamo magic animations for it.
An easy Way would be to make your own magic Animations. Just something more then only moving slightly back and forth, especially for the fireball lobbing etc, where you have a lot of windup time to do fancy casting. Don't know how well that'd work for VR tho.