Yeah but otherwise, I think you're on a good path. The game was actually fun this early already! However, I wouldn't use a RigidBody player controller (you're using that right?), as it is kind of hard to control the movement.
Yes I‘m using the Unity Standard Rigidbody First Person Controller and I don’t have Problems with it ,till yet
But what else should I use then? I tried to make a Player Controller on my own, but that’s very hard .
Ehm and sry for my bad English, I am a student in Germany and i only have English in school
Do you have Skype, discord, ts or mumble? Then we could talk a bit