Thanks Mitnik!
Yeh I tried different control setups, but with the mouse buttons or controller triggers it just all started to feel much better. Which is also when I decided to drop the rhtym lines down to 2 as 4 was a bit too much while also walking around :)
Cut scenes were pretty last minute, but happy I managed to get them in, in time...