No, I guess I should have experiemented more! :0 I clonked agains multiple walls after one mega charged swing and then sat on the ground for 3 seconds charging my next one, repeat.
With another well timed attack you can not only keep your speed up, you can even add the speed you managed to charge on the way.