It does reduce performance but it's kinda optional!
If you want a class to be garbage collected then you have to derive it from UObject and give it the UClass macro, else it will not be garbage collected.
I said 'kinda' because if you want a class to be recognized by Unreal editor then you have to enable garbage collection for it!