Just making his dialogue more scummy in nature would be enough, most people will hate him right there and then without having to make him so absurdly unpleasant to look at.
if you see someone sayin "i hate nigg-",you would call him scum but not get into a figth, now if he punches and kickes someone hes pretty much gonna get rapped,