There's no difference, we just try to stay with one and only that one.. you know.. loyal.
On my other playthrough I'm also going almost after everyone, there I have hearts with Bernard, Caleb, Rhot, Roushk, Korg and Ste currently :p (still no Gunnar yet)