make sure you've exhausted all of the talk options too, there may also be other requirements (make sure you exhaust all talk options for all characters, esp the bartender for hints)
[edit] even talk options that don't have the question mark next to them can change/result in affection increases after revisiting them, so be sure to do so after every major event (especially the "ask if you can help" option for Bernard, the outcome of which can change depending on your character's STR)