Ellen was the MC's grandfather's love of her life, not the MC. The "Sleep with" is done with characters that have an intimate relationship with the MC. Also, she has expressed not wanting a child at this point in her life. Whether or not that will change, and she grows to love the MC as it is stated she is closer in age to the MC than she was to his grandfather could be an idea in the future. Ellen's character design is very beautiful to be honest.