At that point, it really becomes a question of balance rather than realism. Adding too much benefit to a single activity (which already has two, raising both affection and accuracy) will skew the balance, so I will likely not change that. The MC will have to make do with the dirt he gets on him :)
That said, there will be other ways of increasing MCs own attributes.