Either you don't have the quest for gathering the spores yet, or you only interacted with them, as you only need to interact with them once to get the spores. If you already have the spores, go back to the guard who gave you the quest.
