Got here trying to find it again after finding it on another save - as suspected I seem to be able to access the glafire caves with just 10 perception. For any other time travellers looking for information, just shave away any debuffs and try it. The requirement must have been lowered.