So if you get into combat you'll be able to see what's modifying your agility by highlighting over your character portrait - probably low health. You can also see your actual unmodified agility on the character screen viewable from the world map.
I guess, that might be it. I'll check it a bit later.