In version 18 (current public) I had 3(or maybe 2?) basic agility and got a "Faster(1)" per, but during the encounter with goblin it said that I only had 2 agility (in reality I should have 3 or 4).
So if you get into combat you'll be able to see what's modifying your agility by highlighting over your character portrait - probably low health. You can also see your actual unmodified agility on the character screen viewable from the world map.