Honestly, I am not really a big fan of the maximum potential mechanic.
It will essentially limit your combat capabilities if you want to take sidegig like weapon smithing or tailoring.
For example, a 2H macefighter needs Mace (duh), Athletics (escaping bad fights and survuvability), and Leadership (to get an army). As far as I know, you will only be able to max out 4 skills. Add Adventuring, which I think is mandatory for every character, and there is no room anymore for any sidejobs.
I understand what you were going for in that you wanted to limit the player into a "class" they can build instead of just a master of everything, but in practice it just feels limiting.