It's intended that the skill points are limited, and that skillstones are rare. You'll get more skill stones in the future content of the game, that's why they might feel a bit fewer than necessary.
Keep in mind that skills can be reset at all times, and setups for characters can be saved and loaded. This way, you can quickly change what character is fully buffed.