This happens because the weapon you were holding was not exactly correct for a Specific Sword Customer.
If you hold a weapon with a single piece wrong (currently) they will cause the map to generate the guard related to the request
It should hopefully be addressed.
As long as the sword is 100% correct, it shouldn't happen, but, they will do it ANYWHERE - which means if you are just holding the item without even moving and not even near the shop, there will be guards spawned, even if you can't tell there's a Specific Sword customer at the counter.
This becomes particularly even more problematic if you sell ANYTHING to Shield Customers who will cause additional customers to spawn due to an error (that should hopefully also be addressed)
Unlike Specific Sword Customers, You should immediately refuse any and all Shield customers, it isn't the shield that causes it, its the waiting they do while waiting to accept the next item. If they trigger that, they'll trigger a second customer, regardless of what comes after that. - Then customers can start stacking, and if they do that, they can grab the same item, and it can be hard/impossible to see when you have a Specific Sword customer, which will then cause the game to lag when more guards get spammed