You're right; there was a mechanic about reshuffling the deck in an earlier version and while I left the rule about removing diamonds in as I like the symbolism of it - feeling that, like the face cards, removing them highlights their 'importance' - it isn't necessary.
You've given me something to mull on though, so thanks for that!
[edit] I tried to put some more of my thoughts on the matter down in words here.