Yep, click the same texture again to remove it. If you're unsure which texture you've used, click a random one twice.
