Oooh, pressing 'C' (or '/' or Triangle on a pad) opens the collection menu. I included the controls in the description but should have had them explained in-game too!
By design, the collectables do respawn. I just wanted to keep it easy to collect them to prevent too much backtracking.