You can't. You can only remove the belt in one of the endings. There are three endings you can get. In the "Goblin King" ending you will be able to remove the belt completely.
Ah, got it. Thanks.
