have you tried to eat plenty of strange fruits? It should be fine to keep those buffs as long as you don't exit or enter the dungeons. I suppose it is a way to make you competitive when facing those elite opponents while keeping your level low(You should notice that the exp needed to level up is way less when your level is extremely low after the sacrifice, right? So you could sacrifice all of your levels every time you finish a battle to maximize the gain)
Or maybe you could cultivate serval powerful followers instead of having buffs, the principle is the same.