I generally like trying to come up with my own system, sometimes just as a challenge and sometimes cause it matches a theme. I'll highlight a success and failure here:
I made a game called Kintsugi that is like an emotional back to the future: 2 characters have some sort of fractured relationship and they travel through the timeline to change it, but successes/failures can have ripple effects that change time. The resolution is whenever either character wants to do something challenging, BOTH character players do a sort of rock paper scissors and shoot either 1 or 2 fingers out. If the sum of fingers is odd, they succeed, if its even, they fail. They're not allowed to communicate or strategize, so my idea is that the two people really have to be working together and want the same thing for it to work. The GM role also rotates in this game, so as the timeline changes, you might not have a sort of foundational strategy on how to game the system anymore. I think this is a good one!
Another game of mine, Protest Singer, is about musicians casting spells through magic songs, but admittedly the resolution isn't great. I REALLY wanted to use d12s because of the 12 notes in an octave, so there is a songwriting meta-game where you see how many times you can roll a d12 and not get the same result, a push your luck gambling thing, and you write those down on your setlist. then, in the game, you roll a pool of d12s based on your relevant stat trying to get one of the right notes. I think this is super themey but admittedly pretty clunky, would be difficult for new players to understand, and requires a bunch of d12s which is just sorta weird. I don't know why I was so married to the idea the whole time.
In general I find it hard when you're deep in a design to think critically about that sort of thing, so resolution is one of the first things I want to figure out. Unfortunately, it can lead to stubbornness and not wanting to change something that might not actually be the best idea. My advice for custom systems would be 1) question why you're doing it and who you're doing it for and 2) take a couple looks at it over the course of development and be willing to change it if the game evolves!