I definetly think the weapon swap should stay aswell, I haven't used it in the Demo a lot, but its a very uncomplicated system that can easily change combat up a bit and it feels fairly unique for a JRPG.
I think your best bet would probably be to see which you can make more interesting. using relics to properly alter/improve your skills instead of just numbers going up, or more uniqueness in the shieldbreak system? Not every enemy having it, skills giving/taking shield points, only particular weapons affecting it (this ones already in if I recall correctly). If you get both to synergize well with each other, instead of feeling so much like 2 different systems you could potentially keep both aswell.
Personally I'd probably go with the shield break. If you expand that as mentioned above, it can work quite well also with the weapon swapping, since you can only break shields with certain weapons/weaponskills, maybe forcing you into your secondary weapon if you want to break that specific shield to be able to use a spell which would otherwise just be ignored or something.
*Edit: Be sure to play and test around with your current setup first tho. It'd be a shame if my stupid Ideas ruined your Combat.