Rome wasn't built in a day. Refactoring an entire project takes a lot of time, I know from experience, and comes with a plethora of complications. Don't be discouraged by the vocal minority, you have a large community that is ready to wait patiently for your best work. Take care.