honestly my biggest problem on the framework side was just getting the bundler to work since I just wanted a vanilla typescript app; ended up going with Vite after switching between like 5 different approaches.
I think Three.js is pretty nice if you don't mind making lots of stuff from scratch, and have a good handle on linear algebra and transformation matrices. Makes for a nice lightweight app