Great start! I ran into a bug where the block colliders stayed around after they were destroyed, but it was an interesting concept!
Viewing post in Buildings Eraser jam comments
I'm using queue_free() but the issue comes from where I use it.
Walls are RigidBody2D with Polygon2D and CollisionPolygon2D inside.
Currently when walls are fractured I use queue_free() on the polygon (that gets split into fragments). Simply using get_parent().queue_free() solves the issue (because whole wall including CollisionPolygon2D gets removed).