I'm using queue_free() but the issue comes from where I use it.
Walls are RigidBody2D with Polygon2D and CollisionPolygon2D inside.
Currently when walls are fractured I use queue_free() on the polygon (that gets split into fragments). Simply using get_parent().queue_free() solves the issue (because whole wall including CollisionPolygon2D gets removed).