I watched the trailer for the 1.15 features and see there is a FXAA node now. Tried that and it did do what I hoped it would do.
The FXAA node doesn't have any properties like "2x, 4x, 8x etc." so I just added a second FXAA node and it cleaned up the edges even further.
So there still is a question about what the "Anti alising" button does for the Outline node.