You could still open source the other parts of the game and just not include the assets from this asset pack.
That sounds like a good way to handle it! And refer anyone who would want to build it about how to acquire and use the assets themselves.
