If I include a "license.txt" for the libraries being used, would that count toward the 4 MB?
This is a good question. Although you absolutely should include license files in your final submission, they are not crucial for the game to run and are mostly just a notice "Hey I use this and here is the document saying that I can". Depending on the license, I would maybe just include the copyright notice in the game and wouldn't count the whole license text.