Yes, absolutely leave Windows Defender on. I see Windows Defender as a part of Windows and don't even put it on the same level as all these thirdparty anti-virus 'products'.
Companies who distribute anti-virus programs are motivated to take your money and want you to believe you need them. Windows Defender is made by Microsoft and they already got your money when you bought Windows, they don't care and don't tread on a fine line between legit business and flat out scamming you.
I will edit my post to exclude Windows Defender more clearly. I have to deal with computers not functioning properly due to these non-Microsoft (thirdparty) anti-virus programs every day through work. Frankly they clearly cause more damage than actual viruses in my experience.
Also, Windows 10 will get a built-in sandbox feature soon. Just run anything you're uncertain about in that sandbox. This is safer than any such anti-virus could ever hope to be and won't get in your way either.