Well, it depends. Its part of Explorer, and Windows uses some of its files (rather then the other way around...) However, not using Internet Explorer to browse the internet is still safer, as IE is a piece of crap.
It would be if not using it got rid of the vulnerabilities, but it doesn't. One program can be used to exploit another then another. If it is on your system, and it always will be, then it can be exploited.
There is actually a way to "uninstall" it, which I won't go through as it can seriously fuck up your computer. If you're interested in how to do it, google it. You could actually completely remove IE from your computer before XP, but as I stated, Windows XP now actually use parts of IE, rather then IE using parts of Windows.
Yeah, I know you can remove it from XP and older, but on vista, IE7 is there and there is nothing you can do about it. You can't even "downgrade" to 6.
Also, like I said, you have to use IE to update windows. The windows update site uses IE only Scripts, so getting rid of it would actually cripple your computer.
Also, every program has security holes. All of them. If you have more programs you will have more security problems. You can have IE with no Firefox, but you can't have Firefox with no IE.