Short answer? Yes.
I used to do network/desktop support before I became a DBA, which is why I emphasize this so strongly. So, please don't take this as a flame, because it's not intended that way.
Just because you have all that stuff doesn't mean your machine isn't vunerable. Windows 2003 Server has a lot of stuff turned off by default but it's still got a lot of issues. Subscribe to the Microsoft Security Alert bulletins if you don't believe me. @=) And because it is a Server OS, its nature dictates that it can reach out to other computers and do "bad stuff". If you get an email on your PC that has a trojan designed to turn your box into a zombie, then you get to explain to the rest of IT why you installed a server OS on a PC.
If you're going to leave your PC attached to the network, you're going to have to practice all kinds of restraint in visiting web pages and opening emails. Restraint you may never had needed to worry about before. Cookies can track your information and a Server OS can "sense" other servers on the network. If someone managed to hijack your IE browser and put a sniffer or keystroke logger on it, then it could compromise your entire network.
Yes, I'm paranoid. Maybe a little more than the average user. But I've seen too many other people go down this path and lose their jobs over the resulting chaos. Therefore, I don't recommend this to anyone.
If you're not going to remove your machine from the network, you should at least contact your network / server / security team (if you have one) to make sure your box is locked down tight. Also, you'll probably want to do internet browsing and email on another machine so you don't tempt fate. Lastly, you'll need to be very careful about using USB drives / floppies / CDs & DVDs to install files, music or pictures that you got from home or friends.
Anyway, I hope my advice helps you out. And remember, just because I'm paranoid doesn't mean the hackers *aren't* out to get me. @=)
Catadmin - MCDBA, MCSA
"No, no. Yes. No, I tried that. Yes, both ways. No, I don't know. No again. Are there any more questions?"
-- Xena, "Been There, Done That"