I am a consultant and tend to work in a variety of environments. I carry a Thinkpad running VMWare Server over Ubuntu64 with 4GB of RAM. I've got a 320GB secondary hard drive that I use just for VM's and have 25 or so different virtual machines that I boot up as the circumstances demand.
They're a mix of Linux servers and workstations, Vista workstations and XP Workstations. I rarely use the VMWare server console. I access every one of them via one of the remote access methods.
For Linux, I usually install FreeNX or NXServer for desktop access and just SSH for commandline. On Windows, I always use Remote Desktop (RDP), but, on XP, that only works on the "Pro" versions, not the "Home" versions. If all else fails, I install VNC and use that. VNC is at the bottom of my list because it really is a last resort. The only thing it's better than is not actually being able to use the machine.
However, NX on Linux and RDP on Windows work WAY better than VNC. Other than little things like font smoothing and fancy desktop effects, the only big glitch would be if you are doing much with video or audio or DirectX-based stuff. Things like YouTube or other video do NOT like to work with any remote desktop protocol that I know of.
As far as performance, using Linux as a host for VMWare provides really good management of system resources. The Windows-based VM's aren't able to just gobble up memory, but still get it when they need to.
I do C# development all day in a virtual Vista workstation on Visual Studio 2008 and have absolutely no problems having 3-4 different solutions all open at once along with the normal stuff alongside over RDP on another machine, connected via wireless VPN.
I can flip over to the host OS and it won't even be touching swap space at all. As far as I'm concerned, it's a great way to work.