Using an arbitrary Windows machine (2000/XP or later), I can
install Eclipse CDT to a USB drive
move that USB drive onto a different Windows
It can definitely be done! I've seen a technician with a copy of it on a USB stick. the only visible flaw was that when you run on a different PC it requires you to enter the license. I could not see any other problems (speed/debugger etc. it all worked on his copy). check this out: http://technet.microsoft.com/en-us/appvirtualization/dd334515.aspx I believe the virtualized package I saw was made by this means: http://spoon.net/Studio/ Unfortunately it would be matter of experimenting with it...
No VM or extra software was needed!
Eclipse is apparently designed to be what Microsoft calls an XCOPY deployment...meaning that it doesn't require any special entries in the Windows Registry (or any other "installation identity" on the target machine) in order to work properly.
Visual Studio is most decidedly not designed like this. It makes extensive modifications to the registry during installation, and those entries (and any other resources like special folder locations) will be missing on any other computer.
So you might be able to install Visual Studio on a thumb drive, but some artifacts of the installation will be put on the C: drive, and you will only be able to use the thumb drive with that machine.
Maybe you could install VS Express in a VM running from the USB drive using Portable VirtualBox or VMPlayer. Not the best performance but its usable for not too big projects or learning.