Is there a measurable difference in skill, productivity, or programming enjoyment between people who depend on IDEs and those who don't, or is it all just fanboyism?
Let's just narrow this down to Visual Studio vs. Emacs---the question is too broad otherwise, and I suspect most people (at least here on StackOverflow) are familiar with those two. [I am, and I use and prefer emacs]
Next, let's break the three components of the question down.
Programming enjoyment boils down to primarily liking programming in itself, and secondarily to not have annoyance factors pop up when you do it. Using tools that you struggle with is going to subtract from the enjoyment. So, I would guesstimate, the enjoyment is likely maximized by using tools you like and are familiar with.
Of course, once you do get familiar with a new tool, you may find that you like that tool better, and thus will like programming more if you do it with your new favorite tool. It isn't clear to me that people in general will like VS over emacs (or vice versa) when they know them both well.
Next, programming skill. If there's any connection between skill and choice of VS/emacs, I think it's skill causing choice, not choice causing skill. None of {VS, emacs}, in my experience, seem to make me learn anything about writing better code.
A good tool may make me able to write the same code (and thus same quality of code) in smaller amounts of time; if we assume that productivity = quality of code `times` code per time, then it stands to reason that any editor that'll let you write good code in less time is a productivity boost.
I may be too unfamiliar with VS, but here I think emacs has some strengths that I haven't found in VS. It may sound ridiculous, but something as simple as cursor movement with Ctrl-[fbnp] is a real boon---it means you don't have to move your hands to the arrow keys. Another thing: you can do a search or replace which includes newlines, which I use often enough that I'd hate to miss it.
Another thing I like is the ability to embed a shell in my editor. In my experience, it's much more valuable on non-windows platforms (and my first "programming" experience, IIRC, was writing DOS .bat files, so I'm not just a flaming fanboy), but on those platforms it's a big win. It makes emacs "integrate" with "everything" (file search with find/locate, text search with grep, version control with svn/git/hg/..., you name it).
Final verdict--should you learn emacs or not? And is it all fanboyism? If you got the time, I'd say it's always worth learning new ways of doing the same thing, because the new way just might turn out to work better for you. And I encourage you to not reject it for what it appears to be when you're all new to it. In that sense, it's like switching to the Dvorak keyboard layout: you're slower at first, but once you get up to speed you're likely to be at least as fast as on qwerty, and more comfortable. I recommend the same approach to learning new (programming) languages as well: because you don't know the vocabulary (standard library), saying things is hard, but the reward is still there in the horizon waiting for you.
