I never use the My namespace (I'm a C# developer), but my VB co-worker doesn't as well. I found the My members not necessary, because in many cases, they're counter-intuitive for me, e.g. in my opinion opening a file has something to do with IO (hence System.IO.File) and not with my computer (My.Computer.FileSystem). They always seem so scattered and bunched together.
It's just some re-roll of functionality that is already available otherwise, from all languages. And I don't like depending on Microsoft.VisualBasic.dll when I'm developing for .NET - I always prefer System.*.
And then, it's always kind of limited. I see VB developers struggle with their app when they can't find something in the My namespace, because they can't imagine that you can use something in the System namespace. That of course is not a problem of the My namespace itself.