Firstly,
document.querySelector("#elem");
Has an advantage in the fact that, unlike document.getElementId, it can return classes. However, the usefulness of this is far diminished by the fact that it only returns the first object with that class name, so you might as well just use an id if you're not specifically looking for the first object with that classname. if you use,
document.querySelectorAll
However, I believe (I may be wrong), it returns all items with that classname as an array, where regular querySelector is equivalent to querySelectorAll[0]. One more advantage, is that you can run css3 queries through it, which can be quite useful.
Secondly,
document.getElementById("elem");
Has a very good advantage over queryselector in the sense that it is almost 5 times faster, so if you're sitting there with several thousand lines of code and you want to optimise said code, then getElementById is the way to go.
Lastly,
document.querySelector("[id=elem]");
I, personally, don't see the need to use this in any situation. If you needed a querySelector, why not just use a # ? This is exactly equivalent to your first example of querySelector, however it has a lot of useless charaters.
Edit: Just to be clear, in summary, you're probably better off using document.getElementById.