This is not a bug and it has nothing to do with the shebang, just the exclamation point.
Enclosing characters in double quotes preserves the literal value of
all characters within the quotes, with the exception of $, `, \, and,
when history expansion is enabled, !.
So either escape it, use single quotes, or turn off history expansion.
e.g.
> echo "How dare you put an '!' in this string?"
bash: !: event not found
> set +o histexpand
> echo "How dare you put an '!' in this string?"
How dare you put an '!' in this string?