If you want to write a verbatim string containing a double-quote you must write two double-quotes.
string s2 = @"This is \t a ""verbatim"" string";
This is covered in section 2.4.4.5 of the C# specification:
2.4.4.5 String literals
C# supports two forms of string literals: regular string literals and verbatim string literals.
A regular string literal consists of zero or more characters enclosed in double quotes, as in "hello", and may include both simple escape sequences (such as \t for the tab character) and hexadecimal and Unicode escape sequences.
A verbatim string literal consists of an @ character followed by a double-quote character, zero or more characters, and a closing double-quote character. A simple example is @"hello". In a verbatim string literal, the characters between the delimiters are interpreted verbatim, the only exception being a quote-escape-sequence. In particular, simple escape sequences and hexadecimal and Unicode escape sequences are not processed in verbatim string literals. A verbatim string literal may span multiple lines.