Parsing T-SQL statements to tokens

Question

You can see here how nicely C#code statement is parse to tokens. For example, the following code:

namespace MyNamespace
{
    class MyClass
             


        

Answer 1

I had to add the reference manually in the csproj

Microsoft.SqlServer.Management.SqlParser, Version=12.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91

Like

<Reference Include="Microsoft.SqlServer.Management.SqlParser, Version=12.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91" />

A simple example:

string sql = "IIF(COALESCE([Col001], [Col002], [Col003]) > [Col004], [Col005] * [Col006] + ISNULL([Col007], [Col008]), CONCAT(SUBSTRING([Col009], 0, 3), 'sample text', [Col010]))";

var po = new ParseOptions { };
var scanner = new Scanner(po);
scanner.SetSource(sql, 0);

Tokens token;
int state = 0;
int start;
int end;
bool isPairMatch;
bool isExecAutoParamHelp;

while ((token = (Tokens)scanner.GetNext(ref state, out start, out end, out isPairMatch, out isExecAutoParamHelp)) != Tokens.EOF)
{
    string str = sql.Substring(start, end - start + 1);
    Console.WriteLine("{0}: {1}", token, str);
}

Taken from http://www.sqlservercentral.com/blogs/dave_ballantynes_blog/2012/03/13/parsing-t-sql-the-easy-way/

Taken from Parsing T-SQL – The easy way

Note that this parser recognizes a certain number of functions (like IIF, COALESCE, ...). Unrecognized functions are simply marked as TOKEN_ID, like column names.