How to parse / tokenize an SQL statement in Node.js [closed]

Answer 1

If you want to develop your own SQL parser, I'll recommend a PEG design parser. I've used a PEG parser for a compile-to-js/compile-to-c language, and it resulted into a much clear and easy to mantain code. Check: https://github.com/luciotato/LiteScript

You can start from LiteScript parser if: a) this parser is a important part of your application, b) eventually you'll need native-exe-speed parsing.

But if this is not a important part of the application you're developing, contributing to a existent specific sql parser could be the best choice.

Answer 2

I recently published the sql-ast package, which parses SQL scripts and returns an array of AST nodes. (I couldn't find anything similar that's maintained)

It's very limited at the moment. I've only implemented what I need from it. If you have time, please contribute. I've tried hard to make the codebase understandable.

Give it a star if you're interested in seeing it developed further. I will update this answer as the library is improved.

Answer 3

Good luck.

You didn't mention which SQL, but most of the production SQL languages are huge (check out PL/SQL ignoring the Ada part), complicated and not the same so you'll have to worry about dialect variants, too. You are facing building a full SQL front end to do what you want; the incompleteness of other parsers you found is a hint about the level of effort it takes to do this.

After you get the parser part right, then you'll have to do a symbol table and type analysis (meaning of every symbol) before you can begin to find out what a SQL query reads or writes (consider determining columns read by SELECT * from .. ...).

I understand there are commercial SQL parsers out there. You might consider using one of those.

Answer 4

You can look at the SQLite/WebSQL JavaScript parser and Jison grammar file, which can be used for verification tool.

Now it supports full SQLite/WebSQL syntax, and can be modified for other SQL syntax.