Using ANTLR Parser and Lexer Separatly

Question

I used ANTLR version 4 for creating compiler.First Phase was the Lexer part. I created \"CompilerLexer.g4\" file and putted lexer rules in it.It works fine.

Compiler

Answer 1

Lexer rules start with a capital letter, and parser rules start with a lowercase letter. In a parser grammar, you can't define tokens. And since ANTLR thinks all your upper-cased rules lexer rules, it produces theses errors/warning.

EDIT

user2998131 wrote:

General Questions: What is difference between using combined grammar and using lexer and parser separately?

Separating the lexer and parser rules will keeps things organized. Also, when creating separate lexer and parser grammars, you can't (accidentally) put literal tokens inside your parser grammar but will need to define all tokens in your lexer grammar. This will make it apparent which lexer rules get matched before others, and you can't make any typo's inside recurring literal tokens:

grammar P;

r1 : 'foo' r2;

r2 : r3 'foo '; // added an accidental space after 'foo'

But when you have a parser grammar, you can't make that mistake. You will have to use the lexer rule that matches 'foo':

parser grammar P

options { tokenVocab=L; }

r1 : FOO r2;

r2 : r3 FOO;


lexer grammar L;

FOO : 'foo';

user2998131 wrote:

How should join separate grammar and lexer files?

Just like you do in your parser grammar: you point to the proper tokenVocab inside the options { ... } block.

Note that you can also import grammars, which is something different: https://github.com/antlr/antlr4/blob/master/doc/grammars.md#grammar-imports