Antlr v3 error with parser/lexer rules

走远了吗. 提交于 2019-12-17 21:13:45

问题


I am having problems with my Antlr grammar. I'm trying to write a parser rule for 'typedident' which can accept the following inputs:

'int a' or 'char a'

The variable name 'a' is from my lexer rule 'IDENT' which is defined as follows:

IDENT : (('a'..'z'|'A'..'Z') | '_') (('a'..'z'|'A'..'Z')|('0'..'9')| '_')*;

My 'typedident' parser rule is as follows:

typedident : (INT|CHAR) IDENT;

INT and CHAR having been defined as tokens.

The problem I'm having is that when I test 'typedident' the variable name has to be more than one character. For example:

'int a' isn't accepted while 'int ab' is accepted.

The outputed error I get is:

"MismatchedTokenException: mismatched input 'a' expecting '$'"

Any idea why I'm getting this error? I'm fairly new to Antlr so apologies if the error is trivial.

EDIT

I literally just got it working, and I don't know why. I also had two other lexer rules defined as follows:

ALPH : ('a'..'z'|'A'..'Z'); 
DIGIT : ('0'..'9'); 

I realised these weren't being used at all so I deleted them and everything now works fine! My guess why this works is because ALPH and DIGIT were overriding my other Lexer rules:

NUMBER : ('0'..'9')+; 
CHARACTER : '\'' (~('\n' | '\r' |'\'')) '\''; 

Does anyone know if this is the case? I'm curious as to why this problem has now been solved.


回答1:


'int a' isn't accepted while 'int ab' is accepted. ... My guess why this works is because ALPH and DIGIT were overriding ...

Yes, it appears ALPH was defined before the IDENT rule, in which case single letters were tokenized as ALPH tokens. If IDENT was defined before ALPH, it should all go okay (in your case).

To summarize how ANTLR's lexer rules work:

  • lexer rules match as much characters as possible (greedy);
  • if 2 (or more) lexer rules match the same input, the rule defined first will "win".

You must realize that the lexer does not produce tokens based on what the parser (at that time) needs. The lexer operates independently from the parser.



来源:https://stackoverflow.com/questions/9251048/antlr-v3-error-with-parser-lexer-rules

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!