IlegalStateException when I type unfinished code into a JEditorPane with syntax highlighting support

问题

Using ANTLR4 and the Netbeans Platform I have created a grammar for which I have implemented syntax highlighting in my Netbeans Platform application. Everything works fine up to the point when I try to alter my code.

grammar MyRule;

my_rule : '(' my_rule ')'
             | binary
             | binary_hyst
             | my_rule (AND  my_rule)+
             | my_rule (OR  my_rule)+
             ;

binary : '(' binary ')'
       | unary (EQ | NE) unary 
       ;

binary_hyst : '(' binary_hyst ')'
            | unary hyst? ( GT | LT | GTE | LTE ) unary hyst?
            ;

hyst : (HYST '(' unary ')');

unary : '(' unary ')'
      | NUMBER
      | STRING
      | (MOV | DEC | INC) '(' unary ',' NUMBER ')'
      | unary '*' unary 
      | unary '/' unary
      | unary '+' unary
      | unary '-' unary
      ;


// TOKENS 
HYST    : ('hyst');
MOV     : ('mov');
INC     : ('inc');
DEC     : ('dec');

AND : ('&&');
OR  : ('||');

EQ : ('==');
NE : ('!=');

GT  : ('>');
LT  : ('<');
GTE : ('>=');
LTE : ('<=');

NUMBER
    :   '-'? INT '.' [0-9]+  // 1.35, 0.3, -4.5
    |   '-'? INT             // -3, 45
    ;


fragment INT :   '0' | [1-9] [0-9]* ; // no leading zeros
STRING : '"' ( ESC | . )*? '"' ;
ID : [a-zA-Z_]+;
fragment ESC: '\\'  [btnr"\\] ;

WS : [ \t\n\r]+ -> channel(HIDDEN) ;

This grammar allows code like this: ("some_string" >= 20). But when I try to alter my code (i.e. ("some_id" >= 20) && ("some_other_id < 10) I receive an IllegalStateException:

java.lang.IllegalStateException: Lexer     de.nordsys.catelogconfig.rulelanguagesupport.lexer.TriggerRuleNBLexer@181848
      returned null token but lexerInput.readLength()=2
      lexer-state: null
      tokenStartOffset=20, readOffset=22, lookaheadOffset=23
   Chars: "&\n" - these characters need to be tokenized.
    Fix the lexer to not return null token in this state.
        at         org.netbeans.lib.lexer.LexerInputOperation.checkLexerInputFinished(LexerInputOperation.java:457)
        at org.netbeans.lib.lexer.LexerInputOperation.nextToken(LexerInputOperation.java:217)
        at org.netbeans.lib.lexer.inc.TokenListUpdater.relex(TokenListUpdater.java:627)
        at 

...

    line 1:1 token recognition error at: '&\n'

This happens right when I type the first '&' for the '&&' token. It seems that Netbeans wants to parse my code and the lexer recognizes that this is not a valid token, which is correct. If I simply copy and paste a '&&' it works fine and Netbeans doesn't throw an exception. So this seems to be coming from me typing in incomplete tokens. How can I resolve this?

回答1:

You need to ensure that tokens are created for all possible input sequences. The most straightforward way to do this is add an ERR_CHAR rule to the end of your lexer.

ERR_CHAR
  : .
  ;

This allows the syntax highlighter to work, and defers reporting of the error to the parsing phase.

来源：https://stackoverflow.com/questions/26100737/ilegalstateexception-when-i-type-unfinished-code-into-a-jeditorpane-with-syntax

标签

syntax-highlighting

antlr4

netbeans-platform