ANTLR4: TokenStreamRewriter output doesn't have proper format (removes whitespaces)

Question

I am using Antlr4 and java7 grammar (source) for modifying an input Java Source file. More specifically, I am using the TokenStreamRewriter class to modify some tokens. The

Answer 1

The issue is that the lexer is not sending white space to the parser, which means that the rewrite stream doesn't have access to the tokens either. It is because of the skip lexer command:

WS : [ \t\r\n\u000C]+ -> skip ;

You have to change all those to -> channel(HIDDEN) which will send them to the parser on a different channel, making them available in the token stream, but invisible to the parser.