How can I modify the text of tokens in a CommonTokenStream with ANTLR?

Question

I\'m trying to learn ANTLR and at the same time use it for a current project.

I\'ve gotten to the point where I can run the lexer on a chunk of code and output it to a C

Answer 1

I've used the sample Java grammar to create an ANTLR script to process an R.java file and rewrite all the hex values in a decompiled Android app with values of the form R.string.*, R.id.*, R.layout.* and so forth.

The key is using TokenStreamRewriter to process the tokens and then output the result.

The project (Python) is called RestoreR

The modified ANTLR listener for rewriting

I parse with a listener to read in the R.java file and create a mapping from integer to string and then replace the hex values as a I parse the programs java files with a different listener containing a rewriter instance.

class RValueReplacementListener(ParseTreeListener):
    replacements = 0
    r_mapping = {}
    rewriter = None

    def __init__(self, tokens):
        self.rewriter = TokenStreamRewriter(tokens)

    // Code removed for the sake of brevity

    # Enter a parse tree produced by JavaParser#integerLiteral.
    def enterIntegerLiteral(self, ctx:JavaParser.IntegerLiteralContext):
        hex_literal = ctx.HEX_LITERAL()
        if hex_literal is not None:
            int_literal = int(hex_literal.getText(), 16)
            if int_literal in self.r_mapping:
                # print('Replace: ' + ctx.getText() + ' with ' + self.r_mapping[int_literal])
                self.rewriter.replaceSingleToken(ctx.start, self.r_mapping[int_literal])
                self.replacements += 1