How can I introspect a freemarker template to find out what variables it uses?

Question

I\'m not at all sure that this is even a solvable problem, but supposing that I have a freemarker template, I\'d like to be able to ask the template what variables it uses.<

Answer 1

I had the same task to get the list of variables from template on java side and don't found any good approaches to that except using reflection. I'm not sure whether there is a better way to get this data or not but here's my approach:

public Set<String> referenceSet(Template template) throws TemplateModelException {
    Set<String> result = new HashSet<>();
    TemplateElement rootTreeNode = template.getRootTreeNode();
    for (int i = 0; i < rootTreeNode.getChildCount(); i++) {
        TemplateModel templateModel = rootTreeNode.getChildNodes().get(i);
        if (!(templateModel instanceof StringModel)) {
            continue;
        }
        Object wrappedObject = ((StringModel) templateModel).getWrappedObject();
        if (!"DollarVariable".equals(wrappedObject.getClass().getSimpleName())) {
            continue;
        }

        try {
            Object expression = getInternalState(wrappedObject, "expression");
            switch (expression.getClass().getSimpleName()) {
                case "Identifier":
                    result.add(getInternalState(expression, "name").toString());
                    break;
                case "DefaultToExpression":
                    result.add(getInternalState(expression, "lho").toString());
                    break;
                case "BuiltinVariable":
                    break;
                default:
                    throw new IllegalStateException("Unable to introspect variable");
            }
        } catch (NoSuchFieldException | IllegalAccessException e) {
            throw new TemplateModelException("Unable to reflect template model");
        }
    }
    return result;
}

private Object getInternalState(Object o, String fieldName) throws NoSuchFieldException, IllegalAccessException {
    Field field = o.getClass().getDeclaredField(fieldName);
    boolean wasAccessible = field.isAccessible();
    try {
        field.setAccessible(true);
        return field.get(o);
    } finally {
        field.setAccessible(wasAccessible);
    }
}

Sample project that I made for demonstrating template introspection can be found on github: https://github.com/SimY4/TemplatesPOC.git

Answer 2

public static Set<String> getNecessaryTemplateVariables(String templateName) throws TemplateModelException {
        Set<String> result = new HashSet<>();
        TemplateElement rootTreeNode = getTemplate(templateName).getRootTreeNode();
        if ("IteratorBlock".equals(rootTreeNode.getClass().getSimpleName())) {
            introspectFromIteratorBlock(result, rootTreeNode);
            return result;
        }
        for (int i = 0; i < rootTreeNode.getChildCount(); i++) {
            TemplateModel templateModel = rootTreeNode.getChildNodes().get(i);
            if (!(templateModel instanceof StringModel)) {
                continue;
            }
            Object wrappedObject = ((StringModel) templateModel).getWrappedObject();
            if ("DollarVariable".equals(wrappedObject.getClass().getSimpleName())) {
                introspectFromDollarVariable(result, wrappedObject);
            } else if ("ConditionalBlock".equals(wrappedObject.getClass().getSimpleName())) {
                introspectFromConditionalBlock(result, wrappedObject);
            } else if ("IfBlock".equals(wrappedObject.getClass().getSimpleName())) {
                introspectFromIfBlock(result, wrappedObject);
            } else if ("IteratorBlock".equals(wrappedObject.getClass().getSimpleName())) {
                introspectFromIteratorBlock(result, wrappedObject);
            }
        }
        return result;
    }

    private static void introspectFromIteratorBlock(Set<String> result, Object wrappedObject) {
        try {
            Object expression = getInternalState(wrappedObject, "listExp");
            result.add(getInternalState(expression, "name").toString());
        } catch (NoSuchFieldException | IllegalAccessException ignored) {
        }
    }

    private static void introspectFromConditionalBlock(Set<String> result, Object wrappedObject)
        throws TemplateModelException {
        try {
            Object expression = getInternalState(wrappedObject, "condition");
            if (expression == null) {
                return;
            }
            result.addAll(dealCommonExpression(expression));
            String nested = getInternalState(wrappedObject, "nestedBlock").toString();
            result.addAll(getNecessaryTemplateVariables(nested));
        } catch (NoSuchFieldException | IllegalAccessException ignored) {
        }
    }

    private static Set<String> dealCommonExpression(Object expression)
        throws NoSuchFieldException, IllegalAccessException {
        Set<String> ret = Sets.newHashSet();
        switch (expression.getClass().getSimpleName()) {
            case "ComparisonExpression":
                String reference = dealComparisonExpression(expression);
                ret.add(reference);
                break;
            case "ExistsExpression":
                reference = dealExistsExpression(expression);
                ret.add(reference);
                break;
            case "AndExpression":
                ret.addAll(dealAndExpression(expression));
            default:
                break;
        }
        return ret;
    }

    private static String dealComparisonExpression(Object expression)
        throws NoSuchFieldException, IllegalAccessException {
        Object left = getInternalState(expression, "left");
        Object right = getInternalState(expression, "right");
        String reference;
        if ("Identifier".equals(left.getClass().getSimpleName())) {
            reference = getInternalState(left, "name").toString();
        } else {
            reference = getInternalState(right, "name").toString();
        }
        return reference;
    }

    private static String dealExistsExpression(Object expression) throws NoSuchFieldException, IllegalAccessException {
        Object exp = getInternalState(expression, "exp");
        return getInternalState(exp, "name").toString();
    }

    private static Set<String> dealAndExpression(Object expression)  throws NoSuchFieldException,
        IllegalAccessException{
        Set<String> ret = Sets.newHashSet();
        Object lho = getInternalState(expression, "lho");
        ret.addAll(dealCommonExpression(lho));
        Object rho = getInternalState(expression, "rho");
        ret.addAll(dealCommonExpression(rho));
        return ret;
    }

    private static void introspectFromIfBlock(Set<String> result, Object wrappedObject)
        throws TemplateModelException {
        result.addAll(getNecessaryTemplateVariables(wrappedObject.toString()));
    }

    private static void introspectFromDollarVariable(Set<String> result, Object wrappedObject)
        throws TemplateModelException {
        try {
            Object expression = getInternalState(wrappedObject, "expression");
            switch (expression.getClass().getSimpleName()) {
                case "Identifier":
                    result.add(getInternalState(expression, "name").toString());
                    break;
                case "DefaultToExpression":
                    result.add(getInternalState(expression, "lho").toString());
                    break;
                case "BuiltinVariable":
                    break;
                default:
                    break;
            }
        } catch (NoSuchFieldException | IllegalAccessException e) {
            throw new TemplateModelException("Unable to reflect template model");
        }
    }

    private static Object getInternalState(Object o, String fieldName) throws NoSuchFieldException, IllegalAccessException {
        Field [] fieldArray = o.getClass().getDeclaredFields();
        for (Field field : fieldArray) {
            if (!field.getName().equals(fieldName)) {
                continue;
            }
            boolean wasAccessible = field.isAccessible();
            try {
                field.setAccessible(true);
                return field.get(o);
            } finally {
                field.setAccessible(wasAccessible);
            }
        }
        throw new NoSuchFieldException();
    }

    private static Template getTemplate(String templateName) {
        try {
            StringReader stringReader = new StringReader(templateName);
            return new Template(null, stringReader, null);
        } catch (IOException e) {
            throw new IllegalStateException("Failed to Load Template: " + templateName, e);
        }
    }

I optimized SimY4's answer, and supported <#list> and <#if> block. Code is not fully tested

Answer 3

This is probably late, but in case anyone else encountered this problem : you can use 'data_model' and 'globals' to inspect the model - data_model will only contain values provided by the model while globals will also contain any variables defined in the template. You need to prepend the special variables with a dot - so to access globals, use ${.globals}

For other special variables see http://freemarker.sourceforge.net/docs/ref_specvar.html

Answer 4

one other way to get the variables from java. This just tries to process the template and catch the InvalidReferenceException to find all the variables in a freemarker-template

 /**
 * Find all the variables used in the Freemarker Template
 * @param templateName
 * @return
 */
public Set<String> getTemplateVariables(String templateName) {
    Template template = getTemplate(templateName);
    StringWriter stringWriter = new StringWriter();
    Map<String, Object> dataModel = new HashMap<>();
    boolean exceptionCaught;

    do {
        exceptionCaught = false;
        try {
            template.process(dataModel, stringWriter);
        } catch (InvalidReferenceException e) {
            exceptionCaught = true;
            dataModel.put(e.getBlamedExpressionString(), "");
        } catch (IOException | TemplateException e) {
            throw new IllegalStateException("Failed to Load Template: " + templateName, e);
        }
    } while (exceptionCaught);

    return dataModel.keySet();
}

private Template getTemplate(String templateName) {
    try {
        return configuration.getTemplate(templateName);
    } catch (IOException e) {
        throw new IllegalStateException("Failed to Load Template: " + templateName, e);
    }
}

Answer 5

I solved this for my very simple usecase (only using flat datastructure, no nesting (for example: ${parent.child}), lists or more specific) with dummy data provider:

public class DummyDataProvider<K, V> extends HashMap<K, V> {
    private static final long serialVersionUID = 1;

    public final Set<String> variables = new HashSet<>();

    @SuppressWarnings("unchecked")
    @Override
    public V get(Object key) {
        variables.add(key.toString());
        return (V) key;
    }
}

You can give this for processing to a template and when it finishes Set variables contains your variables.

This is very simplistic approach, which certainly needs improvement, but you get the idea.

Answer 6

Execute following regex on the template:

(?<=\$\{)([^\}]+)(?=\})

• (?<=\$\{) Matches everything followed by ${
• ([^\}]+) Matches any string not containing }
• (?=\}) Matches everything before }