How __future__ imports work under the hood

Question

I\'ve been fascinated by the __future__ module - in particular, its ability to change the way statements are parsed in python.

What\'s most interesting

Answer 1

from __future__ import print_function tells the parser to not treat print as a keyword (leaving it as a name instead). That way the compiler treats it as the function and not a statement.

To track this, the compiler struct has a c_future field that holds a PyFutureFeatures object that tracks which future directives have been enabled. Various parts of the parser and compiler check the flags and alter behaviour.

This is mostly handled in the future.c source file, which has a future_parse() function that checks for import from AST objects with the module parameter set to __future__, and sets flags based on what is found.

For example, for the barry_as_FLUFL 'feature', the parser refuses != as syntax but accepts <> instead:

if (type == NOTEQUAL) {
    if (!(ps->p_flags & CO_FUTURE_BARRY_AS_BDFL) &&
                    strcmp(str, "!=")) {
        PyObject_FREE(str);
        err_ret->error = E_SYNTAX;
        break;
    }
    else if ((ps->p_flags & CO_FUTURE_BARRY_AS_BDFL) &&
                    strcmp(str, "<>")) {
        PyObject_FREE(str);
        err_ret->text = "with Barry as BDFL, use '<>' "
                        "instead of '!='";
        err_ret->error = E_SYNTAX;
        break;
    }
}

You can find the other examples by grepping for the FUTURE_* flags listed in compile.h.

Note that there is a __future__ Python module, but it is not directly involved in the parsing and compilation of code; it is merely there to give Python code easy access to metadata about directives (including bitfield values to pass to the flags argument of the compile() function), nothing more.