Why does ast.literal_eval('5 * 7') fail?

Question

Why does the literal evaluation of 5 * 7 fail, while 5 + 7 doesn\'t?

import ast

print(ast.literal_eval(\'5 + 7\'))
# -> 12

pri         


        

Answer 1

ast.literal_eval() accepts + in the evaluated data because 5+2j (complex number^*) are valid literals. The same applies to -. To keep the code simple, no attempt is made to exclude + or - as a binary operators.

No other operators are allowed; the function is supposed to only accept literals, not expressions.

In other words, that 5 + 7 works is a bug, but one that is hard to fix without breaking support for constructing complex numbers. The implementation limits the use to operands that are numbers, unary + and -, or other binary operators (so you can't use these to concatenate lists or produce a set difference).

Also see several related Python bugtracker entries: #25335 ast.literal_eval fails to parse numbers with leading "+", #22525 ast.literal_eval() doesn't do what the documentation says and #4907 ast.literal_eval does not properly handled complex numbers

---

^* Technically speaking, 2j is a valid literal; Python parses 5+2j as int(5) binop(+) complex(0, 2), and only later produces a complex(5, 2) object from the result, when actually executing the addition.

Answer 2

The question is not "why is * not accepted" but rather "why is + accepted at all".

ast.literal_eval can parse literals, but not expressions. However, in Python, complex numbers are not expressed as a single literal value; instead they consist of the real part and imaginary part added together; the imaginary part is signalled with j. literal_eval thus needs to support binary + and - to support complex number constants such as 1 + 2j or -3.4e-5 - 1.72e9j.

In many versions, including Python 3.5, literal_eval is much more lax than it needs to be - it accepts any chain of additions and subtractions for as long as both the left and right-hand sides evaluate to any number, thus (1 + 3) + 2 + (4 - 5) is still parsed, even if it is not complex constant consisting of real + imaginary part.

---

+ and - are not accepted unconditionally: if you try to add 2 lists together, it will fail, even though it can parse list literals, and addition is defined for lists:

>>> ast.literal_eval('[1] + [2]')
Traceback (most recent call last):
...
ValueError: malformed node or string: <_ast.BinOp object at 0x7fffffdbe785f8>
>>> ast.literal_eval('[1, 2]')
[1, 2]
>>> [1] + [2]
[1, 2]