ValueError: malformed string using ast.literal_eval

Question

I\'m doing a loop to get json api, here is what I have in my loop:

response_item = requests.request(\'GET\',url_item,params=None,verify=False)
response_item          


        

Answer 1

You should not use ast.literal_eval() on JSON data. JSON and Python literals may look like the same thing, but they are very much not.

In this case, your data contains a boolean flag, set to false in JSON. A proper Python boolean uses title-case, so False:

>>> import json, ast
>>> s = '{"no_sell_or_sort": false, "size": 20}'
>>> json.loads(s)
{u'no_sell_or_sort': False, u'size': 20}
>>> ast.literal_eval(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py", line 80, in literal_eval
    return _convert(node_or_string)
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py", line 63, in _convert
    in zip(node.keys, node.values))
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py", line 62, in <genexpr>
    return dict((_convert(k), _convert(v)) for k, v
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py", line 79, in _convert
    raise ValueError('malformed string')
ValueError: malformed string

Other differences include using null instead of None, and Unicode escape sequences in what to Python 2 looks like a plain (bytes) string, using UTF-16 surrogates when escaping non-BMP codepoints.

Load your data with json.loads(), not ast.literal_eval(). Not only will it handle proper JSON just fine, it is also faster.

In your case, it appears you are using json.dumps() then try to load the data again with ast.literal_eval(). There is no need for that step, you already had a Python object.

In other words, the line:

response_item = ast.literal_eval(json.dumps(response_item, ensure_ascii=False).encode('utf8'))

is redundant at best, and very, very wrong, at worst. Re-encoding response_item to a JSON string does not produce something that can be interpreted as a Python literal.

Answer 2

ast.literal_eval is safe with SQL injection if you are using this. because when an unwanted charter is inserted it will show Syntex error which prevents from an injection.