Backlash error. Pandas filter dataframe using dynamic query string.

Question

Hi all, The problem is related to the Python backlash error. I am creating a dynamic query string for filtering in pandas. The code is:

Answer 1

The reason is that you are wrapping your key in python format string ( "'{}'".format ) as well. Try this solution:

query_string = ""
index = 0
for (k,v) in filters.iteritems():
  for i in v:
    if (index == 0):
      query_string += str(k) + " == " + "'{}'".format(i)
    else:
      query_string += " & " + str(k) + " == " + "'{}'".format(i)
  index += 1

Answer 2

You can write your own helper function (similar to what you're trying now but making use of **kwargs) and use the @varname syntax as the value placeholder.

def my_filter(df, **kwargs):
    qs = ' & '.join('{0} == @{0}'.format(k) for k in kwargs)
    return df.query(qs, local_dict=kwargs)

Then use as follows:

new_df = my_filter(df, wlbWellType='EXPLORATION', otherColumn='SOMETHING')

This method is safer than manually escaping values as the @varname syntax will do that appropriately for you depending on the value's type.

Answer 3

Consider the following approach:

In [44]: filters
Out[44]:
           col          val
0  wlbWellType  EXPLORATION
1          bbb          BBB

In [45]: qry = filters['col'].add(' == "').add(filters['val']).add('"').str.cat(sep=' & ')

In [46]: print(qry)
wlbWellType == "EXPLORATION" & bbb == "BBB"

slightly different syntax:

In [50]: qry = (filters['col'] + ' == "' + filters['val'] + '"').str.cat(sep=' & ')

In [51]: qry
Out[51]: 'wlbWellType == "EXPLORATION" & bbb == "BBB"'