Build SQL query string using IN with a python list

Question

I built a list of values of interest in pandas.

table1 = pd.read_csv(\"logswithIPs.csv\")
cips = data_dash[\'ip\'].unique().tolist()
print(cips[:10])
[\'111.111.         


        

Answer 1

I would export/save data_dash['ip'].unique() as an SQL table, so that it could be efficiently used for subqueries:

pd.DataFrame({'ip':data_dash['ip'].unique()}).to_sql('tmp_ip', conn, if_exists='replace')

now you can use it on the SQL DB side:

qry = """
select count(*) as count, url
from tab_name
where c_ip in (select ip from tmp_ip)
group by url
"""

filterIPs = pd.read_sql(qry, conn)