Pandas to_sql returning 'relation already exists' error when using if_exists='append'

烈酒焚心 提交于 2019-12-25 02:54:02

问题


I am trying to insert a data frame daily into a table in Redshift. The to_sql command works to create the table, but returns an error when I try to append to the existing table even when using if_exists = 'append' argument.

Versions: pandas: 0.23.4 sqlalchemy: 1.2.15 psycopg2: 2.7.6.1 Python: 3.6.7

I am also using the monkey patch to speed up inserts outlined here: https://github.com/pandas-dev/pandas/issues/8953 but without this patch the insert takes prohibitively long (several hours).

#monkey patch to_sql for redshift
from pandas.io.sql import SQLTable

def _execute_insert(self, conn, keys, data_iter):
    print ("Using monkey-patched _execute_insert")
    data = [dict((k, v) for k, v in zip(keys, row)) for row in data_iter]
    conn.execute(self.insert_statement().values(data))

SQLTable._execute_insert = _execute_insert
import pandas as pd

myDF.to_sql('my_table', engine, if_exists='append', schema = 'my_schema', index=False, chunksize = 10000)

This returns the following error:

"ProgrammingError: (psycopg2.ProgrammingError) Relation "my_table" already exists [SQL: '\nCREATE TABLE my_schema."my_table" (\n\tactual_weight FLOAT(53),..."

来源:https://stackoverflow.com/questions/54240688/pandas-to-sql-returning-relation-already-exists-error-when-using-if-exists-ap

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!