Access database UPSERT with pyodbc

Question

I am using pyodbc to update an Access database.

I need the functionality of an UPSERT.

ON DUPLICATE KEY UPDATE doesn\'t exist in Access SQL, and

Answer 1

As noted in this question and others, Access SQL does not have an "upsert" statement, so you will need to use a combination of UPDATE and INSERT. However, you can improve your current implementation by

• using proper parameters for your query, and
• using Python string manipulation to build the SQL command text.

For example, to upsert into a table named [Donor]

Donor ID  Last Name  First Name
--------  ---------  ----------
       1  Thompson   Gord

You can start with a list of the field names. The trick here is to put the key field(s) at the end, so the INSERT and UPDATE statements will refer to the fields in the same order (i.e., the UPDATE will refer to the ID field last because it will be in the WHERE clause).

data_fields = ['Last Name', 'First Name']
key_fields = ['Donor ID']

The parameter values will be the same for both the UPDATE and INSERT cases, e.g.

params = ('Elk', 'Anne', 2)

The UPDATE statement can be constructed like this

update_set = ','.join(['[' + x + ']=?' for x in data_fields])
update_where = ' AND '.join(['[' + x + ']=?' for x in key_fields])
sql_update = "UPDATE [Donor] SET " + update_set + " WHERE " + update_where
print(sql_update)

which shows us

UPDATE [Donor] SET [Last Name]=?,[First Name]=? WHERE [Donor ID]=?

Similarly, the INSERT statement can be constructed like this

insert_fields = ','.join(['[' + x + ']' for x in (data_fields + key_fields)])
insert_placeholders = ','.join(['?' for x in (data_fields + key_fields)])
sql_insert = "INSERT INTO [Donor] (" + insert_fields + ") VALUES (" + insert_placeholders + ")"
print(sql_insert)

which prints

INSERT INTO [Donor] ([Last Name],[First Name],[Donor ID]) VALUES (?,?,?)

So, to perform our upsert, all we need to do is

crsr.execute(sql_update, params)
if crsr.rowcount > 0:
    print('Existing row updated.')
else:
    crsr.execute(sql_insert, params)
    print('New row inserted.')
crsr.commit()

Answer 2

Consider using parameterized queries from prepared statements that uses ? placeholders. The str.format is still needed for identifiers such as table and field names. Then unpack dictionary items with zip(*dict.items()) to pass as parameters in the cursor's execute call: cursor.execute(query, params).

for table_name in data_source:
    table = data_source[table_name]

    for y in table:
        keys, values = zip(*y.items())    # UNPACK DICTIONARY INTO TWO TUPLES

        if table_name == "whatever":
            SQL_UPDATE = "UPDATE {} set [Part Name] = ?, [value] = ?, [code] = ?," + \
                       " [tolerance] = ? WHERE [Unique Part Number]= ?".format(table_name)

            SQL_INSERT = "INSERT INTO {} ([Part Name], [Unique Part Number], [value]," + \
                     " [code], [tolerance]) VALUES (?, ?, ?, ?, ?);".format(table_name)

            res = cursor.execute(SQL_UPDATE, values)
            if res.rowcount == 0:
                cursor.execute(SQL_INSERT, values)
       ...