Psycopg2 - How to include 'Null' values in inserting dictionary of list to table in postgres?

问题

I'm parsing my xml files and store them to a dictionary of list, where I will insert them into table at posgres using psycopg2. However, not all rows got inserted in the tables (it only inserted to the lowest number of values in the list). Here is the snippet of the dictionary of list:

dict_songs = {'title' : ['Need You Now', 'GTFO'...], 'format': ['MP4', 'MP3'...], 'type' : ['Country Pop', 'R&B Pop'..], 'year': [2010,2018..]}

dict_movie = {'title' : ['Searching', 'Sidewalk of New York'...], 'format': ['DVD', 'Blue Ray'...], 'type' : ['Thriller', 'Romcom'..], 'year': [2018..]

When I counted the length of each list in the dictionary it come up that not all the list has the same length, for example:

for key, value in dict_songs.items():
    #print value
    print(key, len([item for item in value if item]))

# The result is:
title 300000
format 189700
type 227294
year 227094

Title would be the primary key in the song table. When I inserted this dictionary to postgres it only shows 189700 records and not 300000. I want it to be 300000 and put Null for the null (none) values. The same goes to dict_movie

This is the code I use to insert the dict list into table:

keys = ['title', 'format', 'type','year']
insert_statement = 'insert into song_table (%s) values %s'
for t in zip(*(dict_songs[key] for key in keys)):
   cur.execute(insert_statement3, (AsIs(','.join(keys)),t))
myConnection.commit()

Any ideas why or how to go about this? Thank you!

回答1:

I think the problem here is that you don't know where the None/NULL values are. Imagine these to lists:

dict_songs = {
  'title' : ['Need You Now', 'GTFO', 'Titletest']
  'type' : ['Country Pop', 'R&B Pop']
}

Your table could have NULL values in three positions and there is no data in the lists that could hint to the correct one:

+ -------------+-------------+-------------+-------------+
| title        | type        | type        | type        |
+--------------+-------------+-------------+-------------+
| Need You Now | Country Pop | Country Pop | NULL        |
| GTFO         | R&B Pop     | NULL        | Country Pop |
| Jinglebells  | NULL        | R&B Pop     | R&B Pop     |
+--------------+-------------+-------------+-------------+

You lists need to have None values, so you know where to put NULL into the database table. Like this:

dict_songs = {
  'title' : ['Need You Now', 'GTFO', 'Titletest']
  'type' : ['Country Pop', None, 'R&B Pop']
}

来源：https://stackoverflow.com/questions/52436197/psycopg2-how-to-include-null-values-in-inserting-dictionary-of-list-to-table

标签

python

postgresql

null

psycopg2

insert-into