In MySQL I\'ve used LOAD DATA LOCAL INFILE which works fine. At the end I get a message like:
LOAD DATA LOCAL INFILE
Records: 460377 Deleted: 0 Skipped: 145280 Warnings
I use bash command-line to find the duplicate row in the csv file:
awk -F\, '{print $1$2}' /my/source/file.csv| sort -n| uniq -c| grep -v "^\ *1"
when the two first columns are the primary key.