How to remove duplicate items in MySQL with a dataset of 20 million rows?

Question

I\'ve got big MySQL database. I need to delete the duplicate item quickly. Here\'s how it looks:

id | text1 | text2|    
1  | 23    |  43  |   
2  | 23    |          


        

Answer 1

Run this:

SELECT COUNT(*), text1, text2
GROUP BY text1, text2
HAVING COUNT(*) > 1;

When you find rows here, delete one row for each match, and then run it again.

I'm not sure what it will be like in terms of performance - perhaps it doesn't matter, if you do this offline?