Remove duplicates in large MySql table

Question

I have a question about MySql. I have a table with 7.479.194 records. Some records are duplicated. I would like to do this:

insert into new_table 
  select *         


        

Answer 1

You don't need to group data. Try this:

 delete from old_table
    USING old_table, old_table as vtable  
    WHERE (old_table.id > vtable.id)  
    AND (old_table.city=vtable.city AND 
old_table.post_code=vtable.post_code 
AND old_table.short_code=vtable.short_code) 

I can't comment posts becouse of my points ... repair table old_table; next: show:

EXPLAIN SELECT old_table.id FROM   old_table, old_table as vtable  
        WHERE (old_table.id > vtable.id)  
        AND (old_table.city=vtable.city AND 
    old_table.post_code=vtable.post_code 
    AND old_table.short_code=vtable.short_code

Show: os~> ulimit -a; mysql>SHOW VARIABLES LIKE 'open_files_limit';

next: Remove all os restrictions form the mysql process.

ulimit -n 1024 etc.