How to (generically) count duplicate rows (and remove duplicates)?

问题

Is there a generic SELECT statement to detect duplicate rows ("identical", where all columns are equal)? E.G, columns 2 & 4 in the following table

 titel                             | interpret        | jahr
-----------------------------------+------------------+-----
 Beauty                            | Ryuichi Sakamoto | 1990
 Goodbye Country (Hello Nightclub) | Groove Armada    | 2001
 Glee                              | Bran Van 3000    | 1997
 Goodbye Country (Hello  Nightclub)| Groove Armada    | 2001

Or do I need a SELECT which is specific to the table?

Someone has given me an Sqlite d/b with multiple tables each of which look like like they have multiple identical rows (with different columns in each table), so I would prefer a generic solution.

After that, I have to figure out how to delete the duplicates. Maybe I can use DISTINCT on the SELECT, store in a temp table, delete the original and rename the temp table?

回答1:

You had the right idea all along. You will have to run the following on each table:

select distinct titel, interpret, jahr from table1

You could dump the distinct rows into another table like so:

create table table2 as
select distinct titel, interpret, jahr from table1

You can then drop the intial table like so:

drop table table1

Rename the newly created table to table1 like so:

alter table table2 rename to table1

To find row number of each record in the table:

select a.rowid, a.* from table1 a

To find row number of only the records that are duplicate:

select a.rowid, a.* 
from table1 a 
inner join 
(
      select titel, interpret, jahr 
      from table1 
      group by titel, interpret, jahr 
      having count(*) > 1
) b on 
      a.titel = b.titel 
      and a.interpret = b.interpret 
      and a.jahr = b.jahr

来源：https://stackoverflow.com/questions/18680531/how-to-generically-count-duplicate-rows-and-remove-duplicates