Deleting duplicate record from table - SQL query

前端 未结 7 1037
有刺的猬
有刺的猬 2021-02-03 15:09

I need to delete duplicate rows only from the table, like I have 3 duplicate rows in the table, my query will delete 2 rows from 3 duplicated rows.

How can I get this? P

相关标签:
7条回答
  • 2021-02-03 15:21

    I think each table has unique identifier. So if it exists then you can write following query: Delete Table1 from Table1 t1 where 2 >= (select count(id) from Table1 where dupColumn = t1.dupColumn) and t1.id not in (select max (id) from Table1 where dupColumn = t1.dupColumn)

    OOps. It seems it is possible to use second filter only Delete Table1 from Table1 t1 where t1.id not in (select max (id) from Table1 where dupColumn = t1.dupColumn)

    0 讨论(0)
  • 2021-02-03 15:22
    DELETE FROM Table t1, Table t2 WHERE t1.colDup = t2.colDup AND t1.date < t2.date
    

    Will delete every duplicate row from Table (on column colDup) except the oldest (i.e. lowset date).

    0 讨论(0)
  • 2021-02-03 15:22
      -- Just to demonstrates Marks example          
        . 
            -- START === 1.0.dbo..DuplicatesTable.TableCreate.sql
        /****** Object:  Table [dbo].[DuplicatesTable] 
            Script Date: 03/29/2010 21:24:02 ******/
          IF EXISTS (SELECT * FROM sys.objects 
         WHERE 
    object_id = OBJECT_ID(N'[dbo].[DuplicatesTable]') 
    AND type in (N'U'))
            DROP TABLE [dbo].[DuplicatesTable]
        GO
    
        /****** Object:  Table [dbo].[DuplicatesTable]    
    Script Date: 03/29/2010 21:24:02 ******/
        SET ANSI_NULLS ON
        GO
    
        SET QUOTED_IDENTIFIER ON
        GO
    
        CREATE TABLE [dbo].[DuplicatesTable](
            [ColA] [varchar](10) NOT NULL, -- the name of the DuplicatesTable
            [ColB] [varchar](10) NULL,  -- the description of the e DuplicatesTable 
         ) 
    
    
        /* 
        <doc> 
        Models a DuplicatesTable for 
        </doc>
    
        */
    
    
        GO
    
    
        --============================================================ DuplicatesTable START
        declare @ScriptFileName varchar(2000)
        SELECT @ScriptFileName = '$(ScriptFileName)'
        SELECT @ScriptFileName + ' --- DuplicatesTable START =========================================' 
        declare @TableName varchar(200)
        select @TableName = 'DuplicatesTable'
    
        SELECT 'SELECT name from sys.tables where name =''' + @TableName + ''''
        SELECT name from sys.tables 
        where name = @TableName
    
        DECLARE @TableCount INT 
        SELECT @TableCount  = COUNT(name ) from sys.tables 
            where name =@TableName
    
        if @TableCount=1
        SELECT ' DuplicatesTable PASSED. The Table ' + @TableName + ' EXISTS ' 
        ELSE 
        SELECT ' DuplicatesTable FAILED. The Table ' + @TableName + ' DOES NOT EXIST ' 
        SELECT @ScriptFileName + ' --- DuplicatesTable END =========================================' 
        --============================================================ DuplicatesTable END
    
        GO
    
    
        -- END ===  1.0.dbo..DuplicatesTable.TableCreate.sql
    
        . 
        -- START === 1.1..dbo..DuplicatesTable.TableInsert.sql
    
        BEGIN TRANSACTION;
        INSERT INTO [dbo].[DuplicatesTable]([ColA], [ColB])
        SELECT   N'ColA', N'ColB' UNION ALL
        SELECT N'ColA', N'ColB' UNION ALL
        SELECT  N'ColA', N'ColB' UNION ALL
        SELECT  N'ColA', N'ColB' UNION ALL
        SELECT  N'ColA', N'ColB' UNION ALL
        SELECT  N'ColA', N'ColB' UNION ALL
        SELECT  N'ColA', N'ColB' UNION ALL
        SELECT  N'ColA1', N'ColB1' UNION ALL
        SELECT  N'ColA1', N'ColB1' UNION ALL
        SELECT  N'ColA1', N'ColB1' UNION ALL
        SELECT  N'ColA1', N'ColB1' UNION ALL
        SELECT  N'ColA1', N'ColB1' UNION ALL
        SELECT  N'ColA1', N'ColB1' UNION ALL
        SELECT  N'ColA1', N'ColB1'
        COMMIT;
        RAISERROR (N'[dbo].[DuplicatesTable]: Insert Batch: 1.....Done!', 10, 1) WITH NOWAIT;
        GO
    
    
        -- END ===  1.1..dbo..DuplicatesTable.TableInsert.sql
    
        . 
        -- START === 2.0.RemoveDuplicates.Script.sql
        ALTER TABLE dbo.DuplicatesTable ADD
                DuplicatesTableId int NOT NULL IDENTITY (1, 1)
        GO
    
        -- Then the delete is trivial:
        DELETE FROM dbo.DuplicatesTable WHERE DuplicatesTableId NOT IN 
             (SELECT MAX(DuplicatesTableId) FROM dbo.DuplicatesTable GROUP BY ColA , ColB)
    
             Select * from DuplicatesTable ;  
        -- END ===  2.0.RemoveDuplicates.Script.sql
    
    0 讨论(0)
  • 2021-02-03 15:23

    If you have the id's of the rows you want to delete then...

    DELETE FROM table WHERE id IN (1, 4, 7, [id numbers to delete...])
    
    0 讨论(0)
  • 2021-02-03 15:29
    DELETE FROM `mytbl`
        INNER JOIN (
            SELECT 1 FROM `mytbl`
            GROUP BY `duplicated_column` HAVING COUNT(*)=2
        ) USING(`id`)
    

    Edit:

    My bad, the above query won't work.

    Assuming table structure:

    id int auto_increment

    num int # <-- this is the column with duplicated values

    The following query would work in MySQL (i checked):

    DELETE `mytbl` FROM `mytbl` 
        INNER JOIN 
        (
            SELECT `num` FROM `mytbl`
            GROUP BY `num` HAVING COUNT(*)=2
        ) AS `tmp` USING (`num`)
    

    The query would delete the rows that have 2 (not more or else) duplicated values in the num column.

    Edit (again):

    I suggest to add a key on the num column.

    Edit(#3):

    In case that the author wanted to delete the duplicated rows, the following should work for MySQL (it worked for me):

    DELETE `delete_duplicated_rows` FROM `delete_duplicated_rows`
        NATURAL JOIN (
            SELECT *
            FROM `delete_duplicated_rows`
            GROUP BY `num1` HAVING COUNT(*)=2
        ) AS `der`
    

    While assuming table structure is:

    CREATE TABLE `delete_duplicated_rows` (
      `num1` tinyint(4) DEFAULT NOT NULL,
      `num2` tinyint(4) DEFAULT NOT NULL
    ) ENGINE=MyISAM;
    
    0 讨论(0)
  • 2021-02-03 15:36

    This works in SQL Server although it isn't a single statement:

    Declare @cnt int; 
    Select @cnt=COUNT(*) From DupTable Where (Col1=1);  -- Assumes you are trying to delete the duplicates where some condition (e.g. Col1=1) is true.
    Delete Top (@cnt-1) From DupTable
    

    It also doesn't require any extra assumptions (like the existance of another column that makes each row unique). After all, Santanu did say that the rows were duplicates and not just the one column.

    However, the right answer, in my view, is to get a real table structure. That is, add an IDENTITY column to this table so that you can use a single SQL command to do your work. Like this:

    ALTER TABLE dbo.DupTable ADD
        IDCol int NOT NULL IDENTITY (1, 1)
    GO
    

    Then the delete is trivial:

    DELETE FROM DupTable WHERE IDCol NOT IN 
       (SELECT MAX(IDCol) FROM DupTable GROUP BY Col1, Col2, Col3)
    
    0 讨论(0)
提交回复
热议问题