Without using CTE
and ROW_NUMBER()
you can just delete the records just by using group by with MAX
function here is and example
DELETE
FROM MyDuplicateTable
WHERE ID NOT IN
(
SELECT MAX(ID)
FROM MyDuplicateTable
GROUP BY DuplicateColumn1, DuplicateColumn2, DuplicateColumn3)