How can i remove duplicated lines (txt file)?

问题

today I was trying to remove duplicate lines on a simple text file, something like:

input (list.txt):

hello
hello
try

output (list.txt):
try

i was trying with notepad++ to remove duplicate rows and remove the remaining one but nothing. is there a software o some function for do this with notepad++?

thanks.

回答1:

Assuming the file is sorted, to have all duplicate lines together.

• Ctrl+H
• Find what: ^(.+(?:\R|$))\1+
• Replace with: LEAVE EMPTY
• check Wrap around
• check Regular expression
• DO NOT CHECK . matches newline
• Replace all

Explanation:

^           : beginning of line
  (         : start group 1
    .+      : 1 or more any character but newline
    (?:     : start non capture group
      \R    : any kind of linebreak
     |      : OR
      $     : end of line
    )       : end group
  )         : end group 1
  \1+       : back-reference to group 1, may appear 1 or more times

Result for given example:

try

回答2:

you can do it with php by exploding each line to an array then using the array_unique to get rid of duplicate values then implode the array using \n as a seperator. It can be done in php with 6 lines of code or less readfile explode file unique_array file implode file write file close file return file

来源：https://stackoverflow.com/questions/48039999/how-can-i-remove-duplicated-lines-txt-file