Remove Duplicates from Text File

Question

I want to remove duplicate word from a text file.

i have some text file which contain such like following:

None_None

ConfigHandler_56663624
ConfigHa         


        

Answer 1

this way get same file out that was put in

import uuid

def _remove_duplicates(filePath):
  f = open(filePath, 'r')
  lines = f.readlines()
  lines_set = set(lines)
  tmp_file=str(uuid.uuid4())
  out=open(tmp_file, 'w')
  for line in lines_set:
    out.write(line)
  f.close()
  os.rename(tmp_file,filePath)