Remove Duplicate Line in Vim?

Question

I\'m trying to use VIM to remove a duplicate line in an XML file I created. (I can\'t recreate the file because the ID numbers will change.)

The file looks something

Answer 1

First of all, you can use awk to remove all duplicate lines, keeping their order.

:%!awk '\!_[$0]++'

If you not sure if there are some other duplicate lines you don't want remove, then just add conditions.

:%!awk '\!(_[$0]++ && /tag/ && /natural/ && /water/)'

But, parsing a nested structure like xml with regex is a bad idea, IMHO. You are going to care them not to be screwed up all the time. xmllint gives you a list of specific elements in the file:

:!echo "cat //tag[@k='natural' and @v='water']" | xmllint --shell %

You can slash duplicate lines step by step.