How to delete from a text file, all lines that contain a specific string?

Question

How would I use sed to delete all lines in a text file that contain a specific string?

Answer 1

I have made a small benchmark with a file which contains approximately 345 000 lines. The way with grep seems to be around 15 times faster than the sed method in this case.

I have tried both with and without the setting LC_ALL=C, it does not seem change the timings significantly. The search string (CDGA_00004.pdbqt.gz.tar) is somewhere in the middle of the file.

Here are the commands and the timings:

time sed -i "/CDGA_00004.pdbqt.gz.tar/d" /tmp/input.txt

real    0m0.711s
user    0m0.179s
sys     0m0.530s

time perl -ni -e 'print unless /CDGA_00004.pdbqt.gz.tar/' /tmp/input.txt

real    0m0.105s
user    0m0.088s
sys     0m0.016s

time (grep -v CDGA_00004.pdbqt.gz.tar /tmp/input.txt > /tmp/input.tmp; mv /tmp/input.tmp /tmp/input.txt )

real    0m0.046s
user    0m0.014s
sys     0m0.019s

Answer 2

To get a inplace like result with grep you can do this:

echo "$(grep -v "pattern" filename)" >filename

Answer 3

echo -e "/thing_to_delete\ndd\033:x\n" | vim file_to_edit.txt

Answer 4

There are many other ways to delete lines with specific string besides sed:

AWK

awk '!/pattern/' file > temp && mv temp file

Ruby (1.9+)

ruby -i.bak -ne 'print if not /test/' file

Perl

perl -ni.bak -e "print unless /pattern/" file

Shell (bash 3.2 and later)

while read -r line
do
  [[ ! $line =~ pattern ]] && echo "$line"
done <file > o
mv o file

GNU grep

grep -v "pattern" file > temp && mv temp file

And of course sed (printing the inverse is faster than actual deletion):

sed -n '/pattern/!p' file

Answer 5

I was struggling with this on Mac. Plus, I needed to do it using variable replacement.

So I used:

sed -i '' "/$pattern/d" $file

where $file is the file where deletion is needed and $pattern is the pattern to be matched for deletion.

I picked the '' from this comment.

The thing to note here is use of double quotes in "/$pattern/d". Variable won't work when we use single quotes.

Answer 6

You can also use this:

 grep -v 'pattern' filename

Here -v will print only other than your pattern (that means invert match).