Delete lines in a text file that contain a specific string

橙三吉。 提交于 2019-11-25 22:06:41

问题


How would I use sed to delete all lines in a text file that contain a specific string?


回答1:


To remove the line and print the output to standard out:

sed '/pattern to match/d' ./infile

To directly modify the file – does not work with BSD sed:

sed -i '/pattern to match/d' ./infile

Same, but for BSD sed (Mac OS X and FreeBSD) – does not work with GNU sed:

sed -i '' '/pattern to match/d' ./infile

To directly modify the file (and create a backup) – works with BSD and GNU sed:

sed -i.bak '/pattern to match/d' ./infile



回答2:


There are many other ways to delete lines with specific string besides sed:

AWK

awk '!/pattern/' file > temp && mv temp file

Ruby (1.9+)

ruby -i.bak -ne 'print if not /test/' file

Perl

perl -ni.bak -e "print unless /pattern/" file

Shell (bash 3.2 and later)

while read -r line
do
  [[ ! $line =~ pattern ]] && echo "$line"
done <file > o
mv o file

GNU grep

grep -v "pattern" file > temp && mv temp file

And of course sed (printing the inverse is faster than actual deletion):

sed -n '/pattern/!p' file



回答3:


You can use sed to replace lines in place in a file. However, it seems to be much slower than using grep for the inverse into a second file and then moving the second file over the original.

e.g.

sed -i '/pattern/d' filename      

or

grep -v "pattern" filename > filename2; mv filename2 filename

The first command takes 3 times longer on my machine anyway.




回答4:


The easy way to do it, with GNU sed:

sed --in-place '/some string here/d' yourfile



回答5:


You may consider using ex (which is a standard Unix command-based editor):

ex +g/match/d -cwq file

where:

  • + executes given Ex command (man ex), same as -c which executes wq (write and quit)
  • g/match/d - Ex command to delete lines with given match, see: Power of g

The above example is a POSIX-compliant method for in-place editing a file as per this post at Unix.SE and POSIX specifications for ex.


The difference with sed is that:

sed is a Stream EDitor, not a file editor.BashFAQ

Unless you enjoy unportable code, I/O overhead and some other bad side effects. So basically some parameters (such as in-place/-i) are non-standard FreeBSD extensions and may not be available on other operating systems.




回答6:


I was struggling with this on Mac. Plus, I needed to do it using variable replacement.

So I used:

sed -i '' "/$pattern/d" $file

where $file is the file where deletion is needed and $pattern is the pattern to be matched for deletion.

I picked the '' from this comment.

The thing to note here is use of double quotes in "/$pattern/d". Variable won't work when we use single quotes.




回答7:


I have made a small benchmark with a file which contains approximately 345 000 lines. The way with grep seems to be around 15 times faster than the sed method in this case.

I have tried both with and without the setting LC_ALL=C, it does not seem change the timings significantly. The search string (CDGA_00004.pdbqt.gz.tar) is somewhere in the middle of the file.

Here are the commands and the timings:

time sed -i "/CDGA_00004.pdbqt.gz.tar/d" /tmp/input.txt

real    0m0.711s
user    0m0.179s
sys     0m0.530s

time perl -ni -e 'print unless /CDGA_00004.pdbqt.gz.tar/' /tmp/input.txt

real    0m0.105s
user    0m0.088s
sys     0m0.016s

time (grep -v CDGA_00004.pdbqt.gz.tar /tmp/input.txt > /tmp/input.tmp; mv /tmp/input.tmp /tmp/input.txt )

real    0m0.046s
user    0m0.014s
sys     0m0.019s



回答8:


To get a inplace like result with grep you can do this:

echo "$(grep -v "pattern" filename)" >filename



回答9:


You can also use this:

 grep -v 'pattern' filename

Here -v will print only other than your pattern (that means invert match).




回答10:


SED:

  • '/James\|John/d'
  • -n '/James\|John/!p'

AWK:

  • '!/James|John/'
  • /James|John/ {next;} {print}

GREP:

  • -v 'James\|John'



回答11:


perl -i    -nle'/regexp/||print' file1 file2 file3
perl -i.bk -nle'/regexp/||print' file1 file2 file3

The first command edits the file(s) inplace (-i).

The second command does the same thing but keeps a copy or backup of the original file(s) by adding .bk to the file names (.bk can be changed to anything).




回答12:


echo -e "/thing_to_delete\ndd\033:x\n" | vim file_to_edit.txt




回答13:


cat filename | grep -v "pattern" > filename.1
mv filename.1 filename



回答14:


Just in case someone wants to do it for exact matches of strings, you can use the -w flag in grep - w for whole. That is, for example if you want to delete the lines that have number 11, but keep the lines with number 111:

-bash-4.1$ head file
1
11
111

-bash-4.1$ grep -v "11" file
1

-bash-4.1$ grep -w -v "11" file
1
111

It also works with the -f flag if you want to exclude several exact patterns at once. If "blacklist" is a file with several patterns on each line that you want to delete from "file":

grep -w -v -f blacklist file


来源:https://stackoverflow.com/questions/5410757/delete-lines-in-a-text-file-that-contain-a-specific-string

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!