问题
How would I use sed to delete all lines in a text file that contain a specific string?
回答1:
To remove the line and print the output to standard out:
sed '/pattern to match/d' ./infile
To directly modify the file – does not work with BSD sed:
sed -i '/pattern to match/d' ./infile
Same, but for BSD sed (Mac OS X and FreeBSD) – does not work with GNU sed:
sed -i '' '/pattern to match/d' ./infile
To directly modify the file (and create a backup) – works with BSD and GNU sed:
sed -i.bak '/pattern to match/d' ./infile
回答2:
There are many other ways to delete lines with specific string besides sed
:
AWK
awk '!/pattern/' file > temp && mv temp file
Ruby (1.9+)
ruby -i.bak -ne 'print if not /test/' file
Perl
perl -ni.bak -e "print unless /pattern/" file
Shell (bash 3.2 and later)
while read -r line
do
[[ ! $line =~ pattern ]] && echo "$line"
done <file > o
mv o file
GNU grep
grep -v "pattern" file > temp && mv temp file
And of course sed
(printing the inverse is faster than actual deletion):
sed -n '/pattern/!p' file
回答3:
You can use sed to replace lines in place in a file. However, it seems to be much slower than using grep for the inverse into a second file and then moving the second file over the original.
e.g.
sed -i '/pattern/d' filename
or
grep -v "pattern" filename > filename2; mv filename2 filename
The first command takes 3 times longer on my machine anyway.
回答4:
The easy way to do it, with GNU sed
:
sed --in-place '/some string here/d' yourfile
回答5:
You may consider using ex (which is a standard Unix command-based editor):
ex +g/match/d -cwq file
where:
+
executes given Ex command (man ex
), same as-c
which executeswq
(write and quit)g/match/d
- Ex command to delete lines with givenmatch
, see: Power of g
The above example is a POSIX-compliant method for in-place editing a file as per this post at Unix.SE and POSIX specifications for ex.
The difference with sed
is that:
sed
is a Stream EDitor, not a file editor.BashFAQ
Unless you enjoy unportable code, I/O overhead and some other bad side effects. So basically some parameters (such as in-place/-i
) are non-standard FreeBSD extensions and may not be available on other operating systems.
回答6:
I was struggling with this on Mac. Plus, I needed to do it using variable replacement.
So I used:
sed -i '' "/$pattern/d" $file
where $file
is the file where deletion is needed and $pattern
is the pattern to be matched for deletion.
I picked the ''
from this comment.
The thing to note here is use of double quotes in "/$pattern/d"
. Variable won't work when we use single quotes.
回答7:
I have made a small benchmark with a file which contains approximately 345 000 lines. The way with grep
seems to be around 15 times faster than the sed
method in this case.
I have tried both with and without the setting LC_ALL=C, it does not seem change the timings significantly. The search string (CDGA_00004.pdbqt.gz.tar) is somewhere in the middle of the file.
Here are the commands and the timings:
time sed -i "/CDGA_00004.pdbqt.gz.tar/d" /tmp/input.txt
real 0m0.711s
user 0m0.179s
sys 0m0.530s
time perl -ni -e 'print unless /CDGA_00004.pdbqt.gz.tar/' /tmp/input.txt
real 0m0.105s
user 0m0.088s
sys 0m0.016s
time (grep -v CDGA_00004.pdbqt.gz.tar /tmp/input.txt > /tmp/input.tmp; mv /tmp/input.tmp /tmp/input.txt )
real 0m0.046s
user 0m0.014s
sys 0m0.019s
回答8:
To get a inplace like result with grep
you can do this:
echo "$(grep -v "pattern" filename)" >filename
回答9:
You can also use this:
grep -v 'pattern' filename
Here -v
will print only other than your pattern (that means invert match).
回答10:
SED:
- '/James\|John/d'
- -n '/James\|John/!p'
AWK:
- '!/James|John/'
- /James|John/ {next;} {print}
GREP:
- -v 'James\|John'
回答11:
perl -i -nle'/regexp/||print' file1 file2 file3
perl -i.bk -nle'/regexp/||print' file1 file2 file3
The first command edits the file(s) inplace (-i).
The second command does the same thing but keeps a copy or backup of the original file(s) by adding .bk to the file names (.bk can be changed to anything).
回答12:
echo -e "/thing_to_delete\ndd\033:x\n" | vim file_to_edit.txt
回答13:
cat filename | grep -v "pattern" > filename.1
mv filename.1 filename
回答14:
Just in case someone wants to do it for exact matches of strings, you can use the -w
flag in grep - w for whole. That is, for example if you want to delete the lines that have number 11, but keep the lines with number 111:
-bash-4.1$ head file
1
11
111
-bash-4.1$ grep -v "11" file
1
-bash-4.1$ grep -w -v "11" file
1
111
It also works with the -f
flag if you want to exclude several exact patterns at once. If "blacklist" is a file with several patterns on each line that you want to delete from "file":
grep -w -v -f blacklist file
来源:https://stackoverflow.com/questions/5410757/delete-lines-in-a-text-file-that-contain-a-specific-string