bash - remove multiple different lines of text from a text file

问题

I'm working with lots of log files, and most log files have lots of repeating strings that are logged multiple times. To make the logs easily viewable for others who don't have much to do with such things (for myself also), i've wanted to make a script that rips out some text lines that can cause a 'false alarm' to other people. ("Hey admin, i have these errors here multiple times"; > "Sigh, these errors don't mean anything" kind of way)

Is there some bash code with grep, cat or awk that can get rid of lots of different text lines, without having to go through the document over and over again for each line to be removed? (basically remove all garbage lines in one swoop)

Example, i'll mark the lines that i want removed in bold:

One thing I don't know why

It doesn’t even matter how hard you try

Keep that in mind, I designed this rhyme

To explain in due time

All I know

time is a valuable thing

Watch it fly by as the pendulum swings

Watch it count down to the end of the day

The clock ticks life away

It’s so unreal

Didn’t look out below

Watch the time go right out the window

Trying to hold on but didn’t even know

Wasted it all just to

Watch you go

Sorry about the Linkin Park Lyrics, listening to the Radio while trying to solve a problem gives some bad examples sometimes :P

Are all these lines removable in one command? Many thanks if somebody knows how.

回答1:

grep -v "<string1>\|<string2>\|<stringN>" /path/to/file

回答2:

It removes the lines provided in not_wanted array.

#!/bin/bash
    exec < example.txt
    not_wanted[0]="It doesn’t even matter how hard you try"
    not_wanted[1]="time is a valuable thing"
    not_wanted[2]="The clock ticks life away"
    not_wanted[3]="It’s so unreal"
    not_wanted[4]="Trying to hold on but didn’t even know"

    while read line; do
        for i in "${not_wanted[@]}"; do
            if [ "$line" == "$i" ]; then unset line; break; fi
        done 
        if [ "$line" ]; then echo "$line"; fi
    done

回答3:

Put the lines you don't want in a file, then

grep -v -f not.wanted filename > smaller.file

来源：https://stackoverflow.com/questions/5600076/bash-remove-multiple-different-lines-of-text-from-a-text-file

标签

macos

bash

text

lines