How to 'grep' a continuous stream?

Question

Is that possible to use grep on a continuous stream?

What I mean is sort of a tail -f command, but with grep on t

Answer 1

In most cases, you can tail -f /var/log/some.log |grep foo and it will work just fine.

If you need to use multiple greps on a running log file and you find that you get no output, you may need to stick the --line-buffered switch into your middle grep(s), like so:

tail -f /var/log/some.log | grep --line-buffered foo | grep bar

Answer 2

Yes, this will actually work just fine. Grep and most Unix commands operate on streams one line at a time. Each line that comes out of tail will be analyzed and passed on if it matches.

Answer 3

I think that your problem is that grep uses some output buffering. Try

tail -f file | stdbuf -o0 grep my_pattern

it will set output buffering mode of grep to unbuffered.

Answer 4

Turn on grep's line buffering mode when using BSD grep (FreeBSD, Mac OS X etc.)

tail -f file | grep --line-buffered my_pattern

You don't need to do this for GNU grep (used on pretty much any Linux) as it will flush by default (YMMV for other Unix-likes such as SmartOS, AIX or QNX).

Answer 5

sed would be a better choice (stream editor)

tail -n0 -f <file> | sed -n '/search string/p'

and then if you wanted the tail command to exit once you found a particular string:

tail --pid=$(($BASHPID+1)) -n0 -f <file> | sed -n '/search string/{p; q}'

Obviously a bashism: $BASHPID will be the process id of the tail command. The sed command is next after tail in the pipe, so the sed process id will be $BASHPID+1.

Answer 6

you certainly won't succeed with

tail -f /var/log/foo.log |grep --line-buffered string2search

when you use "colortail" as an alias for tail, eg. in bash

alias tail='colortail -n 30'

you can check by type alias if this outputs something like tail isan alias of colortail -n 30. then you have your culprit :)

Solution:

remove the alias with

unalias tail

ensure that you're using the 'real' tail binary by this command

type tail

which should output something like:

tail is /usr/bin/tail

and then you can run your command

tail -f foo.log |grep --line-buffered something

Good luck.