Performance issue with parsing large log files (~5gb) using awk, grep, sed

Question

I am currently dealing with log files with sizes approx. 5gb. I\'m quite new to parsing log files and using UNIX bash, so I\'ll try to be as precise as possible. While searchi

Answer 1

If you don't know the sequence of your strings, then:

awk '/str1/ && /str2/ && /str3/ && /str4/' filename

If you know that they will appear one following another in the line:

grep 'str1.*str2.*str3.*str4' filename

(note for awk, {print} is the default action block, so it can be omitted if the condition is given)

Dealing with files that large is going to be slow no matter how you slice it.