sed + remove “#” and empty lines with one sed command

Question

how to remove comment lines (as # bal bla ) and empty lines (lines without charecters) from file with one sed command?

THX lidia

Answer 1

On (one of) my linux boxes, sed understands extended regular expressions with the -r option, so:

sed -r '/(^\s*#)|(^\s*$)/d' squid.conf.installed

is very useful for showing all non-blank, non comment lines. The regex matches either start of line followed by zero or more spaces or tabs followed by either a hash or end of line, and deletes those matching lines from the input.

Answer 2

First example(paxdiablo) is very good except its not change file, just output result. If you want to change it inline:

sudo sed -i 's/#.*$//;/^$/d' inputFile

Answer 3

If you're worried about starting two sed processes in a pipeline for performance reasons, you probably shouldn't be, it's still very efficient. But based on your comment that you want to do in-place editing, you can still do that with distinct commands (sed commands rather than invocations of sed itself).

You can either use multiple -e arguments or separate commands with a semicolon, something like (just one of these, not both):

sed -i 's/#.*$//' -e '/^$/d' fileName
sed -i 's/#.*$//;/^$/d' fileName

The following transcript shows this in action:

pax> printf 'Line # with a comment\n\n# Line with only a comment\n' >file

pax> cat file
Line # with a comment

# Line with only a comment

pax> cp file filex ; sed -i 's/#.*$//;/^$/d' filex ; cat filex
Line

pax> cp file filex ; sed -i -e 's/#.*$//' -e '/^$/d' filex ; cat filex
Line

Note how the file is modified in-place even with two -e options. You can see that both commands are executed on each line. The line with a comment first has the comment removed then all is removed because it's empty.

In addition, the original empty line is also removed.

Answer 4

This tiny jewel removes all # comments, no matter where they begin in a line:

sed -e 's/\s*#.*$//'

Example:

text="
this is a # test
#this is a test
#this is a #test
this is # another #test
"

$echo "$text" | sed -e 's/\s*#.*$//'

this is a


this is

Next this removes any resulting blank lines:

$echo "$text" | sed -e 's/\s*#.*$//' | sed -e '/^\s*$/d'

Answer 5

Alternative variant, using grep:

cat file.txt | grep -Ev '(#.*$)|(^$)'

Answer 6

@paxdiablo has a good answer but it can be improved.

(1) The '/^$/d' clause only matches 100% blank lines.

If you want to also match lines that are entirely whitespace (spaces, tabs etc.) use this instead:

'/^\s*$/d'

(2) The 's/#.*$//' clause only matches lines that start with the # character in column 0.

If you want to also match lines that have only whitespace before the first # use this instead:

'/^\s*#.*$/d'

The above criteria may not be universal (e.g. within a HEREDOC block, or in a Python multi-line string the different approaches could be significant), but in many cases the conventional definition of "blank" lines include whitespace-only, and "comment" lines include whitespace-then-#.

(3) Lastly, on OSX at least, the @paxdiablo solution in which the first clause turns comment lines into blank lines, and the second clause strips blank lines (including what were originally comments) doesn't work. It seems to be more portable to make both clauses /d delete actions as I've done.

The revised command incorporating the above is:

sed -e '/^\s*#.*$/d' -e '/^\s*$/d' inputFile