bash sed fail in while loop

Question

#!/bin/bash
fname=$2
rname=$1
echo \"$(<$fname)\" | while read line ; do
    result=`echo \"$(<$rname)\" | grep \"$line\"; echo $?`
    if [ $result != 0 ]
    the         


        

Answer 1

Part 1: What's wrong

The reason your sed expression "doesn't work" is because you used single quotes. You said

sed  '/$line/d' $fname > newkas

Supposing fname=input.txt' and line='example text' this will expand to:

sed  '/$line/d' input.txt > newkas

Note that $line is still literally present. This is because bash will not interpolate variables inside single quotes, thus sed sees the $ literally.

You could fix this by saying

sed  "/$line/d/" $fname > newkas

Because inside double quotes the variable will expand. However, if your sed expression becomes more complicated you could run into difficulty in cases where bash interprets things which you intended to be interpreted by sed. I tend to use the form

sed '/'"$line"'/d/' $fname > newkas

Which is a bit harder to read but, if you look carefully, single-quotes everything I intend to be part of the sed expression and double quotes the variable I want to expand.

Part 2: How to improve it

Your script contains a number things which could be improved.

echo "$(<$fname)" | while read line ; do
    :
done

In the first place you're reading the file with "$(<$fname)" when you could just redirect the stdin of the while loop. This is a bit redundant, but more importantly you're piping to while, which creates an extra subshell and means you can't modify any variables from the enclosing scope. Better to say

while IFS= read -r line ; do
    :
done < "$fname"

Next, consider your grep

echo "$(<$rname)" | grep "$line"

Again you're reading the file and echoing it to grep. But, grep can read files directly.

grep "$line" "$rname"

Afterwards you echo the return code and check its value in an if statement, which is a classic useless construct.

result=$( grep "$line" "$rname" ; echo $?)

Instead you can just pass grep directly to if, which will test its return code.

if grep -q "$line" "$rname" ; then
    sed  "/$line/d" "$fname" > newkas
fi

Note here that I have quoted $fname, which is important if it might ever contain a space. I have also added -q to grep, which suppresses its output.

There's now no need to suppress error messages from the if statement, here, because we don't have to worry about $result containing an unusual value or grep not returning properly.

The final result is this script

while IFS= read -r line ; do
    if grep -q "$line" "$rname" ; then
        sed  "/$line/d" "$fname" > newkas
    fi
done < "$fname"

Which will not work, because newkas is overwritten on every loop. This means that in the end only the last line in $fname was used. Instead you could say:

cp "$fname" newkas
while IFS= read -r line ; do
    if grep -q "$line" "$rname" ; then
        sed  -i '' "/$line/d" newkas
    fi
done < "$fname"

Which, I believe, will do what you expect.

Part 3: But don't do that

But this is all tangential to solving your actual problem. It appears to me that you want to simply create a file newkas which contains the all the lines of $fname except those that appear in $rname. This is easily done with the comm utility:

comm -2 -3 <(sort "$fname") <(sort "$rname") > newkas

This also changes the sort order of the lines, which may not be good for you. If you want to do it without changing the ordering then using the method @fge suggests is best.

grep -F -v -x -f "$rname" "$fname"

Answer 2

If I understand your need correctly, you want a file newaks which contains the lines in $fname which are also in $rname.

If this is what you want, using sed is overkill. Use fgrep:

fgrep -x -f $fname $rname > newkas

Also, there are problems with your script:

• you capture the output of grep in result, which means it will never be exactly 0; what you want is executing the command and simply check for $?
• your echoes are convoluted, just do grep whatever thefilename, or while...done <thefile;
• finally, you take the line as is from the source file: the line can potentially be a regex, which means you will try and match a regex in $rname, which may yield to unexpected results.

And others.