How can I use a file in a command and redirect output to the same file without truncating it?

Question

Basically I want to take as input text from a file, remove a line from that file, and send the output back to the same file. Something along these lines if that makes it any

Answer 1

One liner alternative - set the content of the file as variable:

VAR=`cat file_name`; echo "$VAR"|grep -v 'seg[0-9]\{1,\}\.[0-9]\{1\}' > file_name

Answer 2

You can do that using process-substitution.

It's a bit of a hack though as bash opens all pipes asynchronously and we have to work around that using sleep so YMMV.

In your example:

grep -v 'seg[0-9]\{1,\}\.[0-9]\{1\}' file_name > >(sleep 1 && cat > file_name)

• >(sleep 1 && cat > file_name) creates a temporary file that receives the output from grep
• sleep 1 delays for a second to give grep time to parse the input file
• finally cat > file_name writes the output

Answer 3

Use sed instead:

sed -i '/seg[0-9]\{1,\}\.[0-9]\{1\}/d' file_name

Answer 4

Try this

echo -e "AAA\nBBB\nCCC" > testfile

cat testfile
AAA
BBB
CCC

echo "$(grep -v 'AAA' testfile)" > testfile
cat testfile
BBB
CCC

Answer 5

This is very much possible, you just have to make sure that by the time you write the output, you're writing it to a different file. This can be done by removing the file after opening a file descriptor to it, but before writing to it:

exec 3<file ; rm file; COMMAND <&3 >file ;  exec 3>&-

Or line by line, to understand it better :

exec 3<file       # open a file descriptor reading 'file'
rm file           # remove file (but fd3 will still point to the removed file)
COMMAND <&3 >file # run command, with the removed file as input
exec 3>&-         # close the file descriptor

It's still a risky thing to do, because if COMMAND fails to run properly, you'll lose the file contents. That can be mitigated by restoring the file if COMMAND returns a non-zero exit code :

exec 3<file ; rm file; COMMAND <&3 >file || cat <&3 >file ; exec 3>&-

We can also define a shell function to make it easier to use :

# Usage: replace FILE COMMAND
replace() { exec 3<$1 ; rm $1; ${@:2} <&3 >$1 || cat <&3 >$1 ; exec 3>&- }

Example :

$ echo aaa > test
$ replace test tr a b
$ cat test
bbb

Also, note that this will keep a full copy of the original file (until the third file descriptor is closed). If you're using Linux, and the file you're processing on is too big to fit twice on the disk, you can check out this script that will pipe the file to the specified command block-by-block while unallocating the already processed blocks. As always, read the warnings in the usage page.

Answer 6

Use sponge for this kind of tasks. Its part of moreutils.

Try this command:

 grep -v 'seg[0-9]\{1,\}\.[0-9]\{1\}' file_name | sponge file_name