Problem with Bash output redirection

Question

I was trying to remove all the lines of a file except the last line but the following command did not work, although file.txt is not empty.

$cat file.txt |ta         


        

Answer 1

tmp=$(tail -1 file.txt); echo $tmp > file.txt;

Answer 2

You can use sed to delete all lines but the last from a file:

sed -i '$!d' file

• -i tells sed to replace the file in place; otherwise, the result would write to STDOUT.
• $ is the address that matches the last line of the file.
• d is the delete command. In this case, it is negated by !, so all lines not matching the address will be deleted.

Answer 3

Just for this case it's possible to use

cat < file.txt | (rm file.txt; tail -1 > file.txt)

That will open "file.txt" just before connection "cat" with subshell in "(...)". "rm file.txt" will remove reference from disk before subshell will open it for write for "tail", but contents will be still available through opened descriptor which is passed to "cat" until it will close stdin. So you'd better be sure that this command will finish or contents of "file.txt" will be lost

Answer 4

This works nicely in a Linux shell:

replace_with_filter() {
  local filename="$1"; shift
  local dd_output byte_count filter_status dd_status
  dd_output=$("$@" <"$filename" | dd conv=notrunc of="$filename" 2>&1; echo "${PIPESTATUS[@]}")
  { read; read; read -r byte_count _; read filter_status dd_status; } <<<"$dd_output"
  (( filter_status > 0 )) && return "$filter_status"
  (( dd_status > 0 )) && return "$dd_status"
  dd bs=1 seek="$byte_count" if=/dev/null of="$filename"
}

replace_with_filter file.txt tail -1

dd's "notrunc" option is used to write the filtered contents back, in place, while dd is needed again (with a byte count) to actually truncate the file. If the new file size is greater or equal to the old file size, the second dd invocation is not necessary.

The advantages of this over a file copy method are: 1) no additional disk space necessary, 2) faster performance on large files, and 3) pure shell (other than dd).

Answer 5

Before 'cat' gets executed, Bash has already opened 'file.txt' for writing, clearing out its contents.

In general, don't write to files you're reading from in the same statement. This can be worked around by writing to a different file, as above:

$cat file.txt | tail -1 >anotherfile.txt
$mv anotherfile.txt file.txt

or by using a utility like sponge from moreutils:

$cat file.txt | tail -1 | sponge file.txt

This works because sponge waits until its input stream has ended before opening its output file.

Answer 6

When you submit your command string to bash, it does the following:

1. Creates an I/O pipe.
2. Starts "/usr/bin/tail -1", reading from the pipe, and writing to file.txt.
3. Starts "/usr/bin/cat file.txt", writing to the pipe.

By the time 'cat' starts reading, 'file.txt' has already been truncated by 'tail'.

That's all part of the design of Unix and the shell environment, and goes back all the way to the original Bourne shell. 'Tis a feature, not a bug.