Bash read/write file descriptors — seek to start of file

Question

I tried to use the read/write file descriptor in bash so that I could delete the file that the file descriptor referred to afterward, as such:

F=$(mktemp)
exec 3<> "$F"
rm -f "$F"

echo "Hello world" >&3
cat <&3

but the cat command gives no output. I can achieve what I want if I use separate file descriptors for reading and writing:

F=$(mktemp)
exec 3> "$F"
exec 4< "$F"
rm -f "$F"

echo "Hello world" >&3
cat <&4

which prints Hello world.

I suspected that bash doesn't automatically seek to the start of the file descriptor when you switch from writing to reading it, and the following combination of bash and python code confirms this:

fdrw.sh

exec 3<> tmp
rm tmp

echo "Hello world" >&3
exec python fdrw.py

fdrw.py

import os  

f = os.fdopen(3)
print f.tell()
print f.read()

which gives:

$ bash fdrw.sh
12

$ # This is the prompt reappearing

Is there a way to achieve what I want just using bash?

Answer 1

No. bash does not have any concept of "seeking" with its redirection. It reads/writes (mostly) from beginning to end in one long stream.

Answer 2

If you ever do happen to want to seek on bash file descriptors, you can use a subprocess, since it inherits the file descriptors of the parent process. Here is an example C program to do this.

seekfd.c

#define _FILE_OFFSET_BITS 64
#include <string.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>

int main(int argc, char* argv[])
{
    /* Arguments: fd [offset [whence]]
     * where
     * fd: file descriptor to seek
     * offset: number of bytes from position specified in whence
     * whence: one of
     *  SEEK_SET (==0): from start of file
     *  SEEK_CUR (==1): from current position
     *  SEEK_END (==2): from end of file
     */
    int fd;
    long long scan_offset = 0;
    off_t offset = 0;
    int whence = SEEK_SET;
    int errsv; int rv;
    if (argc == 1) {
        fprintf(stderr, "usage: seekfd fd [offset [whence]]\n");
        exit(1);
    }
    if (argc >= 2) {
        if (sscanf(argv[1], "%d", &fd) == EOF) {
            errsv = errno;
            fprintf(stderr, "%s: %s\n", argv[0], strerror(errsv));
            exit(1);
        }
    }
    if (argc >= 3) {
        rv = sscanf(argv[2], "%lld", &scan_offset);
        if (rv == EOF) {
            errsv = errno;
            fprintf(stderr, "%s: %s\n", argv[0], strerror(errsv));
            exit(1);
        }
        offset = (off_t) scan_offset;
    }
    if (argc >= 4) {
        if (sscanf(argv[3], "%d", &whence) == EOF) {
            errsv = errno;
            fprintf(stderr, "%s: %s\n", argv[0], strerror(errsv));
            exit(1);
        }
    }

    if (lseek(fd, offset, whence) == (off_t) -1) {
        errsv = errno;
        fprintf(stderr, "%s: %s\n", argv[0], strerror(errsv));
        exit(2);
    }

    return 0;
}

Answer 3

I found a way to do it in bash, but it's relying on an obscure feature of exec < /dev/stdin which actually can rewind the file descriptor of stdin according to http://linux-ip.net/misc/madlug/shell-tips/tip-1.txt:

F=$(mktemp)
exec 3<> "$F"
rm -f "$F"

echo "Hello world" >&3
{ exec < /dev/stdin; cat; } <&3

The write descriptor isn't affected by that so you can still append output to descriptor 3 before the cat.

Sadly I only got this working under Linux not under MacOS (BSD), even with the newest bash version. So it doesn't seem very portable.

Answer 4

Try changing the sequence of commands:

F=$(mktemp tmp.XXXXXX)
exec 3<> "$F"
echo "Hello world" > "$F"
rm -f "$F"

#echo "Hello world" >&3
cat <&3

Answer 5

When you open a file descriptor in bash like that, it becomes accessible as a file in /dev/fd/. On that you can do cat and it'll read from the start, or append (echo "something" >> /dev/fd/3), and it'll add it to the end. At least on my system it behaves this way. (On the other hand, I can't seem to be able to get "cat <&3" to work, even if I don't do any writing to the descriptor).

Answer 6

#!/bin/bash
F=$(mktemp tmp.XXXXXX)
exec 3<> $F
rm $F

echo "Hello world" >&3
cat /dev/fd/3

As suggested in other answer, cat will rewind the file descriptor for you before reading from it since it thinks it's just a regular file.

Answer 7

To 'rewind' the file descriptor, you can simply use /proc/self/fd/3

Test script :

#!/bin/bash

# Fill data
FILE=test
date +%FT%T >$FILE

# Open the file descriptor and delete the file
exec 5<>$FILE
rm -rf $FILE

# Check state of the file
# should return an error as the file has been deleted
file $FILE

# Check that you still can do multiple reads or additions
for i in {0..5}; do
    echo ----- $i -----

    echo . >>/proc/self/fd/5
    cat /proc/self/fd/5

    echo
    sleep 1
done

Try to kill -9 the script while it is running, you will see that contrary to what happens with the trap method, the file is actually deleted.