How do I atomically create a locked file in Linux?

Question

Scenario: I have many processes running that need to fetch files over the net. If the file is already downloaded, I want it cached on disk. If another process is downloading th

Answer 1

I'm struggling with how to solve a similar problem at the moment, which is what brought me to your question. As I see it, the essence is:

int fd = open(path, O_CREAT|O_RDWR|O_EXCL, mode);
if (fd == -1)
    {
    /* File already exists. */
    the_file_already_exists(fd);
    }
else
    {
    /* I just now created the file.  Now I'll lock it. */

    /* But first I'll deliberately create a race condition!! */
    deliberately_fork_another_process_that_handles_file(path);

    int code = flock(fd,LOCK_EX);
    if (code < 0)
        {
        perror("flock");
        exit(1);
        }

    /* I now have the exclusive lock.  I can write to the file at will --
    or CAN I??  See below. */
    write_to_the_file_at_will(fd);
    }

Obviously in real life I would never deliberately create that race condition, but its equivalent could certainly happen by accident in a real system. That other process might, for example, open the file for reading, obtain a shared lock on it, and read the file. It would see an empty file. That could mean that a write operation is in progress, but it might mean that the file is simply empty and that's the correct and final answer.

If empty files are not allowed, the reader could simply behave exactly the way it would behave if the file was missing. After all, if the reader had started a millisecond earlier it would have failed to open the file anyway. In this case the reader needs to check if the file is empty after it opens it.

If empty files ARE allowed, then you're in a bit of a quandary and I have no ready answer for that.

The problem I have is that when a file is first created, I want to write some sort of default value into it, because I want to "auto-initialize" a fresh system without having to pre-create every possible file it might need. That other process handling the file might itself have already initialized it! For all I know, three other processes might have also run in the meantime and altered the value. In that case I certainly do not want to "write to the file at will" after obtaining the exclusive lock, because I will clobber all those changes.

I suppose the answer is for my code above to ensure that the file is empty before writing to it. If it is NOT empty, then the code should behave exactly as if the file already existed: i.e., it should call:

the_file_already_exists(fd);

Perhaps the bottom line to all of this discussion is that every process which handles the file in any way should check to see if it is empty and behave accordingly. Again though, if empty files ARE allowed, then I can't yet think of any guaranteed solution. None of this would be necessary if there were some way to create the file and lock it as a single atomic sequence, but I don't think there is any way to do that.