PyPy file append mode

Question

I have code like this:

f1 = open(\'file1\', \'a\')
f2 = open(\'file1\', \'a\')

f1.write(\'Test line 1\\n\')
f2.write(\'Test line 2\\n\')
f1.write(\'Test line 3\         


        

Answer 1

The reason in different behavior is different implementation of file I/O operations.

CPython implements it's file I/O on top of fopen, fread and fwrite functions from stdio.h and is working with FILE * streams.

In the same time PyPy implements it's file I/O on top of POSIX open, write and read functions and is working with int file descriptors.

Compare these two programs in C:

#include 

int main() {
    FILE *a = fopen("file1", "a");
    FILE *b = fopen("file1", "a");

    fwrite("Test line 1\n", 12, 1, a);
    fflush(a);
    fwrite("Test line 2\n", 12, 1, b);
    fflush(b);
    fwrite("Test line 3\n", 12, 1, a);
    fflush(a);
    fwrite("Test line 4\n", 12, 1, b);

    fclose(a);
    fclose(b);

    return 0;
}

and

#include 
#include 

int main() {
    int a = open("file1", O_CREAT | O_WRONLY | O_APPEND);
    int b = open("file1", O_CREAT | O_WRONLY | O_APPEND);

    write(a, "Test line 1\n", 12);
    write(b, "Test line 2\n", 12);
    write(a, "Test line 3\n", 12);
    write(b, "Test line 4\n", 12);

    close(a);
    close(b);

    return 0;
}

More info on difference between open and fopen you could find in answers to this question.

UPDATE:

After inspecting PyPy codebase some more, it seems to me it doesn't use O_APPEND flag by some reason, but O_WRONLY | O_CREAT for "a" mode. So it is the real reason in PyPy you need to seek to the end of file after each write call, as J.F. Sebastian mentioned in another answer. I guess a bug should be created at PyPy bugtracker, as O_APPEND flag is available both on Windows and Unix. So, what PyPy does now looks like:

#include 
#include 

int main() {
    int a = open("file1", O_CREAT | O_WRONLY);
    int b = open("file1", O_CREAT | O_WRONLY);

    write(a, "Test line 1\n", 12);
    write(b, "Test line 2\n", 12);
    write(a, "Test line 3\n", 12);
    write(b, "Test line 4\n", 12);

    close(a);
    close(b);

    return 0;
}

Without O_APPEND flag it should reproduce PyPy behavior.