two processes write to one file, prevent mixing the output

问题

I want to get output from two processes and merge them into one file, like:

proc1 >> output &
proc2 >> output &

The problem is that output may be mixed up in the final file. For example if first process writes:

hellow

and the second process writes:

bye

the result may be something like:

hebylloe

but I expect them to be in seperate lines like (order is not important):

bye

hello

So I used flock to synchronize writing to the file with the following script:

exec 200>>output
while read line;
  flock -w 2 200
  do echo $line>>output
  flock -u 200
done

And run the processes like:

proc1 | script &
proc2 | script &

Now the problem is that the performance is decreased significantly. without synchronization each process could write with the speed of 4MB/sec but using the synchronization script the write speed is 1MB/sec.

Can anyone help me how to merge the output from two processes and prevent mixing outputs up?

edit: I realized that there is a relation between line length and std buffer size, if size of each line is less than std buffer size, then every thing works well, nothing is mixed (at least in my tests). so I ran each script with bufsize command:

bufsize -o10KB proc1 | script &
bufsize -o10KB proc2 | script &

Now I want to make sure that this solution is bulletproof. I can not find any relation between buffer size and what happens now!!!

回答1:

Now I want to make sure that this solution is bulletproof. I can not find any relation between buffer size and what happens now!!!

For a fully buffered output stream, the buffer size determines the amount of data written with a single write(2) call. For a line buffered output stream, a line is written with a single write(2) call as long as it doesn't exceed the buffer size.

If the file was open(2)ed with O_APPEND, the file offset is first set to the end of the file before writing. The adjustment of the file offset and the write operation are performed as an atomic step.