How can I achieve parallelism in a program that is writing to the disk in C++?

问题

I have a buffer in main memory that contains a couple of files that I want to write in parallel(if possible?) to the disk. I read and write to different location every time.

This is my code:

#include <thread>
void t1(){//read from the buffer and writes the 1st file to the disk }
void t2(){//same with the second one }
void t3(){//3rd}
void t4(){//4th}

int main(){
  std::thread thread1(t1);
  std::thread thread2(t2);
  std::thread thread3(t3);
  std::thread thread4(t4);

  t1.join();
  t2.join();
  t3.join();
  t4.join();
}

I know that I can do in parallel the reading of the buffer but the write is the bottleneck. Is there I way that I parallelise the write to the disk? Is there anything else that I can do to have better performance ?

Thanks

EDIT: Every thread is writing to a different file.

回答1:

It very much depends on the data you want to write.

Writing fixed-sized data you could split it up into four chunks, and each thread seeks to a specific position in the file and write there. Note that you need four different file stream objects, one per thread.

Writing data without a fixed size, like arbitrary text, is not possible to do in parallel. You need some kind of synchronization for this so only one thread writes at a time.

Also, even if the data is a fixed size, it might not be possible to write in parallel, if the data is streaming and can't be split up into chunks.

---

The above is if you want all threads to write to the same file. If the threads each write to different files then it's no problem. That's no different that multiple processes writing to different files.

来源：https://stackoverflow.com/questions/38973929/how-can-i-achieve-parallelism-in-a-program-that-is-writing-to-the-disk-in-c