How to write in a single file with multiple threads?

Question

I am creating Windows Application in C# in which I want to write in multiple files with multiple threads. I am getting data from different ports and there is one file associ

Answer 1

As mentioned in the comments, this is asking for trouble.

So, you need to have a thread-safe writer class:

public class FileWriter
{
    private ReaderWriterLockSlim lock_ = new ReaderWriterLockSlim();
    public void WriteData(/*....whatever */)
    {
        lock_.EnterWriteLock();
        try
        {
            // write your data here
        }
        finally
        {
            lock_.ExitWriteLock();
        }
    }

} // eo class FileWriter

This is suitable for being called by many threads. BUT, there's a caveat. There may well be lock contention. I used a ReadWriterLockSlim class, because you may want to do read locks as well and hell, that class allows you to upgrade from a read state also.

Answer 2

You'l need to serialize the threads and manage the closing and opening file calls. And probably use append instead of write.. You could use a thread manager class( more like the main thread) to helping syncing all your other threads. Though the best thing would probably be to use one thread dedicated for writing to that file..

Answer 3

You answers to other posts are somewhat out-of-line with your original question - it seems you want to write to one different file per thread.

The easy answer is.. just do it - the file system is thread-safe. The only snag may be performance with only one disk. Concurrently and rapidly writing small chunks of data to many files may result in a lot of disk-thrashing as the file system tries to cope with the distributed files/directories with only one rotor arm. Modern hard disks with large caches and clever controllers will mitigate this to some extent, but you may have an issue. Try it and see!

Your problems will be increased if you plan to continually create and destroy threads, open and close data files for every incoming network read. If you can possibly avoid this, do so.

If the disk cannot keep up, you could implement some clever 'lazy-writing' algorithm of your own to increase the size of the disk writes and so reduce the number, or buy an SSD, or both.