Multiple Threads reading from the same file

Question

I have a xml file that needs to be read from many many times. I am trying to use the Parallel.ForEach to speed this processes up since none of that data being read in is re

Answer 1

When you open the file, you need to specify FileShare.Read :

using (var stream = new FileStream("theFile.xml", FileMode.Open, FileAccess.Read, FileShare.Read))
{
    ...
}

That way the file can be opened multiple times for reading

Answer 2

If you want multiple threads to read from the same file, you need to specify FileShare.Read:

using (var stream = File.Open("theFile.xml", FileMode.Open, FileAccess.Read, FileShare.Read))
{
    ...
}

However, you will not achieve any speedup from this, for multiple reasons:

1. Your hard disk can only read one thing at a time. Although you have multiple threads running at the same time, these threads will all end up waiting for each other.
2. You cannot easily parse a part of an XML file. You will usually have to parse the entire XML file every time. Since you have multiple threads reading it all the time, it seems that you are not expecting the file to change. If that is the case, then why do you need to read it multiple times?

Answer 3

Depending on the size of the file and the type of reads you are doing it might be faster to load the file into memory first, and then provide access to it directly to your threads.

You didnt provide any specifics on the file, the reads, etc so I cant say for sure if it would address your specific needs.

The general premise would be to load the file once in a single thread, and then either directly (via the Xml structure) or indirectly (via XmlNodes, etc) provide access to the file to each of your threads. I envision something similar to:

1. Load the file
2. For each Xpath query dispatch the matching nodes to your threads.

If the threads dont modify the XML directly, this might be a viable alternative.