Say I have an IO-bound task. I\'m using WithDegreeOfParallelism = 10 and WithExecution = ForceParallelism mode, but still the query only uses two threads. Why?
I und
I would agree to Rory, except IO. Haven't tested with disk IO, but network IO definitively may be more effective with more threads, than there are cores on CPU.
Simple test (it would be more correct to run test with each thread count several times, as network speed isn't constant, but still) to prove that:
[Test]
public void TestDownloadThreadsImpactToSpeed()
{
var sampleImages = Enumerable.Range(0, 100)
.Select(x => "url to some quite large file from good server which does not have anti DSS stuff.")
.ToArray();
for (int i = 0; i < 8; i++)
{
var start = DateTime.Now;
var threadCount = (int)Math.Pow(2, i);
Parallel.For(0, sampleImages.Length - 1, new ParallelOptions {MaxDegreeOfParallelism = threadCount},
index =>
{
using (var webClient = new WebClient())
{
webClient.DownloadFile(sampleImages[index],
string.Format(@"c:\test\{0}", index));
}
});
Console.WriteLine("Number of threads: {0}, Seconds: {1}", threadCount, (DateTime.Now - start).TotalSeconds);
}
}
Result with 500x500px image from CDN using 8 core machine with SSD was:
Number of threads: 1, Seconds: 25.3904522
Number of threads: 2, Seconds: 10.8986233
Number of threads: 4, Seconds: 9.9325681
Number of threads: 8, Seconds: 3.7352137
Number of threads: 16, Seconds: 3.3071892
Number of threads: 32, Seconds: 3.1421797
Number of threads: 64, Seconds: 3.1161782
Number of threads: 128, Seconds: 3.7272132
Last result has such time i think firstly because we have to download only 100 images :)
Time differences using 8-64 threads isn't that big, but that is on 8 core machine. If it was 2 core machine (cheap enduser notebook), i think forcing to use 8 threads would have more impact, than on 8 core machine forcing to use 64 threads.