Is there a faster way to check if an external web page exists?

Question

I wrote this method to check if a page exists or not:

protected bool PageExists(string url)
{
try
    {
        Uri u = new Uri(url);
        WebRequest w = WebR         


        

Answer 1

One obvious speedup is to run several requests in parallel - most of the time will be spent on IO, so spawning 10 threads to each check a page will complete the whole iteration around 10 times faster.