Asynchronous Tasks 'Clogging'

Question

Recently I started working on trying to mass-scrape a website for archiving purposes and I thought it would be a good idea to have multiple web requests working asynchronous

Answer 1

I think this is occurring because you have exhausted all available threads in the thread pool. Try starting your tasks using TaskCreationOptions.LongRunning. More details here.

Another problem is that you are using Thread.Sleep, this blocks the current thread and its a waste of resources. Try waiting asynchronously using await Task.Delay. You may need to change your lambda to be async.

Task.Factory.StartNew(async () => {
            t.Stop();
            Console.ForegroundColor = ConsoleColor.Green; //Note that the other tasks might manage to write their lines between these colour changes messing up the colours.
            Console.WriteLine("Task " + i2 + " started after " + t.Elapsed.Seconds + "." + t.Elapsed.Milliseconds + "s");
            await Task.Delay(5000);
            Console.ForegroundColor = ConsoleColor.Yellow;
            Console.WriteLine("Task " + i2 + " finished");
        });