If possible I want to create an async-enumerator for tasks launched in parallel. So first to complete is first element of the enumeration, second to finish is second element of
If I understand your question right, your focus is to launch all tasks, let them all run in parallel, but make sure the return values are processed in the same order as the tasks were launched.
Checking out the specs, with C# 8.0 Asynchronous Streams task queuing for parallel execution but sequential return can look like this.
/// Demonstrates Parallel Execution - Sequential Results with test tasks
async Task RunAsyncStreams()
{
await foreach (var n in RunAndPreserveOrderAsync(GenerateTasks(6)))
{
Console.WriteLine($"#{n} is returned");
}
}
/// Returns an enumerator that will produce a number of test tasks running
/// for a random time.
IEnumerable> GenerateTasks(int count)
{
return Enumerable.Range(1, count).Select(async n =>
{
await Task.Delay(new Random().Next(100, 1000));
Console.WriteLine($"#{n} is complete");
return n;
});
}
/// Launches all tasks in order of enumeration, then waits for the results
/// in the same order: Parallel Execution - Sequential Results.
async IAsyncEnumerable RunAndPreserveOrderAsync(IEnumerable> tasks)
{
var queue = new Queue>(tasks);
while (queue.Count > 0) yield return await queue.Dequeue();
}
Possible output:
#5 is complete
#1 is complete
#1 is returned
#3 is complete
#6 is complete
#2 is complete
#2 is returned
#3 is returned
#4 is complete
#4 is returned
#5 is returned
#6 is returned
On a practical note, there doesn't seem to be any new language-level support for this pattern, and besides since the asynchronous streams deal with IAsyncEnumerable
, it means that a base Task
would not work here and all the worker async
methods should have the same Task
return type, which somewhat limits asynchronous streams-based design.
Because of this and depending on your situation (Do you want to be able to cancel long-running tasks? Is per-task exception handling required? Should there be a limit to the number of concurrent tasks?) it might make sense to check out @TheGeneral 's suggestions up there.
Update:
Note that RunAndPreserveOrderAsync
does not necessarily have to use a Queue
of tasks - this was only chosen to better show coding intentions.
var queue = new Queue>(tasks);
while (queue.Count > 0) yield return await queue.Dequeue();
Converting an enumerator to List
would produce the same result; the body of RunAndPreserveOrderAsync
can be replaced with one line here
foreach(var task in tasks.ToList()) yield return await task;
In this implementation it is important that all the tasks are generated and launched first, which is done along with Queue
initialization or a conversion of tasks
enumerable to List
. However, it might be hard to resist simplifying the above foreach
line like this
foreach(var task in tasks) yield return await task;
which would cause the tasks being executed sequentially and not running in parallel.