Making an IObservable that uses async/await return completed tasks in original order

Question

Suppose you have a list of 100 urls and you want to download them, parse the response and push the results through an IObservable:

public IObservable

        

Answer 1

One way to achieve what you want is to use the Observable.StartAsync method, a SemaphoreSlim, and the Concat operator. The Observable.StartAsync will create hot observables (started immediately), the SemaphoreSlim will throttle the downloading/parsing of the images, and the Concat will collect the images in the original order.

public IObservable GetImages(IEnumerable urls, int maxConcurrency)
{
    return Observable.Using(() => new SemaphoreSlim(maxConcurrency), semaphore =>
        urls
            .ToObservable()
            .Select(url => Observable.StartAsync(async cancellationToken =>
            {
                await semaphore.WaitAsync(cancellationToken);
                try
                {
                    var bytes = await this.DownloadImage(url);
                    var image = await this.ParseImage(bytes);
                    return image;
                }
                finally { semaphore.Release(); }
            }))
            .Concat());
}

You could consider passing the cancellationToken argument to the DownloadImage and ParseImage methods, in order to avoid having fire-and-forget operations running in the background, in case that the resulting IObservable terminates prematurely for any reason (error or unsubscription).