Making an IObservable that uses async/await return completed tasks in original order

Question

Suppose you have a list of 100 urls and you want to download them, parse the response and push the results through an IObservable:

public IObservable

        

Answer 1

Give this a try:

urls.ToObservable()
    .Select(url => Observable.FromAsync(async () => {
        var bytes = await this.DownloadImage(url);
        var image = await this.ParseImage(bytes);
        return image;        
    }))
    .Merge(6 /*at a time*/);

What are we doing here?

For each URL, we're creating a Cold Observable (i.e. one that won't do anything at all, until somebody calls Subscribe). FromAsync returns an Observable that, when you Subscribe to it, runs the async block you gave it. So, we're Selecting the URL into an object that will do the work for us, but only if we ask it later.

Then, our result is an IObservable> - a stream of Future results. We want to flatten that stream, into just a stream of results, so we use Merge(int). The merge operator will subscribe to n items at a time, and as they come back, we'll subscribe to more. Even if url list is very large, the items that Merge are buffering are only a URL and a Func object (i.e. the description of what to do), so relatively small.