How to join multiple IObservable sequences?

Question

        var a = Observable.Range(0, 10);
        var b = Observable.Range(5, 10);
        var zip = a.Zip(b, (x, y) => x + \"-\" + y);
        zip.Subscribe(Conso         


        

Answer 1

I honestly can't think of a solution based on existing operators that works for hot sources of unknown order (that is, xs before ys vs ys before xs). Your solution seems fine (hey, if it works), but I'd make a few changes if it were my code:

• Support cancellation properly using MutableDisposable and CompositeDisposable
• Call OnError for exceptions thrown from the selector (making it more consistant with other operators)
• Consider supporting completion if it's possible for one source to complete before the other

The code below has been tested with your dual-Range input, the same inputs flipped, as well as with Empty<int> + Never<int>:

public static IObservable<string> MergeJoin(
    IObservable<int> left, IObservable<int> right, Func<int, int, string> selector)
{
    return Observable.CreateWithDisposable<string>(o =>
    {
        Queue<int> a = new Queue<int>();
        Queue<int> b = new Queue<int>();
        object gate = new object();

        bool leftComplete = false;
        bool rightComplete = false;

        MutableDisposable leftSubscription = new MutableDisposable();
        MutableDisposable rightSubscription = new MutableDisposable();

        Action tryDequeue = () =>
        {
            lock (gate)
            {
                while (a.Count != 0 && b.Count != 0)
                {
                    if (a.Peek() == b.Peek())
                    {
                        string value = null;

                        try
                        {
                            value = selector(a.Dequeue(), b.Dequeue());
                        }
                        catch (Exception ex)
                        {
                            o.OnError(ex);
                            return;
                        }

                        o.OnNext(value);
                    }
                    else if (a.Peek() < b.Peek())
                    {
                        a.Dequeue();
                    }
                    else
                    {
                        b.Dequeue();
                    }
                }
            }
        };

        leftSubscription.Disposable = left.Subscribe(x =>
        {
            lock (gate)
            {
                if (a.Count == 0 || a.Peek() < x)
                    a.Enqueue(x);

                tryDequeue();

                if (rightComplete && b.Count == 0)
                {
                    o.OnCompleted();
                }
            }
        }, () =>
        {
            leftComplete = true;

            if (a.Count == 0 || rightComplete)
            {
                o.OnCompleted();
            }
        });

        rightSubscription.Disposable = right.Subscribe(x =>
        {
            lock (gate)
            {
                if (b.Count == 0 || b.Peek() < x)
                    b.Enqueue(x);

                tryDequeue();

                if (rightComplete && b.Count == 0)
                {
                    o.OnCompleted();
                }
            }
        }, () =>
        {
            rightComplete = true;

            if (b.Count == 0 || leftComplete)
            {
                o.OnCompleted();
            }
        });

        return new CompositeDisposable(leftSubscription, rightSubscription);
    });
}

Answer 2

How about using the new Join operator in v.2838.

var a = Observable.Range(1, 10);
var b = Observable.Range(5, 10);

var joinedStream = a.Join(b, _ => Observable.Never<Unit>(), _ => Observable.Never<Unit>(), 
    (aOutput, bOutput) => new Tuple<int, int>(aOutput,  bOutput))
    .Where(tupple => tupple.Item1 == tupple.Item2);

joinedStream.Subscribe(output => Trace.WriteLine(output));

This is my first look at Join and I'm not sure if it'd be wise to use the Never operator like this. When dealing with a large volumes of inputs as it'd gernerate a huge amount opertaions the more inputs were revieved. I would think that work could be done to close the windows as matche are made and make the solution more efficient. That said the example above works as per your question.

For the record I think Scott's answer is probably the way to go in this instance. I'm just throwing this in as a potential alternative.

Answer 3

GroupBy may do what you need. It seems that you have no time constraints on when items get "joined", you just need similar items to be together in some fashion.

Observable.Merge(Observable.Range(1, 10), Observable.Range(5, 15))
.GroupBy(k => k)
.Subscribe( go => go.Count().Where(cnt => cnt > 1)
                            .Subscribe(cnt => 
                     Console.WriteLine("Key {0} has {1} matches", go.Key, cnt)));

Two things to note about the above, Merge has the following overloads, so that the your req to have hundreds of joined streams won't present an issue:

Merge<TSource>(params IObservable<TSource>[] sources);
Merge<TSource>(this IEnumerable<IObservable<TSource>> sources);
Merge<TSource>(this IObservable<IObservable<TSource>> source);

Furthermore, GroupBy returns IObservable<IGroupedObservable<TKey, TSource>> which means that you can react to each group, and each new member of each group as they come in - no need to wait till all complete.

Answer 4

This answer is copied from the Rx forums, just so that it will be archived in here as well:

var xs = Observable.Range(1, 10);
var ys = Observable.Range(5, 10);

var joined = from x in xs
    from y in ys
    where x == y
    select x + "-" + y;

Or without using query expressions:

var joined = 
    xs.SelectMany(x => ys, (x, y) => new {x, y})
    .Where(t => t.x == t.y)
    .Select(t => t.x + "-" + t.y);