How to partition (GroupBy) a stream and monitor absence of elements in Rx within some time periods?

Question

For the few previous days I have been trying to compose an Rx query to process a stream of events from a source and check absence of some IDs. The absence is defined so that

Answer 1

EDITED - Improved the code, simplifying the SelectMany to use TakeLast.

I wrote a blog post on detecting disconnected clients - which would work just as well for your scenario here if you replace the timeToHold variable in the post with a function like alarmInterval below to get the throttle Timespan based on the client ID.

e.g.:

// idStream is an IObservable<int> of the input stream of IDs
// alarmInterval is a Func<int, TimeSpan> that gets the interval given the ID
var idAlarmStream = idStream
    .GroupByUntil(key => key, grp => grp.Throttle(alarmInterval(grp.Key)))
    .SelectMany(grp => grp.TakeLast(1));

This gives you the basic functionality of constant monitoring without looking at the active monitoring periods.

To get the monitor window functionality, I'd turn things around and filter the above output with WHERE that checks to see if the ID emitted falls in it's monitoring time window. This makes it is easier to deal with changing monitoring periods.

You could do something fancier by turning each monitoring window into a stream and combining those with the alert stream, but I'm not convinced of the benefits of the extra complexity.

The alarmInterval function will also give you an element of dynamic alarm intervals in that it can return new values, but these will only take effect after an alarm goes off for that ID thus ending its current group.

--- Heading off into some theorising here ---

To get this fully dynamic you will have to end the group somehow - you could do that in a few ways.

One would be to project the idStream using Select into a stream of a custom type that holds the ID plus a global counter value. Give this type an appropriate equality implementation so it will work with the GroupByUntil properly.

Now every time you change the alarm intervals, change the counter. This will cause new groups to be created for every ID. You can then add an additional check in the final filter that makes sure the output events have the most recent counter value.