Implementing WebHooks with ServiceStack

Question

I\'m currently working on a REST service allowing to control and monitor some physical devices.

The corresponding REST API is largely based on principles and ideas you c

Answer 1

I believe you are approaching the solution from the wrong end. You could definitely use ServiceStack to make the Web Hook calls - preferably in JSON.

What you really should be looking into is Message Queues, as they exhibit all the characteristics you would require to implement Web Hook calls (durability, message purging policies, message filtering, delivering policies, routing policies, queuing criteria)

Read more about the properties of message queues on Wikipedia

The workflow an event would follow up to the point where a Web Hook is called:

1. An event occurs in the system; to ensure a Web Hook will be called, you have to durably enqueue the event for processing (via a Service Bus such as RabbitMq, MassTransit, NServiceBus etc.). Let's call the target queue EventsQueue
2. The application would then connect to the EventsQueue and process the messages by:
   1. Finding out who has subscribed to this particular event
   2. For every subscriber enqueue a new message containing a copy of the event data and subscriber details (ex. callback URL) to a WebHookQueue with an initial Time To Live (how long the message is valid for)
3. The application would then connect to the WebHookQueue and process the messages by making callbacks.

So now you have a basic notification system that should ensure a message gets delivered at least once.

Here's a great article detailing how to use TTL (Time To Live) to retry messages at intervals

I would take a somewhat different approach:

• Create different retry level queues (ex. WebHookRetryQueue = WebHookQueue's dead-letter queue, WebHookRetryLvl1Queue = TTL 5 minutes, WebHookRetryLvl2Queue = TTL 15 minutes). The trick is to set these retry level queues' dead-letter queue to the WebHookQueue and to leave messages enqueued to expire (meaning once a message expires in a retry level queue, it's enqueued back into the WebHookQueue).
• You would then need to keep track of the current retry level on the messages in the WebHookRetryQueue and then enqueue the message on the appropriate retry level queue - whereafter the TTL expires, gets inserted back into the WebHookQueue.

Example Workflow: WebHookQueue with Max Retries: 2, TTL: 1 day

Example message: {'event_type': 'Email_Reply', 'callback_url': '...', 'reply_level': 0, 'queued_at': '2013-09-25T22:00:00Z', data: 'json encoded'}

Message -> WebHookQueue (fail) -> WebHookQueue (fail) -> WebHookRetryQueue (incr. reply_level=1 + enqueue) -> WebHookRetryLvl1Queue (5 mins-expire) -> WebHookQueue (fail) -> WebHookQueue (fail) -> WebHookRetryQueue (incr. reply_level=2 + enqueue) -> WebHookRetryLvl2Queue (15 mins-expire) -> WebHookQueue (fail) -> WebHookQueue (fail) -> WebHookRetryQueue (drop message)

---

Edit

Click here to look at simple example using a WebHookQueue and a WebHookRetryQueue using message level TTL's of RabbitMQ. Due to the message level TTL's; it's not necessary to create different Retry Level queues - which might be necessary for less featured message queues.

---

If you had to implement such a queue with the ability to expire individual messages (not via purging policies) or schedule messages ahead of time with the option to reschedule/expire - you would most likely have had to opt for Database based queuing.

Here's a great article on CodeProject on building such a high performance queue for MSSQL Server (portable to other databases such as MySql/Postgresql/Mongodb/Couchbase etc. with effort)

Hope you found this information useful.