.Net framework to manage background running processess on seperate machines

Question

I am having an asp.mvc application which resides on a server.From this application, I want to start a process which is a bit long-running operation and will be resource inte

Answer 1

Push messages to MSMQ from your MVC app and have your windows services listen (or loop) on new messages entering the queue.

In your MVC app create a ID per message queued, so make restful API calls from your windows services back to the mvc app as you make progress on the job?

Answer 2

Give a try to http://easynetq.com/

EasyNetQ is a simple to use, opinionated, .NET API for RabbitMQ.

EasyNetQ is a collection of components that provide services on top of the RabbitMQ.Client library. These do things like serialization, error handling, thread marshalling, connection management, etc.

To publish with EasyNetQ

var message = new MyMessage { Text = "Hello Rabbit" };
bus.Publish(message);

To subscribe to a message we need to give EasyNetQ an action to perform whenever a message arrives. We do this by passing subscribe a delegate:

bus.Subscribe<MyMessage>("my_subscription_id", msg => Console.WriteLine(msg.Text));

Now every time that an instance of MyMessage is published, EasyNetQ will call our delegate and print the message’s Text property to the console.

The performance of EasyNetQ is directly related to the performance of the RabbitMQ broker. This can vary with network and server performance. In tests on a developer machine with a local instance of RabbitMQ, sustained over-night performance of around 5000 2K messages per second was achieved. Memory use for all the EasyNetQ endpoints was stable for the overnight run

Answer 3

Have a look at Hangfire, this can manage background tasks and works across VMs without conflict. We have replaced windows services using this and it works well.

https://www.hangfire.io

Answer 4

Hangfire is really a great solution for processing background tasks, and we have used used it extensively in our projects.

We have setup our MVC application on separate IIS servers which is also a hangfire client and just enqueues the jobs needs to be executed by hangfire server. Then we have two hangfire server instances, which are windows service application. So effectively there is no load on the MVC app server to process the background jobs, as it is being processed by separate hangfire servers.

One of the extremely helpful feature of hangfire is its out of the box dashboard, that allows you to monitor and control any aspect of background job processing, including statistics, background job history etc.

Configure the hangfire in application as well as in hangfire servers

public void Configuration(IAppBuilder app)
{
    GlobalConfiguration.Configuration.UseSqlServerStorage("<connection string or its name>");

    app.UseHangfireDashboard();
    app.UseHangfireServer();
}

Please note that you use the same connection string across. Use app.UseHangfireServer() only if you want to use the instance as hangfire server, so in your case you would like to omit this line from application server configuration and use only in the hangfire servers. Also use app.UseHangfireDashboard() in instance which will serve your hangfire dashboard, which would be probably your MVC application.

At that time we have done it using Windows Service, but if had to do it now, I would like to go with Azure worker role or even better now Azure Web Jobs to host my hangfire server, and manage things like auto scaling easily.

Do refer hangfire overview and documentation for more details.