Solution to the long running query problem in a web application (asynchronous request)

Question

Here is the problem

A user of an enterprise web application is performing a task that results in a long (very long) database query (or other long processing intensiv

Answer 1

Show a message to the user that "Your request is accepted and will take an hour to get updated"

You create a table which stores all these transactions and process these in a batch on server.

User would not have to wait for long and he will be happy to see the message. You can send a confirmation email once your the transaction is processed.

This is the best solution I think.

Answer 2

The solution I have used before involves Jetty Cometd instead of AJAX. The main difference between Ajax and Cometd is that Cometd can uses more of a pub/sub model - the client (web browser in this case) subsribes to the publisher (your app) and the app pushes out updates and notifications to the web browser as apposed to an ajax model of constant polling of the server by the web browser. You can make use of the Cometd solution even if you are not using jetty - you can drop the jetty jars into the lib folder of your respective web server and you should be good to go.

Answer 3

I would do a combination of your so called "bad solution" and the "j2ee solution":

1. The original UI thread sends a asynchronous JMS message to the "backend" and returns.
2. The backend receives the asynchronous request and processes it.
3. When a result (or error) is reached in the backend, it is returned to the controller layer.
4. The UI, still doing AJAX polling or using Bayeux/Cometed, receives and displays the result.

The trick is to match the request / response pair. I would do it like this:

1. Create a "AsyncManagerService" (AMS) that has SESSION, maybe even APPLICATION wide scope so all threads talk to the same instance.
2. The AMS holds a map with an id as key and any object as value.
3. When creating the request JMS message, generate a unique, random key and put it into the jmsCorrelationId of the message as well as into the map, with NULL as a value. Also pass that id to the UI.
4. Let the UI thread poll the AMS with the previously generated id as long as the value in the map is NULL.
5. When the result is ready, let your JMS receiver put it into the AMS' map at the given id.
6. Next time the UI thread polls the map, it will receive the answer and stop polling.

This is clean and well abstracted from any concrete domain. A purely technical solution.

Even if you don't like polling, HTTP is stateless by design and I think this way polling only happens at well defined intervals.

Anyway, I implemented a system with exactly this pattern and it runs just fine...

Answer 4

How long running are these queries?

If you're talking about sessions expiring, perhaps it's best for the user not to be waiting around for it. It will always be annoying for the user to wait 20 minutes peaking every so often in that query's tab.

So, if really long queries are the case, perhaps it's best to change the UI approach and have the user "order" a query that he (she) later comes back to view, or is even notified by mail when it's ready.

In such a case I'd have the query prepared in the DB and cached, in a separate table, there. Meaning, I'd have the web server work once to register the request, have a DB task or a separate process/server prepare queries upon requests and notify the user, and then have the web server display them when the user comes back for the results.