How to use Modules in Google App Engine and add a target to them using Task Queue (Java)?

Question

I have a task that exceeds more than 10 minutes deadline of the Task Queue. Going through different documentations, I found that using modules I could run an instance that would process the long running task but preferably even that should be done using the task queue. I had used backends but they are deprecated.

My question is how do I introduce Modules into my existing App Engine Project and how do I use them to run long-running tasks?

Following is the piece of code :

Queue queue = QueueFactory.getQueue("myqueue");
TaskOptions task = TaskOptions.Builder.withUrl("/submitworker").method(Method.POST);
queue.add(task);

What changes do I have to make in the above code to add a long-running task using a module? [The "submitworker" is a servlet which is the actual long running task]

I had referred this link, but I am unable to get around with the third step:
3. Add service declaration elements to the appengine-application.xml file.

Also, even if I successfully add a module to my project, how can I target this module using Task Queue?

I had gone through this question, but it is a python implementation, my implementation is in Java.

I am looking for a step by step process on how do I use "Target" in the modules and how to use it while adding to the task queue.

Even if I add the long-running module target to the task queue, still would it terminate the execution after 10 minutes or will it complete the task even if the task in the task queue expires?

Please suggest.

Answer 1

Modules and Services are the same thing, they're similar to the old backends (which still work, but are deprecated).

There are two basic ways of getting modules to work:

• Create an EAR and deploy that
• Deploy services independently as WAR files (which is probably what you're doing now to the default module)

The second option is probably easier, because it's just a matter of changing your application-web.xml. You could have a repo or branch per module, or just a build process that changes the module you're targeting.

Right now your application-web.xml probably has something like this:

<application>@appId@</application>
<version>@appVersion@</version>    
<module>default</module>   

change it to something like this

<application>@appId@</application>
<version>@appVersion@</version>    
<module>long-running-service</module>
<instance-class>B1</instance-class>
<manual-scaling>
    <instances>1</instances>
</manual-scaling>

You configure the queue itself to target a specific module in queue.xml See here.

Answer 2

Disclaimer: the answer is based exclusively on docs (I'm actually using python - same concepts but different configs).

To make a service/module allow long-running task you have to configure it for basic or manual scaling. From Scaling types and instance classes (the Deadline row in the table):

• in the Manual scaling column:

Requests can run indefinitely. A manually-scaled instance can choose to handle /_ah/start and execute a program or script for many hours without returning an HTTP response code. Tasks can run up to 24 hours.

• in the Basic scaling column:

Same as manual scaling.

The module scaling configs, done via the respective module's appengine-web.xml file, are described in Scaling elements:

• <manual-scaling>:

Optional. The element enables manual scaling for a module and sets the number of instances for a module.

• <basic-scaling>:

Optional. The element sets the number of instances for a module.

As for the actual conversion to modules, complement the guide you pointed to with the Configuration Files (includes an example) and the appengine-web.xml Syntax (see module and service configs).

About appengine-application.xml, from Configuration Files:

The META-INF directory has two configuration files: appengine-application.xml and application.xml. The appengine-application.xml file contains general information used by App Engine tools when your app is deployed...

...

Note that while every appengine-web.xml file must contain the <application> tag, the name you supply there is ignored. The name of the application is taken from the <application> tag in the appengine-application.xml file.

To direct a certain queue to a certain service/module you use the queue.xml file. From Syntax:

• <target> (push queues):

Optional. A string naming a module/version, a frontend version, or a backend, on which to execute all of the tasks enqueued onto this queue.

The string is prepended to the domain name of your app when constructing the HTTP request for a task. For example, if your app ID is my-app and you set the target to my-version.my-service, the URL hostname will be set to my-version.my-service.my-app.appspot.com.

If target is unspecified, then tasks are invoked on the same version of the application where they were enqueued. So, if you enqueued a task from the default application version without specifying a target on the queue, the task is invoked in the default application version. Note that if the default application version changes between the time that the task is enqueued and the time that it executes, then the task will run in the new default version.

If you are using modules along with a dispatch file, your task's HTTP request might be intercepted and re-routed to another module.