task-queue

Tasks queue up, nothing happens on retry (no log)

谁说胖子不能爱 提交于 2019-12-02 10:12:59
问题 I noticed that over 1,000 tasks queued up in Google App Engine's task queue. When I hit "Run Now" nothing seems to happen, no error in the log entry, etc. I double checked everything, but everything seems fine. The only suspicious thing to me was under the "Previous Run" tab in the task queue console, it showed "Last http response code: 404". Despite the 404, the URL seems fine, I can call it in a browser. Also, tasks just like those also run as expected. Without a log, I do not know what

Tasks queue up, nothing happens on retry (no log)

痞子三分冷 提交于 2019-12-02 05:37:35
I noticed that over 1,000 tasks queued up in Google App Engine's task queue. When I hit "Run Now" nothing seems to happen, no error in the log entry, etc. I double checked everything, but everything seems fine. The only suspicious thing to me was under the "Previous Run" tab in the task queue console, it showed "Last http response code: 404". Despite the 404, the URL seems fine, I can call it in a browser. Also, tasks just like those also run as expected. Without a log, I do not know what else to check. Some tasks may have failed before, but after a fix, that error should be gone. I'm using

How to use Modules in Google App Engine and add a target to them using Task Queue (Java)?

大城市里の小女人 提交于 2019-12-02 02:40:18
问题 I have a task that exceeds more than 10 minutes deadline of the Task Queue. Going through different documentations, I found that using modules I could run an instance that would process the long running task but preferably even that should be done using the task queue. I had used backends but they are deprecated. My question is how do I introduce Modules into my existing App Engine Project and how do I use them to run long-running tasks? Following is the piece of code : Queue queue =

How to use Modules in Google App Engine and add a target to them using Task Queue (Java)?

拟墨画扇 提交于 2019-12-01 23:35:12
I have a task that exceeds more than 10 minutes deadline of the Task Queue. Going through different documentations, I found that using modules I could run an instance that would process the long running task but preferably even that should be done using the task queue. I had used backends but they are deprecated. My question is how do I introduce Modules into my existing App Engine Project and how do I use them to run long-running tasks? Following is the piece of code : Queue queue = QueueFactory.getQueue("myqueue"); TaskOptions task = TaskOptions.Builder.withUrl("/submitworker").method(Method

Google App Engine - About how much quota does a single datastore put use?

冷暖自知 提交于 2019-12-01 11:06:45
The latency for a datastore put is about 150ms - http://code.google.com/status/appengine/detail/datastore/2010/03/11#ae-trust-detail-datastore-put-latency . About how much CPUTime is used by a single datastore put with data size of 100 bytes, into an entity that has only 1 property, and no indexes? Also, does anyone know about how much extra overhead in CPUTime doing this datastore put through the task queue would be? I plan to do some testing with this later today to figure it out, but if anyone already knows that would help me out :). Note: This is kind of a follow up to this question:

Long running program in Google App Engine

爷,独闯天下 提交于 2019-12-01 08:29:25
I have written a servlet code in Java for reading a line from file which is stored in Google Cloud Storage . Once I read each line I pass it to prediction API . Once i get the prediction of the text passed . I append it to original line and store it in some other file in Google cloud storage . This sources file is a csv and has more than 10,000 records . Since I am parsing it individually,passing it to prediction API and then storing back to Cloud Storage . It takes lot of time to do so . Since App Engine has limit of 30 section and also task queues has limitation . Can any suggest me some

Google App Engine - About how much quota does a single datastore put use?

99封情书 提交于 2019-12-01 08:03:09
问题 The latency for a datastore put is about 150ms - http://code.google.com/status/appengine/detail/datastore/2010/03/11#ae-trust-detail-datastore-put-latency. About how much CPUTime is used by a single datastore put with data size of 100 bytes, into an entity that has only 1 property, and no indexes? Also, does anyone know about how much extra overhead in CPUTime doing this datastore put through the task queue would be? I plan to do some testing with this later today to figure it out, but if

Google App Engine Task Queue on GWT

依然范特西╮ 提交于 2019-11-30 23:37:26
I'm looking at Google App Engine's new task queue API for Java and I'm having a hard time relating that to my GWT application. If I wanted to use a task queue to do some asynchronous processing, how should I do that using GWT. The way I see it is, I'd have to send a server request that would then do the submission to the task queue API. If I understand task queues properly, I'd have to create yet another servlet to do the processing from the task queue (be the worker). I'm looking for 2 things: Would the worker be a Servlet (i.e. extends HttpServlet )? If not, can someone give me an example of

Writing to an appengine blob asynchronously and finalizing it when all tasks complete

空扰寡人 提交于 2019-11-30 20:20:00
问题 I have a difficult problem. I am iterating through a set of URLs parameterized by date and fetching them. For example, here is an example of one: somewebservice.com?start=01-01-2012&end=01-10-2012 Sometimes, the content returned from the URL gets truncated (missing random results with a 'truncated error' message attached) because I've defined too large a range, so I have to split the query into two URLs somewebservice.com?start=01-01-2012&end=01-05-2012 somewebservice.com?start=01-06-2012&end

NDB not clearing memory during a long request

[亡魂溺海] 提交于 2019-11-30 15:20:04
问题 I am currently offloading a long running job to a TaskQueue to calculate connections between NDB entities in the Datastore. Basically this queue handles several lists of entity keys that are to be related to another query by the node_in_connected_nodes function in the GetConnectedNodes node: class GetConnectedNodes(object): """Class for getting the connected nodes from a list of nodes in a paged way""" def __init__(self, list, query): # super(GetConnectedNodes, self).__init__() self.nodes =