azure-storage-queues

Azure Worker: Read a message from the Azure queue in a mutex way

时间秒杀一切 提交于 2019-12-05 20:56:06
The run method of my worker role is: public override void Run() { Message msg=null; while (true) { msg = queue.GetMessage(); if(msg!=null && msg.DequeueCount==1){ //delete message ... //execute operations ... } else if(msg!=null && msg.DequeueCount>1){ //delete message ... } else{ int randomTime = ... Thread.Sleep(randomTime); } } } For performance tests I would that a message could be analysed only by a worker (I don't consider failure problems on workers). But seems by my tests, that two workers can pick up the same message and read DequeueCount equals to 1 (both workers). Is it possible?

How to serialize Observables to the cloud and back

烂漫一生 提交于 2019-12-05 20:03:41
I need to split processing sequence (like in this question How to organize sequence of data processors with .net RX ) into several computation units in Azure environment. The idea is to serialize Observable sequence to Azure Queues(or Service Bus) and to deserialize it back. If producer or consumer is failed other party should be able to continue producing/consuming. Could anyone suggest an elegant way to do so and what to use (Azure Queues or Service Bus)? Has anyone used TCP Observable provider - http://rxx.codeplex.com/wikipage?title=TCP%20Qbservable%20Provider for such problems is it safe

Setting the VisibilityTimeout for a Message added to an Azure Queue via Azure Function Output Binding

烈酒焚心 提交于 2019-12-05 08:55:05
I have a TimerTrigger function and the Output binding is a Azure Queue. The idea is that every 10 minutes the timer will run, it will look at a view in my database and iterate through any rows returned adding them to the queue as messages. Below is my sample TimerTrigger. It worked fine adding messages to the Queue. However in my real world scenario some of the rows will require immediate execution while others will have a delay of some minutes (varies per row). I plan on handling the delay by using the VisibilityTimeout for the message. Unfortunately the binding via a string wouldn't let me

Why is the queue domain on my azure storage account missing?

折月煮酒 提交于 2019-12-05 05:58:36
I have successfully create a storage account on Azure with the following settings: Deployment: Resource manager Type: General Purpose (Standard) Replication: ZRS On the Azure portal I can see a "Blobs" service and if I click on it, I can create blob containers under the blob domain: https://[account_name].blob.core.windows.net/ So far so good. When I try to create a queue using the Azure SDK in a C# app I get the error that it can't find the domain for [account_name]. queue .core.windows.net . I've been following the Microsoft tutorials for creating a storage account and getting a simple queue

Azure Storage Queue and multiple WebJobs instances: will QueueTrigger set the message lease time on triggered?

五迷三道 提交于 2019-12-01 19:01:22
问题 Scenario: producer send a message into the Storage Queue, a WebJobs process the message on QueueTrigger, each message must only be processed once, there could be multiple WebJob instances. I've been googling and from what I've read, I need to write the function that processes the message to be idempotent so a message isn't processed twice. I've also read that there is a default lease time of 10 minutes for a message. My question is, when the QueueTrigger is triggered on one WebJob instance,

Azure Storage Queue and multiple WebJobs instances: will QueueTrigger set the message lease time on triggered?

拥有回忆 提交于 2019-12-01 18:38:59
Scenario: producer send a message into the Storage Queue, a WebJobs process the message on QueueTrigger, each message must only be processed once, there could be multiple WebJob instances. I've been googling and from what I've read, I need to write the function that processes the message to be idempotent so a message isn't processed twice. I've also read that there is a default lease time of 10 minutes for a message. My question is, when the QueueTrigger is triggered on one WebJob instance, does it set the lease time on the message so that another WebJob can't pick up the same message? If so

Azure Queue body is too large and exceeds the maximum permissible limit

谁说胖子不能爱 提交于 2019-12-01 11:44:24
I am getting the following error with my storage queue (NOT BLOB - I know others have seen this with the blob) when pushing the message from my c++ app to Azure Storage: The request body is too large and exceeds the maximum permissible limit. I'm aware that I probably need to cut the json down, but are there any other suggestions? (as in increase message size somewhere?) The maximum size for a message in the Azure storage queue is 64KB ( 48 KB when using Base64 encoding) based on the latest Azure Storage Service Limits documentation as below. It is non-configurable and at the moment Azure

does multiple Azure worker role polling same Queue causes Dead Lock or Poison message

送分小仙女□ 提交于 2019-11-30 07:22:07
Scenario: if I've spin off multiple Worker roles or ONE Worker role with multiple threads, which polls the new messages in Azure Queue. Could someone please confirm if the this the correct design approach? The reason I would like to have many worker roles is to speed up the PROCESSJOB. Our application should be near real time, i.e. as soon as there are messages we should get, apply complex business rules and commit to AZURE DB. We are expecting 11,000 message per 3min. Thank you. David Makogon You may have as many queue-readers as you like. It's very common to scale out worker role instances,

Azure Queue body is too large and exceeds the maximum permissible limit

点点圈 提交于 2019-11-30 05:36:54
问题 I am getting the following error with my storage queue (NOT BLOB - I know others have seen this with the blob) when pushing the message from my c++ app to Azure Storage: The request body is too large and exceeds the maximum permissible limit. I'm aware that I probably need to cut the json down, but are there any other suggestions? (as in increase message size somewhere?) 回答1: As others have stated, the Azure Storage Queue message size limit (64K) is a hard limit. Aside from encoding,

Choosing between .NET Service Bus Queues vs Azure Queue Service [closed]

烈酒焚心 提交于 2019-11-29 20:18:50
Just a quick question regarding an Azure application. If I have a number of Web and Worker roles that need to communicate, documentation says to use the Azure Queue Service. However, I've just read that the new .NET Service Bus now also offers queues. These look to be more powerful as they appear to offer a much more detailed API. Whilst the .NSB looks more interesting it has a couple of issues that make me wary of using it in distributed application. (for example, Queue Expiration... if I cannot guarantee that a queue will be renewed on time I may lose it all!). Has anyone had any experience