Preventing azure blob from being accessed by other service while it's being created

时光总嘲笑我的痴心妄想 提交于 2020-01-05 10:33:28

问题


Azure blob API sometimes looks like it's been designed by aliens. Like designed for some very exotic use cases when most simple ones require jumping through the hoops. Here is one such.

I have two worker roles. One is creating blobs, another one is processing them (and moving to "completed" folder when processing is done). The blob size can be moderately big, like 100 MB. Obviously, I don't want second role to start reading the blob before the blob has all the data. Okay, one can expect the help from the Lease API: acquire the lease, copy the blob, release the lease. Then, reader will also try to acquire the lease before processing and therefore will have to wait. But no, lease can't be acquired on a blob that does not exist yet. Also, I couldn't find any method that creates a blob with the lease "on" as an atomic operation.

Please let me know if you know the trick to make it work. Sincerely Yours.


回答1:


Obviously, I don't want second role to start reading the blob before the blob has all the data.

Assuming you're creating a BlockBlob, till the time the blob is committed (or in other words all data is written to the blob) for all intents and purposes the blob will not exist and will not come up in the blob listing result (unless you specifically ask for uncommitted blobs when listing blobs). So I think you don't have to do anything special there.

Okay, one can expect the help from the Lease API: acquire the lease, copy the blob, release the lease. Then, reader will also try to acquire the lease before processing and therefore will have to wait.

There's an alternate solution to this. What you could do is make use of Azure Queues. When the blob creator worker role finishes uploading the blob, it just writes a message in a queue which will be polled by worker role processing the blob. Assuming there are many instances of processor worker role, then each instance can GET the message from this queue which would work similar to blob lease (message is available only to one instance) and then start processing the blob.



来源:https://stackoverflow.com/questions/28099642/preventing-azure-blob-from-being-accessed-by-other-service-while-its-being-crea

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!