tpl-dataflow

Simpler solution than TPL Dataflow for parallel async blob deletion

纵饮孤独 提交于 2019-12-11 22:31:42
问题 I'm implementing a worker role on Azure which needs to delete blobs from Azure storage. Let's assume my list of blobs has about 10K items. The simplest synchronous approach would probably be: Parallel.ForEach(list, x => ((CloudBlob) x).Delete()); Requirements: I want to implement the same thing asynchronously (on a single thread). I want to limit the number of concurrent connections to 50 - so I'll do my 10K deletions when only 50 async ones are being performed at the same time. If one

Throttling asynchronous tasks

寵の児 提交于 2019-12-11 18:49:10
问题 I would like to run a bunch of async tasks, with a limit on how many tasks may be pending completion at any given time. Say you have 1000 URLs, and you only want to have 50 requests open at a time; but as soon as one request completes, you open up a connection to the next URL in the list. That way, there are always exactly 50 connections open at a time, until the URL list is exhausted. I also want to utilize a given number of threads if possible. I came up with an extension method,

Cancellation not returning past whenall using httpclient

匆匆过客 提交于 2019-12-11 16:13:59
问题 Here is an updated version of the code posted under httpclient.GetStringAsync blocking The question is when cancel is done, though the tasks are cancelled, I am expecting Await Task.WhenAll(tasks) to return and print whats in finally , but its not. I can see task cancellations when pressing cancel and I also see the connections reduce to 0, but the finally guessing WhenAll still thinks some tasks are being executed. Here is the code: Private concurrencySemaphore As New SemaphoreSlim(10)

In a JoinBlock, receive a Target when the other Target is filled

会有一股神秘感。 提交于 2019-12-11 08:13:45
问题 I'm connecting a JoinBlock to a WriteOnceBlock and a BufferBlock to fill Targets 1 and 2. My goal is that every time the JoinBlock receives a message from the BufferBlock , it also requests the value that the WriteOnceBlock is holding. My first guess was that I could add a ContinueWith delegate to the Target2 Completion event, but that's not quite right - I need to attach to something like a Filled event that doesn't seem to exist. I also tried using the join block in non-greedy mode as a

ReceiveAsync interrupting/breaking message passing

僤鯓⒐⒋嵵緔 提交于 2019-12-11 07:57:21
问题 This problem raised its head while trying to implement the suggested solution to this problem. Problem summary Performing a ReceiveAsync() call from a TransformBlock to a WriteOnceBlock is causing the TransformBlock to essentially remove itself from the flow. It ceases to propagate any kind of message, whether it's data or a completion signal. System design The system is intended for parsing large CSV files through a series of steps. The problematic part of the flow can be (inexpertly)

TPL Dataflow - how to call action item multiple items

吃可爱长大的小学妹 提交于 2019-12-11 05:58:57
问题 I am a newbie on TPL Dataflow . I have a list of project numbers that I need to process. A project could have about 8000 items and I need to get the data for each item in the project and then push this data into 5 separate servers. Here is what I have coded thus far. I'm stuck at the step of how to load this data into the 5 servers. I am not sure if this is coded correctly. Any advice is much appreciated. public static bool PushData(string projectId) { var linkCompletion = new

Why tasks are not going in parallel?

ぐ巨炮叔叔 提交于 2019-12-11 04:57:37
问题 I have number of locations(called Cells) where I run tests. Tests are implemented as asynchronous tasks and running consequently. User can select to run any tests for each cell. If I select to run same exactly same tests on all cells, then it's going more or less parallel. Having tests A, B, C , if on cell 1 and 2 I select test A, B and on 3 I select only C , then for some reason tests in cell 1 and 2 will start running, but in cell 3 test C will not start, until A and B tests in cell 1 and 2

In TPL Dataflow, is it possible to change DataflowBlockOptions after block is created but before it is used?

冷暖自知 提交于 2019-12-10 20:39:03
问题 ... and have it take effect? I'd like to defer setting the ExecutionDataflowBlockOptions.SingleProducerConstrained property until I'm ready to link the network together. (Because, I want to separate creating the blocks, with their semantics, from linking the network together, with its semantics.) But as far as I can tell you can only set the ExecutionDataflowBlockOptions when the block is created (e.g., for TransformBlock, TransformManyBlock, etc, you pass it in to the constructor and it is

Updating UI Control from ActionBlock

僤鯓⒐⒋嵵緔 提交于 2019-12-10 20:21:11
问题 I have been trying to understand the TPL Dataflow by creating a sample application. One of thing that I have been trying to do is to update a TextBox control from ActionBlock . The reason for using TPL Dataflow is to perform parallel asynchronous operation while keeping the order. The following function is written by me, private TaskScheduler scheduler = null; public Form1() { this.scheduler = TaskScheduler.FromCurrentSynchronizationContext(); InitializeComponent(); } public async void

TPL Dataflow: Flatten incoming collection to sequential items

人盡茶涼 提交于 2019-12-10 17:14:26
问题 I'm building an application using TPL dataflow. Actually I've the following problem. I've one transformblock var tfb1 = new TranformBlock<InMsg, IReadOnlyCollection<OutMsg>> . So tfb1 receives on in message and creates a list of out-messages. This list of out-messages shall be linked to a router datablock, which receives OutMsg as input (and not IReadOnlyCollection<OutMsg> ). How can I flatten the IReadOnlyCollection so that the containing message can be used as input for e.g. a transform