tpl-dataflow

TPL Dataflow local storage or something like it

孤者浪人 提交于 2019-12-06 16:08:36
What I'm trying to accomplish is I have a action block with MaxDegreeOfParallelism = 4 . I want to create one local instance of a session object I have for each parallel path, So I want to total of 4 session objects. If this was threads I would creating something like: ThreadLocal<Session> sessionPerThread = new ThreadLocal<Session>(() => new Session()); I know blocks are not threads so I'm looking for something similar but for blocks. Any way to create this? This block is in a service and runs for months on end. During that time period tons of threads are used for each concurrent slot of the

C# TPL Dataflow - Completion not working

只愿长相守 提交于 2019-12-06 09:41:48
This code never reaches the last line because the completion doesn't propagate from the saveBlock to the sendBlock. What am I doing wrong? var readGenerateBlock = new TransformBlock<int, int>(n => { Console.WriteLine("Read " + n); Thread.Sleep(15); return n; }); var groupingBlock = new BatchBlock<int>(10); var saveBlock = new TransformManyBlock<int[], int>(n => { Console.WriteLine("Saving {0} items [{1}; {2}]", n.Count(), n.First(), n.Last()); Thread.Sleep(150); return n; }); var sendBlock = new TransformBlock<int, int>(n => { Console.WriteLine("Sending {0}", n); Thread.Sleep(25); return n; },

TPL Dataflow duplicate message to all consumers

拜拜、爱过 提交于 2019-12-06 07:55:23
问题 I'm currently writing an application using WPF and TPL Dataflow that should do the following: Load all files in a directory Once it starts processing, log something to the ui and process each file Once completed log something to the ui The problem is that the logging to the UI needs to happen in the UI thread and only log just before it starts processing. The only way I've been able to do this now is by manually calling the dispatcher from inside the TPL Transform block and updating the UI:

TPL Dataflow: design for parallelism while keeping order

徘徊边缘 提交于 2019-12-06 05:57:45
问题 I have never worked with TPL before so I was wondering whether this can be done with it: My application creates a gif image animation file from a lot of frames. I start with a list of Bitmap which represents the frames of the gif file and need to do the following for each frame: paint a number of text/bitmaps onto the frame crop the frame resize the frame reduce the image to 256 colors Obviously this process can be done in parallel for all the frames in the list but for each frame the order

Aggregation and Joins (Inner, Outer, Left, …) with TPL-Dataflow?

半世苍凉 提交于 2019-12-06 01:48:31
Is there any better way to implement functions like aggregation within a TPL-Dataflow mesh than using a BatchBlock to buffer all items until completion, emit them as a collection and then use a transform block to do the actual aggregation? Similarly, is there any other way to do an inner/outer/left/right join of two datasets without using a BatchedJoinBlock to buffer all items of both datasources, emit them as a tuple of two collections and then do the actual join with a Transform block? No . There is no such mechanism out-of-the-box in TPL Dataflow , as aggregation and joining operations aren

How to mark a TPL dataflow cycle to complete?

这一生的挚爱 提交于 2019-12-05 01:05:53
问题 Given the following setup in TPL dataflow. var directory = new DirectoryInfo(@"C:\dev\kortforsyningen_dsm\tiles"); var dirBroadcast=new BroadcastBlock<DirectoryInfo>(dir=>dir); var dirfinder = new TransformManyBlock<DirectoryInfo, DirectoryInfo>((dir) => { return directory.GetDirectories(); }); var tileFilder = new TransformManyBlock<DirectoryInfo, FileInfo>((dir) => { return directory.GetFiles(); }); dirBroadcast.LinkTo(dirfinder); dirBroadcast.LinkTo(tileFilder); dirfinder.LinkTo

TPL DataFlow vs BlockingCollection

南楼画角 提交于 2019-12-05 00:30:24
问题 I understand that a BlockingCollection is best suited for a consumer/producer pattern. However, when do I use a ActionBlock from the TPL DataFlow library? My initial understanding is for IO operations, keep the BlockingCollection while CPU intensive operations are bested suited for an ActionBlock . But I feel like this isn't the whole story... Any additional insight? 回答1: TPL Dataflow is better suited for an actor based design. That means that if you want to chain producers and consumers it's

TransformBlock never completes

陌路散爱 提交于 2019-12-05 00:07:04
I'm trying to wrap my head around "completion" in TPL Dataflow blocks. In particular, the TransformBlock doesn't seem to ever complete. Why? Sample program My code calculates the square of all integers from 1 to 1000. I used a BufferBlock and a TransformBlock for that. Later in my code, I await completion of the TransformBlock . The block never actually completes though, and I don't understand why. static void Main(string[] args) { var bufferBlock = new BufferBlock<int>(); var calculatorBlock = new TransformBlock<int, int>(i => { Console.WriteLine("Calculating {0}²", i); return (int)Math.Pow(i

What is difference between System.Threading.Tasks.Dataflow and Microsoft.Tpl.Dataflow

徘徊边缘 提交于 2019-12-05 00:01:04
There are 2 different official TPL Dataflow nuget package. I am confused to choose which one i should to use. As far as i understand System.Threading.Tasks.Dataflow version is tiny bit newer than other and it seems System.Threading.Tasks.Dataflow is targeted latest versions of .net. Anyone can explain differences between of those ? Microsoft.Tpl.Dataflow was first released as a component separate from the BCL as part of .net 4.5 - here is a blog post announcing the release System.Threading.Tasks.Dataflow as a separate package was created as part of the dotnet core release. This archived

TPL Dataflow Speedup?

泄露秘密 提交于 2019-12-04 13:30:13
问题 I wonder whether the following code can be optimized to execute faster. I currently seem to max out at around 1.4 million simple messages per second on a pretty simple data flow structure. I am aware that this sample process passes/transforms messages synchronously, however, I currently test TPL Dataflow as a possible replacement for my own custom solution based on Tasks and concurrent collections. I know the terms "concurrent" already suggest I run things in parallel but for current testing