tpl-dataflow

What is the difference between TPL dataflow and Akka.net?

馋奶兔 提交于 2020-08-04 03:25:21
问题 I have worked with TPL dataflow. Really liked it. I had heard the term Akka many times from my java/scala friends so I tried to read about it and found out that akka has a .net port too. Great. When I continued reading about what akka is, I was surprised to see that it sounds exactly like TPL dataflow. So coming to my question, what is the difference between TPL dataflow and Akka.net ? When to chose what ? 回答1: Akka is an actor based model, and so is TPL Dataflow. As the latter is described

TPL Dataflow block consumes all available memory

我与影子孤独终老i 提交于 2020-07-28 03:16:08
问题 I have a TransformManyBlock with the following design: Input: Path to a file Output: IEnumerable of the file's contents, one line at a time I am running this block on a huge file (61GB), which is too large to fit into RAM. In order to avoid unbounded memory growth, I have set BoundedCapacity to a very low value (e.g. 1) for this block, and all downstream blocks. Nonetheless, the block apparently iterates the IEnumerable greedily, which consumes all available memory on the computer, grinding

Dataflow TPL Implementing pipeline with precondition

[亡魂溺海] 提交于 2020-07-16 06:17:25
问题 I have a question about implementing pipeline using Dataflow TPL library. My case is that I have a software that needs to process some tasks concurrently. Processing looks like this: first we process album at global level, and then we go inside album and process each picture individually. Let's say that application has got processing slots and they are configurable (for the sake of example assume slots = 2). This means that application can process either: a) two albums on the same time b) one

How to keep track of faulted items in TPL pipeline in (thread)safe way

此生再无相见时 提交于 2020-07-09 05:43:02
问题 I am using TPL pipeline design together with Stephen Cleary's Try library In short it wraps value/exception and floats it down the pipeline. So even items that have thrown exceptions inside their processing methods, at the end when I await resultsBlock.Completion; have Status=RunToCompletion . So I need other way how to register faulted items. Here is small sample: var downloadBlock = new TransformBlock<int, Try<int>>(construct => Try.Create(() => { //SomeProcessingMethod(); return 1; }));

How to keep track of faulted items in TPL pipeline in (thread)safe way

好久不见. 提交于 2020-07-09 05:42:29
问题 I am using TPL pipeline design together with Stephen Cleary's Try library In short it wraps value/exception and floats it down the pipeline. So even items that have thrown exceptions inside their processing methods, at the end when I await resultsBlock.Completion; have Status=RunToCompletion . So I need other way how to register faulted items. Here is small sample: var downloadBlock = new TransformBlock<int, Try<int>>(construct => Try.Create(() => { //SomeProcessingMethod(); return 1; }));

How to keep track of faulted items in TPL pipeline in (thread)safe way

南笙酒味 提交于 2020-07-09 05:42:07
问题 I am using TPL pipeline design together with Stephen Cleary's Try library In short it wraps value/exception and floats it down the pipeline. So even items that have thrown exceptions inside their processing methods, at the end when I await resultsBlock.Completion; have Status=RunToCompletion . So I need other way how to register faulted items. Here is small sample: var downloadBlock = new TransformBlock<int, Try<int>>(construct => Try.Create(() => { //SomeProcessingMethod(); return 1; }));