tpl-dataflow

问题 Consider this example: class Program { private static readonly ITargetBlock<string> Mesh = CreateMesh(); private static readonly AsyncLocal<string> AsyncLocalContext = new AsyncLocal<string>(); static async Task Main(string[] args) { var tasks = Enumerable.Range(1, 4) .Select(ProcessMessage); await Task.WhenAll(tasks); Mesh.Complete(); await Mesh.Completion; Console.WriteLine(); Console.WriteLine("Done"); } private static async Task ProcessMessage(int number) { var param = number.ToString();

问题 I need to construct TPL dataflow pipeline which will process a lot of messages. Because there are many messages I can not simply Post them into infinite queue of the BufferBlock or I will face memory issues. So I want to use BoundedCapacity = 1 option to disable the queue and use MaxDegreeOfParallelism to use parallel task processing since my TransformBlock s could take some time for each message. I also use PropagateCompletion to make all completion and fail to propagate down the pipeline.

问题 So my requirement is to read multiple CSV files (each having a minimum of a million rows) and then parse each line. Currently, the way I have broken up my pipeline, I am first creating a separate pipeline to just read a CSV file into a string[] and then I plan to create the parsing pipeline later. But seeing the results of my File Reading Pipeline, I am dumbfounded because it is considerably slower than just looping through the CSV file and then looping through the rows. static public

问题 The pipeline takes absolute path of file (in Block 1 ) and processes it and saves it to database (in block3 ). The constraint is that certain file types ( .vde) depend on a particular parent file type ( .vd). The parent file type has the metadata required to process the dependent file types. Unless the parent file type is present in the system, I cannot process the dependent file type. Objective - I need the system to somehow wait for the parent file type to enter the system and update the

问题 I have a TPL Dataflow based application, that worked fine using only a batch block, then an action block. I've added in a TransformBlock to try and trasnform the data from the source before posting to the batch block, but my action block is never getting hit. There are no errors or exceptions being thrown. I am unsure if I need to complete my transform block, as it only seems to be being hit once. Is there a step that I need to add to my transform code other than returning an object of the

问题 I have a TPL Dataflow based application, that worked fine using only a batch block, then an action block. I've added in a TransformBlock to try and trasnform the data from the source before posting to the batch block, but my action block is never getting hit. There are no errors or exceptions being thrown. I am unsure if I need to complete my transform block, as it only seems to be being hit once. Is there a step that I need to add to my transform code other than returning an object of the

问题 I'm using TPL dataflow to process items off a queue in an Azure worker role. Should I have a single long running dataflow, or spawn a new flow for every messages I receive? If an error is thrown in a block, that block will stop accepting new messages. That means if there is an exception in a block, the whole dataflow will stop processing. I need to be able to withstand exception from something like invalid queue inputs without locking my dataflow. I see one of two options: I have a start a

问题 I want to set up a TransformBlock that processes its item in parallel. Thus, I'm setting ExecutionDataflowBlockOptions.MaxDegreeOfParallelism to > 1. I don't care about the order of the messages but the documentation says: When you specify a maximum degree of parallelism that is larger than 1, multiple messages are processed simultaneously, and therefore, messages might not be processed in the order in which they are received. The order in which the messages are output from the block will,

问题 Is it possible to get TransformManyBlock s to send intermediate results as they are created to the next step instead if waiting for the entire IEnumerable<T> to be filled? All testing I've done shows that TransformManyBlock only sends a result to the next block when it is finished; the next block then reads those items one at a time. It seems like basic functionality but I can't find any examples of this anywhere. The use case is processing chunks of a file as they are read. In my case there

问题 I have a simple tpl data flow which basically does some tasks. I noticed when there is an exception in any of the datablocks, it wasn't getting caught in the initial parent block caller. I have added some manual code to check for exception but doesn't seem the right approach. if (readBlock.Completion.Exception != null || saveBlockJoinedProcess.Completion.Exception != null || processBlock1.Completion.Exception != null || processBlock2.Completion.Exception != null) { throw readBlock.Completion