tpl-dataflow

TPL Dataflow how to remove the link between the blocks

霸气de小男生 提交于 2019-12-23 09:18:11
问题 I would like to know. How can I remove the link between the blocks? In other words. I want to get opposite of LinkTo. I want to write a logger based on tlp dataflow. I wrote this interface and want to delete a subscription for ILogListener when it needed. public interface ILogManager { void RemoveListener(ILogListener listener); } 回答1: When you link blocks: var link = someSourceBlock.LinkTo(someTargetBlock); you get a return value that implements IDisposable . To kill the link, just dispose

Get batches of messages as available

天大地大妈咪最大 提交于 2019-12-23 03:54:07
问题 I am trying to achieve the following behaviour using the Task Parallel Library: As messages arrive I would like to process them sequentially but in groups. So when the first message arrives it should be processed immediately. If 2 messages come in while the first is being processed then they should be processed in a group of 2. I can almost get what I want using a BatchBlock linked to an ActionBlock var batchBlock = new BatchBlock<int>(100); var actionBlock = new ActionBlock<int[]>(list => {

How to safely use SmtpClient.SendAsync in Multithreaded application

十年热恋 提交于 2019-12-23 03:28:10
问题 In my application, I am using ActionBlock from Dataflow library, to send out email alerts using SmtpClient.SendAsync() method, which does not block calling thread.( ActionBlock is getting it's data from BufferBlock , and blocks are tied together using bufferBlock.LinkTo(actionBlock) ). However, this method will throw InvalidOperationException if another .SendAsync() call is in progress. According to MSDN documentation, there is public event SendCompletedEventHandler SendCompleted that is

TPL Dataflow ProducerConsumer Pattern

好久不见. 提交于 2019-12-21 20:29:19
问题 Just wrote a sample producer consumer pattern using TPL DataFlow. I have some basic questions here. The consumer is active only after all the items are posted from the producer. Does asynchronous means that both produce and consume tasks can run in parallel. Given a sleep time in consumer to verify if its blocking other data items. It seems to be executing sequentially and not getting any parallelism. Am I doing something wrong here? class AscDataBlocks { public Int64 start; public Int64 End;

Parallel scraping in .NET

安稳与你 提交于 2019-12-21 18:05:12
问题 The company I work for runs a few hundred very dynamic web sites. It has decided to build a search engine and I was tasked with writing the scraper. Some of the sites run on old hardware and are not able to take much punishment, while others can handle massive amount of simultaneous users. I need to be able to say use 5 parallel requests for site A, 2 for site B and 1 for site C. I know I can use threads, mutexes, semaphores, etc. to accomplish this, but it will be quite complicated. Are any

How can a TPL Dataflow block downstream get data produced by a source?

非 Y 不嫁゛ 提交于 2019-12-21 12:11:58
问题 I'm processing images using TPL Dataflow. I receive a processing request, read an image from a stream, apply several transformations, then write the resulting image to another stream: Request -> Stream -> Image -> Image ... -> Stream For that I use the blocks: BufferBlock<Request> TransformBlock<Request,Stream> TransformBlock<Stream,Image> TransformBlock<Image,Image> TransformBlock<Image,Image> ... writerBlock = new ActionBlock<Image> The problem is the initial Request is what contains some

tpl dataflow: fixed buffer size without throwing items away

痞子三分冷 提交于 2019-12-21 09:22:30
问题 After playing around with dataflow I encountered a new problem. I would like to limit the inputqueue of all blocks. My producingblock (ActionBlock) is creating 5000 elements really fast and posts them to an broadcastblock. So if i set the BoundedCapacity of the broadcastblock to 100 he throws a lot of data away. But I would prefer the producingblock to wait for new slots in the inputqueue of my bufferblock. Is there any way to get rid of this problem? 回答1: That's exactly what BufferBlock is

TPL Dataflow vs plain Semaphore

巧了我就是萌 提交于 2019-12-21 02:52:10
问题 I have a requirement to make a scalable process. The process has mainly I/O operations with some minor CPU operations (mainly deserializing strings). The process query the database for a list of urls, then fetches data from these urls, deserilize the downloaded data to objects, then persist some of the data into crm dynamics and also to another database. Afterwards I need to update the first database which urls were processed. Part of the requirement is to make the parallelism degree

BroadcastCopyBlock for TPL Dataflow with guaranteed delivery

▼魔方 西西 提交于 2019-12-20 02:54:46
问题 I would be glad for some input on the following implementation of a BroadcastCopyBlock in TPL Dataflow, which copies a received message to all consumers, that registered to the BroadcastCopyBlock and guarantees delivery to all consumers, which are linked to the block at the time it receives the message. (Unlike the BroadcastBlock which does not guarntee delivery of messages, if the next one comes in, before the former message has been delivered to all consumers). My main concern is the

EventSourceException: No Free Buffers available from the operating system

末鹿安然 提交于 2019-12-19 09:09:52
问题 The full exception text is: EventSourceException: No Free Buffers available from the operating system (e.g. event rate too fast). I am calling this method approx a million times because of recursion. It does not stop, I just get the exception text in the Output Debug windown in VS2013. But it is super slow. private static IEnumerable<string> RecursiveFindServices(ISymbol sym, Solution sln) { List<string> list = new List<string>(); var callers = SymbolFinder.FindCallersAsync(sym, sln).Result;