After FileSystemWatcher fires - Thread Pool or Dedicated thread?

Question

I am about to implement the archetypal FileSystemWatcher solution. I have a directory to monitor for file creations, and the task of sucking up created files and inserting

Answer 1

Just be aware that FileSystemWatcher may miss events, there's no guarantee it will deliver all specific events that have transpired. Your design of keeping the work done by the thread receiving events to a minimum, should reduce the chances of that happening, but it is still a possibility, given the finite event buffer size (tops out at 64KB).

I would highly recommend developing a battery of torture tests if you decide to use FileSystemWatcher.

In our testing, we encountered issues with network locations, that changing the InternalBufferSize did not fix, yet when we encountered this scenario, we did not receive Error event notifications either.

Thus, we developed our own polling mechanism for doing so, using Directory.GetFiles, followed by comparing the state of the returned files with the previously polled state, ensuring we always had an accurate delta.

Of course, this comes at a substantial cost in performance, which may not be good enough for you.

Answer 2

The third option is the most logical.

In regards to FSW missing some file events, I implemented this: 1) FSW Object which fires on FileCreate 2) tmrFileCheck, ticks = 5000 (5 seconds) - Calls tmrFileChec_Tick

When the FileCreate event occurs, if (tmrFileCheck.Enabled == false) then tmrFileCheck.Start()

This way, after 10 seconds tmrFileCheck_Tick fires which a) tmrFileCheck.Stop() b) CheckForStragglerFiles

Of tests I've run, this works effectively where there are a < 100 files created per minute.

A variant is to merely have a timer tick ever NN seconds and sweep the directory(ies) for straggler files.

Another variant is to hire me to press F5 to refresh the window and call you when there are straggler files; just a suggestion. :-P

Answer 3

If you know that the second thread will always be required, and you also know that you'll never need more than one worker thread, then option three is good enough.