I have some directories containing test data, typically over 200,000 small (~4k) files per directory.
I am using the following C# code to get the number of files in a di
Create an index every day at midnight. Finding a file will go very fast then. And counting the number of files is just as trivial.
If I see it right, you have one dir for each day. If all files you receive today go in the map of today then this system can be improved. Just index the directory of the previous day at midnight.