SSIS - Polling Files Folder and Trigger ETL Packages

Question

Need advice, see below scenario

We have developed 1 ETL Package which can process 5 files at one time (which usually takes 1 mins). We expect that nearly 100+ files can

Answer 1

What I do is set up an agent to fire the SSIS package on a schedule. The first thing I do in SSIS is determine if there are files to process. I prefer to have a small script task to this for various reasons. If there are files present I set a variable so that my constraint following the script task can go to the next task or if no files are present then end the package reporting success.

If you do not have any ancillary needs like logging, then you can just use a for each file enumerator that will end the package successfully if no files are found.

ps If you set the schedule to say one minute and the package takes longer to run (importing multiple files) then the next schedule will not fire until the original process finishes. So you do not have to worry about multiple processes interfering with each other.