How do I pick the most recently created folder using Foreach loop container in SSIS package?

Question

I've got an interesting challenge with SSIS. Using a for-each file enumerator, I need to pick the subfolder which has been most recently created, and then iterate through each of the files.

Perhaps an example would explain better. The folders look something like this:

c:\data\2011-0703

c:\data\2011-0626

c:\data\2011-0619

How could you get a for each file enumerator to pick the most recent folder? This could either be by looking at the creation date, or comparing the file names.

I'm guessing it would be done with an expression in the enumerator, just can't work out how! Couldn't find anything on the net either.

Thanks

Answer 1

Here is one possible option that you can achieve this with the help of Script Task. Following example shows how this can be done. The example was created in SSIS 2008 R2.

Step-by-step process:

1. Create three folders named 2011-0619, 2011-0626 and 2011-0703 in the folder path C:\temp\ as shown in screenshot #1. Make note of the Date created value of each of the folders.

2. Place few files in each of the folders as shown in screenshots #2 - #4.

3. On the SSIS package, create four variables as shown in screenshot #5. Set the variable RootFolder with value C:\temp\ (in your case this will be c:\data). Set the variable FilePattern with value *.*. Variable RecentFolder will be assigned with the recent folder path in the Script Task. To avoid design time errors, assign the variable RecentFolder with a valid file path. Variable FilePath will be assigned with values when the files are looped through in the recent folder.

4. On the SSIS package, place a Script Task. Replace the Main() method within the Script Task with the script task code give under section Script task code (Get recent folder):. This script gets the list of folders in the root folder and loops through to check the creation datetime to pick the most recently created folder. The recently created folder path is then stored in the variable RecentFolder.

5. On the SSIS package, place a Foreach Loop container and configure it as shown in screenshots #6 and #7.

6. Place a Script Task inside the Foreach Loop container. Replace the Main() method within the Script Task with the script task code give under section Script task code (Display file names):. This script simply displays the names of files within the recently created folder.

7. Once all tasks are configured, the package should look like as shown in screenshot #8.

8. Screenshots #9 - #11 show that the package displays the file names in the recently created folder 2011-0703.

Hope that helps.

Script task code (Get recent folder):

C# code that can be used only in SSIS 2008 and above.

public void Main()
{
    Variables varCollection = null;

    Dts.VariableDispenser.LockForRead("User::RootFolder");
    Dts.VariableDispenser.LockForWrite("User::RecentFolder");
    Dts.VariableDispenser.GetVariables(ref varCollection);

    string rootFolder = varCollection["User::RootFolder"].Value.ToString();
    DateTime previousFolderTime = DateTime.MinValue;
    string recentFolder = string.Empty;

    foreach (string subFolder in System.IO.Directory.GetDirectories(rootFolder))
    {
        DateTime currentFolderTime = System.IO.Directory.GetCreationTime(subFolder);
        if (previousFolderTime == DateTime.MinValue || previousFolderTime <= currentFolderTime)
        {
            previousFolderTime = currentFolderTime;
            recentFolder = subFolder;
        }
    }

    varCollection["User::RecentFolder"].Value = recentFolder;

    Dts.TaskResult = (int)ScriptResults.Success;
}

Script task code (Display file names):

C# code that can be used only in SSIS 2008 and above.

public void Main()
{
    Variables varCollection = null;

    Dts.VariableDispenser.LockForRead("User::FilePath");
    Dts.VariableDispenser.GetVariables(ref varCollection);

    MessageBox.Show(varCollection["User::FilePath"].Value.ToString(), "File Path");

    Dts.TaskResult = (int)ScriptResults.Success;
}

Screenshot #1:

Screenshot #2:

Screenshot #3:

Screenshot #4:

Screenshot #5:

Screenshot #6:

Screenshot #7:

Screenshot #8:

Screenshot #9:

Screenshot #10:

Screenshot #11:

Answer 2

Iterate through the folders. Save the name of the first one. Compare that saved value to the name of each subsequent folder. If the next folder is more recent, swap that name in and keep going. At the end, your saved value will be the name of the most recent folder (if you're comparing creation dates, you'll need to save both the folder name and the creation date).

You can then use the saved value as an argument to your second iteration loop.