Powershell script that can log already scanned files and ignore them on next run?

Question

I am attempting to write a script that will recursively scan a directory, local files with the \'.Error\' extension, then email a group of people with a list of the files. I am

Answer 1

You've got to somehow make the script remember which files it already processed. The best solution would be to move the files you processed to another folder via move-item in your foreach ($file in $array)loop.

You can also write them in a document and check the document every time the script runs - but depending on if the filenames are unique or how many of them are written, that might to errors or slow runtime of your script.

Answer 2

Break out the project into sections. You stated the following requirements:

1. Scan a directory every 20 minutes
2. Exclude previous run's results (only deltas since last run)
3. Email the deltas

To scan every 20 minutes, you can either use a timer yourself or more basic and easier to understand, run the script as a scheduled task to execute every 20 minutes.

To only figure the deltas, you need a way to figure out what the last run was so you can compare it to the new run. The more specific question is, do you want to store every previous run or only the most previous run? If only the most previous run, then just dump the results of the initial run to a importable log file (csv, json, xml, etc.) and then read it on on each subsequent run to compare, and then finally, overwrite. If you want to keep each run indefinitely, you need a way to determine out of all the historical results, which ones were the last run--a timestamp is an easy way to achieve this. Then only grab the results that match the last run's timestamp.

One of the keys to your issue will be, how do you define 'changed'? Is it the existence of a new file/directory at all or is there a possibility of a file that existed before to have it's contents changed? If so, that's a different calculus--do you only want to compare byte size, date changed (a combination of both), or a CRC or hash algorithm? Each will give you a different level of fidelity with different caveats or costs.

As far as the emailing part goes, it looks like you already have that down.

Answer 3

Try breaking the task into simple steps:

• Read list of exclusions from file
• Discover files
• Filter files against list of exclusions
• Send emails
• Append new list of exclusions to file

$exclusionFilePath = '.\path\to\exclusions\file.txt'
if(-not(Test-Path $exclusionFilePath)){
    # Create the exclusion file if it doesn't already exist
    New-Item $exclusionFilePath -ItemType File
}

# read list of exclusions from file
$exclusions = Get-Content -Path $exclusionFilePath

# discover files
$array = @((Get-ChildItem -Path \\SERVER\FOLDER -Recurse -Include *.Error).Fullname)

# filter current files against list of exclusions
$array = $array |Where-Object {$exclusions -notcontains $_}

foreach($file in $array){
  # send emails
  sendEmail

  # append new file path to exclusions file
  $file |Add-Content -Path $exclusionFilePath
}

---

Bonus tip: parameterize your functions

Relying on variables from the calling context is a bit of an anti-pattern, I'd strongly suggest refactoring your sendEmail function to something like:

function Send-ErrorFileEmail {
    param(
        [string]$File
    )
    $MailMessageArgs = @{
        SMTPServer = "relay.EXAMPLE.com"
        From       = "Test@EXAMPLE.com" 
        To         = "user@EXAMPLE.com"
        Subject    = "Error loading XPOLL File - $file" 
        Body       = "This is the body" 
    }
    Send-MailMessage @MailMessageArgs
}

Then use it in the script like:

Send-ErrorFileEmail -File $file