PS Script is exporting an empty CSVs

Question

I am having a helluva time trying to understand why this script is not working as intended. It is a simple script in which I am attempting to import a CSV, select a few colu

Answer 1

So I have figured it out. I was attempting to pipe through the Import-Csv cmdlet directly instead of declaring it as a variable in the o.g. code. Here is the code snippet that gets what I wanted to get done, done. I was trying to pipe in the Import-Csv cmdlet directly before, I simply had to declare a variable that uses the Import-Csv cmdlet as its definition and pipe that variable through to Select-Object then Export-Csv cmdlets. Thank you all for your assistance, I appreciate it!

$RootPath = "\someDirectory\"
$CsvFilePaths = @(Get-ChildItem $RootPath -Recurse -Include *.csv)
$ColumnsWanted = @('Test_Name','Test_DataName','Device_Model','Device_FW','Data_Avg_ms','Data_StdDev')
for($i=0;$i -lt $CsvFilePaths.Length; $i++){
  $csvPath = $CsvFilePaths[$i]
  Write-Host $csvPath
  $importedCsv = Import-CSV $csvPath 
  $importedCsv | Select-Object $ColumnsWanted | Export-CSV $csvPath -NoTypeInformation
}

Answer 2

Unless you read the input file into memory in full, up front, you cannot safely read from and write back to the same file in a given pipeline.

Specifically, a command such as Import-Csv file.csv | ... | Export-Csv file.csv will erase the content of file.csv.

The simplest solution is to enclose the command that reads the input file in (...), but note that:

• The file's content (transformed into objects) must fit into memory as a whole.

• There is a slight risk of data loss if the pipeline is interrupted before all (transformed) objects have been written back to the file.

Applied to your command:

$RootPath = "D:\SomeFolder"

Get-ChildItem $RootPath -Recurse -Include *.csv -OutVariable csvFiles | 
    ForEach-Object{
        (Import-CSV $_.FullName) | # NOTE THE (...)
          Select-Object Test_Name, Test_DataName, Device_Model, Device_FW, 
                        Data_Avg_ms, Data_StdDev | 
            Export-Csv $_.FullName -NoType -Force
}

Note that I've used -OutVariable csvFiles in order to collect the CSV file-info objects in output variable $csvFiles. Your attempt to collect the file paths via $csvFilePaths = ... doesn't work, because it attempts to collects Export-Csv's output, but Export-Csv produces no output.

Also, to be safe, I've changed the Import-Csv argument from $_ to $_.FullName to ensure that Import-Csv finds the input file (because, regrettably, file-info object $_ is bound as a string, which sometimes expands to the mere file name).

---

A safer solution would be to output to a temporary file first, and (only) on successful completion replace the original file.

With either approach, the replacement file will have default file attributes and permissions; if the original file had special attributes and/or permissions that you want to preserve, you must recreate them explicitly.

Answer 3

As Matt commented, your last $PSItem ($_) not related to the Get-ChildItem anymore but for the Select-Object cmdlet which don't have a FullName Property

You can use differnt foreach approach:

$RootPath = "D:\SomeFolder"
$csvFilePaths = Get-ChildItem $RootPath -Recurse -Include *.csv

foreach ($csv in $csvFilePaths)
{
Import-CSV $csv.FullName |
Select-Object Test_Name,Test_DataName,Device_Model,Device_FW,Data_Avg_ms,Data_StdDev | 
Export-Csv $csv.FullName -NoType -Force
}

Or keeping your code, add $CsvPath Variable containing the csv path and use it later on:

$RootPath = "D:\SomeFolder"
Get-ChildItem $RootPath -Recurse -Include *.csv | ForEach-Object{
$CsvPath = $_.FullName
Import-CSV $CsvPath |
Select-Object Test_Name,Test_DataName,Device_Model,Device_FW,Data_Avg_ms,Data_StdDev | 
Export-Csv $CsvPath -NoType -Force
}