Merging csv-files via batch

问题

I need to merge a lot small csv-files from different source into separate bigger csv-files

Usually I use:

for /r E:\MyFolder %%d in (.) do ( 
    copy "%%d\*.csv" "%%d\merge.txt" /B
)

This works fine, but for a certain type of csv file, where a new line at the end missing where this won't work correctly. I can use :

for %%x in (E:\MyFolder\case1\*.csv) do more "%%x" >>E:\MyFolder\case1\merge.txt

But it won't run recursively, only for the top folder, obviously. I tried wrapping the same for loop as in the other statement around it but it still won't work.

Can you help me getting it to work recursively?

回答1:

The following code should work for you:

@echo off
rem // Define constants here:
set "_ROOT=E:\MyFolder"
set "_FILE=merge.csv"
set "_MASK=*.csv"
rem // Resolve target file path for later comparison:
for %%C in ("%_ROOT%\%_FILE%") do (
    rem // Write to target file:
    > "%%~fC" (
        rem // Change into parent directory of target file:
        pushd "%%~dpC." && (
            rem // Loop through all matching files:
            for /R %%F in ("%_MASK%") do (
                rem // Exclude target file:
                if /I not "%%~F"=="%%~fC" (
                    rem // Return file content with final line-break if needed:
                    more "%%~F"
                )
            )
            popd
        )
    )
)

This works when the input CSV files contains less than 64K lines/rows each of which is shorter than 64K characters/bytes.

For input CSV files with 64K lines/rows or more, replace more "%%~F" by < "%%~F" find /V "", given that each of the lines/rows is shorter than 4K characters/bytes.

来源：https://stackoverflow.com/questions/59859486/merging-csv-files-via-batch

标签

csv

batch-file

merge

copy