Add Column to end of CSV file using 'awk' in BASH script

问题

How do you add a column to the end of a CSV file with using a string in a variable?

input.csv

2012-02-29,01:00:00,Manhattan,New York,234
2012-02-29,01:00:00,Manhattan,New York,843
2012-02-29,01:00:00,Manhattan,New York,472
2012-02-29,01:00:00,Manhattan,New York,516

output.csv

2012-02-29,01:00:00,Manhattan,New York,234,2012-02-29 16:13:00
2012-02-29,01:00:00,Manhattan,New York,843,2012-02-29 16:13:00
2012-02-29,01:00:00,Manhattan,New York,472,2012-02-29 16:13:00
2012-02-29,01:00:00,Manhattan,New York,516,2012-02-29 16:13:00

awk.sh

#!/bin/bash

awk -F"," '{$6="2012-02-29 16:13:00" OFS $6; print}' input.csv > output.csv

My attempt above in awk.sh added the string to the end but stripped all the comma separators.

awk.sh result

2012-02-29 01:00:00 Manhattan New York 234 2012-02-29 16:13:00
2012-02-29 01:00:00 Manhattan New York 843 2012-02-29 16:13:00
2012-02-29 01:00:00 Manhattan New York 472 2012-02-29 16:13:00
2012-02-29 01:00:00 Manhattan New York 516 2012-02-29 16:13:00

Appreciate any help!

Updated awk.sh

#!/bin/bash

GAWK="/bin/gawk"
TIMESTAMP=$(date +"%F %T")
ORIG_FILE="input.csv"
NEW_FILE="output.csv"

#Append 'Create' DateTimeStamp to CSV for MySQL logging
$GAWK -v d="$TIMESTAMP" -F"," 'BEGIN {OFS = ","} {$6=d; print}' $ORIG_FILE > $NEW_FILE
rm -f $ORIG_FILE

回答1:

You may add a comma to OFS (Output Field Separator):

awk -F"," 'BEGIN { OFS = "," } {$6="2012-02-29 16:13:00"; print}' input.csv > output.csv

Output:

2012-02-29,01:00:00,Manhatten,New York,234,2012-02-29 16:13:00
2012-02-29,01:00:00,Manhatten,New York,843,2012-02-29 16:13:00
2012-02-29,01:00:00,Manhatten,New York,472,2012-02-29 16:13:00
2012-02-29,01:00:00,Manhatten,New York,516,2012-02-29 16:13:00

EDIT to answer the comment of SirOracle:

From awk man page:

       -v var=val
       --assign var=val
              Assign the value val to the variable var, before execution of the program begins.  Such 
              variable values are available to the BEGIN block of an AWK program.

So assign your date to a shell variable and use it inside awk:

mydate=$(date)
awk -v d="$mydate" -F"," 'BEGIN { OFS = "," } {$6=d; print}' input.csv > output.csv

回答2:

I'd do:

awk '{ printf("%s,2012-02-29 16:13:00\n", $0); }' input.csv > output.csv

This hard codes the value, but so does your code.

Or you can use sed:

sed 's/$/,2012-02-29 16:13:00/' input.csv > output.csv

回答3:

You can set the OFS (output field seperator):

awk -F"," 'BEGIN { OFS = "," } ; {$6="2012-02-29 16:13:00" OFS $6; print}' input.csv >output.csv

which gives me:

2012-02-29,01:00:00,Manhatten,New York,234,2012-02-29 16:13:00,
2012-02-29,01:00:00,Manhatten,New York,843,2012-02-29 16:13:00,
2012-02-29,01:00:00,Manhatten,New York,472,2012-02-29 16:13:00,
2012-02-29,01:00:00,Manhatten,New York,516,2012-02-29 16:13:00,

回答4:

If anyone wants to create csv file through shell with column names: where first input stored in variables from_time, to_time.

example:insert two timestamps with from_time and to_time as column names with respective values -

CODE-

FROM_TIME=2020-02-06T00:00:00
TO_TIME=2020-02-07T00:00:00
{ echo -e "$FROM_TIME,$TO_TIME";}>input1.csv
{ echo -e "from_time,to_time"; cat input1.csv;} > input.csv

first line storing the values second line responsible for adding column name

来源：https://stackoverflow.com/questions/9506810/add-column-to-end-of-csv-file-using-awk-in-bash-script

标签

Linux

bash

parsing

awk