Datetime to epoch conversion

Question

I have a bash question (when using awk). I\'m extracting every single instance of the first and fifth column in a textfile and piping it to a new file with the following cod

Answer 1

Given:

$ cat file
Timestamp                 Stream     Status     Seq         Loss Bytes    Delay
17/02/01.10:58:25.212577  stream_0     OK      80281          0  1000     38473
17/02/01.10:58:25.213401  stream_0     OK      80282          0  1000     38472
17/02/01.10:58:25.215560  stream_0     OK      80283          0  1000     38473
17/02/01.10:58:25.216645  stream_0     OK      80284          0  1000     38472

You can write a POSIX Bash script to do what you are looking for:

while IFS= read -r line || [[ -n "$line" ]]; do
    if [[ "$line" =~ ^[[:digit:]]{2}/[[:digit:]]{2}/[[:digit:]]{2} ]]
    then
        arr=($line)
        ts=${arr[0]}
        dec=${ts##*.}   # fractional seconds
        # GNU date may need different flags:
        epoch=$(date -j -f "%y/%m/%d.%H:%M:%S" "${ts%.*}" "+%s")
        printf "%s.%s\t%s\n" "$epoch" "$dec" "${arr[4]}"
    fi  
    done <file >out_file

$ cat out_file
1485975505.212577   0
1485975505.213401   0
1485975505.215560   0
1485975505.216645   0

For GNU date, try:

while IFS= read -r line || [[ -n "$line" ]]; do
    if [[ "$line" =~ ^[[:digit:]]{2}/[[:digit:]]{2}/[[:digit:]]{2} ]]
    then
        arr=($line)
        ts="20${arr[0]}"
        d="${ts%%.*}"
        tmp="${ts%.*}"
        tm="${tmp#*.}"
        dec="${ts##*.}"    # fractional seconds
        epoch=$(date +"%s" --date="$d $tm" )
        printf "%s.%s\t%s\n" "$epoch" "$dec" "${arr[4]}"
    fi  
    done <file >out_file

For an GNU awk solution, you can do:

awk 'function epoch(s){ 
                        split(s, dt, /[/:. ]/)
                        s="20" dt[1] " " dt[2] " " dt[3] " " dt[4] " " dt[5] " " dt[6]
                        return mktime(s) "." dt[7]} 
     /^[0-9][0-9]/      { print epoch($1), $5 }'    file >out_file

If you don't want the fractional second included in the epoch, they are easily removed.

Answer 2

Using GNU awk

Input

$ cat f
Timestamp                 Stream     Status     Seq         Loss Bytes    Delay
17/02/01.10:58:25.212577  stream_0     OK      80281          0  1000     38473
17/02/01.10:58:25.213401  stream_0     OK      80282          0  1000     38472
17/02/01.10:58:25.215560  stream_0     OK      80283          0  1000     38473
17/02/01.10:58:25.216645  stream_0     OK      80284          0  1000     38472

Output

$ awk '
BEGIN{cyear = strftime("%y",systime())}
function epoch(v,       datetime){
    sub(/\./," ",v);
    split(v,datetime,/[/: ]/); 
    datetime[1] = datetime[1] <= cyear ? 2000+datetime[1] : 1900+datetime[1];
    return mktime(datetime[1] " " datetime[2] " " datetime[3] " " datetime[4]" " datetime[5]" " datetime[6])
}
/stream_0/{
    print epoch($1),$5
}' f



1485926905 0
1485926905 0
1485926905 0
1485926905 0

To write to new file just redirect like below

cut -f4 test170201.rawtxt | awk '
BEGIN{cyear = strftime("%y",systime());}
function epoch(v,       datetime){
    sub(/\./," ",v);
    split(v,datetime,/[/: ]/); 
    datetime[1] = datetime[1] <= cyear ? 2000+datetime[1] : 1900+datetime[1];
    return mktime(datetime[1] " " datetime[2] " " datetime[3] " " datetime[4]" " datetime[5]" " datetime[6])
}
/stream_0/{
    print epoch($1),$5
}'  > testLogFile.txt

Answer 3

awk -F '[.[:blank:]]+' '
   # use separator for dot and space (to avoid trailing time info)

   {
   # for line other than header
   if( NR>1) {
      # time is set for format "YYYY MM DD HH MM SS [DST]"
      #  prepare with valuable info
      T = "20"$1 " " $2
      # use correct separator
      gsub( /[\/:]/, " ", T)
      # convert to epoch
      E = mktime( T)

      # print result, adding fractionnal as mentionned later
      printf("%d.%d %s\n", E, $3, $7)
      }
     else {
        # print header (line 1)
        print $1 " "$7
        }
    }
   ' test170201.rawtxt \
   > Redirected.file

• self commented, code is longer for understanding purpose
• use of gnu awk for the mktime function not available in posix or older version

Oneliner a bit optimized here after

awk -F '[.[:blank:]]+' '{if(NR>1){T="20"$1" "$2;gsub(/[\/:]/," ", T);$1=mktime(T)}print $1" "$7}' test170201.rawtxt