Extract contents from log file for the last 5 minutes

Question

I was trying to use awk command to extract contents from a log file for the last 5 mins. Below was the command I use

`awk -v d1="$(date --date="-5 min" "+%m/%d/%y %k:%M:%S:%3N")" -v d2="$(date "+%m/%d/%y %k:%M:%S:%3N")" '$0 > d1 && $0 < d2 || $0 ~ d2' /tmp/test.log

Found out the condition '$0 > d1 && $0 < d2 || $0 ~ d2' only works on specific date formats. The date format in my log is below: [2/9/17 13:30:35:552 EST] The command I ran above didn't work..

But when I tested the other date format, like below: Feb 9 14:01

the condition worked.

awk -v d1="$(date --date="-30 min" "+%b %_d %H:%M")" -v d2="$(date "+%b %_d %H:%M")" '$0 > d1 && $0 < d2 || $0 ~ d2' /tmp/test2.log

Any idea why the condition doesn't work on this [2/9/17 13:30:35:552 EST] date format?

Answer 1

Let's say [2/9/17 13:30:35:552 EST] is the format, so 17 stands always for 2017 and the timezone conversions are not a problem (EST is ignored).

Let's say this is a sample log:

$ cat test.log 
[2/1/17 10:30:35:000 EST] a
[2/2/17 13:30:31:553 EST] bbb
[2/9/17 09:00:00:000 EST] ccc
[2/9/17 13:30:33:657 EST] ddd

I think you'd like something like that:

$ ./calc.sh '2017-02-01 10:00:00' 2017-02-03
[2/1/17 10:30:35:000 EST] a
[2/2/17 13:30:31:553 EST] bbb

$ ./calc.sh '1 Feb 2017' now
[2/1/17 10:30:35:000 EST] a
[2/2/17 13:30:31:553 EST] bbb
[2/9/17 09:00:00:000 EST] ccc
[2/9/17 13:30:33:657 EST] ddd

$ ./calc.sh '5 Feb 2017' now
[2/9/17 09:00:00:000 EST] ccc
[2/9/17 13:30:33:657 EST] ddd

This is my solution (extended with a lot of variables for clarity):

$ cat calc.sh 
#!/bin/bash

awk -v d1="$(date +%s --date="$1")" -v d2="$(date +%s --date="$2")" '
{
    tDate = substr($1, 2)
    split(tDate, dToken, "/")
    month = dToken[1]
    day = dToken[2]
    year = "20"dToken[3]

    tTime = $2
    tTZ = substr($3, 1, length($3) - 1)
    split(tTime, tToken, ":")
    hh = tToken[1]
    mm = tToken[2]
    ss = tToken[3]

    ts = mktime(year" "month" "day" "hh" "mm" "ss)

    # maybe >= and <= would be better
    if ((ts > d1) && (ts < d2)) {
        print
    }
}
' test.log