0

I want to grep for the block of logs around a particular error message. Say I know my log looks like this:

[----] I, [2014-04-10T19:12:36.294512 #1910:e93004]  INFO -- : Started GET "/ems_infra/new"  for 4.2.2.2 at 2014-04-10 15:12:36 -0400 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] F, [2014-04-10T19:12:36.346380 #1910:e93004] FATAL -- : Error caught: [NoMethodError] undefined method `invert' for nil:NilClass
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.369320 #1910:e93004]  INFO -- : Completed 200 OK in 69ms (Views: 22.5ms | ActiveRecord: 0.0ms)

I'm interested in this whole block. Is there any way to center my grep around the FATAL error, and grab up to Started GET and down to Completed 200 OK?

I know that grep -B 123 -A 456 "FATAL" file.txt will do the trick, but that requires me to know ahead of time how far away FATAL is from the the first line (123) and the last line (456) [1].

[1] http://unixhelp.ed.ac.uk/CGI/man-cgi?grep

Improve this question

3 Answers 3

Reset to default
0

I'm afraid that grep has no such facility. perl and/or python might. Off the top of my head (w/o spending much thought on it) I came up with the following awk-combo.

 awk '/Started GET/{print ""};1' blurble | awk 'BEGIN{RS=ORS="\n\n";FS=OFS="\n"}/FATAL/{print $0}'

Given a block of log concocted from your snippet (x4) and a few extra "other log" lines thrown in and two "FATAL" lines removed I get two segments printed:

[----] I, [2014-04-10T19:12:36.294512 #1910:e93004]  INFO -- : Started GET "/ems_infra/new"  for 4.2.2.2 at 2014-04-10 15:12:36 -0400 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.369320 #1910:e93004]  INFO -- : Completed 200 OK in 69ms (Views: 22.5ms | ActiveRecord: 0.0ms)
[----] I, [2014-04-10T19:12:36.294512 #1910:e93004]  INFO -- : Started GET "/ems_infra/new"  for 4.2.2.2 at 2014-04-10 15:12:36 -0400 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] F, [2014-04-10T19:12:36.346380 #1910:e93004] FATAL -- : Error caught: [NoMethodError] undefined method `invert' for nil:NilClass
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.369320 #1910:e93004]  INFO -- : Completed 200 OK in 69ms (Views: 22.5ms | ActiveRecord: 0.0ms)
[----] I, [2014-04-10T19:12:36.294512 #1910:e93004]  INFO -- : Started GET "/ems_infra/new"  for 4.2.2.2 at 2014-04-10 15:12:36 -0400 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.369320 #1910:e93004]  INFO -- : Completed 200 OK in 69ms (Views: 22.5ms | ActiveRecord: 0.0ms)
[----] I, [2014-04-10T19:12:36.294512 #1910:e93004]  INFO -- : Started GET "/ems_infra/new"  for 4.2.2.2 at 2014-04-10 15:12:36 -0400 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] F, [2014-04-10T19:12:36.346380 #1910:e93004] FATAL -- : Error caught: [NoMethodError] undefined method `invert' for nil:NilClass
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.369320 #1910:e93004]  INFO -- : Completed 200 OK in 69ms (Views: 22.5ms | ActiveRecord: 0.0ms)

Running the two awks against the file (I called it blurble) gives this output:

[----] I, [2014-04-10T19:12:36.294512 #1910:e93004]  INFO -- : Started GET "/ems_infra/new"  for 4.2.2.2 at 2014-04-10 15:12:36 -0400 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] F, [2014-04-10T19:12:36.346380 #1910:e93004] FATAL -- : Error caught: [NoMethodError] undefined method `invert' for nil:NilClass
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.369320 #1910:e93004]  INFO -- : Completed 200 OK in 69ms (Views: 22.5ms | ActiveRecord: 0.0ms)

[----] I, [2014-04-10T19:12:36.294512 #1910:e93004]  INFO -- : Started GET "/ems_infra/new"  for 4.2.2.2 at 2014-04-10 15:12:36 -0400 
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs 
[----] F, [2014-04-10T19:12:36.346380 #1910:e93004] FATAL -- : Error caught: [NoMethodError] undefined method `invert' for nil:NilClass
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.345863 #1910:e93004]  INFO -- : other random logs
[----] I, [2014-04-10T19:12:36.369320 #1910:e93004]  INFO -- : Completed 200 OK in 69ms (Views: 22.5ms | ActiveRecord: 0.0ms)
Improve this answer
0

I have a script that might help: https://github.com/jakl/blocks

./blocks.pl --delim 'Started GET|Completed 200 OK' --term 'FATAL'

It splits your log into blocks starting or ending with --delim and prints those that contain --term

Improve this answer
0

here is a one liner using sed

sed -n '/Started/,/Completed/H;/Completed/{z;x;/FATAL/p}' log

one liner expanded for explanation

# do not print lines unless explicitly told to do so
sed -n '

  # put everything between Started and Completed in hold space
  /Started/,/Completed/ H;

  # if at Completed
  /Completed/ {

    # empty the pattern space
    z;

    # switch hold space with pattern space
    x;

    # if FATAL in pattern space then print
    /FATAL/p

  }

' log

note that this will print an empty line between each block. the empty line comes from the z;x; combo. z empties the pattern space. x switches hold space with pattern space. that means after z;x; there is an empty line in the hold space. if you do not empty the pattern space before switching with hold space you will have the last line from the previous block in the next block. i tried to workaround this to prevent the empty line but that made the code much more convoluted.

if the empty line bothers you much you can filter it out... using another sed command.

sed -n '/Started/,/Completed/H;/Completed/{z;x;/FATAL/p}' log | sed '/^$/d'

or you can put a marker which makes the output of this command almost like grep

sed -n '/Started/,/Completed/H;/Completed/{z;x;/FATAL/p}' log | sed 's/^$/--/'

here is the simplest solution i came up with which does away with empty line

sed -n '

  # replace hold space with pattern space
  /Started/ h

  /Started/,/Completed/ {

    # skip the first line because it is already in hold space
    /Started/ n

    # append pattern space to hold space
    H
  }

  /Completed/ {

    # switch hold space with pattern space
    x

    # if FATAL in pattern space then print
    /FATAL/ p
  }

' log

the same as the obligatory one liner

sed -n '/Started/h;/Started/,/Completed/{/Started/n;H};/Completed/{x;/FATAL/p}' log
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .