Get specific key value in log pattern using the shell script

Question

My log file has below pattern :-

 tx=267c5660-c49a-4ae7-b5ae-c9d43e23b617, rh=163.172.0.0, userId=-1, requestComplete={ requestId=74421156932, entityResourceType=xyz, pageId=homePage, uri=/home/, duration(ms)=422

How can i grep for just a specific field like uri or duration or combination of both like uri and duration.

Basically when i use tail command,I want in the o/p just 2 fields uri and duration.

Answer 1

We can use cut command. According to logs there is one seperator i.e. ",". So we can use "," as a delimeter and we need to give field number, In our case it is 8,9 for uri and duration.

cat "logfile" | cut -d"," -f7,8

Thanks

Answer 2

For a variable like

var=$'2017-04-21 09:04:42,649 +0000 [exec-12056] EventLogger - cid=rio, tx=267c5660-c49a-4ae7-b5ae-c9d43e23b617, rh=163.172.0.0, userId=-1, requestComplete={ requestId=74421156932, entityResourceType=xyz, pageId=homePage, uri=/riokc95758/, duration(ms)=422'

You might consider something like this with GNU grep:

$ grep -Po 'uri=\K.[^,]*' <<<"$var"
/riokc95758/

$ grep -Po 'uri=/\K.[^,/]*' <<<"$var"
riokc95758

$ grep -Po '.*duration\(ms\)=\K.[^,]*' <<<"$var"  #if duration is the last field you can use just grep -Po '.*duration\(ms\)=\K.*'
422

Or even you can use sed with regex groups and backreferences :

$ sed -E "s/(.*uri=)(.[^,]*)(.*duration\(ms\)=)(.[^,]*)(.*)/\2,\4/" <<<"$var" 
/riokc95758/,422

You can assign each of above commands to a variable like $ newvar=$(egrep .... )

Answer 3

You could use cut if the pattern had a consistent number of comma-separated values. The order would have to be consistent as well.

Otherwise, a double grep can get the values you're looking for :

grep -Eo "uri=/.*/" | grep -Eo "/.*/" # gets uri
grep -Eo "duration\(ms\)=[0-9]+" | grep -Eo "[0-9]+" # gets duration

Explanation :

In the first example grep -Eo "uri=/.*/" on your input will return duration(ms)=422. Chaining it with a second grep allows you to isolate 422 alone.

This is somewhat similar to this answer