Grep into variable and maintain stdout?

Question

I've got a long running process and I want to capture a tiny bit of data from the big swath of output.

I can do this by piping it through grep, but then I can't watch it spew out all the other info.

I basically want grep to save what it finds into a variable and leave stdout alone. How can I do that?

Answer 1

With tee, process substitution, and I/O redirection:

{ var=$(cmd | tee >(grep regexp) >&3); } 3>&1

Answer 2

There isn't any way to use a variable like that. It sounds like you want to write your own filter:

./some_long_program | tee >(my_do_stuff.sh)

Then, my_do_stuff.sh is:

#!/bin/bash
while read line; do
    echo "$line" | grep -q 'pattern' || continue
    VARIABLE=$line   # Now it's in a variable
done

If you have the storage space, this is probably more like what you want:

./some_long_program | tee /tmp/slp.log

Then, simply:

grep 'pattern' /tmp/slp.log && VARIABLE=true

or:

VARIABLE=$(grep 'pattern' /tmp/slp.log)

This will let you run the grep at any time. I don't think the variable really adds anything though.

EDIT:

@mpen Based on your last answer above, it sounds like you want to use xargs. Try:

(echo 1 ; sleep 5 ; echo 2) | xargs -L1 echo got:

The -L1 will run the command for every instance found, otherwise it grabs lots of stdin and passes them all (up to some maximum) to the command at once. You'll still want to use this via tee if you want to see all the command output as well:

./some_long_program | tee >(grep 'pattern' | xargs -L1 ./my_do_stuff.sh)

Answer 3

I think this can be done a little more simply if we tee to /dev/tty:

❯ BAR=$(echo foobar | tee /dev/tty | grep -Pom1 b..); echo "[$BAR]"
foobar
[bar]