3

I'm having problems with grep filtering of make output. In particular, 

make target 2>&1 | grep -E --color=never "^make.*"

works as expected, but the following will print no output to the console:

make target 2>&1 | grep -E --color=never "^make.*" | cat

Am I missing something obvious? Why does the first command output but not the second? Is it to do with some sort of IO buffering? Or am I just being stupid?

[EDIT]: cat is just a minimum test case placeholder for the actual command I want to use.

[EDIT]: This does not seem to be a problem with grep, as replacing it with ack leads to the same behaviour.

[EDIT]: Script that cat is a placeholder for:

#!/bin/bash
cat - \
 | grep -E --color=never "^.*warning:.*|^.*error:.*|^make.*[Ee]rror.*|^make.*" \
 | hilite.sh -r "^.*warning:.*" -f yellow -B \
 | hilite.sh -r "^.*error:.*" -f red -B \
 | hilite.sh -r "^make.*[Ee]rror.*" -f red -B \
 | hilite.sh -r "^make.*" -f magenta

[EDIT]: I think it's a buffering/IO issue. I'm leaving the build running over the w/e and will see if it eventually gets the output where it needs to!

Improve this question

3 Answers 3

Reset to default
6

Likely grep is detecting that it's not writing to a TTY, so it buffers more output, rather than operating in its usual line buffered mode (i.e. you see each line as it's found). This behavior is more efficient as it results in fewer write() system calls, but if you're a user who's waiting for pipelined output, then it may be a bit misleading.

If you're using GNU grep (and based on your linux tag I'm assuming you are), check out the --line-buffered option, which will force grep to work in the more familiar line buffered mode. This could technically decrease the performance of grep (as noted in the man page), but since you're looking at the output of a live build, I doubt it will make any difference in that regard.

Improve this answer
0
1

Not sure what your command is supposed to do, cat any files that have make.* in their file name?

Try changing the cat for xargs cat?

Improve this answer
5
  • Cat is just a placeholder for another set of commands. It fails to output anything regardless of what I replace it with.
    – brice
    Commented Feb 3, 2012 at 17:17
  • then try cat `make target 2>&1 | grep -E --color=never "^make.*"`
    – Baldrick
    Commented Feb 3, 2012 at 17:24
  • That didn't help. (aside: the make command takes several minutes)
    – brice
    Commented Feb 3, 2012 at 17:30
  • What about if you turned it into a for loop? for file in `make target 2>&1 | grep -E --color=never "^make.*"`; do cat $file; done
    – Baldrick
    Commented Feb 3, 2012 at 17:32
  • Javano: I'm actually trying to build a highlighting filter for make, rather than do something with the filename. I'm not iterating over lines as much as editing a bytestream. (besides the fact that make output could not be used like this. I've added the script that I use instead of cat for you to get an idea of what I mean.
    – brice
    Commented Feb 3, 2012 at 17:39
1

You might try experimenting with unbuffer, e.g.,

make target 2>&1 | unbuffer -p grep -E --color=never "^make.*" | cat
Improve this answer
1
  • I'd not heard of unbuffer before. I'll check it out. it could come in handy.
    – brice
    Commented Feb 6, 2012 at 11:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .