Multicolored Grep

Question

I'm trying to get each grep command to highlight its results in a different color. I can do it manually with a line like this:

ls -l GREP_COLORS='mt=01;32' grep c | GREP_COLORS='mt=01;31' grep o | GREP_COLORS='mt=01;34' grep n | GREP_COLORS='mt=01;36' grep f

Every c character will be highlighted in green and every o character will be highlighted in red, etc...

For this example to work you'll need to ensure that you always have --color=always on your grep commands. I've set this in my .bashrc so grep will always have colors:

export GREP_OPTIONS='--color=always'

---

What I'm attempting to accomplish is to wrap this functionality with an alias, so I can just call grep and have a different GREP_COLORS value each time. I understand the consideration of multiple shells for each new piped grep and I'm trying to over come this by creating some files (one for each color), to indicate that they have already been used.

I have made some attempts but strangely, this one seems to work the "best". I have this in my .bashrc:

alias mg="mygrep"
mygrep(){
    # define possible colors
    COLORS=("01;32" "01;31" "01;34" "01;36")
    COUNTER=0
    NUM=0
    # as long as the color has already been used, keep searching
    while [ -f /home/lior/Desktop/mygrep_$NUM ]; do
        # get a random index
        let NUM=`shuf --input-range=0-$(( ${#COLORS[*]} - 1 )) | head -1`
        wait ${!}
        $(( COUNTER+=1 ))
        if [ "$COUNTER" -ge ${#COLORS[@]} ]; then
            # remove all color locks
            rm /home/lior/Desktop/mygrep_*
            wait ${!}
        fi
    done
    # mark this color as used
    touch /home/lior/Desktop/mygrep_$NUM
    wait ${!}

    # lets go!
    GREP_COLORS="mt=${COLORS[$NUM]}" grep "$@"
}

I'm using this alias like so:

ll | mg c | mg o | mg n | mg f

The results are quite cool. There are however some errors that are slightly different each time. Here are a couple of screenshots:

Looks like as the shell goes through each pipe command, the previous function did not yet finish its execution. It tries to remove files that don't exist anymore. I'm not to sure where those other command not found errors are coming from.

As you can see, I've put in some wait commands to try let the file manipulation complete but this doesn't seem to be working too well. Another thing I have already tried is to use shared memory /dev/shm but it yielded similar results.

How would I go about getting the results I want?

Note:

I am looking for answers that simply wrap the grep command as it has lots of functionality that I'm wanting to use and intend to insert other logic between the pipes, so I don't want to provide all of the search terms at once. I'm also not looking for other "grep like" tools. Sorry to @terdon who has already posted an awesome perl suggestion.

Answer 1

Here's a different approach. I have a little Perl script which I have already posted in another answer that will highlight the user provided patterns in different colors. A slightly modified version of the script will act like grep:

#!/usr/bin/env perl
use Getopt::Std;
use strict;
use Term::ANSIColor; 

my %opts;
getopts('hic:l:',\%opts);
    if ($opts{h}){
      print<<EoF; 
Use -l to specify the pattern(s) to highlight. To specify more than one 
pattern use commas. 

-l : A Perl regular expression to be colored. Multiple expressions can be
     passed as comma separated values: -l foo,bar,baz
-i : makes the search case sensitive
-c : comma separated list of colors;

EoF
      exit(0);
    }

my $case_sensitive=$opts{i}||undef;
my @color=('bold red','bold blue', 'bold yellow', 'bold green', 
       'bold magenta', 'bold cyan', 'yellow on_blue', 
       'bright_white on_yellow', 'bright_yellow on_red', 'white on_black');
if ($opts{c}) {
   @color=split(/,/,$opts{c});
}
my @patterns;
if($opts{l}){
     @patterns=split(/,/,$opts{l});
}
else{
    $patterns[0]='\*';
}

# Setting $| to non-zero forces a flush right away and after 
# every write or print on the currently selected output channel. 
$|=1;

while (my $line=<>) 
{ 
    my $want=0;
    for (my $c=0; $c<=$#patterns; $c++){
    if($case_sensitive){
        if($line=~/$patterns[$c]/){
           $line=~s/($patterns[$c])/color("$color[$c]").$1.color("reset")/ge;
           $want++;
        }
    }
    else{
        if($line=~/$patterns[$c]/i){
          $line=~s/($patterns[$c])/color("$color[$c]").$1.color("reset")/ige;
          $want++;
        }
      }
    }
print STDOUT $line if $want>0;
}

If you save that script as cgrep somewhere in your PATH and make it executable, you can specify up to 10 different patterns, each of which will be printed in a different color:

$ cgrep -h
Use -l to specify the pattern(s) to highlight. To specify more than one 
pattern use commas. 

-l : A Perl regular expression to be colored. Multiple expressions can be
     passed as comma separated values: -l foo,bar,baz
-i : makes the search case sensitive
-c : comma separated list of colors;

Answer 2

Each invocation of grep in a pipe runs in a separate shell, so you'll need to pass some state between them. The following solution is a crude way to handle that with a file that keeps the colour index and a lock file which ensures that simultaneous calls do not read the same value:

#!/usr/bin/env bash
color_index_file=~/.gitcolor
color_index_lock_file=/tmp/$(basename $0)

colors=()
for index in {31..34}
do
    colors+=("01;$index")
done

until mkdir "$color_index_lock_file" 2>/dev/null
do
    :
done

color_index=$(($(cat "$color_index_file" || echo 0) + 1))

if [[ $color_index -ge ${#colors[@]} ]]
then
    color_index=0
fi

printf "$color_index" > "$color_index_file"
rmdir "$color_index_lock_file"

GREP_COLORS="mt=01;${colors[$color_index]}" grep --color=always "$@"

Test assuming you've named your copy cgrep and put it on your PATH:

echo foobarbaz | cgrep foo | cgrep bar | cgrep baz

Answer 3

If you're good with regular expressions, You may want to check out grc and grcat. grc calls grcat.

grcat uses configuration files where you can add regular expressions to match text to be displayed in each color. It also has a number of other options. It defaults to colorizing system log files.

Depending on what you have in mind for your final script, you may be able to colorize your output with just one command.

The trick is specifying the regexes correctly for each "field" in your data source. This will be a lot easier if your data is relatively uniform in structure.

Last time I tried it, I didn't get too far, but I may have another go at it because I'm a bit better at regexes than I was then.

There is also the tput command which can be used to send information (like color changes) directly to your terminal device.

Both approaches are covered by the post below. It talks about the find command, but it can be applied to the output of any command.

Colored FIND output?

Answer 4

I've often needed this, and I tried several solutions, including scripts implemented in bash like hhighlighter. But none of the solutions I tried worked reliably when using multiple patterns, overlapping matches, or in other corner cases like single character matches.

So I decided to roll my own colorizer:

colorexp

• it has a defined colorization logic: last match determines the color
• if capturing groups are used, only group contents will be colored

N.B. In contrast to grep, it prints all input lines, not just matching ones. So if you need filtering use grep, then pipe into colorexp.

Examples

Basic Usage

• use the -h/-H options to only colorize the text, or only the background

Overlapping matches - last match wins

• all matches are colorized, and the color of the last match will be used

Capturing groups

• when using capturing groups, only the matched group contents will be colorized

Vary colors of groups in patterns

• when exactly one pattern is given, the default is to use different colors for each capturing group
  • in case of multiple patterns, the -G option can be used to enforce varying of the colors for each group

Use the same color for all groups of a pattern

• when multiple patterns are given, the default is to use the same colors for all capturing groups of a pattern
  • in case of a single pattern, the -g option can be used to enforce use of a single color