Performing bash-like string manipulation in GNU parallel

Question

I like GNU parallel, but I find performing string replacements rather complex.

I'm used to simple manipulation such as done in Bash:

$ f="encodes/TearsOfSteel-720-crf41.mp4"
$ echo "$(basename "${f%%-*}")"
TearsOfSteel

Basically, I want to find the basename (everything before the first minus-hyphen) of each input file. With parallel, I'd then want to perform a command with the original filename and the basename:

parallel './do_something {} <basename>' ::: encodes/*.mp4

… where <basename> is what Bash would do if you gave it basename "${f%%-*}", and f obviously being {}.

The non-parallel version would be:

for f in encodes/*.mp4; do
  bn="$(basename "${f%%-*}")"
  ./do_something "$f" "$bn"
done

How can I do that with parallel?

Answer 1

If you do that a lot define your own replacement string by putting this into ~/.parallel/config:

--rpl '{_} s:.*/::; s:-.*::;'

Then you can do:

parallel ./do_something {} {_} ::: encodes/*.mp4

Also remember you can just define a bash function and call that:

doit() {
  f="$1"
  bn="$(basename "${f%%-*}")"
  ./do_something "$f" "$bn"
}
export -f doit
parallel doit {} ::: encodes/*.mp4

Answer 2

I haven't found an easy way to do Bash-style manipulation, but you can use perl expressions to perform string replacements. It's simple enough to remove parts of the string you don't want, subsequently.

One way to do that would be:

parallel "./do_something {} {= s:.*\/::; s:-.*:: =}" ::: encodes/*.mp4

How does that work?

You can run perl commands inside parallel with {= … =}. With perl, you can remove the folder name with the expression:

s/.*\///

That means: replace everything up to the first slash with nothing. We can swap the delimiters from / to : to make it more readable (s:.*\/::).

You can do the same with the trailing part, i.e., removing everything from the first minus-hyphen onwards, which is:

s:-.*::

The whole command is a combination of these two replacements, separated by a semicolon. The command then becomes:

{= s:.*\/::; s:-.*:: =}