Running a program with multiple arguments from list using parallel

Question

I have a bash program that I run via command line (Ubuntu) like this:

./extract_field.sh ABC001

where ABC001 is the field ID that I want to extract from a given shapefile.

To run this script with multiple IDs, I first save one ID per line in a list.txt file:

ABC001
ABC014
ABC213
ABC427

and then invoke the script using parallel:

parallel -a list.txt ./extract_field.sh

So far so good.

However, I plan to change extract_field.sh so it takes two arguments rather than only one. Will the above workflow still work if I just change my text file to accommodate two arguments per line like this?

ABC001 arg2a
ABC014 arg2b
ABC213 arg2c
ABC427 arg2d

With this change, I would expect parallel -a list.txt ./extract_field.sh to behave like

./extract_field.sh ABC001 arg2a
./extract_field.sh ABC014 arg2b

and so on.

Is that right?

I could just test it before asking, but I decided to ask first since this change in the script will probably take me a couple of hours to finish (though it sounds like a simple change).

Answer 1

You can provide multiple arguments to a single command with parallel by specifying a column delimiter in your command syntax To use your example:

parallel --colsep ' ' -a list.txt ./extractfield.sh {1} {2}

Will provide the result of

./extract_field.sh ABC001 arg2a
./extract_field.sh ABC014 arg2b

Given that your file list.txt contains

ABC001 arg2a
ABC014 arg2b

You can test this with cp or mv since these both require multiple positional parameters.

Useful bit of parallel's manpage