If i run a script that processes a path recursively using parallel processing to execute a command on the whole tree if items, am I guaranteed that the commands I use will be executed on all items in a single directory before stepping to the next one?
Get-ChildItem $Path -Recurse -Filter *.some_file_extension | ForEach-Object -Parallel {
my_command $_ --with_options
}
As per the MS Docs on regarding the feature and use case:
https://devblogs.microsoft.com/powershell/powershell-foreach-object-parallel-feature
Normally, when you use the ForEach-Object cmdlet, each object piped to the cmdlet is processed sequentially.
1..5 |
ForEach-Object { "Hello $_"; sleep 1 }
Hello 1
Hello 2
Hello 3
Hello 4
Hello 5
(Measure-Command {
1..5 | ForEach-Object { "Hello $_"; sleep 1 }
}).Seconds
5
But with the new ForEach-Object -Parallel parameter set, you can run all script in parallel for each piped input object.
1..5 |
ForEach-Object -Parallel { "Hello $_"; sleep 1; } -ThrottleLimit 5
Hello 1
Hello 3
Hello 2
Hello 4
Hello 5
(Measure-Command {
1..5 |
ForEach-Object -Parallel { "Hello $_"; sleep 1; } -ThrottleLimit 5
}).Seconds
1
Because each script block in the ForEach-Object example above takes 1 second to run, running all five in parallel takes only one second instead of 5 seconds when run sequentially.
Since the script blocks are run in parallel for each of the 1-5 piped input integers, the order of execution is not guaranteed. The -ThrottleLimit parameter limits the number of script blocks running in parallel at a given time, and its default value is 5.
