bisect-kit: Retry when "repo sync" encounters HTTP 429 error
When many bisectors are executing "repo sync" concurrently, it can reach
the git server quota and HTTP 429 error is returned.
Here we implement a radom sleep + exponential backoff mechanism to
retry. In each retry, a random number is drawn from an exponential
distribution. The mean of distribution grows exponentially when HTTP 429
happens.
The total repo sync time is still capped by a max 3 hours limit,
including the cool down time.
BUG=b:308913816
TEST=./setup_cros_bisect.py sync
Change-Id: Ie7882316213dcff2875ff53f03ff301419a34abf
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/bisect-kit/+/5605862
Reviewed-by: MD Nayeem Jahan Rafi <njrafi@chromium.org>
Commit-Queue: Cheng-Yu Lee <cylee@google.com>
Tested-by: Cheng-Yu Lee <cylee@google.com>
2 files changed
