Repository navigation
Retry functionality fix - #180
renat-sabitov wants to merge 1 commit into
Conversation
related to mitchellh#24 This patch fixes a number of issues with regards to retry functionality: 1. last retry wiping out responce body, which resulted in no error reported 2. retry interval was too small (100 ms), increased to 500 ms avg 3. not enough retries 4. No randomness in exp backoff We had a long history of problems while baking 10 AMIs in a time, this fix along with some fix in packer resolves the issues
d178fad to
72e817f
Compare
There was a problem hiding this comment.
Does this really work better than just changing the 100 in ExpBackoff to 500? (Along with upping MaxTries to 10.)
There was a problem hiding this comment.
The idea is that If you have multiple clients trying to call AWS API, they tend to experience throttling at the same time. Then they wait for exactly the same amount of time and retry, which lead to another round of throttling. Repeat for next retries.
Randomness doesn't allow retries to line up.
|
FYI the official AWS Architecture Blog recently published some tips on backoff algorithms that do the fewest calls with concurrent clients, and jittered wins: The simulation code for 100 clients and a DynamoDB-like Optimistic concurrency control (OCC) server in python is here: The context is DynamoDB but the algorithm is generic. So yes based on the simulation data, adding randomness works better than just increasing the interval. |
|
Is there a status on this PR? We are running into this problem also, and this solution will help us Thx |
|
Same issue here and this seems like a solid solution. @mitchellh Anything else that needs to be done here to get this merged? |
related to #24 and hashicorp/packer#1539
This patch fixes a number of issues with regards to retry functionality:
We had a long history of problems while baking 10 AMIs in a time, this fix
along with some fixes in packer resolves the issues