You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The test scripts use random number generator to e.g. change the sleep times.
This is wonderful for stress tests, but a really bad idea for CI and daily tests, since by definition these tests cannot be reproduced UNLESS the seed is known.
As suggested by @xiulipan : the best option is a command flag that lets the caller choose between constant seed parameter (with a default value) versus truly random seed.
My suggestions:
The test should always print the seed used in any case so it can be re-used.
clearly label CI results using a random seed, e.g. "stress test" not to surprise users.
Activity
marc-hb commentedon Jul 31, 2020
As suggested by @xiulipan : the best option is a command flag that lets the caller choose between constant seed parameter (with a default value) versus truly random seed.
My suggestions:
xiulipan commentedon Jul 31, 2020
thanks @marc-hb for the suggestion. I will take them with the new feature implement.
Another feature:
marc-hb commentedon Jul 31, 2020
Yes that's what I meant by "... so it can be re-used". Rephrasing the previous comment.