Failure Rollout Data Problem #19

Hi, I want to ask about your pattern for collecting failure rollouts.

Do you stop data collection immediately once a failure is detected, or do you wait for some time to let the model retry? If you do wait, how long is the waiting period? Also, do we need to pay attention to the rollout length? Roughly what is the ratio between successful and failed rollouts in your data?

I’m trying to reproduce the method on our task, but the value model does not seem to perform very well. I suspect the issue may come from the quality of the collected rollout data, so I wanted to ask for clarification.

It would also be very helpful if you could upload some failed rollout examples to the sample dataset.

Thanks!