Repository navigation
Reproduce the issue #14
Description
I am facing an OOM (Out Of Memory) issue on an 8-GPU machine with 80GB of memory. The single-node server is consuming a large amount of GPU memory. How can I resolve this?
Metadata
Metadata
Assignees
Labels
No labels
Activity
Luciennnnnnn commentedon Nov 23, 2025
Hi, are you encountering OOM issues when starting the reward server or during RL training? Which model size are you using? Could you please share the detailed configuration so that we can better assist you.
bxhsort commentedon Nov 24, 2025
I trained using only 8 A800 GPUs and the EditScore Qwen2.5-VL 7B version. I found that when starting the server, it already consumes over 55 GB of GPU memory, which prevents OmniGen2’s RL training from running — it OOMs as soon as the server starts. If I reduce the batch size, the code runs normally.
I’d like to ask: when you mention “single machine training,” does that refer specifically to the machine used for RL training and not the machine running the server? If I run in a single-machine setup, how many GPUs should I allocate to the server in order to obtain a normal score?
Luciennnnnnn commentedon Nov 24, 2025
Hi, in our setup, the reward server runs on a separate machine. This means training typically involves at least two machines, and it becomes quite complicated if you want the policy model and the reward model to share GPU memory.
If you only have a single machine available, I would suggest allocating 2 GPUs for the reward server and 6 GPUs for the main training, or alternatively 4 GPUs for the reward server and 4 GPUs for the main training.
To support this, you will need to make a small modification to the reward server code.
bxhsort commentedon Nov 26, 2025
Luciennnnnnn commentedon Nov 26, 2025
This modification is token from TempFlow-GRPO, other parts remains Flow-GRPO.
bxhsort commentedon Nov 26, 2025
Thank you for your reply. When I train using 6 GPUs with sampling, the ratio stays at 1 and does not change.