Repository navigation
[Question]:when conducting a knowledge base search in the chat interface, it was found that ollama has a high cpu usage rate? #7881
Description
Activity
Based on the information available, the high CPU usage and empty GPU load during knowledge base searches with the Ollama model in the RAGFlow chat interface may be due to the Ollama model not utilizing the GPU as expected. This can happen if the environment is not configured to force all model layers to run on the GPU—by default, some layers may fall back to the CPU, leading to high CPU usage and slow performance.
To address this, you can try setting the following environment variables before launching the Ollama service:
export OLLAMA_NUM_GPU=999 export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1 export no_proxy=localhost,127.0.0.1 export ZES_ENABLE_SYSMAN=1 export SYCL_CACHE_PERSISTENT=1
These settings are especially important if you are using Intel Arc™ A-Series Graphics on Linux (Kernel 6.2 or newer). Setting
OLLAMA_NUM_GPU=999ensures all model layers use the GPU, which should reduce CPU load and improve performance. More details can be found in the official deployment guide.If you have already set these variables and are still experiencing issues, there may be other factors at play, such as software limitations, model compatibility, or integration details within RAGFlow. At this time, there is no additional documentation or issue thread with a confirmed solution for this specific problem.
If you need further troubleshooting, please provide details about your hardware, operating system, and the exact Ollama model you are using.
To reply, just mention @dosu.
Share context across your team and agents. Try Dosu.
Reacted by wenqiangleo@KevinHuSh I found the same problem after updating ragflow to 0.19.0. When using agent based on models in ollama, the GPU is 0 but CPU is extremly high. What's going on? There wasn't such situation before updating.
SAME PROBLEM
I have tried a temporary solution, OLLAMA_NUM_GPU=999 in the ollama service,But I don't know the root cause
/etc/systemd/system/ollama.service
[Unit]
Description=Ollama Service
After=network-online.target[Service]
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
.........- Environment="OLLAMA_NUM_GPU=999"
/etc/systemd/system/ollama.service
[Unit] Description=Ollama Service After=network-online.target
[Service] ExecStart=/usr/local/bin/ollama serve User=ollama Group=ollama .........
- Environment="OLLAMA_NUM_GPU=999"
Mine is on windows and i have tried to add enviornment, but the problem is still happening.
"OLLAMA_NUM_GPU=999" doesn't work. Even if this option is set, this problem will recur
The high CPU usage and empty GPU load with Ollama models in RAGFlow after updating to 0.19.0 is likely due to how GPU resources are managed in newer versions. The RAGFlow codebase itself does not control whether Ollama uses the CPU or GPU; it delegates all hardware decisions to the Ollama backend via its API. This means that if Ollama is not configured or able to use the GPU, all inference will run on the CPU, causing the symptoms you describe.
One known cause is the context size (
num_ctxparameter). If this is set too high (for example, 32768), Ollama may run into GPU resource constraints and fall back to CPU, especially in newer versions after v0.17.2. Reducing or removing thenum_ctxparameter can resolve this and allow the GPU to be used again. This issue is documented in recent discussions and is specific to newer versions of RAGFlow and Ollama models (source).Other performance tips include disabling the reranker feature, turning off "Use RAPTOR to enhance retrieval," and disabling "Auto-keyword" and "Auto-question" features, as these can all increase CPU load if left enabled (source).
If you have already set environment variables like
OLLAMA_NUM_GPU=999and the problem persists, double-check the Ollama backend logs and configuration. The RAGFlow client code does not enforce or log the backend device status, so any device selection issues must be resolved in the Ollama server environment.If you continue to see high CPU usage and no GPU load, try lowering or commenting out the
num_ctxparameter in your Ollama model configuration and restart the Ollama service. This has resolved similar issues for other users in recent versions (source).Let me know if you need more specific troubleshooting steps or have questions about your configuration.
To reply, just mention @dosu.
Docs are dead. Just use Dosu.
class OllamaChat(Base):
def init(self, key, model_name, **kwargs):
super().init(key, model_name, base_url=None)self.client = Client(host=kwargs["base_url"]) if not key or key == "x" else Client(host=kwargs["base_url"], headers={"Authorization": f"Bearer {key}"}) self.model_name = model_name def chat(self, system, history, gen_conf): if system: history.insert(0, {"role": "system", "content": system}) if "max_tokens" in gen_conf: del gen_conf["max_tokens"] try: # Calculate context size ctx_size = self._calculate_dynamic_ctx(history)Now the code is calculated dynamically. How can I reduce num_ctx @dosubot
The context size (
num_ctx) in your OllamaChat class is determined by the_calculate_dynamic_ctxmethod, which sums up the estimated token count for all messages in the history, applies a 1.2x buffer, and then rounds up to the nearest multiple of 8192 tokens. This logic is hardcoded and not configurable via environment variables or runtime parameters. To reducenum_ctx, you have two main options:- Reduce the message history size or content length: Fewer or shorter messages in
historywill directly lower the calculated context size. - Modify the source code: You can edit the
_calculate_dynamic_ctxmethod to use a smaller buffer ratio (for example, change1.2to1.0), or set a lower minimum block size (change8192to a smaller value), or even cap the maximum context size. For example:
# In rag/llm/chat_model.py def _calculate_dynamic_ctx(self, history): ... # Apply a smaller buffer ratio total_tokens_with_buffer = int(total_tokens * 1.0) # Lower the minimum context size if total_tokens_with_buffer <= 4096: ctx_size = 4096 else: ctx_multiplier = (total_tokens_with_buffer // 4096) + 1 ctx_size = ctx_multiplier * 4096 # Optionally, cap the maximum context size ctx_size = min(ctx_size, 8192) return ctx_size
Reducing
num_ctxcan help prevent high CPU usage and force Ollama to use the GPU, especially if your hardware has limited GPU memory. This approach has resolved similar issues for other users when large context sizes caused Ollama to fall back to CPU-only inference Reference.For further performance improvements, consider disabling features like the reranker, RAPTOR retrieval, auto-keyword, and auto-question, as these can also increase CPU load Reference.
To reply, just mention @dosu.
Share context across your team and agents. Try Dosu.
- Reduce the message history size or content length: Fewer or shorter messages in
To reduce the context size (
num_ctx) in your Docker-based RAGFlow environment, you need to modify the_calculate_dynamic_ctxmethod in the Python source code, then rebuild and redeploy your Docker image. Here’s how you can do it:- Edit the file
rag/llm/chat_model.pyin your local copy of the RAGFlow source. Locate the_calculate_dynamic_ctxmethod in theOllamaChatclass. Adjust the buffer ratio, block size, or cap the maximum context size. For example, you might change:
# Original logic total_tokens_with_buffer = int(total_tokens * 1.2) if total_tokens_with_buffer <= 8192: ctx_size = 8192 else: ctx_multiplier = (total_tokens_with_buffer // 8192) + 1 ctx_size = ctx_multiplier * 8192
to something like:
total_tokens_with_buffer = int(total_tokens * 1.0) if total_tokens_with_buffer <= 4096: ctx_size = 4096 else: ctx_multiplier = (total_tokens_with_buffer // 4096) + 1 ctx_size = ctx_multiplier * 4096 ctx_size = min(ctx_size, 8192) # Optional cap
This will reduce the context size and help avoid high CPU usage and CPU fallback when GPU memory is limited (reference).
-
Save your changes.
-
Rebuild your Docker image. In your project root, run:
docker build -t your-custom-ragflow .or, if you use Docker Compose:
docker-compose build
This ensures your code changes are included in the new image (reference).
- Restart your container using the new image:
docker-compose up -d
or with plain Docker:
docker run ... your-custom-ragflow
No Docker-specific environment variables are needed for this change; the adjustment is purely in the Python code. After redeploying, the new context size logic will take effect for all OllamaChat operations in your container.
If you use a prebuilt image and cannot rebuild, you would need to mount the modified Python file into the container, but rebuilding is the recommended and cleanest approach.
To reply, just mention @dosu.
Share context across your team and agents. Try Dosu.
- Edit the file
It should be a bug, perhaps will be fixed. @KevinHuSh
What about running Ollama with
OLLAMA_KEEP_ALIVE=-1?ok, I'll have a try. thanks.
i handled my problem after setting OLLAMA_NUM_GPU=1 and updating ollama to latest version
What about running Ollama with
OLLAMA_KEEP_ALIVE=-1?Mine cannot set this because the space for GPU is limited and I have other use for other software which have to free the model of ollama, is there any other plan? @KevinHuSh
i handled my problem after setting OLLAMA_NUM_GPU=1 and updating ollama to latest version
my ollama version is 0.9.0. I will try upgrading ragflow again by setting this.
The latest ollama has just been updated. The displayed version number is 0.9.0. I will test OLLAMA_NUM_GPU=1
i handled my problem after setting OLLAMA_NUM_GPU=1 and updating ollama to latest version
I tried. It doesn't work.
The latest ollama has just been updated. The displayed version number is 0.9.0. I will test OLLAMA_NUM_GPU=1
Have you tried this plan? Mine didn't work.
To be honest, i even dont know how it works etheir, this problem appeared when i updated ollama to 0.7.0 and suddenly disappeared... maybe you can check ollama issues webpage and find something useful
To be honest, i even dont know how it works etheir, this problem appeared when i updated ollama to 0.7.0 and suddenly disappeared... maybe you can check ollama issues webpage and find something useful
my ollama now is 0.9.0 and there is no such problem for ragflow 0.17. When upgrading to 0.19, the problem appears. I have tried twice.
@KevinHuSh
Self Checks
Describe your problem
Currently, when conducting a knowledge base search in the chat interface, it was found that ollama has a high cpu usage rate, but the GPU load is indeed empty, and the search takes more than ten minutes to complete. What happened during this process? Why does such a phenomenon occur?