[Bug]: Import task waits forever for memory when the container memory limit changes at runtime #54106

Environment

• Milvus Version: master (fb7d370050) and 3.0 (693665922d). The code is the same on both branches.
• Deployment Mode: any (seen in the integration-test mini cluster)
• MQ: any
• SDK: any
• OS: Ubuntu 22.04 (CI)

Reproduction

Option B: Steps

This problem does not occur in a normal deployment. It occurs only when the container memory limit of a running DataNode changes, and the process does not restart.

1. Start a DataNode with a container memory limit of 9 GiB. With the default dataNode.import.memoryLimitPercentage (10%), the import memory allocator sets its limit to ~921 MiB at its first use.
2. Raise the container memory limit of the running process to 20 GiB.
3. Run an import whose buffer is 1 GiB or more. The buffer is dataNode.import.readBufferSizeInMB (16 MiB) x vchannels x partitions. For example, 4 vchannels and 16 partitions give 1 GiB.
4. Observed via: the DataNode log shows task waiting for memory allocation... with usedMemory=0. The import job stays in Importing and never completes.

Trigger Conditions

• Frequency: always, when the memory limit increases after the DataNode creates the allocator and before the import task starts.
• Does NOT happen when: the container memory limit does not change while the process runs.

Expected Behavior

The import task gets its memory and the job completes.

Actual Behavior

The import task waits in BlockingAllocate forever. The job never leaves Importing.

Error Logs

From ciloop build #33913 (3.0):

[12:04:01] importv2/memory_allocator.go:60 "new import memory allocator" systemTotalMemory=9663676416
[12:07:13] importv2/task_import.go:185 "start to import" bufferSize=1073741824
[12:07:13] importv2/memory_allocator.go:79 "task waiting for memory allocation..." requestedSize=1073741824 usedMemory=0 availableMemory=966367641

Non-default Configuration

None.

Analysis Hints (Optional)

The allocator and the task read the memory limit at different times:

• GetMemoryAllocator() reads hardware.GetMemoryCount() one time, in sync.Once, and keeps the value.
• ImportTask.GetBufferSize() reads hardware.GetMemoryCount() again for each task, and caps the buffer at that limit.
• hardware.GetMemoryCount() reads the cgroup limit on each call.

If the limit increases between the two reads, the task cap is larger than the allocator limit. BlockingAllocate then waits for usedMemory + size <= limit. With size > limit this condition is never true, even when usedMemory is 0. cond.Wait() wakes up only on Release, so nothing can unblock it.

How CI hits it. ciloop runs UT-Integration in parallel with UT-CPP-Cov (INTEGRATION_OVERLAP=go-done, 2 shards). Each integration shard starts with 9 GiB. When UT-CPP-Cov completes, ciloop changes the limit of the running shards to 20 GiB (Promoted shard 1 to 12.0 CPU / 20480m memory). The Milvus processes do not restart. tests/integration/import TestBulkInsert/TestImportWithAFewRows fails with a 240s timeout when the promotion occurs after its DataNode starts and before the test starts its import.

• On 3.0, this occurred in 12 of 62 ciloop runs from 2026-10-08 to 2026-10-10 (for example [Bug]: [GOSDK] Delete without expr or ids gets unclear error message: delete plan can't be empty or always true #33761, [Bug]: Querynode memory usage is very unbalanced, some report memory quota exceeded. #33913, enhance: Upgrade go version from 1.20 to 1.21 #33940). The same signature (systemTotalMemory=9663676416, requestedSize=1073741824, availableMemory=966367641) is in each failed log.
• On master, it occurred in 0 of 196 runs. The C++ unit tests on master take longer, so the promotion occurs after the import package completes.

Related: the fileSize cap in GetBufferSize does not apply on the DataNode, because the import task never sets FileStats. Because of this, an 8 KB file gets a 1 GiB buffer. This is a separate problem.