Skip to content

[Bug]: container exec -i leaks one host PIPE descriptor per completed invocation (1.6.1) #2439

Open

Description

@nervouna

I have done the following

  • I have searched the existing issues.
  • I have reproduced this using a build of main.

This is a reproduction with the Homebrew 1.6.1 release binary. I inspected the corresponding source and the current main exit-cleanup code, but have not built or tested main.

Problem description

On macOS, each successfully completed container exec -i ... /bin/cat leaves one additional open PIPE file descriptor in the container's host container-runtime-linux process. The CLI and guest command have already exited. The descriptors remain after an idle wait.

Using the same /bin/cat command without -i does not increase the descriptor count. Two consecutive batches of eight interactive executions each increase both the total FD count and PIPE count by exactly eight.

Expected: the runtime releases stdin resources after an exec process exits and the input stream has reached EOF.

Steps to reproduce

Use a quiet, already-running Linux container with /bin/cat. The measurements below were taken on an existing, long-lived Ubuntu 24.04 ARM64-based custom image, not a newly created stock-image container. The reproduction itself only runs /bin/cat; it does not invoke any application client, access application files, or need networking.

  1. Identify the host container-runtime-linux PID belonging to that container, for example using pgrep -fl container-runtime-linux. Use the runtime PID, not a container exec CLI PID or a guest PID.
  2. Save the script below as repro.py.
  3. Run python3 repro.py CONTAINER_ID RUNTIME_PID.

It performs eight noninteractive executions as a control, then two batches of eight interactive executions. Every command must exit successfully, and the interactive output must match the fixed input. Each batch is followed by a two-second wait, with an additional five-second wait at the end. Only descriptor counts are printed.

import json
import subprocess
import sys
import time

container_id, runtime_pid = sys.argv[1:3]


def snapshot(label):
    output = subprocess.check_output(
        ["/usr/sbin/lsof", "-nP", "-a", "-p", runtime_pid, "-Fft"],
        text=True,
        timeout=10,
    )
    lines = output.splitlines()
    result = {
        "fds": sum(line.startswith("f") and line[1:].isdigit() for line in lines),
        "pipes": sum(line == "tPIPE" for line in lines),
    }
    if result["fds"] == 0:
        raise RuntimeError("No descriptors found; check the runtime PID")
    print(label, json.dumps(result), flush=True)
    return result


def run_batch(interactive):
    for _ in range(8):
        args = ["container", "exec"] + (["-i"] if interactive else [])
        args += [container_id, "/bin/cat"]
        result = subprocess.run(
            args,
            input=b"fd-probe\n" if interactive else b"",
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            check=True,
            timeout=10,
        )
        expected = b"fd-probe\n" if interactive else b""
        assert result.stdout == expected, "Unexpected cat output"
    time.sleep(2)


before = snapshot("baseline")
run_batch(False)
control = snapshot("after_8_noninteractive")
run_batch(True)
first = snapshot("after_8_interactive")
run_batch(True)
second = snapshot("after_16_interactive")
time.sleep(5)
final = snapshot("after_5s_idle")
print("deltas", json.dumps({
    "noninteractive_fds": control["fds"] - before["fds"],
    "first_interactive_fds": first["fds"] - control["fds"],
    "second_interactive_fds": second["fds"] - first["fds"],
    "interactive_pipes": final["pipes"] - control["pipes"],
}), flush=True)

Measured leak evidence — confirmed

Output from running that script against the same runtime throughout:

baseline {"fds": 8367, "pipes": 8294}
after_8_noninteractive {"fds": 8367, "pipes": 8294}
after_8_interactive {"fds": 8375, "pipes": 8302}
after_16_interactive {"fds": 8383, "pipes": 8310}
after_5s_idle {"fds": 8383, "pipes": 8310}
deltas {"noninteractive_fds": 0, "first_interactive_fds": 8, "second_interactive_fds": 8, "interactive_pipes": 16}

The baseline was already high because this is a long-lived runtime; the reproducible result is the per-batch delta. An earlier independent check also showed +8 PIPE descriptors for eight interactive executions, versus no increase for twelve noninteractive /bin/true executions.

Relevant source — suspected cleanup gap

In 1.6.1 RuntimeService.startExecProcess, the exit wait handler closes processInfo.io[1] and processInfo.io[2], but does not close processInfo.io[0]. In the same file, FileHandle.stream() clears the readability handler and finishes the stream at EOF, without explicitly closing the handle.

This is a likely ownership/cleanup gap consistent with the measured leak, not a verified patch. The appropriate fix should account for stdin ownership and stream teardown on normal exit, errors, and cancellation.

Operational impact and workaround

A caller issuing one short interactive exec every ten seconds can accumulate roughly 8,640 descriptors per day. The runtime sets its soft and hard RLIMIT_NOFILE to 65,536.

As an application-side workaround, I switched repeated local RPC requests to a single long-lived interactive exec stream. After warming up that stream, 1,000 read-only RPC requests left the runtime FD count unchanged at 8,364. Normal ten-second polling then kept the total at 8,365 throughout a separate 60-second observation. The container and guest service were not restarted. This avoids repeated exec creation; it does not fix the runtime leak or reclaim previously retained descriptors.

Historical crash — correlation only, not a reproduced crash

A separate, older crash report from the same host records EXC_BREAKPOINT / SIGTRAP after approximately 650,616 seconds of runtime (about 7.53 days). Sanitized relevant frames:

Base::log_message_and_trap(...)
Base::report_fixme_and_trap(...)
-[VZVirtioSocketDevice connectToPort:completionHandler:] + 840
VZVirtualMachine.connect(queue:port:)
VZVirtualMachineInstance.dialAgent()
LinuxContainer.exec(_:configuration:)

Inspection of the same installed Virtualization.framework build maps the +840 return location to a trap following a failed socketpair call. Together with the measured leak and the runtime's descriptor limit, this makes descriptor exhaustion a plausible explanation for that older crash.

However:

  • The historical runtime binary differs from the currently tested 1.6.1 binary, and its exact version has not been established. The historical crash is not being reported as a confirmed 1.6.1 crash.
  • The historical errno and FD count at the moment of failure are unavailable.
  • I have not deliberately driven the runtime to its FD limit or reproduced the crash.
  • The stack alone does not establish whether the initiating failure belonged to container, Containerization, Virtualization.framework, or another resource condition.

The confirmed bug in this report is the repeatable PIPE descriptor retention on 1.6.1. The older crash is included only as possible operational impact.

Environment

  • OS: macOS 27.0.1 (26A434), Apple Silicon / arm64.
  • Container client and server: 1.6.1, Homebrew release build.
  • Xcode installed: 27.1 (27A9275); not used to build the tested release binary.
  • Guest: Ubuntu 24.04-based ARM64 custom image, 1 vCPU / 1 GiB; no TTY requested for the repro.
  • Virtualization.framework in the historical report: 308.1.7.

Only aggregate counts, generic commands, software versions, and selected symbols are included. No raw crash report, usernames, private paths, container/image names, process IDs, UUIDs, credentials, or application payloads are attached.

Code of Conduct

  • I agree to follow this project's Code of Conduct.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions