Skip to content

fix(selftest): make the pinned-gate-tests python shim an exec wrapper - #293

Merged
topcoder1 merged 1 commit into
mainfrom
claude/amazing-lehmann-b36509
Oct 8, 2026
Merged

topcoder1 merged 1 commit into
mainfrom
claude/amazing-lehmann-b36509

Conversation

@topcoder1

Copy link
Copy Markdown
Owner

Problem

On macOS, uv run pytest selftest/test_pinned_gate_tests_action.py fails 11 of 25 tests on every local run. That includes a clean origin/main (2815beb). Tests (Python) passes on CI. Every failure is the same error:

Error: Command '[.../runner_temp/pinned-gate-tests.XXXX/venv/bin/python', '-m', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

_run_action() made shim/python a symlink to sys.executable and put shim/ first on PATH. That makes the action's bare python -I -m venv "$venv" run the test's interpreter.

When CPython starts through a symlink (outside a macOS framework build), it keeps the symlink's own path as sys._base_executable. So venv writes home = <tmp>/shim into the new venv's pyvenv.cfg. That directory has no stdlib. The new venv's python looks for its stdlib starting from home, finds none there, and falls back:

Interpreter Fallback Result
CI's setup-python; Docker python:3.13 (/usr/local) compile-time prefix, which is where it is really installed works by accident
uv python-build-standalone with astral-sh/python-build-standalone#896 (merged 2025-12-05; seen in 3.13.12 and 3.13.16) patched getpath re-derives the directory from the symlink-resolved interpreter works
uv python-build-standalone without it (the 3.13.9 build uv cached here, built 2025-10-28) compile-time prefix /install Could not find platform independent libraries <prefix>, then No module named 'encodings', so ensurepip exits 1

So the bug is not specific to macOS. uv's cpython 3.13.9 fails the same 11 tests on Linux too.

Fix

  • The shim is now an exec wrapper. _write_python_shim() writes shim/python as #!/bin/sh + exec <realpath of sys.executable> "$@", with the path shlex.quoted. The interpreter starts from its real path, so venv records the real bin directory as home. No fallback is involved.
  • The action still gets the same interpreter. The old symlinked python also ran as the base interpreter, not inside the uv .venv: its pyvenv.cfg lookup happened next to the shim. Only the path venv sees has changed.
  • _run_action() now checks every venv the action built.
    • It checks that home is not the shim's directory, and that the python in home is this interpreter.
    • It compares interpreters, not directories. A macOS framework build (Homebrew) records /opt/homebrew/opt/python@3.13/bin as home whatever path it starts through, and that is correct there.
    • CI's interpreter hides the bug, so through a symlink shim the cases' own assertions stay green on CI. This check is what fails there, whether the symlink comes back in the helper or at _run_action's own call.
    • It reads each workdir's venv/pyvenv.cfg with no existence check, so if the action's layout changes the check fails loudly instead of being skipped.

The action itself is unchanged, and so are all existing assertions. The suite still has 25 tests.

Verification

All runs below are of the whole file. A mutant is the fixed file with the shim reverted to a symlink, in the helper or at _run_action's call.

Interpreter origin/main Fixed Helper back to symlink Call site back to symlink
macOS (Darwin 25.5, arm64), uv cpython 3.13.9 11 failed, 14 passed 25 passed 15 failed, 10 passed 15 failed, 10 passed
macOS, Homebrew 3.13.12 (framework build) — 25 passed 25 passed (bug can't occur) —
Linux (Docker python:3.13, arm64), /usr/local 3.13.16, standing in for setup-python 25 passed (bug hidden) 25 passed 15 failed, 10 passed 15 failed, 10 passed
Linux (Docker, arm64), uv cpython 3.13.9 11 failed, 14 passed (same ensurepip error) 25 passed — —
  • What the mutants fail on: every mutant failure is the new check: venv home is the python shim's dir. The 10 cases that still pass are input validation and structural checks; none of them builds a venv.
  • Exec-bit mutant (macOS): a shim left non-executable fails loudly (FileNotFoundError on the missing pyvenv.cfg).
  • ruff 0.15.10 (CI's pin): ruff format --check and ruff check are clean.
  • Docker runs: each copies the worktree from a read-only mount, then runs uv sync --frozen --all-extras and uv run --no-sync pytest, like Tests (Python) does.
  • Independent review: a code-reviewer lane returned COMMENT with three findings, all addressed in this version.
    • The first guard compared directories and failed falsely on Homebrew. It now compares interpreters.
    • The guard didn't cover _run_action's own call. The check now runs in _run_action.
    • Two docstring claims weren't true for framework builds. They are now qualified.
  • Codex pre-review: skipped; this is a selftest-only diff, below the threshold.

Not changed here: on origin/main, four must-fail cases passed on macOS only because the venv build failed: test_ungated_recursive_delete_fails and three rows of test_count_cases_fail. They now fail at the new check instead. They still assert only rc != 0, so a failure after a good venv build would still satisfy them (pip unable to install from the wheelhouse, for example). Tightening them is a separate change.

Auto-merge rationale: none. This is a manual merge, as for every topcoder1/ci-workflows PR (no automerge lane is wired on this repo). The diff is selftest-only (safe_test).

🤖 Generated with Claude Code

_run_action() made shim/python a symlink to sys.executable. Started
through a symlink, CPython keeps the symlink's path as
sys._base_executable, so venv wrote `home = <tmp>/shim` and the new
venv's python could find its stdlib only by fallback. CI's setup-python
falls back to a compile-time prefix that is its real install. uv's
python-build-standalone builds without astral-sh/python-build-standalone#896
(cpython 3.13.9, for one) fall back to /install and die with "No module
named 'encodings'", so 11 of 25 cases failed on every local macOS run,
and on Linux with that build too.

- _write_python_shim() writes `#!/bin/sh` + `exec <realpath of
  sys.executable> "$@"`, so venv records the real bin dir as home.
- _run_action() checks every venv the action built: home is not the
  shim's directory, and the python in home is this interpreter. CI masks
  the bug, so this check is what fails there if a symlink comes back,
  in the helper or at the call site.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the risk:safe_test Risk class: safe_test label Oct 8, 2026
@claude

claude Bot commented Oct 8, 2026

Copy link
Copy Markdown

No issues found. Small, well-reasoned fix replacing a symlink shim with an exec wrapper; the new assertion correctly checks the invariant it's meant to enforce.

@topcoder1
topcoder1 merged commit d34c7f6 into main Oct 8, 2026
17 checks passed
@topcoder1
topcoder1 deleted the claude/amazing-lehmann-b36509 branch October 8, 2026 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:safe_test Risk class: safe_test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant