Repository navigation
🐛 fix(discovery): let -p replace the python set in config or env - #3494
Merged
gaborbernat merged 2 commits intoOct 30, 2026
Merged
Conversation
--python and --try-first-with were plain append actions, so a value from virtualenv.ini or VIRTUALENV_PYTHON became the starting list and the command-line values were added after it. Discovery takes the first spec that resolves, so with python = 3.13 in virtualenv.ini, virtualenv -p 3.14 env built a 3.13 environment, and -p 3.12 on a machine without 3.12 succeeded with 3.13 instead of reporting that no interpreter was found. Builtin worked around this for the env var alone by rotating the list by one whenever VIRTUALENV_PYTHON was set. That covered a single env value, but with several values it moved the second one to the front, with or without -p, and it never helped the config file. Both options now use the ExtendAction from pypa#3440, which already does this for --extra-search-dir: the first command-line value replaces the configured list and later ones are added in order. nargs=1 hands the action a one-item list per flag, since it extends with whatever it receives. A configured list is still tried in order when the command line gives none, and several -p are still all tried in order. The rotation in Builtin is gone because the parser now gets the order right. One visible change follows from the precedence: a -p that matches nothing now fails the same way it does without any configuration, instead of falling back to VIRTUALENV_PYTHON.
darrenhuai
marked this pull request as ready for review
October 30, 2026 01:47
Document that -p and the other multi-value options replace the configured list, and why --python needs nargs=1 with ExtendAction. The precedence test becomes a table of expected discovery calls with an autospec mock, so it has no branches.
EliahKagan
added a commit
to EliahKagan/GitPython
that referenced
this pull request
Oct 31, 2026
The `ruff`, `format`, `mypy`, `html`, and `misc` environments are each
meant to run once, on the first available interpreter in a preference
order: Python 3.9 first, then 3.10, 3.11, 3.12, and 3.8. They wrote
that as `base_python = py{39,310,311,312,38}`, a form introduced in
62c024e (gitpython-developers#1693).
tox expands braces in generative environment lists and section names,
such as `env_list = py{38,...}` and `[testenv:py{38,39,310}]` in this
file, and in factor conditions, but not in the values of settings
like `base_python`. There, it just splits on commas, giving the five
specs `py{39`, `310`, `311`, `312`, and `38}`. The first and last are
not valid interpreter specs, so 3.9 and 3.8 were never candidates.
The bare `310`, `311`, and `312` are accepted as version specs, so the
environments still ran if 3.10, 3.11, or 3.12 was found. If none was,
the environment did not run: with tox 4.11.3 running on 3.13 and only
3.9 on `PATH`, it reported "skipped because could not find python
interpreter with spec(s): py{39, 310, 311, 312, 38}". Since tox 4.56.0
changed the default of `skip_missing_interpreters` to false, current
tox fails it instead.
Write each list as `py39, py310, py311, py312, py313, py314, py315`,
the documented form for several `base_python` values, of which the
first one found is used. This writes the intended order, except that
3.8 is left out, and adds the newer versions at the end. virtualenv
20.29 to 21.14 do not try them in that order, as described below.
62c024e listed as fallbacks all the Python versions the project
supported, but the lists were not extended when da58f39 (gitpython-developers#2196) added
3.13, 3.14, and 3.15 to `env_list`. So on a system with only 3.13 or
later, these environments could not run at all. With only 3.13, 3.14,
and 3.15 available, a probe with `py39, py310, py311, py312` was skipped
with tox 4.23.2 and virtualenv 20.28.1 and failed with tox 4.67.1 and
virtualenv 21.14.13, while the full list ran on 3.13 with both.
The brace form never selected 3.8, so leaving it out takes away
nothing that worked, and listing it could cause failures. Since
virtualenv 21.5.0, virtualenv cannot create Python 3.8 environments.
tox 4.56.0 and later work around that by bootstrapping an older
virtualenv, but only when the installed virtualenv can create none of
the `base_python` candidates, which is not the case for a list that
also names newer versions. virtualenv 20.29 to 21.14 also try the
first listed interpreter last, as described below, so 3.8 would be
tried before 3.9. With tox 4.67.1 and virtualenv 21.14.13 and only
3.8 and 3.9 available, a probe with `py39, py310, py311, py312, py38`
accepted 3.8 and then failed with "could not find python interpreter
matching any of the specs". With `py39, py310, py311, py312`, it ran
on 3.9.
Even where 3.8 environments can be created, `mypy` and `html` would
not work as intended on 3.8. The `test` extra installs mypy only on
3.9 and later, so the `mypy` environment would not have the `mypy`
command it runs, and `ignore_outcome` would turn that failure into a
warning. The `doc` group's Sphinx requirement also applies only on
3.9 and later, so `html` would build with whatever Sphinx version the
other documentation dependencies happened to pull in.
Checked with a minimal tox project whose only environment prints its
interpreter's version, with Python 3.9 and 3.12 on `PATH` and tox
itself running on 3.13.
| Tooling | Brace form | Comma form |
|----------------------------------|------------|------------|
| tox 4.11.3, virtualenv 20.24.5 | 3.12 | 3.9 |
| tox 4.23.2, virtualenv 20.28.1 | 3.12 | 3.9 |
`tox config` shows the intended entries for each environment.
With tox 4.23.2 and virtualenv 20.28.1, `tox -e ruff` and
`tox -e format` with this `tox.ini` ran on Python 3.9.6 and passed.
With tox 4.67.1, each of the five environments also ran and passed on
each of Python 3.13.17, 3.14.9, and 3.15.1, selected with
`-x testenv:<name>.base_python=py313` and so on. `html` reports no
Sphinx warnings on 3.13, but 13 on each of 3.14 and 3.15, all about
`__annotate_func__`, which Python 3.14 adds to the `__dict__` of
classes with annotations. The build used Sphinx 7.4.7, as the `doc`
group requires Sphinx below 8. CI's documentation build on 3.14
reports the same 13 warnings.
virtualenv 20.29 to 21.14 do not keep this order. tox passes the
`base_python` list to virtualenv in `VIRTUALENV_PYTHON`, and when that
variable is set, those versions move the first spec to the end of the
list. That change, pypa/virtualenv@bc7a91a5 (pypa/virtualenv#2821), was
meant to let a `--python` option take precedence over a single
interpreter set in the variable. So with tox 4.67.1 and virtualenv
21.14.13, the first listed interpreter was tried last: a probe with
`py39, py312, py311` ran on 3.12, and one with `py312, py39` ran on
3.9. virtualenv 21.15.2 tries the values in the order given again
(pypa/virtualenv#3494). With it and tox 4.67.1, a probe with
`py39, py313` ran on 3.9, where virtualenv 21.14.13 ran it on 3.13.
This was a problem in how tox and virtualenv interacted, not in this
configuration. With the affected virtualenv versions, these
environments are tried on 3.9 only if none of 3.10 through 3.15 is
found.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With
python = 3.13invirtualenv.ini, a-pon the command line loses to the config file. This machine has 3.13 and 3.14 installed and no 3.12. Run against main:Without the config file,
-p 3.14builds 3.14.3 and-p 3.12stops withfailed to find interpreter for Builtin discover of python_spec='3.12'. The env var goes wrong in a different way:VIRTUALENV_PYTHON=3.14,3.13with no-pat all builds 3.13, so the second value wins.-p/--pythonand--try-first-withareappendactions._fix_defaultseeds the namespace with the env var or ini value, argparse appends the command-line values after it, andBuiltin.runreturns the first spec that resolves, so the configured interpreter wins whenever it exists. #2821 patched this forVIRTUALENV_PYTHONonly, by rotatingpython_specby one whenever the variable is set. That puts a single env value behind-p, but with two or more env values it moves the second one to the front, and it never covered the config file. tox passes itsbase_pythonlist as a newline-joinedVIRTUALENV_PYTHONwith no-p, so a tox env with more than onebase_pythongets the rotated order too. In #2285, the issue #2821 closed, the answer was that a CLI value should overwrite the env var.Both options now use the
ExtendActionfrom #3440, which already does this for--extra-search-dir: the first command-line value replaces the configured list and later ones are appended in order.nargs=1is there becauseExtendActionextends the list with whatever it receives, and a bare3.14would otherwise be split into characters. Since the parser now gets the order right, the rotation inBuiltinis gone. A configured list is still tried in order when there is no-p, several-pare still all tried in order, and the help output doesn't change.One behavior does change:
VIRTUALENV_PYTHON=3.13plus a-pthat matches nothing used to fall back to 3.13 and now fails, the same as without the env var. That follows the documented priority (command line, then env vars, then the config file), and the changelog fragment says so. With this branch the two runs above build 3.14.3 and fail withfailed to find interpreter,VIRTUALENV_PYTHON=3.14,3.13builds 3.14.3, and the ini on its own still builds 3.13.7.test_cli_replaces_configured_interpretersrecords what discovery tries for env and ini values with zero, one or two values on the command line, for both options. On main it fails whenever the command line has a value, and for two env values without one. Theenv-var-and-clicase intest_returns_python_specifiedencoded the old merged list plus the rotation, so I replaced it with anenv-var-multiplecase that expects the first env value.Thanks for contributing, make sure you address all the checklists (for details on how see development documentation)
ran the linter to address style issues (
tox -e fix). I ran the same hooks throughprekon the changed files; the semgrep hook only looks atactivate.sh.wrote descriptive pull request text
ensured there are test(s) validating the fix
added a
docs/changelog/+short-description.<type>.rstfragment for each user-facing change, or appliedskip newsupdated/extended the documentation: the multi-value options section of
docs/how-to/usage.rstnow says a command-line value replaces the configured listfor changes to activation scripts,
pyvenv.cfg, wheel downloads, app-data or release workflows: explained in thePR text which outside input the change handles, such as a path, a prompt or a downloaded file, and how it stops that
input from running as code. See the
security scope. (none of these are touched; the change is in how the discovery options are parsed)
ran tests on Python 3.9 and 3.15 (
tox run -e 3.9,3.15). I ran the discovery, config and seed-embed test files on 3.9 and 3.15 and the full suite on 3.13, and left the full 3.9 and 3.15 runs to CI.linked upstream issues beside any temporary interpreter overrides (none)
kept the PR in draft until CI passes