Repository navigation
feat(rules): describe runfiles groups for rules_runfiles_group - #4254
Draft
FrankPortman wants to merge 1 commit into
Draft
FrankPortman wants to merge 1 commit into
FrankPortman wants to merge 1 commit into
Conversation
Give py_library, py_binary and py_test runfiles group describers, so packaging rules that use rules_runfiles_group (e.g. rules_img's image_from_binary) can split a binary's runfiles into layers: the runtime, one group per PyPI package, one per library and per rule target in data, and the binary's own files. Groups are collected on demand by a packager's aspect, so nothing runs otherwise. The runtime group comes from a singleton target resolving the same toolchain as the binary, so a packager builds it once per configuration. A binary whose pyc_collection differs from the configuration's, or that generates legacy __init__.py files, keeps its dependencies in its own group. Describers only see providers, so this adds PyInfo.direct_implicit_pyc_files and PyExecutableInfo.venv_runfiles. Fixes bazel-contrib#3757
This was referenced Oct 12, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #3962, which targeted the provider-based API that rules_runfiles_group has since replaced (see bazelbuild/rules_java#368).
Groups are now collected on demand: a packager's aspect asks each target's describer what it adds and which attributes it merges from. The Python rules get two private attributes, and the describers live in
python/private/runfiles_groups.bzl. Nothing runs unless a packager walks the graph, so the flags from #3962 are gone.A
py_binary/py_testgets these groups:rules_python#runtime: interpreter and stdlib. Comes from a singleton target that resolves the same toolchain, so a packager builds it once per configuration instead of once per binary (same trick as the JDK group in rules_java).rules_python#pypi/<name>: one per wheel, named by package since whl repo names aren't stable.weightis the file count.py_library, and one per rule target indata. Plain files indatago in their owner's group.executable_group.A binary whose
pyc_collectiondiffers from the configuration's, or that generates legacy__init__.pyfiles, keeps its deps in its own group, since the deps' groups wouldn't match its runfiles.Describers only see providers, so this adds two fields:
PyInfo.direct_implicit_pyc_files(the direct twin oftransitive_implicit_pyc_files) andPyExecutableInfo.venv_runfiles.Bazel 8 works too: only
python/private/BUILD.bazelloads rules_runfiles_group, and the rule definitions refer to the describers by label, so there's no autoload cycle.Testing:
runfiles_group_analysis_testwithoverlapping_group_behavior = "error"checks that the groups add up to exactlydefault_runfilesfor binaries with data that has default outputs, data whose outputs differ from its runfiles, a rule target insrcs, a binary indata, precompiling enabled, both fallbacks, and apy_test. Existing suites pass, andwhl_library_testpasses on 8.5.1 and 9.1.0. End to end with rules_img'srgi_aspect_wip: apy_binarywith a pip dep gives 4 layers (runtime,six, its library, the app) on both 9.1.1 and 8.5.1.Blocked on:
.bazelrcpoints at the dev registry (marked to delete before merge), and the nested workspaces (examples, integration tests) fail to resolve it.lib.bzlalso loads the packager code, which needsbazel_features>= 1.51, so loading the describers would add a WORKSPACE setup step that rules_python doesn't need today. @malt3, could the describer side be loadable without the packager?Known limitations: a plain file listed in several targets'
datalands in each of their groups, and apy_libraryindatawith precompiling enabled describes pyc files the binary doesn't get. Both overlap, neither drops files.Written with heavy LLM assistance (Claude), steered by me; I understand it well enough to own it in review.
Fixes #3757