Skip to content

fix(passthrough): honor selectors and custom VFIO IDs - #675

Open
johnahull wants to merge 1 commit into
ROCm:mainfrom
johnahull:fix/passthrough-vfio-validation
Open

johnahull wants to merge 1 commit into
ROCm:mainfrom
johnahull:fix/passthrough-vfio-validation

Conversation

@johnahull

@johnahull johnahull commented Oct 27, 2026 •

Copy link
Copy Markdown

Problem

The operator supports PF/VF passthrough through a node-scoped DeviceConfig and VFIO device IDs. Two generic correctness and safety gaps make those configurations unsafe or incomplete:

  1. An enabled passthrough DeviceConfig without a selector is not sufficiently scoped to the intended nodes.
  2. When users provide spec.driver.vfioConfig.deviceIDs, the bind/unbind command uses those IDs but the VF-detection init container still scans only the built-in defaults. A custom VF configuration can therefore wait indefinitely for a device that the detection command does not look for.
  3. Device IDs are inserted into privileged shell commands. Without quoting, a malformed or unexpected value can change the generated command instead of remaining a literal PCI-ID pattern.

What this changes

  • Require spec.selector for enabled vf-passthrough and pf-passthrough configurations. The validation treats an omitted driver.enable as enabled, matching the CRD default.
  • Centralize VFIO device-ID selection for worker pod generation.
  • Use configured VFIO IDs consistently in both:
    • the host-driver bind/unbind command; and
    • the VF-detection init container.
  • Shell-quote configured and default IDs before inserting them into the generated bind/unbind and detection commands.
  • Preserve the existing default-ID behavior when no custom IDs are configured.
  • Add unit tests covering selector validation, custom-ID propagation into both command paths, and shell metacharacter handling.

Compatibility

This PR is independent of GIM artifact resolution. It applies only to PF/VF passthrough configurations; container-driver behavior is unchanged. The AMD MI3xx default VF IDs and GIM artifact mapping are kept in PR #674. The unrelated -e fix remains in PR #673.

Testing

  • GOTOOLCHAIN=auto go test ./api/... ./cmd/... ./internal/... ./tools/build/copyright
  • GOTOOLCHAIN=auto go vet ./api/... ./cmd/... ./internal/... ./tools/build/copyright
  • git diff --check

The tests verify that configured VFIO IDs appear safely in both the binding command and detection command, and that enabled PF/VF passthrough rejects an empty selector.

Out of scope

@johnahull
johnahull force-pushed the fix/passthrough-vfio-validation branch from aad4024 to c9e99f5 Compare October 27, 2026 16:23
for _, deviceID := range utils.DefaultVFDeviceIDs {
grepArgs = append(grepArgs, "-e "+deviceID)
for _, deviceID := range getVFIODeviceIDs(devConfig) {
grepArgs = append(grepArgs, "-e "+quoteShellArg(deviceID))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Use consistent PCI-ID matching in detection and binding

A valid custom ID such as deviceIDs: ["74B5"] or ["0x74b5"] now leaves the worker stuck in its init container. The binding command’s lspci -d 1002:<ID> accepts both forms, but lspci -nn prints [1002:74b5], so the new case-sensitive grep never matches. Previously the default detector matched 74b5, allowing these configurations to proceed. Please use the same PCI-ID selection semantics for detection and binding, for example by checking for nonempty lspci -d output, and cover uppercase and prefixed IDs in regression tests.

@yansun1996
yansun1996 marked this pull request as ready for review November 4, 2026 02:02
@yansun1996
yansun1996 requested a balanced review from Copilot November 4, 2026 02:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unquoted PCI ID arguments allow configured text to alter host-device selection, while case-sensitive detection fails for uppercase IDs and prevents the binding container from starting.

2 open issues
What changed in this PR

This PR tightens PF/VF passthrough configuration validation and makes worker commands use configured VFIO device IDs consistently. GIM artifact resolution and the separate unbind shell-test fix remain out of scope.

Changes:

  • Require a nonempty node selector for enabled PF/VF passthrough, treating an omitted enable flag as enabled.
  • Centralize VFIO ID selection and shell-quote IDs in generated binding and detection commands.
  • Add tests for selector validation, custom-ID propagation, and shell metacharacter quoting.
File Description
internal/​validator/​specValidators.go Require selectors for enabled passthrough configurations.
internal/​validator/​specValidators_test.go Cover selector requirements and unchanged container-driver validation.
internal/​controllers/​workermgr/​workermgr.go Share VFIO ID selection and quote IDs in generated commands.
internal/​controllers/​workermgr/​workermgr_test.go Check custom-ID quoting in binding and VF detection commands.

Review effort: Balanced

}
quotedDeviceIDs := make([]string, 0, len(deviceIDs))
for _, deviceID := range deviceIDs {
quotedDeviceIDs = append(quotedDeviceIDs, quoteShellArg(deviceID))
for _, deviceID := range utils.DefaultVFDeviceIDs {
grepArgs = append(grepArgs, "-e "+deviceID)
for _, deviceID := range getVFIODeviceIDs(devConfig) {
grepArgs = append(grepArgs, "-e "+quoteShellArg(deviceID))

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants