Skip to content

feat(supervisor): deliver bypass detection logs via NFLOG and drop CAP_SYSLOG - #2383

Closed
gaveezy wants to merge 1 commit into
NVIDIA:mainfrom
gaveezy:feat/nflog-bypass-detection
Closed

gaveezy wants to merge 1 commit into
NVIDIA:mainfrom
gaveezy:feat/nflog-bypass-detection

Conversation

@gaveezy

@gaveezy gaveezy commented Jul 21, 2026

Copy link
Copy Markdown

Summary

Switch sandbox bypass detection from tailing the kernel ring buffer (dmesg, requires CAP_SYSLOG in the initial user namespace) to an nfnetlink_log (NFLOG) socket bound inside the workload netns (covered by namespaced CAP_NET_ADMIN), and drop CAP_SYSLOG from the Kubernetes, Docker, and Podman driver capability sets. This makes bypass detection work in user-namespaced pods (hostUsers: false), where a granted CAP_SYSLOG is a no-op because the kernel checks the ring buffer against the initial user namespace.

Related Issue

Closes #2382

Changes

  • New bypass_monitor::nflog module in openshell-supervisor-process: minimal hand-rolled nfnetlink_log client over raw libc (zero new dependencies) — acked group bind + packet copy-mode config, and parsing of packet messages (prefix attribute, NFULA_UID, raw IPv4/IPv6 + TCP/UDP headers for dst/ports/proto).
  • NetworkNamespace::install_bypass_rules binds the NFLOG socket first (on a short-lived setns thread, during privileged startup before supervisor seccomp hardening) and generates log prefix <p> group <N> rules on success; any failure falls back to the previous log ... flags skuid printk rules + dmesg monitor, and only that path enables the nf_log_all_netns sysctl.
  • bypass_monitor::spawn takes the optional socket and prefers it; OCSF dual-emit / denial-aggregator / activity reporting is factored into a shared BypassReporter so both backends report identically. OCSF ConfigStateChange events record which backend was selected.
  • CAP_SYSLOG removed from the K8s pod spec, Docker cap_add, and Podman cap_add; driver tests now assert its absence. Sidecar topology is unchanged (it has no bypass log listener).
  • Docs: docs/kubernetes/topology.mdx capability table, Podman driver README capability rationale; e2e/rust/tests/user_namespaces.rs capability assertions updated.

Enforcement is unaffected — the nftables REJECT rules do not depend on log delivery; NFLOG only carries the detection/alerting signal. As a side benefit the sandbox no longer reads host-global kernel logs.

Testing

  • mise run pre-commit passes
  • mise run ci passes (lint, compile/type checks, tests)
  • Unit tests added/updated — 14 new tests (netlink message layout, ack parsing, datagram/packet parsing incl. IPv6 and truncation fuzz, NFLOG vs ring-buffer rule generation); full crate suite (184 tests) run on Linux via Docker, cargo clippy (pedantic + nursery) clean for the Linux target
  • E2E tests added/updated (if applicable) — existing user_namespaces.rs e2e updated; requesting test:e2e / test:e2e-kubernetes labels to exercise the live NFLOG path (rule install + packet delivery) on a real kernel

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)

Notes for reviewers

  • The NFLOG group bind deliberately skips PF_BIND: group-targeted log group <N> statements deliver to the bound group directly; protocol-family binding is only required for legacy non-group logging. Called out in a code comment — the e2e run should confirm on a live kernel.
  • Commits are not yet cryptographically signed, so copy-pr-bot will need a maintainer /ok to test.

…P_SYSLOG

Bypass detection previously tailed the kernel ring buffer via dmesg,
which requires CAP_SYSLOG in the initial user namespace and therefore
silently degrades in user-namespaced pods (hostUsers: false). Bind an
nfnetlink_log socket inside the workload netns instead and generate
"log group <N>" nftables rules; delivery is netns-scoped and covered by
namespaced CAP_NET_ADMIN. The printk + dmesg path remains an automatic
fallback for kernels without nfnetlink_log, and CAP_SYSLOG is removed
from the Kubernetes, Docker, and Podman driver capability sets.

Refs #2382

Signed-off-by: Hai Nguyen <haingu@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 21, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

Copy link
Copy Markdown

Thank you for your submission! We ask that you sign our Developer Certificate of Origin before we can accept your contribution. You can sign the DCO by adding a comment below using this text:


I have read the DCO document and I hereby sign the DCO.


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the DCO Assistant Lite bot.

@gaveezy

gaveezy commented Jul 21, 2026

Copy link
Copy Markdown
Author

I have read the DCO document and I hereby sign the DCO.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(supervisor): NFLOG-based bypass detection to drop the CAP_SYSLOG requirement

1 participant