Kubernetes secrets that leave no trace in a memory dump

Deutsche Fassung

Can a Kubernetes secret be used without ever showing up in a dump? This repo answers by measuring rather than arguing: real programs on real Linux, real dumps, and a scanner looking for a canary.

Two scenarios, three language families (C/Rust, Go, Java):

• Delivery — Kubernetes provides the secret as a file or an environment variable.
• Fetch — the application retrieves it at runtime over an API (HashiCorp Vault).

The answer in four sentences

1. For delivery: yes. The secret must never become a String, []byte, or char* on the heap. Against ptrace, only memfd_secret holds.
2. When fetching, there is no original to scrub. A naive fetch leaves three to nine copies, exactly one of which the application owns. Zero is reachable, but only without a standard-library HTTP client.
3. The copies do not decay. Not one disappears over 60 seconds of idling. Under load they grow — naive C adds two per fetch, Java four to five.
4. What matters is not the mechanism but the discipline: a target buffer allocated once and overwritten on every fetch. memfd_secret, an anonymous mapping, and MemorySegment all reach zero.

One route bypasses all four: a CRIU checkpoint of the container writes every memory page to the node as a plaintext tar, and in Kubernetes that interface has been beta and enabled by default since 1.30. No language decides what lands in it. The guide says what still helps. Sourced, not measured in this project — kubelet checkpoint API.

Documents

File	Contents
report/delivery.en.md	Delivery. 535 data points, region matrix checked against fs/coredump.c.
report/api-fetch.en.md	Fetching over HTTP and HTTPS. 3400 data points, load curve, exposure duration.
report/hardening-guide.en.md	For implementers. Per language: what works, what is awkward, what is impossible, what is easy to get wrong. With code.
CONTRACT.md	The frozen interface between all modules — CLI, wire protocols, JSONL schema. (German)

All three reports also exist in German, as .de.md in the same directory.

Layout

Directory	Contents
probe/	Kernel capability probe and region matrix. RESULTS.md is the foundation.
native/ go/ java/	The demos. One hardened and one naive run per use case.
server/	Counterpart for every wire protocol, including the fake Vault.
harness/	Mints canaries, delivers them the way kubelet does, dumps, scans, evaluates.
integration/	Cross-module evaluation, load curve, cross-validation.
results/	One JSONL file per module. The raw measurements.

The rule everything rests on

A hardened result counts only if the paired naive run found the canary.

If the scanner finds nothing in either, the test is broken — not the secret protected. Such cells count as INVALID, never as success. The rule caught two measurement errors that would otherwise have passed as results.

Requirements

Linux only, and a permissive one. The demos hang on coredump_filter, madvise, memfd_secret, AF_ALG, keyctl and kTLS; the flake declares aarch64-linux and x86_64-linux and nothing else. Beyond that:

	Why
kernel with secretmem enabled	otherwise memfd_secret returns ENOSYS and the strongest cells fail silently
ptrace permitted	gcore and /proc/<pid>/mem are four of the six dump methods
core_pattern = core, RLIMIT_CORE set	for the kernel-core method
sudo	one delivery path in the harness; there is a documented fallback without it

Do not guess whether your kernel qualifies — ask it:

nix build .#probe && ./result/bin/capabilities | jq .

Measured here against kernel 7.0.14-orbstack on aarch64, coredump_filter 0x33. For reference, that machine was reached with orb -m k8ssec sh -c 'cd <repo> && nix develop'.

Getting started

nix develop                          # measurement environment (Linux)
nix build .#all                      # every module
python3 integration/aggregate.py     # exits 1 on INVALID

The flake pins nixpkgs. flake.lock is never updated, so measurements stay comparable over time. devenv shell is a second, separate environment holding only the tooling — formatters, linters, git hooks — and it runs on macOS too.

There is no CI, on purpose. GitHub runners cannot run this measurement: secretmem is not enabled, core_pattern and ptrace_scope are not settable. A green badge over a measurement that measured nothing would be worse than no badge.

Two things stay German: the source comments and CONTRACT.md. So do the notes fields in results/*.jsonl — those are recorded measurements, and a measurement log is not rewritten after the fact.

Status

Hobby project and a finished investigation. No warranty, no maintenance promise, no response times. It is CC0 — fork it and adapt it, that is faster than asking here.

License

Everything original in this repo is released under CC0 1.0 Universal (SPDX-License-Identifier: CC0-1.0), see LICENSE. This repo is a measurement artifact, not production code — take it, no need to ask.

Two limits that are easy to miss:

• go/vendor/ is not covered. golang.org/x/sys v0.47.0 is BSD-3-Clause, Copyright 2009 The Go Authors, and keeps its own license. It sits unchanged in go/vendor/golang.org/x/sys/LICENSE.
• CC0 waives no patent or trademark rights. The text says so explicitly: "No trademark or patent rights held by Affirmer are waived, abandoned, surrendered, licensed or otherwise affected by this document."