Repository navigation
Use published SecureBuild images for latest kURL add-ons - #6167
Merged
Merged
Conversation
mikhailswift
force-pushed
the
mikhail/securebuild-addon-images
branch
from
October 1, 2026 17:34
db10932 to
a28ccd2
Compare
This was referenced Oct 5, 2026
banjoh
pushed a commit
that referenced
this pull request
Oct 29, 2026
… (sc-140326) (#6242) * fix(kubernetes): alias canonical pause image for SecureBuild packages (sc-140326) Airgapped upgrades stepping onto k8s 1.36.5/1.37.1 fail deterministically: kubeadm's upgrade-health-check Job uses the canonical pause image (registry.k8s.io/pause:3.10.2), but the SecureBuild switch (#6167) loads pause as docker.io/kurlsh/pause and kubeadm-image-overrides never aliased it back. In airgap the Job cannot pull, never completes, and 'kubeadm upgrade plan' fails its CreateJob preflight (staging run STAGING-release-v2026.10.18-1, 7/7 airgap legs, kurl_upgrade). use_securebuild_images() now appends an idempotent pause alias to kubeadm-image-overrides whenever it rewrites pause to the kurlsh repo, including future versions without a securebuild-<version> dir, and fails generation on unrecognised (e.g. digest-only) pause references. Backfill packages/kubernetes/1.36.5 and 1.37.1. kubernetes_configure_kubeadm_images skips the pause component: it is not a static pod, the line exists only for the load-time retag. * test(kubernetes): match 'multi' kubeadm-image-overrides fixture to the shipped 1.36.5 file The fixture comment claims fidelity to packages/kubernetes/1.36.5/kubeadm-image-overrides but its etcd line used a proxy.replicated.com source while the shipped file uses docker.io/kurlsh/etcd. Align the fixture; the proxy-referenced etcd line stays in the default etcd-only fixture, whose proxy-propagation assertion depends on it. * fix(kubernetes): derive canonical pause alias from the package Manifest at load time Rework per PR review: instead of appending a pause line to kubeadm-image-overrides (which would leave future >=1.36.5 versions that have no securebuild dir with a pause-only overrides file, tripping kubernetes_configure_kubeadm_images into creating an empty patch dir and kustomization stanza on every init/join), alias the pause image to the canonical registry.k8s.io name directly in kubernetes_load_images, derived from the package Manifest's 'image pause' line. kubeadm's upgrade-health-check Job needs the canonical name locally in airgap or CreateJob fails (sc-140326). Reverts the script.sh/kubeadm-image-overrides approach and the configure_kubeadm_images pause skip; no override-file format changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Use published SecureBuild images for the latest Registry, Flannel, Goldpinger, OpenEBS, CoreDNS, the Kubernetes control plane, kube-proxy, etcd and Velero add-ons. Regenerate the add-ons from their source templates, mapping only versions whose SecureBuild tags have been published and verified.
Kubernetes 1.36.5 now packages
kurlsh/kube-apiserver:v1.36.5,kurlsh/kube-controller-manager:v1.36.5,kurlsh/kube-scheduler:v1.36.5andkurlsh/etcd:v3.6.15. Generate kubeadm static-pod image patches during init and control-plane joins, and explicitly select etcd 3.6.15 instead of kubeadm’s default 3.6.8. Keep the existing etcd data directory and TLS settings.Velero 1.18.4 now packages and installs
kurlsh/velero:v1.18.4and the AWS, GCP and Azure plugins atv1.14.4. The server image includes rebuilt Velero/helpers and Restic with patched gRPC, Go crypto and OpenTelemetry dependencies. Explicitly pass the server image tovelero installso both its Deployment and node-agent DaemonSet use the packaged image.MinIO and s3cmd already reference rebuilt tags; refresh their package manifests to invalidate cached archives, and annotate existing MinIO pod templates so the rebuilt image rolls out even when the add-on version is unchanged. Retain the already-merged EKCO 0.28.17 at
replicated/ekco, including its newer HAProxy 3.4.6.Which issue(s) this PR fixes:
No issues automatically closed.
Special notes for your reviewer:
/usr/bin/registry. Preserve the existing root permissions for its HTTPS listener and root-owned PVCs.kurlshreferences.upgrade apply/upgrade nodepatch preservation and coordinated same-version image rollout are deferred to a follow-up; they have not been implemented or validated here.1.18.4-r1, and bundled Restic retains version 0.15.0 at revisionr11. Verified binaries containgolang.org/x/crypto v0.56.0,google.golang.org/grpc v1.83.2andgo.opentelemetry.io/otel/sdk v1.45.0. No source patches were applied.skip-testgriddefers individual add-on matrices to the combined staging run. Required CI remains enabled; cluster install/upgrade results are not claimed here.Published images were independently scanned on amd64 and arm64 with Grype 0.119.0 and the database built 2026-10-01T06:33:48Z:
replicated/ekco:v0.28.17kurlsh/s3cmd:20260825-a9ea1c6kurlsh/minio:RELEASE.2025-10-15T17-29-55Zkurlsh/registry:3.1.2kurlsh/goldpinger:3.11.3kurlsh/coredns:1.14.7kurlsh/openebs-linux-utils:4.6.0kurlsh/openebs-provisioner-localpv:4.6.0kurlsh/flannel-cni-plugin:v1.9.1-flannel3kurlsh/flannel:v0.28.9kurlsh/kube-proxy:v1.36.5kurlsh/kube-apiserver:v1.36.5kurlsh/kube-controller-manager:v1.36.5kurlsh/kube-scheduler:v1.36.5kurlsh/etcd:v3.6.15kurlsh/velero:v1.18.4kurlsh/velero-plugin-for-aws:v1.14.4kurlsh/velero-plugin-for-gcp:v1.14.4kurlsh/velero-plugin-for-microsoft-azure:v1.14.4The four added control-plane/etcd images were anonymously pulled from public Docker Hub for amd64 and arm64, including every layer. Executable hashes match the verified SecureBuild binaries. All three Kubernetes control-plane full-image scans have zero High/Critical findings (one Low each). Etcd’s full-image scan has 33 High and 9 Critical matches across 14 distinct SecureOS package advisories on
etcd-3.6,etcdctl-3.6andetcdutl-3.6; its independently scanned binaries have zero findings. Package-advisory metadata still needs reconciliation, and the raw findings have not been suppressed. The move to upstream etcd 3.6.15 also fixes the actual source-level advisories found when rebuilding 3.6.8. No custom application source patches were applied.The three Velero plugin images report zero CVEs of any severity on both architectures. Velero's 80 High/Critical matches represent 13 distinct CVEs duplicated across eight SecureOS APK packages. Standalone server/helper/Restic package scans and inspection of the published binaries confirm the dependency fixes; binary scans have zero High/Critical findings. The live SecureOS advisory feed, verified October 1 at approximately 18:13 UTC, now records fixed versions for all 80 matches through the packages' origin mapping. Installed Velero/Restic revisions are newer than those fixed versions. The next Grype DB that ingests these records is expected to clear the APK matches; no refreshed-DB scan has yet been observed. Raw findings have not been suppressed.
Verified published image index digests
replicated/ekco:v0.28.17:sha256:94a3da6b2f0ce285c30cf464e42d20e2886fca7ff9fd9e55fae54bca9042582bkurlsh/s3cmd:20260825-a9ea1c6:sha256:adbe3dfc5da8713de6eba48b10472c38b07bf1c3500dd023d17aa63ea5e0d7f5kurlsh/minio:RELEASE.2025-10-15T17-29-55Z:sha256:7d81751f04a90d2c8cbd1742b64d50b0471da9a23ca9343c2c10756764fe75f4kurlsh/registry:3.1.2:sha256:98fab4be3f0b5c486e1abfc6045313edb06b36c02436e11bfed537c373de5c85kurlsh/goldpinger:3.11.3:sha256:3badf8c12c624bcaf91a45c08ff637b3b190bffefdbfa282102ec1d260ddd5e7kurlsh/coredns:1.14.7:sha256:4cb463f4d981cf4a58123a004a3dcbdd7e394bc4d309beb3898d5ec442d8ec48kurlsh/openebs-linux-utils:4.6.0:sha256:0e352dadf905a07332a94df32bc242199a97bd00417c0e243ab6ac08c07fdb1dkurlsh/openebs-provisioner-localpv:4.6.0:sha256:dfa7a7d42f479bb0423f6675b7f69669e1ab19ae178f90f1f1d2d94f2a158be0kurlsh/flannel-cni-plugin:v1.9.1-flannel3:sha256:452b0ca8823926e1a0bba1198b10518bf3b9ddc989a03dec8a89951949cace65kurlsh/flannel:v0.28.9:sha256:409953e589022ae17c777e042b7294a25efec2445e4ec1d8a70811a4531835a1kurlsh/kube-proxy:v1.36.5:sha256:4af2b0126fb5afe9409fb3fbc3abe6224d48d043215bad730d1952319e15d0bdkurlsh/kube-apiserver:v1.36.5:sha256:9236e8824bb5ffa34b524d6448e4bff1a8f1f2914a91f2401adaf87d4ff3f29fkurlsh/kube-controller-manager:v1.36.5:sha256:9b0b24b18df807d2c2f13f4f329167156e70580d9b621794ff40579f89d66606kurlsh/kube-scheduler:v1.36.5:sha256:e2eba0f4da2e88e32057a8f7b45d87a38fb81f6c8af32ef2bacc589e99c5eeedkurlsh/etcd:v3.6.15:sha256:5b03daad1dfeb7bf7fcccfd0014892c0f59833b90e23915fddbf074175420353kurlsh/velero:v1.18.4:sha256:0416792bc7a3e6ff54d792b0ebe36669dd1eabe4ebba66a482c1261a6e0b0c8dkurlsh/velero-plugin-for-aws:v1.14.4:sha256:14ead6bbec46fabcdeda3714c302661a1b5c65d6104372de6b67ee283ae798b8kurlsh/velero-plugin-for-gcp:v1.14.4:sha256:a96fc3cab20d785433817c2b56e3c14b5898f79851e34455af72381a6601c3f5kurlsh/velero-plugin-for-microsoft-azure:v1.14.4:sha256:575a2816b08773f5f08e5247171d79ccd7defbe4216e3da8f76fa8fcd6d0465eSteps to reproduce
Validation completed:
skopeo copy --alland scanned each amd64/arm64 digest (22 scans). Independently pulled and scanned the four added Velero images on both architectures (eight additional scans).skopeo copy --all --src-no-creds, then their actual amd64/arm64 filesystems and binaries were independently scanned.make test-shellpasses with the static-pod additions. Assembled install/join scripts pass shell syntax checks. Actual kubeadm 1.36.5 validates the generated init/join configurations and renders the four static pods with the public image references. Existing Docker/containerd alias checks, generator repeatability and YAML merge checks preserve the etcd data directory and cipher-suite settings. No cluster install/upgrade result is claimed./v2/on port 443 with writable filesystem storage under its configured UID./velero version --client-onlypassed, and offlinevelero install --dry-runrendering verified the server/node-agent and plugin image references.make test-shellsuite passed for the original change in a disposable Ubuntu 24.04 container. Velero additions passed shell syntax, regeneration andgit diff --checkvalidation.kubectl set image --localaccepted the CoreDNS/kube-proxy overrides. The Kubernetes image-package recipe includes shippedimage-overrides.Local metadata generation did not publish shared staging metadata. No new integration tests were added.
Does this PR introduce a user-facing change?
Does this PR require documentation?
NONE