Add bootc apply-live

This is probably mainly draining the logic from rpm-ostree into ostree-ext, then re-using it here.

But...a whole lot of suddenly OS-specific issues come to the fore. For example, should we try to distinguish between "new content" and "changes"?

This one also relates to #165

I know at least some people have also been asking for an "apply live by default" mode. Note that there's a super tricky detail of that in that if there's kernel changes in the new root, we must in general do something like keep /usr/lib/modules/$kver mounted/copied into the running root.

This one also relates to #165

I know at least some people have also been asking for an "apply live by default" mode. Note that there's a super tricky detail of that in that if there's kernel changes in the new root, we must in general do something like keep /usr/lib/modules/$kver mounted/copied into the running root.

Could we bind mount /usr/lib/modules somewhere else like /run/booted-kernel-modules during the initial boot, and then have the kernel also look for modules there? Then we don't have to mess with trying to union the new and old modules dirs?

Could we use the same logic as the DNF needs-restarting plugin? This gives hints whether processes, services or the whole system needs a restart to account for the changes. And this would keep us close to DNF.

It might also be nice to have a --dry-run option to know before applying a change whether it will require a process / service / system restart. It would also be interesting to have the ability to use the same logic to compare two images out of band, e.g. in a CI/CD pipeline.

WIP: bootstrap: pivot into node image before bootstrapping

Could we use the same logic as the DNF needs-restarting plugin? This gives hints whether processes, services or the whole system needs a restart to account for the changes. And this would keep us close to DNF.

It might also be nice to have a --dry-run option to know before applying a change whether it will require a process / service / system restart. It would also be interesting to have the ability to use the same logic to compare two images out of band, e.g. in a CI/CD pipeline.

the intent behind apply-live is to avoid reboots, where a kernel update is actually required a more natural fit would be to have a kexec flow to switch directly to the new kernel or defer that until the next actual reboot is requested.

Having bootc apply-live ever trigger a whole system restart would make it unusable for some use-cases.
i.e. performing software updates when the workload cannot be stopped or moved to a different host.
for example, if you have vms or other workloads on the systems in the context of a cloud computing environment.

A normal user may be okay with downstream, but using this in a data centre context would be very problematic. It would be necessary to live migrate all workloads from a given host to a different one just to update the version of Ping or some other package on the host because of a cve.

so the ability to live application without reboot is required for CVE patching and other use-cases.

I hope this issue is prioritized at some point, as I still consider it very important for bootc to be adopted in data centres.

Just to cross-reference this more explicitly: in OCP we're going to be implementing something similar to this RFE to be able to boot from a disk image which doesn't have e.g. the kubelet and overlay the target node image, which does have it. One success criteria for this RFE should ideally be that we can simplify that logic to use bootc apply-live instead. For example, it should ideally work in live systems as well.

Yeah, when you are ready to work on it feel free to ping in this issue or on the matrix chat, happy to help

Yeah, last time I tried to update the ostree rust bindings I ran into various issues...it needs some love.

That said, the kexec logic is not really complex and we could also reimplement it in Rust directly here.

My patch I posted there should fix it enough to let you regenerate the bindings

PR merged. I have a tree on my machine with new bindings, I'll see if I can get them out today.

@cgwalters

Good news! Updated ostree bindings are available (v0.20.0) thanks to @cgwalters :)
ostreedev/ostree#3376
ostreedev/ostree#3378

Yep and #1069 started using them here

One thing related to this we've been talking about on the MCO side is the idea of lowering Node Disruption Policies down into RHEL, e.g. in bootc. This would work in tandem with apply-live I think, where e.g. bootc apply-live would by default only apply live changes if it respects the policies, or error out.

And/or maybe have a bootc upgrade --apply-or-reboot which would either apply live changes if all changes respect node disruption policies or reboot the machine.

OTOH I can also see this NDPs integrate with #22 instead, since /etc files is probably 95% of what you'd use it for and it's much easier to live apply config changes since those are already distinct from the host image and actually overlaid live.

But on the other other hand, having the concept of NDPs be generic enough so it applies to both OS content and config overlays seems like it'd be useful.

What's the status of this? It would be awesome to have for https://gitlab.com/fedora/bootc/tracker/-/issues/4.

I think it should be very straightforward to just overmount a new / composefs - at least in the case where it's read only. The tricker questions are around /etc because if we just overmount that then anything with an open fd will be silently writing to an old mount, and it'd be pretty easy to get data loss. A lot of the problems here are similar to mutable sysexts; hard to avoid giving people a choice of bugs. The rpm-ostree logic here does an eager 3 way merge which is mostly what people want I think, but does mean that "apply live" also creates a persistent mutation.

For use cases that can have /etc readonly (not just transient, very viable with more stateless systems) then it's all much simpler because in that case /etc isn't even a mount point, it's just part of / too.

Any rough idea when this is going to land? Once it does I will convert all my machines to using bootc hehe. Mind you soft-reboot is already great, but no unnecessary service restarts would make it much more viable for my usecase :)

Would love to see this in ostree, then called out to from bootc if it's practical

We are looking into bootc for immutable base images for our clusters and would really benefit from this feature - rebooting for every update is simply not an option.

For now, I'm trying to hack something together with systemd-sysext, as bootc seems to be the ideal solution for us otherwise.

Another use case that came up for this, is logical bound images.

Another use case that came up for this, is logical bound images.

Yeah. While it is not ergonomic right now, if one wants to run a newer/different version of a LBI, you can do this by having a dynamic configuration which temporarily masks the builtin quadlet and overrides it with e.g. one in /run.

Navigation Menu

Repository navigation

Add `bootc apply-live` #76

8 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Add bootc apply-live #76

Description

Activity

cgwalters commented on Nov 2, 2023

alexlarsson commented on Nov 2, 2023

fabiendupont commented on Nov 6, 2023

SeanMooney commented on Nov 28, 2024

jlebon commented on Dec 5, 2024

8 remaining items

cgwalters commented on Jan 24, 2025

Mstrodl commented on Jan 25, 2025

Mstrodl commented on Jan 30, 2025

Mstrodl commented on Feb 4, 2025

cgwalters commented on Feb 4, 2025

jlebon commented on Feb 27, 2025

evan-goode commented on Dec 11, 2025

cgwalters commented on Dec 16, 2025

MatthiasGrandl commented on May 2, 2026

Mstrodl commented on May 8, 2026

Jannik2099 commented on Aug 19, 2026

jmarrero commented on Sep 3, 2026

cgwalters commented on Sep 3, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Add `bootc apply-live` #76