[nybidari]

Part of #3226.

The API guide says a durableDir is created 0777, like an emptyDir. On gVisor its root is 01777. The volume is a tmpfs inside the sandbox, and its mount hint sets no mode, so it gets the tmpfs default. With the sticky bit, a user can't delete or rename another user's files in the volume's root.
A durable dir created before this keeps 01777 when it is restored from a VOLUMES or ROOTFS snapshot, since the snapshot records its mode.

Breaking Change:

A gVisor actor whose template has a durableDir can't restore a MEMORY snapshot taken before this change. On a MEMORY restore, runsc checks that the sandbox spec's mount hints match the checkpoint, except .source. The new dev.gvisor.spec.mount.<volume>.options key doesn't match, so runsc restore fails and the actor goes to CRASHED. This includes MEMORY golden snapshots of these templates, and actors suspended while their worker rolls during the upgrade. No API restores such a snapshot without its memory.
On upgrade:

• Recreate affected templates so they take new golden snapshots.
• Delete and recreate affected actors that hold a MEMORY snapshot from before the upgrade.
  A rollback breaks the same way. MEMORY snapshots taken after the upgrade fail to restore on the old worker. From v1.0.0, changes must keep upgrades and rollbacks working, so this has to land before then.

AI use:
AI-assisted, I reviewed and tested it.

• Tests pass
• Docs updated

[Fabricio Voznika (fvoznika)]

Please add comment that a tmpfs dir gets 0177 by default and that's why it's overwritten here.

[nybidari]

Done

[Benjamin Elder (BenTheElder)]

We actually have a way to express snapshot compatibility constraints, though nothing in main is using it yet. This might be a good use case. I think we don't want to outright break memory snapshots on v1.0.0-rc.1 but we can incrementally switch to this.

cc Zoe Zhao (@zoez7) Huy Pham (@hdp617)

[Zoe Zhao (zoez7)]

Here's how to expression the snapshot compatibility:

1. Each gvisor-worker needs to start reporting a new attribute, let's call it "root_mode", this will be saved in the worker's status field after RegisterWorkerRequest is processed.
2. During restore, substrate already compares the attributes reported by the worker, with the ones stamped into the snapshots in internal/hardware/hardware.go. However I realized there is a bug in matchesCompat: it currently only iterates over r.compat.GetAttributes() (the snapshot's attributes) and ignores keys that only the worker has, so the new attribute doesn't cause a downgrade to "rootfs". I will add a small PR to fix this.

[Zoe Zhao (zoez7)]

Benjamin Elder (@BenTheElder) If you decide this is M3 blocking, AND don't want to outright break memory snapshots on v1.0.0-rc.1, #3501 becomes M3 blocking as well.