Repository navigation
Adding volumes to StatefulSet after the initial creation #367
Description
Kubernetes StatefulSet itself doesn't allow volumes to be added to StatefulSets after their initial creation.
Trying to add a volume, results in this error:
Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden
The same limitation is encountered via operator.
- create StatefulSet by setting:
spec:
statefulSet: {}
- try to add a volume to it
spec.
statefulSet:
storage:
size: 100Mi
mountPath: /logs
Tasks:
- understand the reasons behind StatefulSet not supporting
- if it's not an anti-pattern to add volume afterwards, support adding it via operator. StatefulSet probably have to be deleted and recreated (meaning there will be some downtime)
Activity
kabicin commentedon Jul 11, 2022
The reason for StatefulSet not supporting modification of other fields (in general):
For this specific use case outlined above, PVs and PVCs are not an issue but ensuring deletion of all pods is necessary to ensure recreation with the correct volume from StatefulSetSpec.
Edit: I am reconsidering to not ensure deletion of all pods since this specific use case will never cause a forced rollback state and so we can depend on StatefulSet controller to gracefully delete a pods with
statefulSet: {}kabicin commentedon Jul 13, 2022
The PR above only implements adding volumes to instances without storage specified, i.e.
statefulSet: {}.Given the above findings, do we want to support modifying volumes to RCO instances which already have StatefulSet + volume provisioned? For example, changing the
mountPathPV size (which currently updates RC spec but does not actually update the StatefulSet spec until it is manually restarted).halim-lee commentedon Jul 14, 2022
I would recommend adding support only for adding volumes for now. Since making changes to StatefulSet volumes also requires deletion of pods, I am not sure if it is safe to make that too available. If we decide to support modifying the existing volumes, we can implement it later.