Unlocking Mutable PersistentVolume Node Affinity in Kubernetes v1.35

Kubernetes v1.35 brings a significant change with mutable PersistentVolume node affinity, addressing a common pain point in resource management. Previously, once set, the node affinity for a PV was immutable, which could lead to inefficiencies when the underlying infrastructure changed. Now, you can modify the node affinity to reflect real-time conditions, enhancing the way your applications interact with the underlying storage.

To leverage this feature, you need to enable the MutablePVNodeAffinity feature gate on the APIServer, which is disabled by default. Once enabled, you can change the PV node affinity using the VolumeAttributesClass API. For example, you might adjust the node affinity from a specific zone to a broader region, allowing for more flexible scheduling. Here's how you might define the node affinity in your PV specification:

1spec:
2  nodeAffinity:
3    required:
4      nodeSelectorTerms:
5      - matchExpressions:
6        - key: topology.kubernetes.io/zone
7          operator: In
8          values:
9          - us-east1-b

However, there are important considerations to keep in mind. Changing the PV node affinity does not alter the actual accessibility of the underlying volume. There's also a race condition when tightening node affinity; the scheduler might still place a Pod on an old node that can no longer access the volume. If you update the PV and immediately start new Pods in a script, you may encounter unexpected behavior. Always ensure the underlying volume is updated in the storage provider before making changes to the PV affinity.

This feature is currently in alpha, so be cautious about its stability and potential edge cases in production environments.