kubernetesschedulingPractitioner

Mastering Mutable Pod Resources in Suspended Kubernetes Jobs

5 min read Kubernetes BlogApr 27, 2026Reviewed for accuracy

Practitioner — Hands-on experience recommended

Kubernetes v1.36 promotes the ability to modify container resource requests and limits in the pod template of a suspended Job to beta. This feature addresses a common pain point: the need to adjust resource specifications on Jobs without having to restart them. By allowing adjustments to CPU, memory, and GPU settings while a Job is suspended, you can fine-tune resource usage based on current cluster conditions or workload demands.

The mechanism behind this feature is straightforward. The Kubernetes API server relaxes the immutability constraint on pod template resource fields specifically for suspended Jobs. You can modify fields like spec.template.spec.containers[*].resources.requests and spec.template.spec.containers[*].resources.limits. To create a suspended Job, simply set spec.suspend: true in your Job manifest. After creating the Job, you can edit the resource requests using kubectl edit and then resume the Job with a patch command. This flexibility allows for better resource management without unnecessary downtime.

In production, you need to be aware of a key limitation: if you suspend a Job that was already running, you must wait for all active Pods to terminate before making any resource modifications. The API server will reject changes if status.active is greater than zero, preventing inconsistencies between running Pods and the updated pod template. This feature was first introduced as alpha in v1.35, and its promotion to beta in v1.36 signals its readiness for broader use in production environments.

Key takeaways

→Leverage the Mutable Pod Resources feature to adjust resource requests and limits for suspended Jobs.
→Use `spec.suspend: true` in your Job manifest to create a suspended Job.
→Remember to wait for all active Pods to terminate before modifying resources on a suspended Job.
→Utilize `kubectl edit` to easily change resource specifications on the fly.
→Patch the Job to resume it after making resource adjustments.

Why it matters

This feature significantly enhances resource management in Kubernetes, allowing you to adapt to changing workload demands without downtime. It helps maintain optimal resource utilization across your cluster.

Code examples

YAML

1apiVersion: batch/v1
2kind: Job
3metadata:
4  name: training-job-example-abcd123
5  labels:
6    app.kubernetes.io/name: trainer
7spec:
8  suspend: true
9  template:
10    metadata:
11      annotations:
12        kubernetes.io/description: "ML training, ID abcd123"
13    spec:
14      containers:
15      - name: trainer
16        image: example-registry.example.com/training:2026-04-23T150405.678
17        resources:
18          requests:
19            cpu: "8"
20            memory: "32Gi"
21            example-hardware-vendor.com/gpu: "4"
22          limits:
23            cpu: "8"
24            memory: "32Gi"
25            example-hardware-vendor.com/gpu: "4"
26      restartPolicy: Never

YAML

1apiVersion: batch/v1
2kind: Job
3metadata:
4  name: training-job-example-abcd123
5  labels:
6    app.kubernetes.io/name: trainer
7spec:
8  suspend: true
9  template:
10    metadata:
11      annotations:
12        kubernetes.io/description: "ML training, ID abcd123"
13    spec:
14      containers:
15      - name: trainer
16        image: example-registry.example.com/training:2026-04-23T150405.678
17        resources:
18          requests:
19            cpu: "4"
20            memory: "16Gi"
21            example-hardware-vendor.com/gpu: "2"
22          limits:
23            cpu: "4"
24            memory: "16Gi"
25            example-hardware-vendor.com/gpu: "2"
26      restartPolicy: Never

Bash

1# Create a suspended Job
2kubectl apply -f my-job.yaml --server-side
3# Edit the resource requests
4kubectl edit job training-job-example-abcd123
5# Resume the Job
6kubectl patch job training-job-example-abcd123 -p '{"spec":{"suspend":false}}'