Mastering Node-Pressure Eviction in Kubernetes
Node-pressure eviction exists to prevent resource starvation in your Kubernetes cluster. When nodes run low on critical resources like memory or disk space, the kubelet steps in to terminate pods proactively. This mechanism ensures that your applications remain responsive and that the cluster operates smoothly, even under heavy load.
The kubelet continuously monitors resource usage on nodes. It uses eviction signals to determine when to act. If memory usage exceeds defined thresholds—like memory.available<1Gi—the kubelet will fail selected pods, setting their phase to Failed. You can configure soft and hard eviction thresholds. Soft thresholds allow for a grace period, while hard thresholds trigger immediate termination. For example, you might set memory.available<10% as a hard threshold to ensure that memory availability never drops too low.
In production, understanding the nuances of node-pressure eviction is essential. The kubelet does not respect PodDisruptionBudgets during these evictions, which can lead to unexpected behavior if you rely on them for availability. Be aware that the CommitLimit can change based on the node's page-file size, which might affect your eviction strategy. As of Kubernetes v1.31, this feature is enabled by default, but ensure you’re aware of version-specific requirements, like enabling the KubeletSeparateDiskGC feature gate in v1.35.
Key takeaways
- →Configure soft eviction thresholds to manage pod terminations gracefully.
- →Set hard eviction thresholds to prevent critical resource exhaustion.
- →Monitor `memory.available` to anticipate node-pressure eviction events.
- →Understand that PodDisruptionBudgets are ignored during node-pressure evictions.
- →Be aware of the impact of CommitLimit changes on resource management.
Why it matters
Node-pressure eviction directly impacts application availability and performance. Properly managing this process can prevent downtime and ensure that your services remain responsive under load.
Code examples
memory.available<1Gimemory.available<10%memory.available=1m30sWhen NOT to use this
The official docs don't call out specific anti-patterns here. Use your judgment based on your scale and requirements.
Want the complete reference?
Read official docsIndustry-standard certifications built by the people behind Linux and Kubernetes. Earn the CKA — the gold standard Kubernetes administrator cert. OpsCanary readers get 30% off year-round with code OPSCANARY3.
Get CKA certified →Tracing Kueue Admission: Unraveling Workload Mysteries with Headlamp
Understanding how Kueue admission works is crucial for optimizing resource allocation in Kubernetes. With the Kueue plugin for Headlamp, you can trace the admission path of a Workload, revealing insights into its resource requests and conditions. Dive in to learn how to leverage this tool effectively.
Navigating the Shift to cgroup v2 in Kubernetes
Kubernetes is evolving, and so is its resource management with the shift to cgroup v2. Understanding how cgroups manage resources is crucial, especially with features like Memory QoS and the new kubelet defaults. This article dives into what you need to know to leverage these changes effectively.
Mastering Node Swap in Kubernetes: Boosting Workload Resilience
Node swap can be a game changer for your Kubernetes workloads, especially during traffic spikes. By enabling the Linux kernel to page out memory to disk, you can effectively manage memory oversubscription and improve application performance.
Get the daily digest
One email. 5 articles. Every morning.
No spam. Unsubscribe anytime.