Mastering Kubernetes Garbage Collection: What You Need to Know

Garbage collection exists to keep your Kubernetes cluster tidy and efficient. It solves the problem of resource bloat by automatically cleaning up objects that are no longer needed. This is especially important during upgrades, where orphaned resources can lead to confusion and wasted resources.

Kubernetes employs several mechanisms for garbage collection. Owner references indicate which objects depend on others, allowing the control plane to clean up related resources before deletion. There are two main types of cascading deletion: foreground and background. In foreground cascading deletion, the owner object first enters a deletion in progress state, while in background cascading deletion, the owner object is deleted immediately, and the garbage collector cleans up dependents in the background. You can also configure parameters like imageMaximumGCAge to control how long unused images are retained, and set limits on the number of dead containers per pod with MaxPerPodContainer.

In production, you need to be aware of some gotchas. Cross-namespace owner references are disallowed, which can lead to issues if not properly managed. Additionally, the kubelet only garbage collects containers it manages, and it doesn't track image usage across restarts. This means that if the kubelet restarts, the age tracking resets, delaying garbage collection. Be cautious when using external garbage collection tools, as they can interfere with kubelet behavior and remove containers that should exist.