Mastering DaemonSets: Ensuring Node-Local Facilities in Kubernetes

DaemonSets exist to solve a common problem in Kubernetes: ensuring that specific services run on every eligible node. This is particularly important for tasks like logging, monitoring, or any service that requires node-local access. By using a DaemonSet, you can automate the deployment of Pods that provide these facilities, ensuring consistency and reliability across your cluster.

A DaemonSet works by creating a Pod for each eligible node. The DaemonSet controller uses the .spec.affinity.nodeAffinity field to match Pods to the target hosts. Once a Pod is created, the default scheduler takes over, binding the Pod to the node by setting the .spec.nodeName field. If a new Pod cannot fit on a node, the scheduler may preempt existing Pods based on priority. This means that if you want to ensure your DaemonSet Pods always run, consider setting the .spec.template.spec.priorityClassName to a higher priority class. Additionally, the DaemonSet automatically adds tolerations to its Pods, allowing them to run on nodes that are marked as unschedulable.

In production, you need to be aware of a few key gotchas. First, the .spec.selector cannot be mutated after creation, which can lead to orphaned Pods if not managed carefully. Also, while DaemonSets can run on unschedulable nodes due to the automatic tolerations, this behavior might not always be desirable. Always evaluate your node configurations and Pod priorities to avoid unexpected evictions or scheduling conflicts.