GPU Autoscaling in Kubernetes: Mastering KEDA with External Scalers

In the world of Kubernetes, efficiently managing GPU resources can be a game changer, especially for workloads that demand high computational power. KEDA (Kubernetes Event-driven Autoscaling) allows you to autoscale based on external metrics, making it ideal for applications that require dynamic scaling based on GPU usage. By leveraging KEDA, you can ensure that your GPU resources are utilized effectively, scaling up when demand spikes and scaling down when it's not needed.

To implement GPU autoscaling, you can build a custom DaemonSet that runs on GPU nodes. Each pod in this DaemonSet will call NVML (NVIDIA Management Library) to read local GPU metrics. It then serves these metrics over gRPC using KEDA's ExternalScaler interface. The KEDA operator connects to your scaler and drives HPA decisions based on the metrics provided. Key configuration parameters include scalerAddress, which defaults to keda-gpu-scaler.gpu-scaler.svc.cluster.local:6000, and profile, which you can set to your specific scaling profile, such as vllm-inference. You can also define minReplicaCount and maxReplicaCount to control the scaling limits.

When deploying this in production, remember that the integration of KEDA with your existing workloads can lead to significant resource savings. However, ensure you thoroughly test your setup, especially the communication between the DaemonSet and KEDA. Use the provided Helm command to install the GPU scaler and the YAML configuration to define your ScaledObject. Keep in mind that as of May 27, 2026, this setup is still evolving, so stay updated with any changes in the KEDA project that may affect your implementation.