Mastering EKS Control Plane Egress: Route Your Traffic with Precision
The introduction of customer-routed control plane egress in Amazon EKS is a game changer for managing Kubernetes traffic. It allows you to route Kubernetes control plane traffic through your own Amazon VPC, providing enhanced security and compliance. This capability is crucial for organizations that need to control how their Kubernetes API Server communicates with external services, ensuring that sensitive data doesn’t traverse the public internet.
When you enable this feature, the Kubernetes API Server is isolated on each control plane instance, and its egress is bound to an Elastic Network Interface (ENI) in your specified subnets. This means that traffic such as admission webhook calls, OIDC discovery requests, and DNS resolutions will flow through your VPC ENI. You can configure the egress mode using the controlPlaneEgressMode parameter, which defaults to AWS_MANAGED. To switch to CUSTOMER_ROUTED, you can use the AWS CLI commands provided. Note that once you set this mode, it is permanent for the life of the cluster, so plan accordingly.
In production, be aware that the setting is irreversible. If you enable CUSTOMER_ROUTED, you cannot revert to AWS_MANAGED. This could lead to complications if your networking needs change. Additionally, consider the implications of your AWS Organizations Service Control Policies (SCPs) to enforce this setting across your clusters. Properly managing these configurations will ensure that your Kubernetes environment remains secure and compliant with your organization's policies.
Key takeaways
- →Enable CUSTOMER_ROUTED mode to control Kubernetes API Server egress traffic.
- →Use Elastic Network Interfaces (ENIs) to bind egress to your VPC subnets.
- →Understand that once set to CUSTOMER_ROUTED, you cannot revert to AWS_MANAGED.
- →Configure AWS Organizations SCPs to enforce control plane egress settings.
- →Utilize the `controlPlaneEgressMode` parameter when creating or updating clusters.
Why it matters
This feature enhances security and compliance by ensuring that sensitive control plane traffic remains within your VPC, reducing the risk of exposure to the public internet.
Code examples
aws eks create-cluster \
--name my-cluster \
--kubernetes-version 1.36 \
--role-arn arn:aws:iam::111122223333:role/eks-cluster-role \
--resources-vpc-config subnetIds=subnet-aaa,subnet-bbb,securityGroupIds=sg-xxx,controlPlaneEgressMode=CUSTOMER_ROUTEDaws eks update-cluster-config \
--name my-cluster \
--resources-vpc-config controlPlaneEgressMode=CUSTOMER_ROUTEDaws eks describe-cluster --name my-cluster \
--query "cluster.resourcesVpcConfig.controlPlaneEgressMode"When NOT to use this
The official docs don't call out specific anti-patterns here. Use your judgment based on your scale and requirements.
Want the complete reference?
Read official docsUnified observability — logs, uptime monitoring, and on-call in one place. Used by 50,000+ engineering teams to ship faster and sleep better.
Try Better Stack free →Zero-Downtime Migration: From Ingress NGINX to Envoy Gateway
Migrating from Ingress NGINX to Envoy Gateway without downtime is crucial for maintaining service continuity. By leveraging weighted DNS records, you can run both systems simultaneously and control traffic flow seamlessly. This article breaks down the practical steps to achieve this migration effectively.
Mastering Ingress Request Tracing for Multi-Tenant SaaS on Kubernetes
In a multi-tenant SaaS environment, understanding request flows is crucial for maintaining performance and reliability. By implementing end-to-end ingress request tracing, you can track customer requests through your services using Trace IDs and Span IDs.
Building a Cloud Native Platform: Kairos, k0rdent, and bindy in Action
Creating a cloud native platform from scratch can be daunting. With Kairos, you get an immutable Linux distribution that boots from OCI images, ensuring consistency. Dive into how k0rdent and bindy enhance your Kubernetes management and DNS operations.
Get the daily digest
One email. 5 articles. Every morning.
No spam. Unsubscribe anytime.