Mastering EKS Control Plane Egress: Route Your Traffic with Precision
The introduction of customer-routed control plane egress in Amazon EKS is a game changer for managing Kubernetes traffic. It allows you to route Kubernetes control plane traffic through your own Amazon VPC, providing enhanced security and compliance. This capability is crucial for organizations that need to control how their Kubernetes API Server communicates with external services, ensuring that sensitive data doesn’t traverse the public internet.
When you enable this feature, the Kubernetes API Server is isolated on each control plane instance, and its egress is bound to an Elastic Network Interface (ENI) in your specified subnets. This means that traffic such as admission webhook calls, OIDC discovery requests, and DNS resolutions will flow through your VPC ENI. You can configure the egress mode using the controlPlaneEgressMode parameter, which defaults to AWS_MANAGED. To switch to CUSTOMER_ROUTED, you can use the AWS CLI commands provided. Note that once you set this mode, it is permanent for the life of the cluster, so plan accordingly.
In production, be aware that the setting is irreversible. If you enable CUSTOMER_ROUTED, you cannot revert to AWS_MANAGED. This could lead to complications if your networking needs change. Additionally, consider the implications of your AWS Organizations Service Control Policies (SCPs) to enforce this setting across your clusters. Properly managing these configurations will ensure that your Kubernetes environment remains secure and compliant with your organization's policies.
Key takeaways
- →Enable CUSTOMER_ROUTED mode to control Kubernetes API Server egress traffic.
- →Use Elastic Network Interfaces (ENIs) to bind egress to your VPC subnets.
- →Understand that once set to CUSTOMER_ROUTED, you cannot revert to AWS_MANAGED.
- →Configure AWS Organizations SCPs to enforce control plane egress settings.
- →Utilize the `controlPlaneEgressMode` parameter when creating or updating clusters.
Why it matters
This feature enhances security and compliance by ensuring that sensitive control plane traffic remains within your VPC, reducing the risk of exposure to the public internet.
Code examples
aws eks create-cluster \
--name my-cluster \
--kubernetes-version 1.36 \
--role-arn arn:aws:iam::111122223333:role/eks-cluster-role \
--resources-vpc-config subnetIds=subnet-aaa,subnet-bbb,securityGroupIds=sg-xxx,controlPlaneEgressMode=CUSTOMER_ROUTEDaws eks update-cluster-config \
--name my-cluster \
--resources-vpc-config controlPlaneEgressMode=CUSTOMER_ROUTEDaws eks describe-cluster --name my-cluster \
--query "cluster.resourcesVpcConfig.controlPlaneEgressMode"When NOT to use this
The official docs don't call out specific anti-patterns here. Use your judgment based on your scale and requirements.
Want the complete reference?
Read official docsIndustry-standard certifications built by the people behind Linux and Kubernetes. Earn the CKA — the gold standard Kubernetes administrator cert. OpsCanary readers get 30% off year-round with code OPSCANARY3.
Get CKA certified →Deploying Dragonfly Lightweight: P2P Distribution Without the Database Overhead
Tired of heavyweight database stacks slowing down your deployments? Discover how a lightweight Dragonfly deployment leverages Kubernetes primitives like ConfigMaps and headless Services for efficient P2P distribution. This approach simplifies your architecture while maintaining performance.
K8gb: Elevating Global Load Balancing in Kubernetes
K8gb is now a CNCF incubating project, marking a significant step in cloud-native global server load balancing. It automates traffic management and ensures seamless failover using Kubernetes-native health checks and CoreDNS.
TCPRoute and UDPRoute in Gateway API v1.6: What You Need to Know
The Gateway API v1.6 marks a significant step forward with TCPRoute and UDPRoute graduating to standard status. These resources allow you to route traffic based solely on protocol and port, simplifying your networking configurations in Kubernetes.
Get the daily digest
One email. 5 articles. Every morning.
No spam. Unsubscribe anytime.