Taming Secret Sprawl in Multi-Account Kubernetes with External Secrets Operator
Managing secrets in a multi-account Kubernetes setup can lead to chaos if not handled properly. Secret sprawl occurs when secrets are scattered across various accounts and clusters, making it difficult to maintain security and compliance. The External Secrets Operator (ESO) addresses this challenge by providing a Kubernetes-native reconciliation model that synchronizes secrets from external secret management systems, like Bitwarden, directly into the Kubernetes Secret API.
At its core, ESO operates through three main components: a centralized secret management system (Bitwarden), the External Secrets Operator running in each Kubernetes cluster, and the Kubernetes Secrets that ESO generates and maintains. When you create an ExternalSecret resource, ESO retrieves the specified secret from Bitwarden and creates or updates the corresponding Kubernetes Secret. This process is governed by a reconciliation loop that continuously checks for updates based on a configured refresh interval, which defaults to 15 minutes. You can customize this behavior using parameters like refreshInterval to ensure your secrets are always current.
In production, you need to be aware of a few key points. First, ensure you have a Kubernetes cluster (EKS, AKS, GKE, etc.) and that you’ve installed both kubectl and helm. The installation of ESO involves adding the external-secrets Helm repository and deploying it with the necessary configurations. Be cautious with your access tokens; granting write access can lead to unintended secret creation in Bitwarden. The version of this tool is set to evolve, so keep an eye on updates and changes to ensure compatibility with your existing systems.
Key takeaways
- →Utilize the External Secrets Operator to synchronize secrets from Bitwarden into Kubernetes seamlessly.
- →Set the `refreshInterval` parameter to control how often secrets are updated, with a default of 15 minutes.
- →Deploy ESO using Helm to simplify installation and management of Custom Resource Definitions.
Why it matters
In production, managing secrets effectively is crucial for maintaining security and operational efficiency. ESO helps eliminate the risks associated with secret sprawl, ensuring that your applications have the necessary credentials without manual overhead.
Code examples
1helm repo add external-secrets https://charts.external-secrets.io
2
3helm install external-secrets external-secrets/external-secrets \
4 --namespace external-secrets \
5 --set installCRDs=true \
6 --set "bitwarden-sdk-server.enabled=true"1cat <<EOF | kubectl apply -f -
2apiVersion: external-secrets.io/v1
3kind: ExternalSecret
4metadata:
5 name: app-payment-creds
6 namespace: app-backend
7spec:
8 refreshInterval: 15m # Automatically rotate every 15 minutes
9 secretStoreRef:
10 name: bitwarden-global-store
11 kind: ClusterSecretStore
12 target:
13 name: payment-creds # The na
14EOF1cat <<EOF | kubectl apply -f -
2apiVersion: external-secrets.io/v1
3kind: ClusterSecretStore
4metadata:
5 name: bitwarden-global-store
6spec:
7 provider:
8 bitwardensecretsmanager:
9 apiURL: https://vault.bitwarden.eu./api
10 identityURL: https://vault.bitwarden.eu./identity
11 auth:
12 secretRef:
13 credentials:
14 key: token
15 name: bitwarden-access-token
16 namespace: external-secrets
17 bitwardenServerSDKURL: https://bitwarden-sdk-server.external-secrets.svc.cluster.local:9998
18 caProvider:
19 type: Secret
20 name: bitwarden-ca-certs
21 key: ca.crt
22 namespace: external-secrets
23 organizationID: <YOUR_ORGANIZATION_ID>
24 projectID: <YOUR_PROJECT_ID>
25EOFWhen NOT to use this
The official docs don't call out specific anti-patterns here. Use your judgment based on your scale and requirements.
Want the complete reference?
Read official docsIndustry-standard certifications built by the people behind Linux and Kubernetes. Earn the CKA — the gold standard Kubernetes administrator cert. OpsCanary readers get 30% off year-round with code OPSCANARY3.
Get CKA certified →Feature Flags in Kubernetes: Mastering AWS AppConfig
Feature flags are essential for decoupling feature releases from deployments, especially in dynamic environments like Kubernetes. With AWS AppConfig, you can implement these flags using a sidecar pattern, allowing for seamless configuration management. Dive in to learn how this integration works and what you need to watch out for in production.
Harnessing the Power of Cloud-Native Agents in Kubernetes
Cloud-native agent harnesses are revolutionizing how we manage distributed systems. With a core engine called the agent loop, these harnesses streamline operations across various environments, including Kubernetes. Dive in to understand how this architecture can enhance your deployment strategy.
Mastering Custom Resources in Kubernetes: A Guide for Operators
Custom resources in Kubernetes allow you to extend the API and tailor it to your needs. By combining them with custom controllers, you create a powerful declarative API that can manage complex applications. Dive into how this works and what you need to watch out for in production.
Get the daily digest
One email. 5 articles. Every morning.
No spam. Unsubscribe anytime.