SealedSecrets: storing encrypted kubeconfigs in git

Published: 2026-02-17

Every kubeconfig, registry credential, and TLS certificate in the cluster lives in git — encrypted. SealedSecrets from Bitnami makes this possible with asymmetric encryption: public key in the repo, private key only in the hub cluster. No Vault required for this pattern, and no CI/CD secret injection.

How it works

copyplaintext Secret
       │
       ▼  kubeseal + pub cert
  SealedSecret (encrypted, safe to commit)
       │
       ▼  git push → Flux → hub cluster
  SealedSecret CRD applied to hub
       │
       ▼  sealed-secrets-controller decrypts with private key
  plain Secret in cluster

The controller runs in flux-system on hub. It decrypts every SealedSecret it sees and creates a corresponding plain Kubernetes Secret. No human ever stores the private key locally. The encryption happens at sealing time — the public key is the only thing needed to seal, and it can be stored in the repo.

Why this is safe

The encryptedData field in a SealedSecret is encrypted with RSA-4096/OAEP + AES-256-GCM. Even if someone clones your GitOps repo, they can't decrypt the secrets without the controller's private key. The private key never leaves the cluster (unless you intentionally back it up to Vault or a key management system).

Installing SealedSecrets

bashcopy# Via Helm (recommended)
helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
helm install sealed-secrets -n flux-system \
  sealed-secrets/sealed-secrets \
  --set fullnameOverride=sealed-secrets-controller

# kubeseal CLI
brew install kubeseal  # macOS
# or download from https://github.com/bitnami-labs/sealed-secrets/releases

Or as a FluxCD HelmRelease:

yamlcopyapiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: sealed-secrets
  namespace: flux-system
spec:
  chart:
    spec:
      chart: sealed-secrets
      version: "2.*"
      sourceRef:
        kind: HelmRepository
        name: sealed-secrets
        namespace: flux-system
  values:
    fullnameOverride: sealed-secrets-controller

Sealing a new secret

bashcopy# 1. Write the plaintext secret to a temp file (DO NOT COMMIT)
cat > /tmp/my-secret.yaml <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: my-secret
  namespace: dev
type: Opaque
stringData:
  password: "hunter2"
EOF

# 2. Seal it (fetches pub cert from hub)
kubeseal \
  --format yaml \
  --controller-namespace flux-system \
  --kubeconfig .kubeconfigs/infra.yaml \
  < /tmp/my-secret.yaml \
  > fluxcd/projects/dev/kustomization/spoke/secrets/my-secret.yaml

# 3. Delete the plaintext
rm /tmp/my-secret.yaml

# 4. Add to kustomization.yaml and commit

The SealedSecret is namespace-bound — dev namespace encrypted data cannot be decrypted in a different namespace. If you need the same secret in multiple namespaces, you must seal it separately for each namespace.

Sealing with a local cert (offline)

If you don't have cluster access at sealing time:

bashcopy# First, fetch and save the cert
kubeseal \
  --fetch-cert \
  --controller-namespace flux-system \
  --kubeconfig .kubeconfigs/infra.yaml \
  > .kubeconfigs/sealed-secrets-infra.pem

# Later, seal offline using the saved cert
kubeseal \
  --format yaml \
  --cert .kubeconfigs/sealed-secrets-infra.pem \
  < /tmp/my-secret.yaml \
  > sealed-my-secret.yaml

Store the .pem cert alongside your kubeconfigs. It's not sensitive — anyone can seal with it, but only the cluster controller can unseal.

Updating an existing secret

SealedSecrets cannot be partially updated. You always reseal the entire secret:

bashcopykubeseal \
  --format yaml \
  --controller-namespace flux-system \
  --kubeconfig .kubeconfigs/infra.yaml \
  < /tmp/new-values.yaml \
  > fluxcd/projects/dev/kustomization/spoke/secrets/my-secret.yaml

The new encryptedData replaces the old one. The controller detects the update and recreates the plain Secret.

Anti-pattern: editing the encryptedData field manually in spec.encryptedData. The ciphertext is not separable by key — the entire blob changes when any value changes. Just reseal from scratch.

Where secrets live in the repo

Spoke kubeconfigs embedded in env-kustomization

The most important secrets are the spoke kubeconfigs. Each one is embedded directly in fluxcd/{env}-kustomization.yaml alongside the Namespace and Flux Kustomization objects:

yamlcopy# fluxcd/dev-kustomization.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: dev
---
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: dev-kubeconfig
  namespace: dev
spec:
  encryptedData:
    value: AgB3eVxk...very-long-base64...
  template:
    metadata:
      name: dev-kubeconfig
      namespace: dev
    type: Opaque
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: dev-apps
  namespace: dev
spec:
  path: ./fluxcd/projects/dev/kustomization/hub/
  kubeConfig:
    secretRef:
      name: dev-kubeconfig

The Namespace, SealedSecret, and Kustomization referencing it are all in one file. Flux creates them in order; the kubeConfig secret becomes available as soon as the controller decrypts the SealedSecret.

This structure makes each spoke cluster self-contained in a single file: adding a new spoke is cp dev-kustomization.yaml staging-kustomization.yaml + seal the staging kubeconfig.

Controller key rotation

The sealed-secrets-controller rotates its signing key every 30 days by default. Old keys are kept so existing SealedSecrets can still be decrypted. This means SealedSecrets sealed a year ago still work.

However, if you delete the controller and reinstall it, the old private keys are gone and all SealedSecrets become undecryptable. Always back up the controller secrets:

bashcopy# Back up all controller keys
kubectl get secret \
  -n flux-system \
  -l sealedsecrets.bitnami.com/sealed-secrets-key \
  -o yaml > sealed-secrets-keys-backup.yaml

Store this backup in a secure location (Vault, offline storage). If you need to restore:

bashcopykubectl apply -f sealed-secrets-keys-backup.yaml
kubectl rollout restart deployment/sealed-secrets-controller -n flux-system

Cluster migration

When migrating to new hardware with a fresh cluster, the private key changes. Every SealedSecret must be re-sealed:

bashcopy# 1. Fetch the new cluster's cert
kubeseal --fetch-cert \
  --kubeconfig new-cluster.yaml \
  --controller-namespace flux-system \
  > sealed-secrets-new.pem

# 2. Re-seal each secret
# (requires the original plaintext — keep it in a separate secure location during migration)
kubeseal --cert sealed-secrets-new.pem --format yaml < original-secret.yaml > new-sealed-secret.yaml

There's no shortcut here. The alternative is to copy the controller keys from the old cluster to the new one before recreating secrets.

Troubleshooting decryption failures

bashcopy# Check controller logs
kubectl logs \
  -n flux-system \
  -l app.kubernetes.io/name=sealed-secrets-controller \
  --context=infra-k8s

# List all sealed-secrets CRDs
kubectl get sealedsecret -A

# Check status of a specific sealed secret
kubectl describe sealedsecret my-secret -n dev

Common error messages:

• no key could decrypt secret — sealed with a different controller cert; reseal with current cert
• failed to decrypt — namespace mismatch; reseal for the correct namespace
• controller not found — wrong --controller-namespace; the controller name must match fullnameOverride

If the controller logs show decryption failures for all secrets simultaneously after an upgrade, the controller may have lost its keys. Check if the key secrets still exist:

bashcopykubectl get secret -n flux-system -l sealedsecrets.bitnami.com/sealed-secrets-key