ConfigMaps and Secrets Done Right

Introduction¶

ConfigMaps hold non-sensitive configuration; Secrets hold sensitive values. They have nearly identical APIs. The interesting parts are how pods consume them and what "Secret" actually protects.

Two ways to consume¶

As environment variables¶

envFrom:
  - configMapRef:
      name: app-config
  - secretRef:
      name: app-secrets
env:
  - name: DB_PASSWORD
    valueFrom:
      secretKeyRef:
        name: app-secrets
        key: db-password

• Simple, universal.
• Not updated while the pod runs — a changed ConfigMap needs a pod restart.
• Env vars can leak: kubectl describe pod, crash dumps, child processes, /proc/<pid>/environ.

As mounted files¶

volumes:
  - name: config
    configMap:
      name: app-config
  - name: secrets
    secret:
      secretName: app-secrets
containers:
  - volumeMounts:
      - name: config
        mountPath: /etc/app
      - name: secrets
        mountPath: /etc/app/secrets
        readOnly: true

• Each key becomes a file.
• Auto-updated in the volume within ~a minute (unless subPath is used — subPath mounts don't update). Your app still has to re-read the file.
• Preferred for secrets: files aren't inherited by child processes, don't show in describe, can be readOnly.

Secrets are base64, not encrypted¶

kubectl get secret app-secrets -o jsonpath='{.data.db-password}' | base64 -d

Anyone with get secret RBAC, or etcd access, reads them in plaintext. To actually protect them:

1. Encryption at rest — configure the API server with an EncryptionConfiguration (aescbc/kms provider). Now etcd holds ciphertext. Cloud-managed clusters often enable this; verify.
2. RBAC — get/list on Secrets is powerful; restrict it. Don't grant list secrets cluster-wide.
3. External secret managers — External Secrets Operator or the Secrets Store CSI driver pull from Vault / AWS Secrets Manager / GCP SM at pod start, so the source of truth is outside the cluster.
4. Never commit Secret YAML — use Sealed Secrets (encrypts to a cluster-specific key, safe in git) or SOPS.

Rolling pods on config change¶

Kubernetes does not restart pods when a ConfigMap/Secret changes. Options:

• Put a hash of the config in a pod annotation so the Deployment's pod template changes → rollout. Helm: checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}.
• kubectl rollout restart deployment/app after applying config.
• Use a tool like Reloader (watches referenced ConfigMaps/Secrets, triggers rollouts).
• Mounted-file config + an app that reloads on file change (SIGHUP, inotify).

Immutable ConfigMaps/Secrets¶

immutable: true

Prevents accidental edits, and lets the kubelet stop watching them (less API load at scale). You create a new one (app-config-v2) and update the reference.

Verification and troubleshooting¶

kubectl get cm app-config -o yaml
kubectl describe pod mypod | grep -A5 'Environment\|Mounts'
kubectl exec mypod -- env | grep DB_
kubectl exec mypod -- cat /etc/app/secrets/db-password

• Pod stuck CreateContainerConfigError — the ConfigMap/Secret or a referenced key doesn't exist. kubectl describe pod names it.
• Changed the ConfigMap, app still sees old values — env vars don't update (restart needed), or it's a subPath mount (never updates), or the app cached it at startup.
• Mounted secret file has odd permissions — default is 0644; set defaultMode: 0400 on the volume for tighter perms. Note files are owned by root unless fsGroup is set.
• base64 value has a trailing newline — echo -n when creating, or use kubectl create secret generic ... --from-literal / --from-file which handle it.
• Secret visible in kubectl describe — you put it in env inline or in a ConfigMap. Move to secretKeyRef or a mounted volume.
• App in CrashLoop reading config — the file exists but is a directory (whole ConfigMap mounted where a single file was expected) — use subPath or items: to project one key.

Related tools and reading¶

• On-site: Base64 encoder/decoder, YAML validator.
• Related posts: Secrets in CI without leaking them, Debugging CrashLoopBackOff.