Problem Context

Helm is Kubernetes' de-facto package manager. You will love it for the first 5 charts, fight it for the next 10, and either adopt disciplined patterns or migrate to Kustomize/Carvel/Timoni for the rest. The framework is not the problem โ€” the problem is that teams adopt Helm without a chart strategy and end up with 200-line values.yaml files nobody understands.

๐Ÿค” Sound familiar?
  • Your values.yaml is so deeply nested you can't override anything cleanly
  • helm upgrade drifts from git diff; nobody knows what's actually in the cluster
  • A failed upgrade left the release in pending-upgrade and you can't roll back

The Mental Model

A Helm chart is a templated bundle of Kubernetes manifests. A release is one installation of that chart with specific values, tracked by name + revision in a Kubernetes secret. helm upgrade --install is the verb you actually use; helm rollback is the panic button.


flowchart LR
    C[Chart] --> T[templates/*.yaml]
    V[values.yaml] --> R[helm template / upgrade]
    O[--values override.yaml] --> R
    T --> R
    R --> M[Rendered manifests]
    M --> K[Kubernetes API]
    K --> S[Release secret<br/>helm.sh/release.v3.<name>.<rev>]
      

Chart Layout That Doesn't Rot

my-app/
  Chart.yaml          # name, version, appVersion, dependencies
  values.yaml         # documented defaults; every key has a comment
  values.schema.json  # JSON schema โ€” fail fast on typos
  templates/
    _helpers.tpl      # named templates (fullname, labels, selectors)
    deployment.yaml
    service.yaml
    hpa.yaml
    NOTES.txt         # printed on install
  charts/             # vendored dependencies, never edit
  ci/                 # ci-values-*.yaml for chart-testing

The values.schema.json is the single best Helm habit. It rejects typos at helm install time instead of producing a silently broken Pod.

Three Rules for Templates

  • One concern per file โ€” Deployment in deployment.yaml, Service in service.yaml. Don't multiplex.
  • Helpers for labels and names โ€” {{ include "my-app.labels" . | nindent 4 }} everywhere; never hand-write.
  • Minimal templating โ€” if-blocks are debug nightmares. Prefer flat values with a default and explicit overrides.

Values Strategy

Layer values, don't copy them:

helm upgrade --install api ./my-app \
  -f values.yaml \
  -f environments/prod/values.yaml \
  -f environments/prod/secrets.dec.yaml \
  --set image.tag=sha-3f2a9c8
  • Base values in the chart.
  • Per-environment overlay file in your deploy repo.
  • Per-release --set only for things that change per deploy (image tag).
  • Secrets via SOPS-encrypted YAML, External Secrets, or sealed-secrets โ€” never plain.

Dependencies and Umbrella Charts

Subcharts are great for reusing community charts (Postgres, Redis, kube-prometheus-stack). Umbrella charts that bundle "your whole platform" into one release tend to age badly โ€” one chart upgrade blocks everything. Prefer many small releases orchestrated by Argo CD / Flux instead.

GitOps: Helm Without Running Helm

In a modern setup you don't run helm from your laptop. Argo CD (with the Helm source type) or Flux (with HelmRelease) renders charts in-cluster, reconciles to git, and gives you drift detection and atomic rollbacks. The Helm CLI becomes a dev/debug tool.

Upgrades, Hooks, and Atomicity

  • --atomic rolls back on failure โ€” always use it in CI.
  • --wait --timeout 10m waits for Pods to be Ready before marking success.
  • Hooks (helm.sh/hook: pre-upgrade) for migrations โ€” but they can leave clusters in weird states; prefer Job manifests with proper dependencies via Argo.
  • helm history + helm rollback NAME N recovers most bad upgrades; if release stuck in pending-upgrade, patch the secret to deployed.

Failure Modes

  • Templates without a schema โ€” typos in values.yaml succeed silently and break Pods.
  • Capabilities-based templates โ€” .Capabilities.APIVersions.Has evaluated at render time differs across clusters; pin to versions you support.
  • Chart-as-database โ€” embedding business config in values; use ConfigMaps / external config stores instead.
  • Mass helm dependency update โ€” pulls latest subchart versions; pin and review.

Production Checklist

  • Every chart has values.schema.json and CI runs helm lint + helm template | kubeconform.
  • Releases managed by Argo CD or Flux; helm CLI is a dev tool.
  • Layered values: chart โ†’ env overlay โ†’ SOPS secrets โ†’ per-deploy image tag.
  • Pinned subchart versions; renovate PRs reviewed, not auto-merged.
  • --atomic --wait in any direct CI install path.

Key Takeaways

  • Helm pays off if you bring schema validation, layered values, and GitOps. Without those it becomes a YAML factory.
  • Many small releases beat one umbrella chart for blast radius and ergonomics.
  • The CLI is for debugging; production lives in Argo/Flux.