Debugging Flux reconciliation: a field guide
Published: 2026-05-17
Flux is declarative and eventually consistent. When something doesn't reconcile, the error is always visible — you just need to know where to look. This post covers the common failure modes and how to diagnose them quickly.
Quick overview
bash# See everything Flux manages, all namespaces
flux get all -A
# Same but only failures
flux get all -A | grep -v "True\|Unknown"
The READY column shows True, False, or Unknown. False means something is actively wrong. Unknown means Flux is waiting (e.g., for a dependency to become ready).
Kustomization not ready
bashflux get kustomizations -A
# dev dev-apps False kustomize build failed: ...
Get the full error:
bashkubectl describe kustomization dev-apps -n dev | tail -30
Common causes:
load-restrictor error
A Kustomize file references something outside its directory:
Error: accumulating resources: accumulation err='merging resources from
'../../../base/app': must build at directory
Fix: don't use ../ paths in resources:. Reference only directories that contain their own kustomization.yaml. If you need to reference a parent directory's file, use a symlink or restructure.
Missing CRD
A CRD from a HelmRelease hasn't been installed yet:
no matches for kind "HelmRelease" in version "helm.toolkit.fluxcd.io/v2"
Fix: add dependsOn so the Kustomization waits for the chart that installs the CRD.
YAML validation error
error: error validating data: ValidationError(Deployment.spec.template.spec.containers[0].resources)
Fix: run kubectl kustomize fluxcd/projects/dev/kustomization/hub/ locally to reproduce, then fix the manifest.
HelmRelease not ready
bashflux get helmreleases -A
# dev elasticsearch False Helm upgrade failed: ...
Full Helm error:
bashkubectl describe helmrelease elasticsearch -n dev
# Look at Status.Conditions → Message field for exact Helm error
Remediation stuck
If the previous upgrade failed and Helm left the release in a failed state, subsequent retries also fail:
bash# Force a reset
flux suspend helmrelease elasticsearch -n dev
helm rollback elasticsearch -n dev # or helm uninstall if rollback fails
flux resume helmrelease elasticsearch -n dev
Values rendering error
A valuesFrom secret doesn't exist or has the wrong key:
unable to get values: secret "apm-es-credentials" not found
Fix: check the ExternalSecret status and ensure the secret exists in the namespace.
Version constraint not satisfied
chart version ">=1.0.0 <2.0.0" not found
Fix: check the chart repo is synced:
bashflux reconcile source helm prometheus-community -n flux-system
flux get source helm -A
Source not syncing
bashflux get source git -A
# flux-system flux-system False failed to checkout and determine revision
bashkubectl describe gitrepository flux-system -n flux-system
Common causes:
- SSH key rotated in GitLab but not updated in the
flux-systemsecret - Network issue: Flux pod can't reach GitLab
- Branch name changed
Fix for SSH key rotation:
bash# Re-create the secret with the new key
kubectl create secret generic flux-system \
--from-file=identity=~/.ssh/flux_ed25519 \
--from-file=identity.pub=~/.ssh/flux_ed25519.pub \
--from-literal=known_hosts="$(ssh-keyscan github.com)" \
-n flux-system --dry-run=client -o yaml | kubectl apply -f -
Force reconcile
bash# Force reconcile a specific Kustomization (re-fetches source first)
flux reconcile kustomization dev-apps -n dev --with-source --timeout=5m
# Force reconcile a HelmRelease
flux reconcile helmrelease elasticsearch -n dev --timeout=10m
# Force re-pull the git source
flux reconcile source git flux-system -n flux-system
--with-source first reconciles the GitRepository (re-fetches from git), then the Kustomization. Without it, Flux uses the cached source.
Suspend and resume
Useful when you need to manually change something that Flux would immediately overwrite:
bash# Pause reconciliation
flux suspend kustomization dev-apps -n dev
flux suspend helmrelease elasticsearch -n dev
# Make manual changes...
kubectl edit deployment elasticsearch-master -n dev
# Resume
flux resume kustomization dev-apps -n dev
flux resume helmrelease elasticsearch -n dev
Without suspending, Flux will revert your manual changes on the next reconciliation interval.
Events stream
bash# Watch Flux events in real time
flux events --watch -A
# Or using kubectl, filter to reconciliation failures only
kubectl get events -n dev --watch --field-selector reason=ReconciliationFailed
# All Flux-related events across all namespaces
kubectl get events -A --field-selector source=flux-system
The flux-system Kustomization
This is the root Kustomization that bootstraps everything else. If it fails, nothing reconciles anywhere in any cluster. It applies fluxcd/kustomization.yaml which registers all environment Kustomizations.
If you add a new file to fluxcd/kustomization.yaml with a syntax error, the entire root Kustomization fails. Always test locally first:
bashkubectl kustomize fluxcd/ --dry-run=client
Useful one-liners
bash# All HelmReleases that aren't ready
flux get helmreleases -A | grep -v "True"
# Conditions summary for all Kustomizations
kubectl get kustomization -A -o custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,READY:.status.conditions[0].status,MSG:.status.conditions[0].message'
# Which HelmReleases are managed by a Kustomization
kubectl get helmrelease -n dev -o yaml | grep "kustomize.toolkit.fluxcd.io/name"
# Latest reconcile timestamp for all resources
flux get all -A -o json | jq '.[] | {name: .name, lastApplied: .status.lastAppliedRevision}'
Summary
- Start with
flux get all -A | grep -v "True\|Unknown"to find failing resources across all namespaces - Kustomization failures: check
kubectl describe kustomizationforload-restrictor, missing CRDs, or YAML validation errors - HelmRelease failures: check
Status.Conditions.Message— remediation stuck meansflux suspend+helm rollback+flux resume - Source failures: most often a rotated SSH key or network issue; re-create the
flux-systemsecret with the new key flux reconcile ... --with-sourceforces both a git fetch and reconciliation in one command- Suspend before manual changes; Flux will revert anything not in git on the next reconciliation cycle