Kafka in Kubernetes with the Bitnami chart

Published: 2026-02-23

The Bitnami kafka chart is one of the most feature-complete options for running Kafka on Kubernetes. It bundles ZooKeeper as a sub-chart dependency, ships JMX exporter as a sidecar, and generates NetworkPolicy, ServiceMonitor, and PrometheusRule manifests out of the box. This post walks through the key knobs we turned to make it production-worthy.

Chart setup

yamlcopy# Chart.yaml (wrapper chart)
apiVersion: v2
name: kafka
version: 0.1.0
dependencies:
  - name: kafka
    version: "22.x.x"
    repository: "oci://registry-1.docker.io/bitnamicharts"
  - condition: zookeeper.enabled
    name: zookeeper
    version: "11.x.x"
    repository: "oci://registry-1.docker.io/bitnamicharts"

Pin the chart version in Chart.lock and commit it. Floating 22.x.x is fine for helm dependency update but the lock file ensures reproducible deploys. Run helm dependency update ./kafka to regenerate the lock file after bumping the version.

StatefulSet sizing

yamlcopyreplicaCount: 3

heapOpts: "-Xmx2g -Xms2g"

persistence:
  enabled: true
  storageClass: "yc-network-ssd"
  size: 50Gi

podManagementPolicy: Parallel

resources:
  requests:
    cpu: 500m
    memory: 3Gi
  limits:
    cpu: 2
    memory: 4Gi

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchLabels:
            app.kubernetes.io/name: kafka
        topologyKey: kubernetes.io/hostname

Parallel pod management is safe for Kafka because brokers don't form quorum during startup — ZooKeeper does. It cuts rolling-restart time significantly on 3+ broker clusters.

Set JVM heap to ~75% of container memory limit. With a 4Gi limit, -Xmx2g -Xms2g leaves 2Gi for OS page cache, which Kafka uses heavily for read performance.

The podAntiAffinity rule ensures each broker lands on a different node. Without it, two brokers on the same node means a single node failure takes down quorum.

Listeners and authentication

For intra-cluster communication we use PLAINTEXT (encrypted at the network level by Cilium). For clients that connect from outside the namespace we enforce SASL/SCRAM:

yamlcopyauth:
  clientProtocol: sasl
  interBrokerProtocol: plaintext
  sasl:
    mechanisms: scram-sha-256
    interBrokerMechanism: plain
    jaas:
      clientUsers:
        - app-user
        - monitoring-user
      clientPasswords:
        - ""   # injected via ESO; leave empty here
        - ""

The passwords are mounted from a Kubernetes Secret created by External Secrets Operator from Vault. The chart looks for kafka-jaas Secret in the same namespace and mounts it as a volume — so we populate only the secret, not the values file.

Why SCRAM-SHA-256 over SCRAM-SHA-512

SCRAM-SHA-256 is widely supported by all Kafka client libraries. SHA-512 provides marginally better security but has compatibility issues with some older Java clients. For new deployments, SHA-256 is the pragmatic choice.

JMX exporter sidecar

The chart ships a bitnami/jmx-exporter sidecar and a matching jmx-configmap.yaml. Enabling it exposes broker metrics on port 5556 in the Prometheus format:

yamlcopymetrics:
  jmx:
    enabled: true
    image:
      registry: docker.io
      repository: bitnami/jmx-exporter
      tag: 0.17.2-debian-11-r34
    containerSecurityContext:
      runAsUser: 1001
      runAsNonRoot: true
    resources:
      requests:
        cpu: 50m
        memory: 128Mi
      limits:
        cpu: 200m
        memory: 256Mi
  serviceMonitor:
    enabled: true
    namespace: monitoring
    labels:
      release: kube-prometheus-stack
  kafka:
    enabled: true
    serviceMonitor:
      enabled: true

The JMX config file (mounted from jmx-configmap.yaml) whitelists broker metrics: kafka.server:type=BrokerTopicMetrics, kafka.network:type=RequestMetrics, etc.

Key metrics to watch:

• kafka_server_replicamanager_underreplicatedpartitions — should be 0
• kafka_controller_kafkacontroller_activecontrollercount — exactly 1 across the cluster
• kafka_network_requestmetrics_requestspersec — traffic patterns per request type
• kafka_server_brokertopicmetrics_messagesinpersec — throughput per topic

PrometheusRule

yamlcopymetrics:
  prometheusRule:
    enabled: true
    rules:
      - alert: KafkaUnderReplicatedPartitions
        expr: kafka_server_replicamanager_underreplicatedpartitions > 0
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "Under-replicated partitions on {{ $labels.pod }}"
      - alert: KafkaOfflinePartitions
        expr: kafka_controller_kafkacontroller_offlinepartitionscount > 0
        for: 5m
        labels:
          severity: critical
      - alert: KafkaActiveControllers
        expr: sum(kafka_controller_kafkacontroller_activecontrollercount) != 1
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "Wrong number of active Kafka controllers (not exactly 1)"
      - alert: KafkaConsumerGroupLag
        expr: kafka_consumergroup_lag_sum > 10000
        for: 15m
        labels:
          severity: warning
        annotations:
          summary: "Consumer group {{ $labels.consumergroup }} is lagging on topic {{ $labels.topic }}"

NetworkPolicy

The chart generates networkpolicy-ingress.yaml and networkpolicy-egress.yaml when you set networkPolicy.enabled: true. We extend them with a CiliumNetworkPolicy to allow only specific consumer namespaces to reach port 9092:

yamlcopynetworkPolicy:
  enabled: true
  allowExternal: false
  additionalRules:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: orders
      ports:
        - port: 9092
          protocol: TCP
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: notifications
      ports:
        - port: 9092
          protocol: TCP

Provisioning topics

The chart has a provisioning Job that runs after the brokers are ready and creates topics via the Kafka CLI:

yamlcopyprovisioning:
  enabled: true
  numPartitions: 6
  replicationFactor: 3
  topics:
    - name: orders.created
      partitions: 6
      replicationFactor: 3
      config:
        retention.ms: "604800000"   # 7 days
        cleanup.policy: delete
    - name: payments.completed
      partitions: 3
      replicationFactor: 3
      config:
        retention.ms: "2592000000"  # 30 days
    - name: dead-letter
      partitions: 3
      replicationFactor: 3
      config:
        retention.ms: "2592000000"  # 30 days
        cleanup.policy: delete

The provisioning Job uses a dedicated ServiceAccount with a Role that is allowed only to exec into the kafka pod.

Partition count guidelines

More partitions allow higher parallelism but increase ZooKeeper/broker overhead. For most microservice traffic, 6 partitions per topic is a reasonable default.

PodDisruptionBudget

yamlcopypdb:
  create: true
  minAvailable: 2

With 3 brokers and minAvailable: 2, Kubernetes will refuse to drain more than one broker node at a time. Combined with offsets.topic.replication.factor=3 this ensures consumer group offsets survive a rolling node drain.

ZooKeeper sub-chart

We run ZooKeeper as a 3-node quorum co-located in the same Helm release:

yamlcopyzookeeper:
  enabled: true
  replicaCount: 3
  persistence:
    enabled: true
    storageClass: "yc-network-ssd"
    size: 8Gi
  resources:
    requests:
      cpu: 100m
      memory: 512Mi
    limits:
      cpu: 500m
      memory: 1Gi

For clusters where ZooKeeper is shared across multiple Kafka clusters, disable the sub-chart (zookeeper.enabled: false) and point externalZookeeper.servers at the shared ensemble.

ZooKeeper needs to elect a leader before Kafka brokers can register. If brokers crash-loop on startup, check ZooKeeper logs first — failed ZK quorum is the most common cause.

Flux HelmRelease

yamlcopyapiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: kafka
  namespace: kafka
spec:
  interval: 10m
  chart:
    spec:
      chart: ./kafka
      sourceRef:
        kind: GitRepository
        name: infra-repo
      interval: 10m
  valuesFrom:
    - kind: Secret
      name: kafka-jaas
      valuesKey: jaasPasswords
      targetPath: auth.sasl.jaas.clientPasswords
  values:
    replicaCount: 3

valuesFrom injects the SASL passwords from the Secret at reconcile time. The values file never contains the actual password string.

Debugging

bashcopy# Check broker logs
kubectl logs -n kafka kafka-0 -f

# List topics
kubectl exec -n kafka kafka-0 -- \
  kafka-topics.sh --bootstrap-server localhost:9092 \
  --command-config /opt/bitnami/kafka/config/consumer.properties \
  --list

# Describe under-replicated partitions
kubectl exec -n kafka kafka-0 -- \
  kafka-topics.sh --bootstrap-server localhost:9092 \
  --describe --under-replicated-partitions

# Consumer group lag
kubectl exec -n kafka kafka-0 -- \
  kafka-consumer-groups.sh --bootstrap-server localhost:9092 \
  --describe --group orders-consumer

Kafka on Kubernetes works best when you treat it like the stateful beast it is: fixed storage class, PDB, conservative resource limits, pod anti-affinity, and JMX monitoring from day one.