Vector: log pipeline from Kubernetes to Elasticsearch

Published: 2026-03-15

Fluentd and Logstash are the traditional log shippers. Vector is the modern alternative: written in Rust, significantly lower CPU and memory footprint, unified sources/transforms/sinks model, and a configuration format that's readable without consulting a wiki.

Architecture

Pods (stdout/stderr)
  → Vector DaemonSet (per node)
  → Transform (parse, enrich, filter)
  → Elasticsearch / Loki / S3

Vector runs as a DaemonSet. Each pod collects logs from container files on the same node. Transformations happen in the pipeline before forwarding.

Install via Helm

yamlapiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: vector
  namespace: observability
spec:
  chart:
    spec:
      chart: vector
      version: "0.x"
      sourceRef:
        kind: HelmRepository
        name: vector
        namespace: flux-system
  values:
    role: Agent
    customConfig:
      data_dir: /vector-data-dir
      sources:
        kubernetes_logs:
          type: kubernetes_logs
          namespace_labels_field: namespace_labels
          node_metadata_fields: [node_ip]
      transforms:
        parse_json:
          type: remap
          inputs: [kubernetes_logs]
          source: |
            if is_string(.message) {
              parsed, err = parse_json(.message)
              if err == null {
                . = merge(., parsed)
              }
            }
        add_env_label:
          type: remap
          inputs: [parse_json]
          source: |
            .environment = "${VECTOR_ENV:-unknown}"
            .cluster = "${VECTOR_CLUSTER:-unknown}"
      sinks:
        elasticsearch:
          type: elasticsearch
          inputs: [add_env_label]
          endpoints:
            - http://elasticsearch.elk.svc.cluster.local:9200
          index: "k8s-%F"
          bulk:
            action: index
    env:
      - name: VECTOR_ENV
        value: "prod"
      - name: VECTOR_CLUSTER
        value: "infra"

VRL: Vector Remap Language

VRL is Vector's expression language for transforms. Purpose-built for log processing — no Lua, no Groovy, no JVM.

Parse JSON logs:

parsed, err = parse_json(.message)
if err == null {
  . = merge(., parsed)
}

If the log line is JSON, merge its fields into the event root. Structured logs from .NET (Microsoft.Extensions.Logging with JSON formatter) are handled automatically.

Drop noisy health check logs:

if starts_with(string!(.message), "GET /healthz") {
  abort
}

abort drops the event entirely. Health probes make up 30–40% of container logs in a busy cluster. Dropping them before shipping saves significant Elasticsearch storage.

Filter by namespace:

if !includes(["app", "production"], .kubernetes.namespace) {
  abort
}

Only ship logs from production namespaces; skip kube-system, monitoring, etc.

Redact sensitive fields:

if exists(.headers.authorization) {
  .headers.authorization = "REDACTED"
}

Redact token values at the pipeline level so you don't have to trust every app to sanitize its logs.

ServiceMonitor for Vector metrics

yamlvalues:
  serviceMonitor:
    enabled: true
    additionalLabels:
      release: kube-prom-stack

Useful metrics:

promql# Log processing rate
rate(vector_component_received_events_total[5m])

# Sink errors (failed deliveries to Elasticsearch)
rate(vector_component_errors_total{component_kind="sink"}[5m])

# Buffer size (backpressure)
vector_buffer_byte_size

Alert on sink errors:

yaml- alert: VectorSinkErrors
  expr: rate(vector_component_errors_total{component_kind="sink"}[5m]) > 0
  for: 5m
  labels:
    severity: warning
  annotations:
    description: Vector is failing to deliver logs to the sink.

Multiple sinks: Elasticsearch and S3

For long-term retention, all logs go to both Elasticsearch (hot, searchable) and S3 (cold, cheap):

yamlsinks:
  elasticsearch:
    type: elasticsearch
    inputs: [add_env_label]
    endpoints: [http://elasticsearch.elk.svc.cluster.local:9200]
    index: "k8s-%F"

  s3_archive:
    type: aws_s3
    inputs: [add_env_label]
    bucket: logs-archive-example
    region: ru-central1
    endpoint: https://storage.yandexcloud.net
    key_prefix: "k8s/%Y/%m/%d/"
    compression: gzip
    encoding:
      codec: ndjson
    auth:
      access_key_id: "${S3_ACCESS_KEY}"
      secret_access_key: "${S3_SECRET_KEY}"

S3 logs are gzip-compressed NDJSON. 1 GB/day of raw logs becomes ~120 MB.

Troubleshooting

Logs not appearing in Elasticsearch:

bash# Check Vector is running
kubectl get pods -n observability -l app.kubernetes.io/name=vector

# Check Vector logs for sink errors
kubectl logs -n observability daemonset/vector --tail=50

# Test the pipeline locally
vector test /etc/vector/vector.yaml

High memory usage: Vector buffers in memory by default. Add disk buffers for high-volume scenarios:

yamlsinks:
  elasticsearch:
    buffer:
      type: disk
      max_size: 268435488  # 256 MiB per node

Missing Kubernetes metadata: Ensure the ServiceAccount has permissions to read pods and namespaces. The Helm chart handles this by default.

weblog