VictoriaLogs + Vector: replacing EFK with a lightweight log stack

Published: 2026-03-09

Elasticsearch handles application logs. VictoriaLogs handles cluster-level observability logs — errors and warnings from every namespace, with 30-day retention and 2-node high availability. Vector is the single collector that feeds both.

Architecture

copyPod stdout/stderr
       │
       ▼
  Vector DaemonSet
       │
  ┌────┴─────────────┐
  │                  │
  ▼                  ▼
Elasticsearch    VictoriaLogs
(app/aurora      (error + warn
 namespaces)      from all ns)

Vector runs as a DaemonSet, reads from the Kubernetes log API, parses JSON logs, and routes based on namespace and log level.

VictoriaLogs cluster

Deployed via victoria-logs-cluster chart:

yamlcopyvlstorage:
  replicaCount: 2
  retentionPeriod: 30d
  persistentVolume:
    size: 10Gi
  resources:
    requests:
      cpu: 200m
      memory: 512Mi
    limits:
      cpu: 1
      memory: 2Gi

vlinsert:
  replicaCount: 1
  resources:
    requests:
      cpu: 100m
      memory: 128Mi

vlselect:
  replicaCount: 1
  resources:
    requests:
      cpu: 100m
      memory: 128Mi

Three components: insert, select, storage. Two storage replicas for redundancy. The vlinsert and vlselect components are stateless — scale them independently.

Vector configuration

The Vector config lives inside the victoria-logs HelmRelease values under vector.customConfig. It's structured as a pipeline: sources → transforms → sinks.

Sources:

yamlcopysources:
  k8s:
    type: kubernetes_logs
    exclude_paths_glob_patterns:
      - "/var/log/pods/observability_vector-*/**"  # don't collect Vector's own logs
  internal_metrics:
    type: internal_metrics

Transforms — label enrichment:

yamlcopytransforms:
  dedot_labels:
    type: remap
    inputs: [k8s]
    source: |
      # Kubernetes labels use dots in keys, which break JSON.
      # Convert dots to underscores in all kubernetes.pod_labels keys.
      .kubernetes.pod_labels = map_keys(.kubernetes.pod_labels) -> |k| {
        replace(k, ".", "_")
      }

Transforms — JSON parsing:

yamlcopy  parser:
    type: remap
    inputs: [dedot_labels]
    source: |
      .log = parse_json(.message) ?? .message
      ._msg = string(.log.message) ?? string(.log.msg) ?? string(.message) ?? ""
      del(.message)

Transforms — routing to Elasticsearch (app namespaces):

yamlcopy  aurora_filter:
    type: filter
    inputs: [parser]
    condition: |
      includes(["app", "aurora"], .kubernetes.pod_namespace)

Transforms — routing to VictoriaLogs (errors/warnings from all namespaces):

yamlcopy  error_filter:
    type: filter
    inputs: [parser]
    condition: |
      lvl = downcase(string(.log.level ?? .log.lvl ?? .log.severity ?? ""))
      includes(["error", "warn", "warning", "fatal", "panic", "err"], lvl)

Sinks:

yamlcopysinks:
  elasticsearch:
    type: elasticsearch
    inputs: [aurora_filter]
    endpoints:
      - "http://dev-master.elasticsearch.svc.cluster.local:9200"
    index: "vector-%Y-%m-%d"
    auth:
      strategy: basic
      user: elastic
      password: "${ES_PASSWORD}"

  victorialogs:
    type: elasticsearch
    inputs: [error_filter]
    endpoints:
      - "http://victoria-logs-vlinsert.observability.svc.cluster.local:9428/insert/elasticsearch"
    mode: bulk

VictoriaLogs exposes an Elasticsearch-compatible bulk insert API, so Vector's elasticsearch sink works without modification.

Why Vector instead of Fluentd or Filebeat

VRL (Vector Remap Language) is more readable than Fluentd's record_transformer. The elasticsearch sink works for both Elasticsearch and VictoriaLogs — one config type covers both.

VictoriaLogs query

VictoriaLogs has a web UI and a LogsQL query language:

bashcopykubectl port-forward -n observability svc/victoria-logs-vlselect 9471:9471
# Open http://localhost:9471/select/vmui/

LogsQL examples:

copy# All errors from app namespace in last hour
kubernetes.pod_namespace: app AND level: error

# Specific pod
kubernetes.pod_name: my-app-xxx AND level: error

# Text search
"database connection refused"

# Combined
kubernetes.pod_namespace: app AND "connection refused" AND _time: 1h

Excluding Vector's own logs

The Vector DaemonSet pod has a label vector.dev/exclude: "true". Configure the source to skip it:

yamlcopysources:
  k8s:
    type: kubernetes_logs
    pod_annotation_fields:
      pod_labels: "kubernetes.pod_labels"
    # Exclude pods with this label
    extra_label_selector: "vector.dev/exclude != true"

Without this, Vector logs its own log ingestion, which creates a feedback loop that fills the log store with Vector's own debug output.

Resource sizing

VictoriaLogs is significantly lighter than Elasticsearch for the same data volume:

The disk savings come from VictoriaLogs' compression (~10x vs Elasticsearch ~3x).