series: Prometheus Monitoring Stack
- vmsingle: one VictoriaMetrics binary instead of the whole Prometheus stack VictoriaMetrics single-node on k0s: embedded scraper with static targets, cAdvisor RBAC, hostPath PV, and a stable ClusterIP to survive CoreDNS restarts.
- Multi-cluster Prometheus: remote_write to central VictoriaMetrics Spoke Prometheus instances remote_write to a central VictoriaMetrics on the infra cluster. externalLabels.cluster, writeRelabelConfigs for traffic reduction, cross-cluster dashboards.
- Blackbox exporter and Prometheus Probes for endpoint monitoring Using prometheus-blackbox-exporter and Probe CRDs to monitor HTTP endpoints, TCP ports, and TLS certificate expiry across clusters.
- kube-prometheus-stack: the full monitoring setup kube-prometheus-stack HelmRelease with Prometheus persistence and retention tuning, Grafana admin secret, ServiceMonitor for custom apps, sidecar dashboard loading, and storage sizing estimates.
- Grafana datasource and dashboard provisioning via ConfigMap additionalDataSources in kube-prom-stack values, sidecar dashboard discovery from ConfigMaps with grafana_dashboard label, dashboards in git, folder structure.
- Prometheus recording rules: pre-aggregating expensive queries PrometheusRule with record: rules for CPU/memory/HTTP rate. Naming convention level:metric:operations. SLO burn rate multi-window pre-computation.
- VictoriaMetrics Operator: VMAgent, VMSingle, VMServiceScrape VMSingle for 6-month retention, VMAgent as scraper with remote_write only, VMServiceScrape as drop-in for ServiceMonitor, VMRule, Grafana datasource setup.
- kube-prometheus-stack on multi-cluster: spoke Prometheus, central Grafana Grafana on one cluster, Prometheus on all six. Spoke kube-prom-stack with Grafana disabled, externalLabels.cluster for cross-cluster dashboards.
- Prometheus exporters in Kubernetes: kafka, mongodb, redis, and more HelmRelease configs for kafka-exporter, mongodb-exporter, and redis-exporter. RabbitMQ built-in Prometheus support. helm-exporter for tracking chart versions. ServiceMonitor patterns for each.
- node-exporter alerting: clocks, disks, and inotify limits PrometheusRule alerts for clock skew, CPU saturation, memory pressure, disk fullness prediction, inotify watch exhaustion, and network error rates — with practical thresholds from production.
- Kafka alerting in Kubernetes: from broker down to consumer lag PrometheusRule alerts for Kafka: BrokerDown, UnderReplicatedPartitions, OfflinePartitions, consumer group lag, disk fill prediction, and JVM memory pressure with kafka-exporter metrics.
- Writing good PrometheusRules: structure, labels, and unit tests PrometheusRule anatomy with required helm-chart/namespace labels, Grafana deep-link annotations, humanizeTimestamp and humanizeDuration in alert messages, and promtool rule unit test framework.
No posts match the selected filters.