Prometheus Monitoring Stack
12 posts in reading order. All series →
Tags, archive, series1 active
Posts 12
-
Writing good PrometheusRules: structure, labels, and unit tests
PrometheusRule anatomy with required helm-chart/namespace labels, Grafana deep-link annotations, humanizeTimestamp and humanizeDuration in alert messages, and promtool rule unit test framework.
-
Kafka alerting in Kubernetes: from broker down to consumer lag
PrometheusRule alerts for Kafka: BrokerDown, UnderReplicatedPartitions, OfflinePartitions, consumer group lag, disk fill prediction, and JVM memory pressure with kafka-exporter metrics.
-
node-exporter alerting: clocks, disks, and inotify limits
PrometheusRule alerts for clock skew, CPU saturation, memory pressure, disk fullness prediction, inotify watch exhaustion, and network error rates — with practical thresholds from production.
-
Prometheus exporters in Kubernetes: kafka, mongodb, redis, and more
HelmRelease configs for kafka-exporter, mongodb-exporter, and redis-exporter. RabbitMQ built-in Prometheus support. helm-exporter for tracking chart versions. ServiceMonitor patterns for each.
-
kube-prometheus-stack on multi-cluster: spoke Prometheus, central Grafana
Grafana on one cluster, Prometheus on all six. Spoke kube-prom-stack with Grafana disabled, externalLabels.cluster for cross-cluster dashboards.
-
VictoriaMetrics Operator: VMAgent, VMSingle, VMServiceScrape
VMSingle for 6-month retention, VMAgent as scraper with remote_write only, VMServiceScrape as drop-in for ServiceMonitor, VMRule, Grafana datasource setup.
-
Prometheus recording rules: pre-aggregating expensive queries
PrometheusRule with record: rules for CPU/memory/HTTP rate. Naming convention level:metric:operations. SLO burn rate multi-window pre-computation.
-
Grafana datasource and dashboard provisioning via ConfigMap
additionalDataSources in kube-prom-stack values, sidecar dashboard discovery from ConfigMaps with grafana_dashboard label, dashboards in git, folder structure.
-
kube-prometheus-stack: the full monitoring setup
kube-prometheus-stack HelmRelease with Prometheus persistence and retention tuning, Grafana admin secret, ServiceMonitor for custom apps, sidecar dashboard loading, and storage sizing estimates.
-
Blackbox exporter and Prometheus Probes for endpoint monitoring
Using prometheus-blackbox-exporter and Probe CRDs to monitor HTTP endpoints, TCP ports, and TLS certificate expiry across clusters.
-
Multi-cluster Prometheus: remote_write to central VictoriaMetrics
Spoke Prometheus instances remote_write to a central VictoriaMetrics on the infra cluster. externalLabels.cluster, writeRelabelConfigs for traffic reduction, cross-cluster dashboards.
-
vmsingle: one VictoriaMetrics binary instead of the whole Prometheus stack
VictoriaMetrics single-node on k0s: embedded scraper with static targets, cAdvisor RBAC, hostPath PV, and a stable ClusterIP to survive CoreDNS restarts.
No posts match the selected filters.