sorbandClaude Opus 4.8
e9c13dcf22
Alertmanager was configured as an alerting target but never scraped, so its own metrics were absent: a silently breaking alert chain could not report itself — the same blind spot as a missing series, now at the end of the chain. Adds the operating_alertmanager scrape job and AlertDeliveryFailing on alertmanager_notifications_failed_total. The README still claimed alert delivery was deliberately muted via a room=security null receiver. That route is gone; alertmanager.yml routes everything to the matrix receiver, so the backup alerts added yesterday do get delivered. Documentation asserting the opposite is dangerous in both directions, so it now states the current wiring and keeps the alert-storm history as background. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
63 lines
1.7 KiB
YAML
63 lines
1.7 KiB
YAML
global:
|
|
scrape_interval: 15s
|
|
|
|
rule_files:
|
|
- /etc/prometheus/alerts.yml
|
|
|
|
alerting:
|
|
alertmanagers:
|
|
- static_configs:
|
|
- targets: ["alertmanager:9093"]
|
|
|
|
# Zusaetzlich zu diesen Scrape-Targets kommen Metriken per Remote-Write rein
|
|
# (k3s-Cluster: flux, kube_state_metrics; Matrix-Server: synapse).
|
|
scrape_configs:
|
|
## Operating-Server (dieser Host)
|
|
- job_name: "operating_prometheus"
|
|
static_configs:
|
|
- targets: ["localhost:9090"]
|
|
|
|
- job_name: "operating_node-exporter"
|
|
static_configs:
|
|
- targets: ["node-exporter:9100"]
|
|
|
|
- job_name: "operating_cadvisor"
|
|
# cadvisor laeuft im Portainer-Stack "thread-net-git", selbes traefik-Netz
|
|
static_configs:
|
|
- targets: ["cadvisor:8080"]
|
|
|
|
# Alertmanager-Eigenmetriken. Ohne diesen Job sind ZUSTELLfehler unsichtbar:
|
|
# Prometheus kennt alertmanager zwar als Alarm-Ziel (oben unter alerting:),
|
|
# scrapt ihn aber nicht - eine still reissende Alarmkette waere also genau das,
|
|
# was niemand meldet. Liefert u.a. alertmanager_notifications_failed_total.
|
|
- job_name: "operating_alertmanager"
|
|
static_configs:
|
|
- targets: ["alertmanager:9093"]
|
|
|
|
- job_name: "operating_traefik"
|
|
metrics_path: "/metrics"
|
|
static_configs:
|
|
- targets: ["traefik:8080"]
|
|
|
|
## Externe Hosts
|
|
- job_name: "k3s_host_node"
|
|
static_configs:
|
|
- targets: ["10.0.0.2:9100"]
|
|
labels:
|
|
instance: "k3s-host"
|
|
|
|
- job_name: "pterodactyl_host_node"
|
|
static_configs:
|
|
- targets: ["157.90.155.206:9100"]
|
|
labels:
|
|
instance: "pterodactyl"
|
|
|
|
- job_name: "gameserver_cadvisor"
|
|
static_configs:
|
|
- targets: ["157.90.155.206:8080"]
|
|
|
|
# CVE-Exporter (gitops#47)
|
|
- job_name: "cve_exporter"
|
|
static_configs:
|
|
- targets: ["cve-exporter:9101"]
|