Files
threadnet-operating/monitoring/prometheus
sorbandClaude Opus 4.8 e0808bad90 fix(alerts): a missing series must not silence the backup alerts
Verification on CFGMON showed RestoreDrillStale could never fire: restore-drill
has no last_schedule_time series until its first scheduled run (a manually
triggered job does not set it), and an expression over a missing series yields
nothing. BackupNotRunning shares the flaw — deleting a CronJob removes the very
series the alert reads, so it goes quiet instead of firing.

Fall back to kube_cronjob_created, but aggregate with max by(namespace, cronjob):
'or' matches including __name__, so a bare fallback would return BOTH series and
the never-updating created timestamp would fire permanently once the window
elapsed. Add BackupCronJobMissing so a vanished CronJob is itself the alert.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-14 12:00:00 +00:00
..