Counterpart to restore-drill.yaml: that one covers the databases, this the
Synapse media store. Separate job on purpose - the database probe needs a
throwaway Postgres, this one the production PVC read-only, and folding two
different permission and failure pictures into one job makes an emergency
harder to diagnose, not easier.
The name is deliberate. BackupJobFailed already matches restore-drill.*,
so a failure is covered without a new rule and reaches the maintenance
room through the single alertmanager route.
It re-proves its own comparison every run. After the check passes, one
shared file is altered by a byte and the comparison must report it -
otherwise the job fails with "this probe proves nothing". A comparison
that has only ever said "equal" is a guess, and that stays true when it
runs monthly rather than once.
The intersection carries the proof, not the totals: the backup is a
snapshot while production keeps running, and Synapse prunes its own
preview caches. One-sided files are therefore tolerated in url_cache and
url_cache_thumbnails and are an error anywhere else.
The production PVC is mounted read-only and only emptyDir is written. On
today's single node a second read-only mount of the ReadWriteOnce volume
is unproblematic; the comment says what would have to change if a second
node ever appeared, because that failure would alert without anything
being wrong with the data.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F2Q4Ri8NGwyTZzScvKnWFM
Thore CimbalandClaude Opus 5
59b7d288dc