sorb hat zu Recht nachgehakt: Meine Aussage, nur der 19. und 20. seien
AAR-pflichtig gewesen, war unbelegt. Die Commits dieser beiden Tage zeigen
zwei weitere Vorfaelle.
1. Safari sendete nach der Freischaltung von v0.5.3 ungefiltert weiter -
LiveKits sender?.replaceTrack uebersprang den Tausch stumm, das rohe Mikrofon
blieb auf der Leitung. Datenschutzbezug: Wer den Filter einschaltete, durfte
annehmen, dass genau das nicht passiert.
2. Alle geplanten Pruefungen waren dauerhaft rot und meldeten damit nichts mehr.
Beide waren in ihren Issues ausfuehrlich dokumentiert - und beide ohne AAR.
Wichtiger als die Nacharbeit: #0054 nennt den Safari-Fall selbst das VIERTE
Vorkommen des Musters 'meldet Erfolg, ist aber blind'. Viermal erkannt, nie als
eigenstaendige Lehre abgelegt - und genau deshalb bin ich am 19./20. dreimal neu
hineingelaufen. Das Muster steht jetzt als Wiki-Seite mit sieben belegten
Vorkommen, Erkennungsmerkmalen und Gegenmitteln.
Der Validator hat die Seite prompt als verwaist gemeldet, bis sie im Index
verlinkt war. Wo ein Werkzeug widerspricht, halte ich den Prozess ein - das ist
exakt der Befund aus #0040.
Nicht als Vorfall gewertet und geprueft: die Boje-Aufraeumarbeit in #0074.
roadmap.md verlangt einen AAR nach jedem Deploy mit Uebergabe und nach
Incidents. Gestern und heute gab es vier solche Ereignisse - Produktionsausfall
durch rc.2, Verlust der DKIM-Eintraege, Game-Host-Deploy ueber zwei Uebergaben,
volle Wurzelpartition auf rohana mit gerissenem Spiegel. Geschrieben wurde
zunaechst keiner. Der letzte AAR im Repo stammte vom 16.08.
Dokumentiert habe ich stattdessen ausfuehrlich IN den Issues. Das ist nicht
wertlos, aber es ist nicht der Ort, an dem AGENTS.md die Lehren sucht - und die
Folge war messbar: dieselben Fallen wurden an einem Tag mehrfach neu entdeckt.
#0040 um eine Rueckmeldung ans Framework ergaenzt: Die Vorgaben wurden dort
eingehalten, wo ein Werkzeug widerspricht (validate, gen_status, Spiegel), und
uebergangen, wo nur Prosa sie verlangt. Mit Vorschlaegen fuers Refinement -
einschliesslich der ehrlichen Alternative, die Definition von Vorfall enger zu
fassen statt strenger zu kontrollieren.
Sperre beisst (wikijs-postgres und element-web-docs erreichen weder Internet noch
CFGMON), erlaubte Pfade funktionieren (DNS, Pod zu Postgres), breite Pods sind
unberuehrt (synapse erreicht rohana, CFGMON und 1.1.1.1).
Zwei eigene Fehlmessungen dokumentiert, weil sie wiederkommen: 10.43.0.1:443 ist
auch aus einem Pod mit breitem Ausgang nicht erreichbar, und /dev/tcp gibt es in
dash nicht - im Synapse-Container meldet es immer Fehlschlag.
Korrektur zur Zaehlung: zehn, nicht neun.
Nebenfund aus sorbs Frage zur URL-Vorschau: Synapses eigene Sperrliste enthaelt
169.254.0.0/16 nicht - heute durch die NetworkPolicy gedeckt, beim Umbau des
Egress-Modells aber nachzutragen.
Arbeitslasten durchgegangen: neun von zwanzig brauchen gar keinen Ausgang, sechs
genau ein Ziel (Storage Box, rohana), fuenf tatsaechlich breiten Zugang - Synapse
wegen aktiver URL-Vorschau, clamav wegen freshclam, coturn und SFU wegen Medien.
Wichtiger technischer Punkt: NetworkPolicies sind additiv. egress-block-metadata
waehlt ALLE Pods und erlaubt 0.0.0.0/0 - eine zusaetzliche enge Policy fuer
einen einzelnen Pod aendert daran nichts. Der Rueckbau muss das Modell drehen:
default-deny-egress plus ausdrueckliche Erlaubnis je Arbeitslast, so wie es
beim Ingress bereits ist.
Zuordnungen sind aus Konfiguration und Aufgabe abgeleitet, nicht gemessen - das
steht auch so im Issue.
#0083: Die drei im Issue genannten Dateien (prometheus.yml, alerts.yml,
alertmanager.yml) sind laengst Verzeichnis-Mounts - der vorgeschlagene Umbau ist
dort umgesetzt. Offen sind noch fuenf Mounts: loki-config.yaml, config.alloy und
drei .py-Skripte.
Bei den Skripten ist die Falle schlimmer, nicht harmloser: Python liest beim
Start, ein Neustart ist ohnehin noetig - und wird deshalb fuer ausreichend
gehalten. Der Prozess startet neu, der Mount zeigt weiter auf den alten Inode,
der Container meldet Erfolg und fuehrt alten Code aus. Ob die laufende
cve-exporter-Instanz die Aenderung vom 19.08. ausfuehrt, ist ungeprueft.
#0088: Beide Zahlen im Issue sind ueberholt - es sind 15 Ingress-Policies mit 21
Regeln, und eine Egress-Policy existiert inzwischen in drei Namespaces. Sie
sperrt aber genau ein Ziel: den Metadaten-Dienst. Alles andere darf weiterhin
ins Internet.
Der Kern des Befunds gilt damit weiter, nur die Zuspitzung 'unbeschraenkt'
nicht mehr. Beide Issues sagten Dinge, die so nicht mehr stimmen - genau die
Klasse, die heute schon mehrfach Zeit gekostet hat.
Zehn von zehn Targets up, 5 Server mit Klarnamen, Recording Rules liefern die
drei laufenden, Serienzahl von 3615 auf 1336 gefallen.
Die Ursache war nie die Firewall, wie ueber Wochen vermutet: Die Ports waren
gar nicht veroeffentlicht - nur im Docker-Netz erreichbar.
Der Exporter auf 9531 hatte nie funktioniert (Client-API mit Application-Key
ueber http), und das zugehoerige Dashboard war deshalb seit jeher leer, ohne
dass es jemandem auffiel. Beides ersetzt.
Zwei Messfallen festgehalten, weil sie wiederkommen: Der Entwicklungstunnel
fuehrt bei cfgmon.lab:9090 auf eine ANDERE Prometheus-Instanz, der echte
operating-Stack ist nur aus dem Cluster ueber 10.0.0.3 erreichbar. Und
--force-recreate ist fuer Prometheus ueberholt - das Verzeichnis wird gemountet.
Offen und unabhaengig: flux2.json nutzt gotk-Metriken, die es in dieser
Flux-Fassung nicht mehr gibt.
Aufraeumteil durch sorb erledigt (wiki-zugang geloescht, die vermutete tote App
gab es nicht). Letztes Abnahmekriterium umgesetzt: taeglicher CronJob zaehlt
Wiki-Nutzer ohne Gruppe und schreibt das Ergebnis nach Loki.
Beide Zweige geprueft - der Warn-Zweig laeuft nur, wenn etwas kaputt ist, also
genau dann, wenn er funktionieren muss.
Nebenfund mit Reichweite ueber dieses Issue hinaus: Die NetworkPolicy-Regeln fuer
einen neu erzeugten Pod sind beim Containerstart noch nicht programmiert. Im
selben Pod gemessen - sofort abgewiesen, nach 20 s erfolgreich. Betrifft
moeglicherweise auch wikijs-backup, synapse-backup und restore-drill.
Eigener TXT statt IONOS-CNAME, p=quarantine mit rua; sieben Service-Namen auf
Null-MX und -all; Apex unveraendert. Abnahme erfuellt - Authentik-Testmail kam
nachweislich an, nicht angenommen sondern geprueft.
Berichtsadresse ist gamemaster@axion1337.chat statt eines neuen Postfachs. Das
umgeht den Autorisierungs-Record, an dem eine zonenfremde Adresse still
gescheitert waere: ohne ihn senden die meisten Empfaenger gar nichts, und
ausbleibende Berichte sehen aus wie keine Probleme.
Zwischenfall festgehalten: Bei der Aenderung verschwanden alle drei
DKIM-Selektoren. Kein Ausfall - direkte Zustellung traegt ueber SPF -, aber
weitergeleitete Mail waere bei p=quarantine still im Spam gelandet. Wieder
aktiviert, s1/s2 loesen bis zum Schluessel auf.
Daraus zwei Werkzeug-Lehren, beide umgesetzt (notfallhandbuch 91d7106, b6ee7fd):
pruefe-dns.sh kannte DKIM nicht und haette den Verlust nicht gesehen; und es
misst im Lab den abgefangenen Resolver statt der Zone - eine Anfrage an eine
TEST-NET-Adresse lieferte ein Ergebnis. Beides jetzt abgedeckt, die Gegenprobe
laeuft vor den Messungen.
docker ps zeigt es eindeutig: Die Exporter veroeffentlichen ihre Ports gar nicht,
sind also nur im Docker-Netz erreichbar. Weder Firewall noch Bind-Adresse.
Zentrale Seite auf 10.0.0.4 umgestellt und der seit drei Monaten laufende
pterodactyl_exporter erstmals angebunden - das vorhandene Dashboard nutzt genau
dessen Metriken und war deshalb dauerhaft leer.
Host-Seite mit Reihenfolge und Stolpersteinen dokumentiert; Compose uebergeben.
#0077: ClamAV-Dashboard liegt in threadnet-operating (7ee42f9). Zwei Korrekturen
am Issue-Text, beide gemessen: das vorgeschlagene Label app existiert in dieser
Loki nicht (Cluster-Logs haengen an job=loki.source.kubernetes.k8s_logs mit
instance=<ns>/<pod>:<container>), und der Text kennt nur das Synapse-Modul -
die wichtigere Quelle ist der Client-Scanner, der auch verschluesselte Raeume
abdeckt. In sieben Tagen: 7 Treffer beim Client, 0 beim Modul.
Zusaetzlich ein Panel, das im Issue nicht stand: Fail-open-Faelle. Laesst der
Scanner mangels clamd Dateien durch, ist das sonst unsichtbar.
#0002: Der Host liegt inzwischen im vSwitch unter 10.0.0.4 - Ports 22/80/443
offen, 9100 und 8080 zu. Eine Umstellung der Targets auf die private IP genuegt
also NICHT; die Exporter binden nicht auf der privaten Schnittstelle. Der noetige
Handgriff liegt auf dem Game-Host. Mit Kontrollmessungen belegt, damit ein
stiller Fehlschlag nicht wie ein Befund aussieht.
#0032 (gameserver ohne Push-Mirror) auf rejected: Der Game-Server ist nicht Teil
des Arbeitsumfangs. Der Befund bleibt sachlich richtig, er ist nur nicht unserer.
STATUS.md listete alle offenen Issues flach; die Frage "was gehoert zu M1" war
darin nur durch Suchen zu beantworten. Jetzt nach Meilenstein gruppiert und
innerhalb dessen nach Prioritaet - ersetzend, nicht ergaenzend: derselbe Bestand
zweimal untereinander waere genau die Doppelpflege, die diese Datei vermeidet.
roadmap.md bekommt die Bedeutung der fuenf Meilensteine in einem Satz je Zeile.
Die stand bisher nur in den GitLab-Milestone-Beschreibungen und war im Repo
nirgends nachlesbar. Zahlen bleiben draussen - die Datei sagt in ihrem eigenen
Kopf, warum (F-001/F-010).
Zwei Folgen des Merges, die im Issue noch als offen bzw. enger beschrieben waren.
Das Tor war am 19.08. bewusst auf apps/web beschraenkt, weil pnpm -r lint:types
an apps/desktop/electron-builder.ts abbrach und apps/web nie erreichte. Der
Fehler ist behoben (ThreadNet-Web:ac343e9), der Job prueft jetzt pnpm -r, und
desktop_linux/desktop_windows fuehren ihn ebenfalls als needs.
Der Gitea-Spiegel steht wieder auf 8ca03fe. ADR-0023 fuehrt ihn in den
Konsequenzen als ungeloest - die ADR bleibt unangetastet, eine angenommene ADR
wird nicht editiert; die Aufloesung steht hier.
Zwei meiner Diagnosen dazu waren falsch und stehen als solche im Text: weder
Paketgroesse noch ein Zeitlimit, sondern der Zustand des Ziel-Repos (9,6 GB
gegen 649 MB lokal) bei gleichzeitig voller Wurzelpartition.
v0.6.0-rc.1 scheiterte am 2026-08-19 mit genau dem 404, den dieses Issue behoben
haben sollte. Die .npmrc stand nie in der COPY-Zeile von apps/web/Dockerfile und
kam im Docker-Build nie an.
Die Abnahme hat gemessen, ob die Konfiguration wirkt, wenn sie vorhanden ist -
nicht, ob sie ueberall ankommt, wo installiert wird. Gutgegangen ist es nur,
weil --frozen-lockfile unter pnpm 10 die gepinnte Tarball-URL nahm und gar nichts
aufloeste; der Schutz kam vom Lockfile, nicht von der .npmrc. pnpm 11 prueft mit
minimumReleaseAgeStrict jeden Eintrag, loest wieder auf und landet ohne .npmrc
bei npmjs. Genau diesen Ablauf sagt die .npmrc in ihrem eigenen Kommentar voraus.
Behoben in ThreadNet-Web:c90ac4d. Status bleibt done - korrigiert wurde die
Behauptung, nicht der Zustand.
Lehre: Fuer Konfiguration, die ueber ihre Position im Dateisystem wirkt, ist die
Liste der Orte die Pruefung, nicht das Verhalten an einem davon.
Der Upstream-Merge holte 70.265 fremde Commits nach ThreadNet-Web; 39 davon
faerbten die Gruppenpruefung als Echtzeit-Stempel rot. Sie verletzen die
Konvention wirklich, konnten ihr aber nie folgen und werden sich nie aendern -
also ein Dauerrot, und ein Dauerrot meldet nichts mehr (#0104).
Neues Feld fremdhistorie in docs/components/*.md. Wo gesetzt, prueft die
Hygiene nur Commits, die von eigenen Identitaeten COMMITTET wurden. Der
Trennschnitt ist gemessen, nicht geraten: 18 eigene Commits, alle von uns
committet; 39 fremde von GitHub/RiotRobot; keine Ueberschneidung. Der Autor
taugt nicht - eigene Commits koennen fremde Autoren tragen (Cherry-Picks).
Der Feldwert ist die Begruendung, kein Schalter - Muster der Quittungen aus
ADR-0020. Die Ausnahme gilt nur, wo sie deklariert ist, nicht global; der Preis
(ein Commit unter voellig unbekannter Identitaet faellt dort durchs Raster)
steht in den Konsequenzen.
Belegt: nach dem Fix 0 offene Befunde bei 20 Quittungen, und in ThreadNet-Web
werden weiterhin 18 Commits geprueft, alle auf 12:00:00.
Ausserdem zurueckgenommen: mein Nachtrag an ADR-0022. Eine angenommene ADR wird
nicht editiert (Regel in der Vorlage) - die Erkenntnis steht jetzt in #0099.
ThreadNet-Web:main auf 8ca03fe, Merge-Commit 88c4e15 mit beiden echten Eltern.
GHSA-wrcp-5v3v-3j6v ist mit dem Versionssprung erledigt.
Drei Anlaeufe: rc.1 erzeugte kein Image (.npmrc fehlte im Docker-Kontext, unter
pnpm 11 fatal), rc.2 ging live und brach die Raumliste, rc.3 bestand die Abnahme.
ADR-0022 um zwei Korrekturen ergaenzt, die erst die Ausfuehrung gezeigt hat:
Die stille Klasse verschwindet nicht ganz, sie dreht sich um. Der Merge meldet,
wenn Upstream eine Datei verschiebt - das hat gehalten. Er meldet nicht, wenn
beide Seiten die Aufloesung ueberleben und nur eine noch Sinn ergibt. Genau das
brach rc.2.
Und: ein gruener Build war nie eine Abnahme. Der web-Job baut nur, webpack
wirft Typen weg. tsc meldete den Fehler durchgehend, gefragt hatte ihn niemand.
Seit 8ca03fe fuehrt docker_web den Job typecheck als needs.
Daraus die stehende Regel fuer kuenftige Merges: nach der Konfliktaufloesung auf
ueberlebende Reste pruefen, nicht nur auf verlorene Zeilen. Wo Zeichenketten
statt Typen im Spiel sind, braucht es einen Abgleich gegen die Registry - fuer
Einstellungen einmalig gefahren, 135 abgefragte gegen 152 registrierte.
Decision sorb: option B, a one-time real merge rather than a shared replace ref. What
decided it was visibility, not effort - a replace ref works only while everyone
remembers to fetch it, and for a repo whose core problem is "git says nothing", a
mechanism that silently differs per clone is the wrong shape.
The test corrected the option's own description. --allow-unrelated-histories on its own
gives a two-way comparison and 1757 conflicts; with the graft set locally it is 32. So
the graft is not the alternative to B, it is how B is performed: set it, let the merge
compute against it, commit, and the merge commit then carries the real parents so the
graft can go.
Also recorded because it cost time and looked like a fundamental problem: tags fetched
with --depth=1 leave a shallow boundary, so v1.12.26 was walled off at a single commit
even though develop carried the same commit in full. Every merge attempt failed with
"refusing to merge unrelated histories" until fetch --unshallow.
The merge itself is measured but deliberately not executed. apps/web/package.json
carries two product decisions rather than conflicts - our Element Call fork against
upstream's, and a matrix-js-sdk git pin against a released version - and resolving the
first one wrongly would silently delete the noise suppression work from #0054. Neither
is safe without a build and the ClamAV functional test.
With the full upstream history the common ancestor could be searched rather than
guessed: measuring tree distance, not dates, over every develop commit in the import
window. deadd548 from 2026-05-08 wins with 43 differing files against 783 for the tag
we wrongly assumed. Thirty-one of those are mode-only, the historical import bug, and
the content remainder is precisely our own Discord feature plus LICENSE and CHANGELOG.
Tested in a throwaway clone rather than asserted: without the graft there is no
merge-base, with it deadd548 becomes one, and merging three months of develop yields
32 conflicting files. Only four are source, and they are exactly our patches - the
rest are GitHub workflows we do not use and the lockfile.
One intermediate result was worthless and nearly passed as success: merging v1.12.26
reported zero conflicts while not running at all, because the tag was fetched shallow
and carries a single commit. Checking MERGE_HEAD is what caught it.
The real gain is not the number. The issue warns that if Element moves a viewmodels
file during the MVVM rework, our lines vanish without a conflict and git says nothing.
With a genuine ancestor, git says something.
Adopting this is ADR-worthy either way - a shared replace ref or a one-time unrelated-
histories merge - so the decision stays with sorb.
upstream is configured and both tags are fetched, so updates can finally be looked at
as a diff. The first diff disproved what this issue and the fork doc have both said
since August: the import is not the v1.12.17 tag. 783 files differ, 115 exist only on
our side, and two of them date the tree - TextualBodyViewModel.tsx and
RovingTabIndex.tsx are absent from the tag, present in v1.12.26 and present in our
import. The changelog stops at 1.12.17 only because it is written at release time.
So the base is a develop state after the 1.12.17 release. Fittingly it is the MVVM
files that prove it - the area this issue already warned about.
Step 3 changes accordingly: grafting onto v1.12.17 would put every future merge on an
invented ancestry, and worse than raising false conflicts, it could hide real ones.
Finding the actual ancestor means searching develop history, which needs the full
upstream repository rather than a shallow tag fetch.
The advisory assessment is unchanged - the base sits below 1.12.22 either way.
Decision sorb, and it needed two files rather than one: the desktop client loads its
own config.json, so changing only element-values.yaml would have hardened the web
client and left the distributed builds alone - the themes-rollout trap again.
Read out of our own code rather than assumed: the edit button beside the server name
disappears, and the 401/403 login error names the server. That is a surface
restriction. MatrixChat still accepts hs_url from the query string in two registration
flows without consulting the setting, so a crafted link is untouched.
Recorded as a narrowing, not a mitigation. GHSA-wrcp-5v3v-3j6v still applies to
1.12.17 and the upstream update in steps 2-4 remains the actual fix.
Pipeline 540 has both management checks passing, and canonize_rotation has been green
since yesterday. That is the condition AGENTS.md's alarm rule silently assumed, and it
holds again.
#0031 is done because the Authentik blueprint check now runs rather than skipping:
sorb supplied the variables, the token carries exactly one permission on its own
service account, and the run says "Geprueft: 11 Projekte" with no skip line. The blind
spot it named - a blueprint discarded on every pass while Flux reported green - would
now surface.
#0104 closes on all four criteria. Twice the route was the cause rather than an
acknowledgement: gitops had no workflow block and created pipelines with no jobs,
which is red without a fault, and management had the same gap for API triggers and was
closed pre-emptively. Exactly one thing is acknowledged, because it cannot be unmade -
pipeline 518 exists in history and sits inside the check's eight-day window, so the
entry expires with the window on 2026-08-28.
From 25 open findings this morning to none, with nothing hidden: 26 entries carry a
reason and a date in the log.
gitops produced a red pipeline with no jobs today because nothing guarded pipeline
creation. This repo has the same shape: its three jobs cover schedule, web and push,
so an API-triggered pipeline matches none of them and would be created empty and
marked failed.
Nobody triggers this repo by API today, which is why it has not happened yet - not a
reason to leave it. One block, and the class is gone in both places.
The workflow rules stop new empty pipelines; they do not remove pipeline 518, which
already exists and stays inside the check's eight-day window. Acknowledged until
2026-08-28 so the entry expires with the window rather than outliving it - if it is
still red afterwards, the fix was incomplete and the check says so again.
This is the case the mechanism was built for: known, dated, self-clearing, with the
root cause fixed rather than papered over.
The issue insists on measuring before building, so: the import was Element Web
1.12.17 in May, upstream is at v1.12.26, and GHSA-wrcp-5v3v-3j6v has been open since
2026-07-20 with "affected: < v1.12.22". We are inside that range. The other five
advisories concern 1.11.x and do not.
Severity is medium and the attack needs the client pointed at a malicious homeserver,
which our users have no reason to do. But the precondition is reachable:
disable_custom_urls is not set, so Element accepts an arbitrary homeserver by default,
and the desktop builds are handed out.
So the honest answer is overdue rather than urgent: a fork without a merge path took
four weeks to not close a known hole, which is exactly the latency this issue names as
the real problem.
One mitigation is available today and independent of the fork question - setting
disable_custom_urls removes the precondition for our users in one line. With
federation closed it is a reasonable setting on its own merits.
Running pruefe_prosa with a complete clone set resolved every SHA citation in the
repo except one, and that one is an Alertmanager silence id from 2026-08-15, sitting
in #0002 next to two siblings that were already listed as exceptions.
With it entered the repo reaches 0 errors and 0 unverified citations - every commit
named in documentation demonstrably exists in the repository it claims to be in.
sorb tested a group call after the restart and it works. That proves what curl could
not: the full OpenID token check through /_matrix/federation/v1/openid/userinfo still
completes with federation closed, so the whitelist genuinely does not reach that
endpoint.
It is also where option C finally died. Blocking /_matrix/federation at the edge would
have removed exactly this path, and nothing before the call would have shown it.
sorb chose C, fully disabling federation at the edge. Building it showed that would
have killed group calls: lk-jwt-service verifies OpenID tokens through
/_matrix/federation/v1/openid/userinfo and reaches it over the public name, so a path
block on /_matrix/federation is the mrtc outage again with a different cause. C was
dropped and B implemented.
The ADR records the measurement the decision rests on - zero destinations, zero remote
users, zero rooms with outside participation in four months - and the trap, so nobody
completes C later as a quick follow-up. Doing that safely means binding the auth
service to Synapse in-cluster first, which is its own undertaking with a call
acceptance.
Also recorded because it nearly slipped through: Flux applied the ConfigMap while
Synapse kept running its old config from 2026-08-01. The config is rendered at pod
start, so the change was inert until a restart - the same class as #0044. Verified
afterwards inside the running process rather than in the ConfigMap, with the client
API and the OpenID endpoint still answering.
The question sat open since July because nobody knew what closing would cost. It is
now measured: zero. No destinations, no remote users, no rooms with outside
participation, across four months of operation - while the federation API answers
HTTP 200 from the public internet, since the well-known delegation routes federation
over 443 and port 8448 being shut changes nothing.
So the trade is not reach against safety but an unused capability against Synapse's
largest remote-facing surface. Recommended B, an empty federation_domain_whitelist:
one line, versioned, reversible in minutes, and honest about what it does not do -
the endpoints keep answering, so the server becomes unengaged rather than invisible.
Decision remains sorb's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Rolled out to matrix, authentik and monitoring, verified in two namespaces rather
than assumed: DNS resolves, federation still reaches matrix.org, the metadata service
is blocked, and the private path to 10.0.0.3 is intact with Alloy reporting no errors.
The issue stays open on purpose. Its actual demand - default deny with explicit
allows - is not met; egress is still open everywhere except link-local. Federation
makes that unwritable as a list while it stays open (#0060). What is now on record is
the inventory that makes the next decision possible: monitoring and authentik have a
nameable set of destinations and could take default deny, matrix cannot.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Before anything else: NetworkPolicy really is enforced here. k3s runs flannel and the
policy controller inside its own process, so the absence of pods in kube-system
proves nothing - measured instead, with a counter-probe. synapse-main reaches
matrix's postgres and authentik-server does not, exactly as allow-ingress-postgres
says. So the existing ingress rules are real protection and egress rules will bite.
The textbook first step - allow 0.0.0.0/0 but except RFC1918 and the metadata
service - would have cut two things here, both over 10.0.0.3, CFGMON on the private
Hetzner network. Alloy remote-writes metrics and logs there, so monitoring would have
gone silent, and nothing reports the failure of the reporting path. And the TURN
rotation reaches Gitea through a hostAlias to that address because the public route
was unreliable, so the rotation push would have died a day after we got it working
again.
That also settles #0008 harder than the absence of a firewall rule could: the cluster
reaches Prometheus and Loki privately, so 9090 and 3100 are demonstrably not needed
publicly.
Proposed step 2 is therefore narrower and nearly risk-free: block only the metadata
service, 169.254.0.0/16, while leaving federation, ACME, SMTP, registries and the
private path open. Nobody here needs the metadata service, and the hard part of this
issue - federation to arbitrary servers - is not touched at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Alert delivery is disconnected" was true when the issue was filed and untrue by
that evening. I carried it as a production blocker on the strength of the issue text
alone, which was wrong: the rules aggregate per target, save_state runs inside the
loop including the failure branch, the security null route is gone, and coturn is
pinned. All of it verified in the code, not inferred.
What genuinely remained were the two smaller review points, and both are done now.
The exporter no longer erases first-seen timestamps when a report fails to parse -
pruning is limited to targets actually read this round, proven in both directions.
And because TrivyScanStale cannot by construction report a target that never
produced a report, the exporter now emits trivy_reports_total and
trivy_report_read_errors with an alert on each.
Priority corrected from high to medium to match what was actually open.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
sorb created the project access token with exactly what the job needs: Maintainer,
because main is protected with push=Maintainers, and write_repository only, because
the job pushes and never calls the API. Checked through the API without touching the
value.
Proven rather than assumed: a throwaway job pushed a ref with that token and removed
it again, leaving main untouched and no branch behind. So the value reaches the job -
protected variable on a protected branch - and it may write. The first real test is
the rotation on 2026-09-01.
Recorded as a future silent failure: the token expires 2027-08-19, and because the
job runs green while idle, nobody would notice until the next real rotation after
that date. Exactly the #0104 class, so it belongs in a calendar rather than in hope.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Decision sorb. Both questions are answered: there is no rule for 9090 or 3100 on
CFGMON and Hetzner denies inbound by default, so Weg A never had anything to
restrict; and the state now has ports-soll.md plus pruefe-ports.sh holding it.
The reservation is written down rather than glossed over: from sorb's network the
must-be-shut half cannot be proven, because the site coupling grants privileged
access there. The script says so instead of reporting false green, and an external
run - judged excessive for routine use - is named for the case that warrants it.
Whether the GAME push survives the vSwitch move belongs to #0002.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured from the cluster the port is shut; measured from sorb's Mac it answered,
because the lab reaches CFGMON over the site coupling, which the cloud firewall does
not filter. The rule is fine and so is the service.
What was actually broken is my check: its counter-probe proves the method works, not
that the vantage point is external. pruefe-ports.sh now probes its own location
first and refuses to score the closed-port half from a privileged network.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The CFGMON firewall has no rule for 9090 or 3100 at all. Hetzner denies inbound by
default, so the ports were never closed - they were never opened. That matches the
2026-08-01 observation exactly and makes Weg A moot for the public path.
While measuring the ports the rules do permit, port 53 answered from a public
address and resolved google.com recursively. The visible rule names only
10.58.73.17 as its source. Recorded here as a separate finding rather than decided
inside this issue.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three corrections to questions I should not have asked again.
#0103 is implemented: autoEnrollGroups points at wiki-anwender, verified in the live
database rather than from the job log. The decision was already in #0049 - the role
model was built, only the way in was missing.
#0102's blocker was in the documentation all along: maintenance-notify sends under
.de, not .chat, via IONOS on 587. The MAIL_FROM in config.example is an example, not
the operating state. So hardening .chat cannot break maintenance mail, and Authentik
remains its only .chat sender - DKIM-covered.
#0008 is not resolved by the console screenshots: they show fw-matrix-cx42, the
Matrix host, while the issue asks which rule keeps 9090 and 3100 shut on CFGMON.
Still open. What they did show is worth keeping: SSH and the Kubernetes API are
properly source-restricted, and several rules open ports to everyone where nothing
listens - including an inbound smtp 587 that cannot help the outbound sending it was
presumably added for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The 2026-08-15 recheck found the ports closed but nobody had decided that and
nothing recorded it, so the next firewall change could have reopened them unseen.
notfallhandbuch:pruefe-ports.sh plus ports-soll.md close that half, measured rather
than derived, and check both directions - a reachability-only check would never
notice telemetry standing open.
Still needs sorb: the shared look at the Hetzner console. The script says what is
reachable from outside; which rules produce that, and whether they stand there on
purpose, only the console says.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Agreed with sorb. Two additions, both inside the project section - sections 1-5 are
the neckbeard baseline and stay byte-identical, which is also why the repo map's
scripts row was left alone rather than corrected in place.
"The red pipeline IS the alarm" now carries what it silently assumed: that green is
the normal state. The counter-example is named with its date, because the abstract
rule did not stop this from happening - the job stood red for nine days and that is
precisely why nobody looked.
A Prüfungen section lists the eight scripts the repo actually has against the two
the frozen map names, says when each runs, and states the acknowledgement rule:
known findings are acknowledged, not tolerated, every entry carries an expiry,
"permanent" is only expressible as an ADR reference, and leaving a check red is a
finding in itself rather than a neutral state.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Found while auditing whether everything is documented per the framework. The
automated checks were all green, which is the point: they check artifacts against
the schema, not prose against reality.
The README's structure table named four directories that do not exist - decisions/,
vision/, hosts/, shared/ - and sent readers to axionwiki.lab, which answers 404 and
was retired by ADR-0014, citing the superseded ADR-0006 as if current. AGENTS.md
says plainly that wiki.lab is gone and not to look for it; the repo's own entry
document said the opposite. Replaced with the layout that is actually there, plus
the checks and where acknowledgements live.
roadmap.md carried the same two staleness: "Docusaurus läuft" with an end-of-August
deadline for a question ADR-0014 already answered, and the board build-out from
gitops#46, which was decided the other way round and closed today. Both marked done
with what was actually decided, since a roadmap that claims open work is finished is
worse than one that is merely behind.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The framework audit found this missing. Acknowledging known findings is a process
decision about how the alarm system treats exceptions, and AGENTS.md is explicit
that documenting an exception instead of deciding it is itself the error. It lived
only inside #0104 and #0105, which are issues, not decision records.
The ADR carries the reasoning the issues could not: why option C beat working the
backlog down first or tolerating red, and why the obvious objection - an exception
list is a candidate for the next blind spot - is answered by the three rules rather
than waved away. It also records what is deliberately not acknowledged, the
transient mirror divergence, because that message is the only signal if a mirror
truly stops.
One consequence is stated plainly rather than discovered later: acknowledgements
bind to substrings of the finding text, so fixing or moving a cause can change the
wording and require the entry to follow. Stable finding IDs would avoid that and
would make the file unreadable without special knowledge; the trade is taken
knowingly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first CI run after the acknowledgement list went in was still red, so the
"all three green" line in #0104 was premature and is corrected rather than left
standing.
Two things it showed. Disabling the wiki mirror changed the finding's wording from
"Mirror wiki: counterpart unreadable" to "wiki: no active push mirror", and since
acknowledgements match on substrings, the entry no longer applied. Added. That is
the cost of a list readable without special knowledge: fix or move a cause and the
acknowledgement has to follow, otherwise it goes red - correctly.
The second finding, management mirror divergence, is deliberately NOT acknowledged.
It appears on any run starting minutes after a push because the mirror lags, and
acknowledging it is tempting - but that same message is the only signal if a mirror
ever really stops (#0028). It stays sharp; a run right after a push is briefly red.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Decision sorb. The mirror from axion1337.chat/wiki to sorb/wiki is off. It pointed
the wrong way: per ADR-0015 Wiki.js writes to sorb/wiki from the cluster and
canonize_wiki pulls it back to git.lab, so this mirror would have overwritten the
wiki content with git.lab's state. It had never run once - status none, no error -
which was luck rather than design.
Whether the wiki project stays at all is deliberately not decided now. sorb wants
that conversation immediately before project completion, once production readiness
is reached on everything else, so #0105 carries it as waiting with that as its
stated reason rather than as an open task somebody might pick up.
The group check's finding is acknowledged until 2026-12-31 rather than permanently.
A permanent acknowledgement would need an ADR, and writing that ADR is exactly the
decision being postponed; a date instead means that if the project outlives it, the
question asks itself again.
Both scheduled checks now reach zero open findings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The group check carried 17 commit-hygiene findings that sorb deliberately deferred
to the history pass, plus two known items. Acknowledged until 2026-11-30, which
takes the check from 20 open findings to one.
Each commit is acknowledged by its own SHA rather than by a pattern. A rule like
": Echtzeit-Stempel" would have been one line instead of seventeen and would have
swallowed every future violation with it - the blind spot this whole mechanism
exists to avoid. As a side effect the entries clean themselves up: once the history
is rewritten they match nothing and report themselves for removal.
What remains open in both checks is the same single real finding, and it is new:
the GitLab project "wiki" is undeclared, and its push mirror to sorb/wiki has never
run once - status none, no error, no successful update. Per ADR-0015 that direction
is wrong: Wiki.js writes to sorb/wiki from the cluster and canonize_wiki pulls it
back to git.lab. Had this mirror ever fired it would have overwritten the wiki
content with git.lab's state. It stays red on purpose, because it needs a decision.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
All three scheduled checks were permanently red, which is how a nine-day outage of
the canonize job went unnoticed: one more red cross among red crosses is invisible.
A check that can only ever be red cannot report anything.
An acknowledgement takes a known finding out of the red verdict without hiding it -
it still prints, with its reason and its deadline. Red is reserved for what is not
acknowledged, which is to say: for the new.
Three rules keep the list from becoming the next blind spot, which is the obvious
objection to this whole idea:
- Every entry needs a deadline. Once it passes, the entry stops acknowledging and
says so, so the finding counts again.
- "Permanent" is only expressible as an ADR reference. A permanent exception
without a decision record is already an error per AGENTS.md; here it cannot even
be written down.
- An entry that matches nothing reports itself, so the file cannot quietly
accumulate lines for problems that no longer exist.
Four entries to start: notfallhandbuch (ADR-0016, deliberately unmirrored),
threadnet-wiki (ADR-0015, the wiki mirrors the other way round), gameserver (sorb:
not part of ThreadNet - dated, so the ADR-or-move decision does not drift), and the
zero-job pipeline artifact.
Verified rather than argued, including the counter-proofs #0104 asks for: today
four are acknowledged and one real finding stays red; adding a fresh finding still
turns it red; and with the clock moved past the deadlines the dated entries stop
acknowledging and report themselves. The scope column was added after the first run
showed each check reporting the other's entries as ineffective.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Decision sorb. AGENTS.md makes a red pipeline the alarm, with no second channel by
design. That only works while green is the normal state, and right now none of the
three scheduled checks reach it: canonize_rotation was red for nine days,
gruppenpruefung is red daily on 20 findings of which 17 are deliberately deferred,
and stillstandspruefung aborts daily for a missing GITEA_TOKEN. The management
pipeline has failed every day since at least 2026-08-12.
canonize_rotation is the proof rather than the anecdote: it failed for nine days on
a conflict touching both TURN secrets and the client image tag, and nobody noticed,
because one more red cross among red crosses is invisible. It surfaced only because
someone looked for an unrelated reason.
The issue asks how "known and deferred" gets distinguished from "new" without the
deferred work blocking the channel, and recommends acknowledging #0053's findings
with an expiry date while fixing #0031 outright. Acceptance requires showing a
freshly introduced finding still turns the pipeline red - demonstrated, not assumed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Decision sorb. mas-cli locked the account and killed one OAuth and two browser
sessions, so signing in is no longer possible. deactivated_at stays empty though:
lucky now sits where scanner-test sits, not where @bojeledoggo does.
This MAS version's CLI has no deactivate-user - only lock/unlock. Full deactivation
runs through the admin API or Element Admin and therefore through an admin token,
which does not belong in a session. Locking takes the effect forward until sorb
makes it final with a click.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Decision sorb. The account had no name, no email, no group, one login on the day
it was created in May and nothing since. The delete removed exactly one row with no
dependent objects, because nothing was ever attached to it.
sorb's caveat was right but did not apply here: Matrix accounts cannot truly be
deleted - Synapse only deactivates, and the localpart stays reserved so nobody
inherits a former identity's history and mentions. That is why @bojeledoggo is
deactivated rather than gone, and that is the end state rather than a half measure.
Boje was never a Matrix account: no row in Synapse, none in MAS, no link. The
limitation governs the other account, not this one.
Two of the checklist's four lines are now done. The test rooms and clark/lucky
remain; both accounts are still active in Synapse.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Checking whether Boje is deactivated in Synapse produced the answer to three of
this checklist's four lines. @bojeledoggo is deactivated, so that item is done.
clark and lucky are still active, so that one is not.
Boje is the odd one: the account exists only in Authentik, created 2026-05-15 with
a single login that same day and nothing since, no email and no group. There is no
@boje or @Boje in Synapse and no upstream link in MAS, so the identity never
completed a Matrix login. The open question is less "add the email" than "does this
identity still belong anywhere".
Recorded because the name similarity has misled once already: @bojeledoggo,
Boje and elbojoloco are three different accounts.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Decision sorb. elbojoloco joins apo in wiki-anwender, so both read /anwender and
the start page while betrieb/* stays closed. Recorded because the addendum's table
would otherwise still list the account as having no effect.
The grant takes hold at the next login - the groups claim is minted during sign-in,
not continuously - and elbojoloco has never signed in to the wiki, so the account
gets created and mapped on first attempt.
Boje is still undecided, and clark and lucky remain test accounts slated for
deletion in #0074. The silent dead end persists for them, which is why the grants
do not close this issue.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Thore Cimbal
