fix(matrix-stack): shorten HelmRelease reconcile interval to 1m

Live-tested Issue #38's rotation: after merging the automated rotation
PR, coturn restarted quickly (Kustomization-level, 1m interval), but
synapse-main lagged behind since it depends on this separate HelmRelease
with its own 5m interval - a real (self-healing, but avoidable) window
where coturn had the new TURN secret and Synapse still had the old one,
which would reject each other's credentials/relayed media. Matching the
interval to production-apps's 1m tightens that window without needing
any new automation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

.devcontainer/Dockerfile

@@ -16,11 +16,14 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
     zsh \
     sudo \
     openssh-client \
+    gosu \
     && rm -rf /var/lib/apt/lists/*
 
-# Install kubectl
-RUN curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && \
-    echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | tee /etc/apt/sources.list.d/kubernetes.list && \
+# Install kubectl (apt.kubernetes.io was deprecated/shut down by Google in 2023;
+# pkgs.k8s.io is the current community-owned repo, versioned per k8s minor release)
+RUN mkdir -p /etc/apt/keyrings && \
+    curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.34/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg && \
+    echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.34/deb/ /" | tee /etc/apt/sources.list.d/kubernetes.list && \
     apt-get update && apt-get install -y kubectl && \
     rm -rf /var/lib/apt/lists/*
 
@@ -30,9 +33,10 @@ RUN curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | b
 # Install Flux CLI
 RUN curl -s https://fluxcd.io/install.sh | bash
 
-# Install sops
-RUN SOPS_VERSION=$(curl -s https://api.github.com/repos/getsops/sops/releases/latest | grep tag_name | cut -d '"' -f 4) && \
-    curl -sL -o /usr/local/bin/sops https://github.com/getsops/sops/releases/download/${SOPS_VERSION}/sops-${SOPS_VERSION}.linux.amd64 && \
+# Install sops (arch resolved at build time, same reasoning as the Docker CLI step below)
+RUN SOPS_ARCH=$(dpkg --print-architecture) && \
+    SOPS_VERSION=$(curl -s https://api.github.com/repos/getsops/sops/releases/latest | grep tag_name | cut -d '"' -f 4) && \
+    curl -sL -o /usr/local/bin/sops https://github.com/getsops/sops/releases/download/${SOPS_VERSION}/sops-${SOPS_VERSION}.linux.${SOPS_ARCH} && \
     chmod +x /usr/local/bin/sops
 
 # Install age
@@ -40,17 +44,31 @@ RUN apt-get update && apt-get install -y age && \
     rm -rf /var/lib/apt/lists/*
 
 # Install Docker CLI (for interacting with Docker daemon)
+# arch is resolved at build time so this works on both amd64 (cloud/CI) and arm64 (Apple Silicon) hosts
 RUN curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg && \
-    echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null && \
+    echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null && \
     apt-get update && apt-get install -y docker-ce-cli && \
     rm -rf /var/lib/apt/lists/*
 
 # Create a non-root user 'vscode' for development
-RUN useradd -m -s /bin/bash -G docker vscode && \
+# groupadd is needed because only the Docker CLI (not the daemon) is installed above,
+# so the 'docker' group is never created as a package side effect
+RUN groupadd docker && \
+    useradd -m -s /bin/zsh -G docker vscode && \
     echo "vscode ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers.d/vscode
 
 # Install oh-my-zsh for better shell experience
-RUN su - vscode -c "sh -c '$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)' '' --unattended"
+RUN curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh -o /tmp/install-omz.sh && \
+    su - vscode -c "sh /tmp/install-omz.sh --unattended" && \
+    rm /tmp/install-omz.sh
+
+# Entrypoint runs as root to reconcile the docker group's GID against the mounted
+# socket (see docker-init.sh), then drops to 'vscode' for the actual session/command.
+# Stays root-owned at the PID 1 level; VS Code's own `docker exec -u vscode` sessions
+# and the entrypoint's `gosu vscode` both end up correctly grouped either way.
+COPY docker-init.sh /usr/local/bin/docker-init.sh
+RUN chmod +x /usr/local/bin/docker-init.sh
 
-USER vscode
 WORKDIR /workspace
+ENTRYPOINT ["/usr/local/bin/docker-init.sh"]
+CMD ["/bin/zsh"]

.devcontainer/README.md

@@ -91,7 +91,7 @@ Der Container mounted `~/.age` automatisch. Setze die Umgebungsvariable:
 ```bash
 # Im Container-Terminal (SOPS_AGE_KEY_FILE ist bereits automatisch gesetzt!)
 # Jetzt kannst du Secrets bearbeiten (wird transparent ver-/entschlüsselt):
-sops apps/production/custom-configs/mas-secrets.sops.yaml
+sops apps/production/custom-configs/mas-secret.yaml
 ```
 
 ### Schritt 3: VSCode Integration (optional)
@@ -130,7 +130,7 @@ kubectl get pods -n matrix
 flux get helmreleases -A
 
 # Secrets bearbeiten (mit verschlüsselung)
-sops apps/production/custom-configs/mas-secrets.sops.yaml
+sops apps/production/custom-configs/mas-secret.yaml
 
 # FluxCD Sync erzwingen
 flux reconcile kustomization production-apps --with-source
@@ -193,6 +193,44 @@ Siehe `README.md` → **Issue 3**. Kurz:
 - `wellKnownDelegation: enabled: false` setzen
 - Oder `.well-known/matrix/server` manuell auf `elementWeb` weiterleiten
 
+## ⚠️ Wartungshinweis: Warum dieser Container regelmäßig getestet werden muss
+
+Der Dockerfile installiert mehrere Tools über externe apt-Repos und Install-Skripte
+(`pkgs.k8s.io`, `download.docker.com`, GitHub-Releases, `fluxcd.io`/`ohmyzsh.sh`
+Installer). **Diese Quellen sind nicht unter unserer Kontrolle und können jederzeit
+brechen** — genau das ist am 2026-07-28 passiert: der Container konnte seit
+Fertigstellung nie erfolgreich gebaut werden, ohne dass es jemand bemerkt hat, weil
+niemand ihn zwischenzeitlich tatsächlich gebaut hat. Gefundene und behobene Probleme:
+
+| # | Problem | Ursache | Fix |
+|---|---------|---------|-----|
+| 1 | `apt.kubernetes.io` → `404 Not Found` | Google hat das alte Kubernetes-apt-Repo 2023 abgeschaltet | Umgestellt auf das offizielle Nachfolge-Repo `pkgs.k8s.io` (versioniert pro k8s-Minor-Version, aktuell `v1.34`) |
+| 2 | `docker-ce-cli` "has no installation candidate" auf Apple Silicon | Repo-Zeile hatte `arch=amd64` hartkodiert, Build lief aber auf arm64 | `arch=$(dpkg --print-architecture)` zur Build-Zeit ermitteln |
+| 3 | `useradd: group 'docker' does not exist` | Nur die Docker-**CLI** wird installiert (kein Daemon), daher legt kein Paket die `docker`-Gruppe automatisch an | `groupadd docker` explizit vor `useradd` |
+| 4 | oh-my-zsh-Install schlägt mit Quoting-Fehler fehl | Verschachtelte `sh -c '...'`-Anführungszeichen in einer Zeile | Install-Skript erst in eine Datei laden, dann sauber mit `su - vscode -c "sh /tmp/install-omz.sh --unattended"` ausführen |
+| 5 | `sops`-Binary war hart auf `linux.amd64` gepinnt | Lief auf Apple Silicon nur zufällig per QEMU-Emulation von Docker Desktop mit, nicht nativ | Arch dynamisch über `dpkg --print-architecture` auflösen (`linux.arm64` / `linux.amd64`) |
+| 6 | `docker.sock`-Zugriff im Container: `permission denied` | Der gemountete Host-Socket gehört (je nach Docker-Setup) einer Gruppe/GID, die im Container nicht existiert oder nicht der `docker`-Gruppe entspricht (auf Docker Desktop für Mac/Windows z.B. GID 0/root statt einer eigenen `docker`-Gruppe) | `docker-init.sh`: Root-Entrypoint gleicht beim Container-Start die GID der `docker`-Gruppe an den tatsächlich gemounteten Socket an (bzw. tritt der GID-Inhaber-Gruppe bei, falls die GID schon vergeben ist), wechselt danach per `gosu` zu `vscode` |
+
+**Konsequenz für die Zukunft:** Vor jeder größeren Änderung an `.devcontainer/` (oder
+mindestens vierteljährlich) einmal real bauen und laufen lassen:
+
+```bash
+docker build -f .devcontainer/Dockerfile -t ess-gitops-devcontainer-test .devcontainer
+docker run --rm \
+  -v ~/.kube:/home/vscode/.kube \
+  -v ~/.age:/home/vscode/.age \
+  -v /var/run/docker.sock:/var/run/docker.sock \
+  ess-gitops-devcontainer-test bash -c '
+    kubectl version --client && helm version --short && flux --version && \
+    sops --version && age --version && docker version --format "{{.Server.Version}}" && \
+    id vscode
+  '
+```
+
+Wenn `docker version` hier den echten Server, nicht nur die Client-Version zeigt, und
+`id vscode` die passende Docker-Gruppe/GID auflistet, funktioniert der Socket-Zugriff
+tatsächlich — nicht nur der Build.
+
 ## 📚 Weitere Ressourcen
 
 - [Dev Containers Docs](https://containers.dev)

.devcontainer/docker-init.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+# Runs as root at container start (before any `docker exec -u vscode` from VS Code).
+# The docker.sock's GID is only known once the host socket is actually bind-mounted,
+# so it can't be baked in at image build time - it must be reconciled here, at runtime.
+set -e
+
+if [ -S /var/run/docker.sock ]; then
+  SOCK_GID=$(stat -c '%g' /var/run/docker.sock)
+  CURRENT_GID=$(getent group docker | cut -d: -f3)
+  if [ -n "$SOCK_GID" ] && [ "$SOCK_GID" != "$CURRENT_GID" ]; then
+    EXISTING_GROUP=$(getent group "$SOCK_GID" | cut -d: -f1)
+    if [ -n "$EXISTING_GROUP" ]; then
+      # GID is already taken by another group (e.g. GID 0/root - Docker Desktop for
+      # Mac/Windows owns the socket this way inside its VM), so join that group
+      # instead of trying to reassign it to 'docker'.
+      usermod -aG "$EXISTING_GROUP" vscode
+    else
+      groupmod -g "$SOCK_GID" docker
+    fi
+  fi
+fi
+
+exec gosu vscode "$@"

.devcontainer/postCreateCommand.sh

@@ -26,7 +26,7 @@ echo ""
 echo "📚 Useful commands:"
 echo "  - kubectl get pods -n matrix          (check pod status)"
 echo "  - flux get helmreleases -A            (check helm releases)"
-echo "  - sops apps/production/custom-configs/mas-secrets.sops.yaml  (edit secrets)"
+echo "  - sops apps/production/custom-configs/mas-secret.yaml  (edit secrets)"
 echo ""
 echo "🔗 For kubeconfig setup:"
 echo "  - Copy your ~/.kube/config to access the cluster"

.gitignore

@@ -0,0 +1,2 @@
+.DS_Store
+.claude/

apps/authentik/authentik-backup-secret.yaml

@@ -0,0 +1,23 @@
+apiVersion: v1
+kind: Secret
+metadata:
+    name: authentik-backup-credentials
+    namespace: authentik
+stringData:
+    borg-passphrase: ENC[AES256_GCM,data:5PApz4TqSNN2vVXeFSuomd051nl+cYk+a+STViwddG/Hj7XWQ099vvTlKSE=,iv:MFG04/66YqtOjZWsLpy236MYwR05z91ngOQ0BmLNxzA=,tag:gxD5prW8Ted3Q2ZY3sYSYQ==,type:str]
+    ssh-private-key: ENC[AES256_GCM,data:87PrkqAIH2xRCu3jlAK3Ts24XKRDoXNCWLomRhizCY43hwLeY5fNzfsGnMTKBt5IxL2bIljx+pbh/Y8IO38ISUUbUAczfi1KeN1gg69wKgqNAlckcmD0dkA2N2F0ZnEFX/rkad68YR8Hw1/A5h3qGUd5FwoVWzIFD/diY86s/LiQBYzwD4bYLUACHqIL7+QyJ6jEsFzA3tLwDySpUpM9OAjFQF0EazObChNYqe/qrc8i1gI8CptWOFkqiZ8Pen1E79gg/zgleOBqSmlcefgZk+JFPAfvORTHkDrHW9X/bLygaTQvskmGNzaKHpe2s/bVafWb7G60ROrpdjYIJfS7c0baRCSrd9kwdM4JQC2gSsMkI4v6oeFSvkTrb684B10zacZdiTKfySDV+Ry4hCZofF2RhAnsTbOpo0XANoOi4InoxUqGPfctVfUCVb6X+HIR5XYxpLrNwWqRpscMP2P3RJsaUZOinmQUKrHX/nMtPZ/ltpwjhQeAXZTtdyPxTCjHAh6fzouuHk66vlCdukXsMQDBdIfdA/3jjGbW,iv:yFNKkdegLLq8jq1Ya6v67urSJTdG3Ge4ZbmKizqQhmk=,tag:6l/qMb1d1oD8sfVEebW6gQ==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBWMllTL2hRcEhCT2MxcWZy
+            dkxndmlZRW5FZ2NiWWdFckxqYTFKS1B6SXlvCjJySXRsZThvczNnWjlyM0N2Snha
+            eitvWUhhdmU4U0V3OG55WjdLbU9KcUEKLS0tIEM3MStzNUJhaGg4M0hKMVF1bnBB
+            RWVqVDRBNmJ0b1Bwd0l3dWxPT3Q0SU0KOEoyejkH4RC0p8ka3FjI7MyzRJg+uu7h
+            j3wf1q+Hgg73djDBSPYJkrB6Bdl4YMwo8SzbtW8O9elDE0qAioR0bQ==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
+    encrypted_regex: ^(data|stringData)$
+    lastmodified: "2026-07-28T18:20:58Z"
+    mac: ENC[AES256_GCM,data:lwwNbxSxtgDTDmaWMU1uf7TBOEw8gFBFKb982VIsGMeM0fIPHvX12Qts90MNgYIJliOWgAWrwyvAgfXWKuE37RNo+BtyfHCWi4IESKSN/RJrd/yMpRKx+02rifH3nl26ZCAQT1Pa0fjI1SfMhbVzfnD9a/AMARXZMhLRc0OqczY=,iv:SgP2iMtENRtZfw6I9EaOsmvecFNYCIZWqVj+cZ+T7EI=,tag:s4oEyxqyUx6ibqsk4g9xpw==,type:str]
+    version: 3.13.3

apps/authentik/authentik-backup.yaml

@@ -0,0 +1,85 @@
+# Nightly Borg backup of the authentik Postgres database to a Hetzner Storage Box
+# (issues #6 + #15). See apps/authentik/authentik-backup-secret.yaml for the SSH key +
+# Borg repo passphrase, and apps/production/synapse-backup.yaml for the matrix-side job
+# (same Storage Box, separate repo/passphrase, offset schedule).
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: authentik-backup-known-hosts
+  namespace: authentik
+data:
+  known_hosts: |
+    [u641795.your-storagebox.de]:23 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIICf9svRenC/PLKIL9nk6K/pxQgoiFC41wTNvoIncOxs
+---
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+  name: authentik-backup
+  namespace: authentik
+spec:
+  schedule: "15 3 * * *"
+  concurrencyPolicy: Forbid
+  successfulJobsHistoryLimit: 3
+  failedJobsHistoryLimit: 3
+  jobTemplate:
+    spec:
+      backoffLimit: 2
+      template:
+        metadata:
+          labels:
+            app.kubernetes.io/name: authentik-backup
+            app.kubernetes.io/component: backup
+        spec:
+          restartPolicy: OnFailure
+          containers:
+            - name: backup
+              image: rohana.axion1337.de/sorb/axion-backup:v2
+              env:
+                - name: BORG_REPO
+                  value: "ssh://u641795@u641795.your-storagebox.de:23/./authentik-backup"
+                - name: BORG_PASSPHRASE
+                  valueFrom:
+                    secretKeyRef:
+                      name: authentik-backup-credentials
+                      key: borg-passphrase
+                - name: SSH_PRIVATE_KEY_FILE
+                  value: /secrets/ssh/ssh-private-key
+                - name: SSH_KNOWN_HOSTS_FILE
+                  value: /secrets/known-hosts/known_hosts
+                - name: DB_HOSTS
+                  value: "authentik:authentik-postgresql"
+                - name: PGUSER
+                  value: authentik
+                - name: PGPASSWORD
+                  valueFrom:
+                    secretKeyRef:
+                      name: authentik-credentials
+                      key: pg-password
+              volumeMounts:
+                - name: ssh-key
+                  mountPath: /secrets/ssh
+                  readOnly: true
+                - name: known-hosts
+                  mountPath: /secrets/known-hosts
+                  readOnly: true
+                - name: scratch
+                  mountPath: /scratch
+              resources:
+                requests:
+                  cpu: 50m
+                  memory: 128Mi
+                limits:
+                  memory: 256Mi
+          volumes:
+            - name: ssh-key
+              secret:
+                secretName: authentik-backup-credentials
+                items:
+                  - key: ssh-private-key
+                    path: ssh-private-key
+                    mode: 0400
+            - name: known-hosts
+              configMap:
+                name: authentik-backup-known-hosts
+            - name: scratch
+              emptyDir: {}

apps/authentik/authentik-blueprints.yaml

@@ -226,3 +226,58 @@ data:
           domain: authentik-default
         attrs:
           default_application: !Find [authentik_core.application, [slug, matrix]]
+  matrix-oidc-provider.yaml: |
+    # yaml-language-server: $schema=https://goauthentik.io/blueprints/schema.json
+    version: 1
+    metadata:
+      name: matrix-oidc-provider
+      labels:
+        blueprints.goauthentik.io/instantiate: "true"
+    entries:
+      # The OIDC Provider + Application linking Authentik to MAS was originally
+      # clicked together by hand in the UI and existed nowhere as code (issue
+      # #36): losing the Authentik DB would have meant re-creating this from
+      # scratch, including a new client_secret that MAS would then no longer
+      # match. client_secret is read from AUTHENTIK_MAS_OIDC_CLIENT_SECRET
+      # (see authentik.yaml HelmRelease values) rather than inlined here,
+      # since this ConfigMap itself is not SOPS-encrypted - the actual value
+      # lives in the authentik-credentials Secret instead.
+      - model: authentik_providers_oauth2.oauth2provider
+        state: present
+        identifiers:
+          name: Matrix Authentication Service
+        id: matrix_mas_provider
+        attrs:
+          client_type: confidential
+          client_id: dHbTAgAgXvjh3VALh220mB3dxcVXAifiXU2ZO3U6
+          client_secret: !Env AUTHENTIK_MAS_OIDC_CLIENT_SECRET
+          # Path includes MAS's own upstream-provider ID, not Authentik's -
+          # must match MAS's config exactly or the OIDC callback breaks.
+          redirect_uris:
+            - matching_mode: strict
+              url: https://account.axion1337.chat/upstream/callback/01KQDJTR1ZVTG8JQ220F5BNBFZ
+          # Stable across username renames - this is what keeps
+          # upstream_oauth_links rows valid after e.g. the elbojoloco rename.
+          sub_mode: hashed_user_id
+          include_claims_in_id_token: true
+          access_code_validity: minutes=1
+          access_token_validity: minutes=5
+          signing_key: !Find [authentik_crypto.certificatekeypair, [name, "authentik Self-signed Certificate"]]
+          authorization_flow: !Find [authentik_flows.flow, [slug, default-provider-authorization-implicit-consent]]
+          invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
+          property_mappings:
+            - !Find [authentik_core.propertymapping, [managed, "goauthentik.io/providers/oauth2/scope-openid"]]
+            - !Find [authentik_core.propertymapping, [managed, "goauthentik.io/providers/oauth2/scope-email"]]
+            - !Find [authentik_core.propertymapping, [managed, "goauthentik.io/providers/oauth2/scope-profile"]]
+
+      - model: authentik_core.application
+        state: present
+        identifiers:
+          slug: matrix
+        attrs:
+          name: aXion1337.chat Accountverwaltung
+          provider: !KeyOf matrix_mas_provider
+          meta_description: Matrixclient tailored for aXionCommunity
+          meta_publisher: aXionGaming
+          policy_engine_mode: any
+          open_in_new_tab: false

apps/authentik/authentik-secret.yaml

@@ -7,10 +7,10 @@ stringData:
     secret_key: ENC[AES256_GCM,data:yIyQapbFtFM11LynFtkV3ffExhaDfN9QHeFbI1T0xkIhgsV+9sjg3qwMVmeBlAe7xZl8gsAM4kDj2Q6O91OdDg==,iv:+Cl8vOcxG9/mgRheaCO0bLWyCJXN+f1F2DD3oeHbPFY=,tag:711ytyKf6/tmXomBLoffGA==,type:str]
     pg-password: ENC[AES256_GCM,data:3w8R9mRjMXMJDLjrC8QYaXFHsCU3yYZs2PcaFQNp3Z4=,iv:G/aXgoGz3vBOzZ5K3Y+DDJsqer4F5gvcMmtkzRx93CU=,tag:dXPs1pY/APvnMlxdvB1EkA==,type:str]
     smtp-password: ENC[AES256_GCM,data:JpMgaQFPkBzOg5WjvpmhM0kPwvZkH+4tQjT17RJHjG14WjmWtfG9Bg==,iv:zjQRLIlrxKv5hbd4JZowNUEiibiCUMf79acZY0+dYAc=,tag:ORPafTPyOQJvVvHWQGmqhA==,type:str]
+    mas-oidc-client-secret: ENC[AES256_GCM,data:0yx55FroLSxlnuYgfNwczu3PnbPm1kW74JtiU9oFevVqeQDZc385wU6x5X5TN7owXDO7QaOfGTTMvqIpbwQb6Q5Vt1VMToR+0f44oJcktYoTiDFU9Sy6lR/y6nlvBCNqeJg7vIyVpkIqxwqty15EekyqMpkIMp1fT6Pxmek0SO0=,iv:Ey06ljnqbVARDLVt2sLe8R776VEWpTlzI/+Nka5NocA=,tag:I+GNLHz4V8TFa2ijzK5y2Q==,type:str]
 sops:
     age:
-        - recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
-          enc: |
+        - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
             YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBRekJuZythYzliTFJ3RlhS
             R2p6TG9NeFdabFlPRWtpNHJMYVVxTWZEcmlRClk0WUorSzdxNlcyWHYwWFBTMnlq
@@ -18,7 +18,8 @@ sops:
             QXVrY1NTeHZkeTlPRWNlVThzWno3T0kKC0KBoLT64GNqb8Ri9u69G7nqb1KftwwP
             /24aVHrPxKi9d4ij9n3bvCYDF4rhtfexhrE4n7CfuKn2DcSiuTniuw==
             -----END AGE ENCRYPTED FILE-----
-    lastmodified: "2026-04-29T21:43:59Z"
-    mac: ENC[AES256_GCM,data:Y+dJppkaVZ5NOhlvwbbsF5+vDFqGUI1Ps8IcE4J7FIW4HIdMVf6RKM0EInvPUW1LaBlmelCitcE30w0As7ysNRhLY8yUDaKUvuU6mRejlNUIF8wAHzhciL2jTvAQsArHjybJatEig28+wM9VcY8JEa/d/CmuiB9Nq4WbIV+JXlA=,iv:UQj2rIVLNPjtYp3d/jRyNfJyyyUsZ3+NDCgpI4aztzc=,tag:cwiCzG/A+rfRFfLjXVt82w==,type:str]
+          recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
     encrypted_regex: ^(data|stringData)$
+    lastmodified: "2026-07-28T15:54:53Z"
+    mac: ENC[AES256_GCM,data:P6IF+jukwzldK92nHl6s4h6sS4ldXLwpyLpwv2tpI3vFWgTLEnGCnowi2k5lmWUlITEVmLLC0HvsBuduTiGI2sIHHt+r3RdqkV88HGn6oYDVq5a+Ax7ESfqti/4B7ClQCSxl/tU6hBUFe812DiBXJgA03UJQZn8uHY/dP/RgRpc=,iv:V8sqhbJcKglkKsQmJBdgoxDaCYJ3Wt/qRa18jEviH60=,tag:EiNotrYAKIzKndgjU/kTFQ==,type:str]
     version: 3.12.2

apps/authentik/authentik.yaml

@@ -40,6 +40,15 @@ spec:
     global:
       security:
         allowInsecureImages: true
+      # Read by the matrix-oidc-provider blueprint via !Env, so the OAuth2
+      # Provider's client_secret can be captured as code without ever
+      # inlining the live credential into a plain (non-SOPS) ConfigMap.
+      env:
+        - name: AUTHENTIK_MAS_OIDC_CLIENT_SECRET
+          valueFrom:
+            secretKeyRef:
+              name: authentik-credentials
+              key: mas-oidc-client-secret
 
     authentik:
       log_level: info
@@ -89,3 +98,8 @@ spec:
             memory: 256Mi
           limits:
             memory: 512Mi
+        # Chart's own generated policy allows ANY pod in ANY namespace on 5432
+        # (see issue #37) - disabled in favor of our own scoped policy in
+        # apps/authentik/networkpolicy.yaml.
+        networkPolicy:
+          enabled: false

apps/authentik/kustomization.yaml

@@ -9,3 +9,6 @@ resources:
   - authentik.yaml
   - ingress.yaml
   - networkpolicy.yaml
+  # Backup zur Hetzner Storage Box (Issues #6 + #15)
+  - authentik-backup-secret.yaml
+  - authentik-backup.yaml

apps/authentik/networkpolicy.yaml

@@ -4,9 +4,10 @@
 # for upstream OIDC calls. Egress is intentionally untouched (federation-equivalent
 # outbound calls like SMTP aren't restricted here).
 #
-# Note: authentik-postgresql already has its own NetworkPolicy from the Bitnami
-# postgresql subchart (port 5432, no source restriction) - left alone, not duplicated,
-# since it would get reset on the next Helm upgrade anyway.
+# authentik-postgresql: the Bitnami postgresql subchart's own generated NetworkPolicy
+# restricted the port (5432) but not the source - any pod in any namespace could reach
+# it (issue #37). Disabled via postgresql.primary.networkPolicy.enabled: false in
+# authentik.yaml and replaced below with a policy scoped to authentik-server/-worker.
 apiVersion: networking.k8s.io/v1
 kind: NetworkPolicy
 metadata:
@@ -47,6 +48,35 @@ spec:
 ---
 apiVersion: networking.k8s.io/v1
 kind: NetworkPolicy
+metadata:
+  name: allow-ingress-authentik-postgresql
+  namespace: authentik
+spec:
+  podSelector:
+    matchLabels:
+      app.kubernetes.io/name: postgresql
+      app.kubernetes.io/component: primary
+  policyTypes:
+    - Ingress
+  ingress:
+    - from:
+        - podSelector:
+            matchLabels:
+              app.kubernetes.io/name: authentik
+              app.kubernetes.io/component: server
+        - podSelector:
+            matchLabels:
+              app.kubernetes.io/name: authentik
+              app.kubernetes.io/component: worker
+        - podSelector:
+            matchLabels:
+              app.kubernetes.io/name: authentik-backup
+      ports:
+        - protocol: TCP
+          port: 5432
+---
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
 metadata:
   name: allow-ingress-acme-solver
   namespace: authentik

apps/production/coturn-secret.yaml

@@ -4,19 +4,28 @@ metadata:
     name: coturn-secret
     namespace: matrix
 stringData:
-    TURN_SECRET: ENC[AES256_GCM,data:SILIqMB+fmAMFITAL7lG1hOgICec6BJf1mOcK0gdmnCHWYqRuJv7jgjfGylG25xzQKi+zE7Qual9PnkZG2KiOA==,iv:+GZqLGusE4Q0x2jEEtFxj06rryyQmQhXdkTy4eE8ZHw=,tag:OpSZkinPTAi1ZKWyo8OX3A==,type:str]
+    TURN_SECRET: ENC[AES256_GCM,data:gab4MSNlRANQz/T/Xn0Z45tkUvQ3Uxn5u0MyYfhZTan49P865UbX4rYPf/KmKsekdwFdOGBzvCgcwxhddVbZ1w==,iv:INbz5UUxgA6xdsPBp5W9Mwe+kd9dhHh2Rx5l3yeNRIQ=,tag:dgk/iFjvst7Wul9Lai2QEQ==,type:str]
 sops:
     age:
-        - recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
-          enc: |
+        - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAyRk1mK3NWc1l4T0JCOFpF
-            S0RuQ3ViZmo3QTNVL2JvZ0hzMy91R2l0TEhzCk01a1VGdk1sdVg4aWswTzRibXI4
-            ZlJtNFF5MjBONEZOaWVpeU5taHl2bkEKLS0tIGxpUHY3NUFLWFBaWm1QSlZiVFkx
-            MEJleHFnd3oyT3VPL2dsYkpMUlRkOWMKcKUIgsQ/ff49pGGXMnYwJmwqPVC7woAR
-            IEzvhcNX97xx746SnrxZe5t2YadsYMkYIl0nvqBPJhSlvqMNafpQbQ==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBQMncxOEhRZC9jcHpjT3dW
+            Q1RNaW5pc00zVnJhWHRnZHd3TGhURWNZU2dJCk85bXJkbTEyd1ZybjhDT05pL2c3
+            ZU9EUSt1eDlSQWJyVGtsak1oS3FSR2sKLS0tIEFUdXVHL0V1ZW5VMVVBZEJaYUIw
+            U1BrYlJyQVZkZFhBdmdwbDMyK3lTQkkKEaSy1o+IICf2uaT6olapRJa/duXxjOBg
+            OqRS9axnJ71XxEnHjLTsCbkI5b+8Fux08qKaH9sMsJrWOiSHDdTXXQ==
             -----END AGE ENCRYPTED FILE-----
-    lastmodified: "2026-04-29T21:06:21Z"
-    mac: ENC[AES256_GCM,data:UhyR5m1HYWrZHwNLW5sg2PxbpaydWbP5cekghGlzSpQg7CYEcvZw3tJ/qB8zA19xZSM7tdSHOXdD+QytRq6qW59M1unqMaumA43B6JxQg1C1NdXAW0mkSc2WiNchvgpVii9P/TVlzSSIRwC3YGCQUsfa3SSfNzI4Z6fMuBnhYLE=,iv:4HYxbrYSRJLe1KcQ6q8bpee8/Lx1m3pPmisb/L2Mu64=,tag:l7n3u+Pg6533OzwtNUZvNw==,type:str]
+          recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAwQk82eGdWRmdRNFJQR0xF
+            SWhNRUtNWThZNFM0aXU3V0N4UkhtTy9NTFdnCi9pa3dsVXRja1dTL1pZTnoxQ0JT
+            UUIxekZnVGUvdFgyblFiS0JLMjU3L2cKLS0tIE1IRTJ0M3kvMFZPWVVDYjJlVkk0
+            eGJQOTVUc1NsNE5GdmJtODlmdHp3c2cKHTP6YRMTdYE/iBuSZs/Tjt4TwKCxHEIu
+            f3jTblKIqWwRHKCgOIkC16QDbpMBlNLH3JknJEdIjkB2HIrXrw1MNA==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1x4jjwc8nuttwr8us924pvdc6dll5npkc6c8f4zf2hx5d2qu75dtqx0fm0d
     encrypted_regex: ^(data|stringData)$
+    lastmodified: "2026-07-28T19:26:57Z"
+    mac: ENC[AES256_GCM,data:zU3i1WGY1X4igBYzfC1WAWdtMQZWGBSacaJpXnUVt6+BCfucqaH807IRWpWbTR0+xuLteaqcdXuVuAlStqs/VV586Q4cD2FfWDeBq/O09HEwTrfekTjAyyu7zCFyZ2ja+kB4DQ83NoscCiNwNmfTZF+6RzMqx7yphPxK5go1r7c=,iv:6FIYWsNvz78zJscbxyDgy1u6ur20MnOMkMcsA4329OI=,tag:uSDpAWEixGcVmtOloMoKuw==,type:str]
     version: 3.12.2

apps/production/coturn.yaml

@@ -65,6 +65,17 @@ metadata:
   namespace: matrix
 spec:
   replicas: 1
+  # hostNetwork pods bind directly to the node's ports (3478/5349) - on this single-node
+  # cluster, RollingUpdate's default "bring up the new pod before removing the old one"
+  # can never schedule (port conflict). Recreate kills the old pod first.
+  # Note: switching to Recreate on an existing Deployment that already had the
+  # RollingUpdate defaults recorded required a one-time manual
+  # `kubectl patch --type=merge -p '{"spec":{"strategy":{"rollingUpdate":null,"type":"Recreate"}}}'`
+  # (2026-07-28) - a YAML `rollingUpdate: null` in this file is dropped before reaching the
+  # API server (client-side omits null keys) rather than sent as an explicit field deletion,
+  # so it can't clear an already-set field on its own.
+  strategy:
+    type: Recreate
   selector:
     matchLabels:
       app: coturn
@@ -74,6 +85,10 @@ spec:
         app: coturn
       annotations:
         prometheus.io/scrape: "false"
+        # Bumped on every TURN_SECRET rotation (Issue #38) to force a new pod, since
+        # Kubernetes doesn't restart running pods when a referenced Secret's content
+        # changes and the initContainer that reads it only runs once at pod start.
+        rotated-at: "2026-07-28T19:26:57Z"
     spec:
       hostNetwork: true
       dnsPolicy: ClusterFirstWithHostNet

apps/production/custom-configs/synapse-values.yaml

@@ -58,15 +58,6 @@ data:
             room_list_publication_rules:
               - user_id: "*"
                 action: allow
-        turn:
-          config: |
-            turn_uris:
-              - "turn:turn.axion1337.chat?transport=udp"
-              - "turn:turn.axion1337.chat?transport=tcp"
-              - "turns:turn.axion1337.chat?transport=tcp"
-            turn_shared_secret: "cab3c8408363515d9b4cdc3384a1f76ca17a973242fdfdc72b67ac4d86158527"
-            turn_user_lifetime: 86400000
-            turn_allow_guests: false
         oembed:
           config: |
             oembed_enabled: true

apps/production/element-server-suite.yaml

@@ -4,7 +4,11 @@ metadata:
   name: matrix-stack
   namespace: matrix
 spec:
-  interval: 5m
+  # Shortened from 5m to match production-apps Kustomization's 1m interval - narrows the
+  # window between coturn (Kustomization-only, no Helm indirection) and synapse-main
+  # (behind this HelmRelease) picking up a rotated TURN secret after Issue #38's
+  # automated-rotation PR gets merged. Self-heals either way, just faster now.
+  interval: 1m
   chart:
     spec:
       chart: matrix-stack
@@ -25,7 +29,10 @@ spec:
     - kind: Secret
       name: ess-mas-values-secret
       valuesKey: values.yaml
-      
+    - kind: Secret
+      name: synapse-turn-secret
+      valuesKey: values.yaml
+
   values:
     # Top-Level: serverName – das ist dein Matrix-Homeserver-Name
     serverName: axion1337.chat
@@ -74,7 +81,7 @@ spec:
       image:
         registry: rohana.axion1337.de
         repository: sorb/threadnet-web
-        tag: v0.2.1-elementcall-noquotavp9
+        tag: v0.2.3-elementcall-h264
       ingress:
         host: axion1337.chat
 

apps/production/kustomization.yaml

@@ -1,8 +1,10 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 
-# Patch: Fügt einen Checksum der element-values.yaml zur HelmRelease hinzu
-# Damit wird Flux die HelmRelease neu-synced wenn sich die ConfigMap ändert
+# Patch: Fügt Checksums der element-values.yaml und des turn_shared_secret zur
+# HelmRelease hinzu. Damit wird Flux die HelmRelease neu-synced (und synapse-main neu
+# gestartet), wenn sich die jeweilige ConfigMap/Secret ändert - siehe Issue #38's
+# Rotations-Mechanismus, der turn-secret-checksum bei jeder Rotation bumpt.
 patches:
   - target:
       kind: HelmRelease
@@ -12,6 +14,9 @@ patches:
       - op: add
         path: /metadata/annotations/element-config-checksum
         value: "401f8a87d0ef5d91d2e5032d4aede42c"
+      - op: add
+        path: /metadata/annotations/turn-secret-checksum
+        value: "d220c0e4ff8f7106328c8827d47e8734"
 
 resources:
   - matrix-postgres-auth.yaml
@@ -26,8 +31,15 @@ resources:
   # TURN Server für WebRTC
   - coturn-secret.yaml
   - coturn.yaml
+  - synapse-turn-secret.yaml
   # HelmRelease (muss ganz unten stehen, damit die ConfigMaps vorher da sind!)
   - element-server-suite.yaml
   # Custom Apex Ingress für Element Web + Well-Known auf axion1337.chat
   - apex-ingress.yaml  # Custom Apex Ingress für Element Web + Well-Known auf axion1337.chat
-  - networkpolicy.yaml
+  - networkpolicy.yaml
+  # Backup zur Hetzner Storage Box (Issues #6 + #15)
+  - synapse-backup-secret.yaml
+  - synapse-backup.yaml
+  # Automatisierte TURN-Secret-Rotation (Issue #38)
+  - turn-secret-rotation-secret.yaml
+  - turn-secret-rotation.yaml

apps/production/networkpolicy.yaml

@@ -258,6 +258,9 @@ spec:
         - podSelector:
             matchLabels:
               app.kubernetes.io/name: matrix-authentication-service
+        - podSelector:
+            matchLabels:
+              app.kubernetes.io/name: synapse-backup
       ports:
         - protocol: TCP
           port: 5432

apps/production/synapse-backup-secret.yaml

@@ -0,0 +1,23 @@
+apiVersion: v1
+kind: Secret
+metadata:
+    name: synapse-backup-credentials
+    namespace: matrix
+stringData:
+    borg-passphrase: ENC[AES256_GCM,data:RRXPwr4UGX30IdozM53abN7ZYztTO3Y1/63dtTh0JSZmU8i8l/ATb4gc3lc=,iv:rIyUr+lOUjo9J53OKZ5ZDmp3d8Nrb9PP2JDK2oCutYU=,tag:MqUmoYgCA03WJQy+RQi04w==,type:str]
+    ssh-private-key: ENC[AES256_GCM,data:320B/lSq7DljCrXZ0BluGv8gLIzYF3KL6VQnR6CM/Vzuf/6qbbZ7lMzrS96XfoubU95OxDeJuOsuNZBQUZNyUGtt4QjQmnl8XHUJzl48xLqh83HLFPtQm2uqU072lscf5OTT7I+JzD2BGks3OIowhrg1q0MdVQdfd4Rhdz8Jphjb+WxvstzNmEX5gxQ9mnBVPmj4DS6ikdXnpe+VDvaCJaVkzD5KgwRmqPy0qbFs9WXziQSo3am5fPeHbwhV3UlRhlok9WrDI40a1T1S5DBhgNhwShq1jAxjr9onuq92REymxAV50oLzsw9ivnH0uimw+3PcplRG1v2xxJ/pimWTCjE17bO7OZ8TyzzwyZ7QA02vSpOMNUdUwVU3N6pSdYQpdTETuVBTVqVc+GKC18Z9fWh2rdwX5eUDTwp0bbDDvPEHdANNxMg3VYD3gwpCgy6/wVnjrO+pVMs8K1CqFn4/H7azhzzeEPkrz27ZBxjlnqvDODsm9tlklr6X4jZDZmTaEmiH+WXuc+1qbNxvsrqtSFfa3CfFGxM1nBOh,iv:sKGsTLsxdQYVUvw7CEARL3YNInSd9LPbFp5Ci5CTgIw=,tag:O1mjNZUsGEqV4uu+LlgD/g==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB3ZUdtN3hyNHlyMkIrbTVS
+            aHQ4OGNUWnA4eTFUVkx4UVp5VWlnMDgxTEU0CkU3M2dMYWgyKytlRVFOVWptZEd2
+            NXlIY0JCd24xcGFzaGpIeks0R2U0U3MKLS0tIHlxYVZ1ZTJsRXNaZ25sVzZtSnp0
+            SEhzQ0tUYzZTRXcwMVNwbG85SHpyb0UKOn3nxy6Y7yQkGargXQ9z6O36vUWW4qJZ
+            D/GbFGmoRi94EtVFdmTGALhjy2D4J9QXy6gHsTapvKyMxF8NEtk+FQ==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
+    encrypted_regex: ^(data|stringData)$
+    lastmodified: "2026-07-28T18:20:58Z"
+    mac: ENC[AES256_GCM,data:Rur32fQdCyM3nr/X+KeSgmPYEi4nKyh8lqTuSW3TPBVDjwTWMDp1I1ZPPyy5syeW6RHbKputFUzBWVnuQmVrfbZaQ6DBBI5kP9InspUAVUjXDRk9XqiWtdg/wYaTMMJ4Nxv/zdwkh6uJQSG2JHQBWce4NZc2hoPokLR0CjcWiZk=,iv:eWTUj48EFjjtuIIuErMltEdDfabLZeolkpInMYtVP5Q=,tag:6DPxkPqLt0ihJ80WnTzHeA==,type:str]
+    version: 3.13.3

apps/production/synapse-backup.yaml

@@ -0,0 +1,96 @@
+# Nightly Borg backup of the shared Postgres instance (synapse + MAS databases) and the
+# Synapse media_store PVC to a Hetzner Storage Box (issues #6 + #15). See
+# apps/production/synapse-backup-secret.yaml for the SSH key + Borg repo passphrase, and
+# apps/authentik/authentik-backup.yaml for the equivalent authentik-side job.
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: synapse-backup-known-hosts
+  namespace: matrix
+data:
+  # Pinned via `ssh-keyscan -p 23 u641795.your-storagebox.de` (2026-07-28) rather than
+  # trusting the host key on first connect in an unattended job.
+  known_hosts: |
+    [u641795.your-storagebox.de]:23 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIICf9svRenC/PLKIL9nk6K/pxQgoiFC41wTNvoIncOxs
+---
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+  name: synapse-backup
+  namespace: matrix
+spec:
+  schedule: "0 3 * * *"
+  concurrencyPolicy: Forbid
+  successfulJobsHistoryLimit: 3
+  failedJobsHistoryLimit: 3
+  jobTemplate:
+    spec:
+      backoffLimit: 2
+      template:
+        metadata:
+          labels:
+            app.kubernetes.io/name: synapse-backup
+            app.kubernetes.io/component: backup
+        spec:
+          restartPolicy: OnFailure
+          containers:
+            - name: backup
+              image: rohana.axion1337.de/sorb/axion-backup:v2
+              env:
+                - name: BORG_REPO
+                  value: "ssh://u641795@u641795.your-storagebox.de:23/./synapse-backup"
+                - name: BORG_PASSPHRASE
+                  valueFrom:
+                    secretKeyRef:
+                      name: synapse-backup-credentials
+                      key: borg-passphrase
+                - name: SSH_PRIVATE_KEY_FILE
+                  value: /secrets/ssh/ssh-private-key
+                - name: SSH_KNOWN_HOSTS_FILE
+                  value: /secrets/known-hosts/known_hosts
+                - name: DB_HOSTS
+                  value: "synapse:matrix-stack-postgres matrixauthenticationservice:matrix-stack-postgres"
+                - name: PGUSER
+                  value: postgres
+                - name: PGPASSWORD
+                  valueFrom:
+                    secretKeyRef:
+                      name: matrix-stack-generated
+                      key: POSTGRES_ADMIN_PASSWORD
+                - name: MEDIA_PATH
+                  value: /media/media_store
+              volumeMounts:
+                - name: ssh-key
+                  mountPath: /secrets/ssh
+                  readOnly: true
+                - name: known-hosts
+                  mountPath: /secrets/known-hosts
+                  readOnly: true
+                - name: media
+                  mountPath: /media
+                  readOnly: true
+                - name: scratch
+                  mountPath: /scratch
+              resources:
+                requests:
+                  cpu: 50m
+                  memory: 128Mi
+                limits:
+                  memory: 256Mi
+          volumes:
+            - name: ssh-key
+              secret:
+                secretName: synapse-backup-credentials
+                items:
+                  - key: ssh-private-key
+                    path: ssh-private-key
+                    mode: 0400
+            - name: known-hosts
+              configMap:
+                name: synapse-backup-known-hosts
+            - name: media
+              persistentVolumeClaim:
+                claimName: matrix-stack-synapse-media
+                readOnly: true
+            - name: scratch
+              emptyDir: {}

apps/production/synapse-turn-secret.yaml

@@ -0,0 +1,31 @@
+apiVersion: v1
+kind: Secret
+metadata:
+    name: synapse-turn-secret
+    namespace: matrix
+stringData:
+    values.yaml: ENC[AES256_GCM,data:An+GypE/btGmfnvL1rPzvbN1sAnZKQnHV20GxXkZiRb+Wi8BymAUA6JRBo5mdtvK1dgDMF6H1wN+Y7wjzwduteaoskVfgd7Xw+lL1SrZ/6pchGxlUIWSJlLTmp6IHZaruleDUsCKdrWfN9TQDE+WYdiECBktlGGXs8ZZMTzjHmHKJ3tNwSimaxuG662Amb0LiHLX+zBUKOuG0L4cWrp6JI3NKQFgNNiF6xwXjHDX550bGOQ2pi5UVPL9+HlH2tPeTzoCCZdJLWF+2kZg31bZ7dzRp8M23eDrzfGC9e9m0NrA2z1ljagbnuT0tixp6J8205qfBOIOVOWbXHmd/LQ8nphA64En6JXqU8XeZ+mgMUmFuGwyewLq4oilLoQfdZujEyGhc9Hd/yonRlc/RDBlONlI8d+IWqZxf0AaUYEAe9uE6MulqRs0CowPUpCmWtZjz68qQlIvJEAqxNdWEnp+SkExDNNe9y8tBOCXy726TRX0MLIlZ+CjPDCHhwDeKw5oQU0wXr9QezY=,iv:bPK1+PViBg046N3q2Eb1ZOJOYv7X/HgpCEU/MLp7o6k=,tag:/2Kyv7aB4baOL+3KdAeQ9w==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBuK2xIT3M0cGtVOHdKYklF
+            OHl1VHFRTkVSaHVQNUUyRlhZUjRWM3JUN0hBCnpUS2YzdG1mSjlRaEVvTHdKVkR4
+            L1hrR05IRUdqdGp2aU95aWxRRXdsQWcKLS0tICtLbnFsK3Zza053VXFWdVErRy92
+            WUUzZTBIRzUyWnp5a1ZScUVqb0NyencKvnFyJCR6j1/aH4gJvFmLPNlk5XpC08wF
+            mTmL981uGfz4NULc+O3sDkonJ827glpefgWPgPW2HmKT88d4A9vyJw==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBVY0NXa1B5c2RzbzRReU1h
+            RHBSbk9aK3d6VjRWZ1hzL3FDZFZJZGFqTVQ0Ck9ib0p3bkR6cU8yc0VFNjEzSFFi
+            T3dWY1luQW1VZ1NjZFNoZFFLSCsvelkKLS0tIHIxV0d6TWhpSEc3d3c0L1VvOWxk
+            eDhTM2pDbTNXZXlWRVYxR2tPQU5iLzgKrLLLSBU/g5ebeRNi7hWYbcuJ/2JOfiUn
+            0DBnzMVJPBfqq/u8THiRYaMajx3k4D9+FN7qc5nBgTd85iGBo+OowA==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1x4jjwc8nuttwr8us924pvdc6dll5npkc6c8f4zf2hx5d2qu75dtqx0fm0d
+    encrypted_regex: ^(data|stringData)$
+    lastmodified: "2026-07-28T19:26:57Z"
+    mac: ENC[AES256_GCM,data:UfD+QJel9FygoKPM02KdOLvP1id/21ghQ0uWAQJxEs/w+B2vVkpe/dXSJ7QnI0e528ns1HW81GAxkxyAupsUDq8/pBdEQgItwDs5DIgHvH+sTPAtj2Kyq2wMOeEt+XDGLRdOlAwbcKDS/LyfeZ5b9QnAbY2w/tTinTkzEvbudT8=,iv:WmmwjAmZmM0W8dvqBcv9XYZkYtan3lg0Sb8ypVuGoJk=,tag:3yzZhSFtbSbLUXSdDWGAtw==,type:str]
+    version: 3.13.3

apps/production/turn-secret-rotation-secret.yaml

@@ -0,0 +1,23 @@
+apiVersion: v1
+kind: Secret
+metadata:
+    name: turn-secret-rotation-credentials
+    namespace: matrix
+stringData:
+    age-key.txt: ENC[AES256_GCM,data:4LAs9LLFo38UMHXCo4lun9RHxGnDyp7GWlaNdIqqkSL9lNv7+ILdlc03CxFVobCYxK65xMOn1xdEty+887JBMlawST04am/5MkAnUivKwXCw8OHmbZhCwKHFqSYH/NsgVNf+btZKIIny8XPVQAPj/vQIi+Ity+BQyPkEZ1WUcsqjDoaK9IFhQTePJtHgWivhOY2WpUt/TP7vTfub4TOrgVpzNd9LIpBkwq+zhpVXKwnYUWOuLMXlPe08kazy,iv:9A24HbTl24slj+qTCfyI01+dGqRFVPDUA0wp2kSUHpc=,tag:iQxU0dWD4Noo1m0HXhV6vQ==,type:str]
+    gitea-token: ENC[AES256_GCM,data:cmZ1GCrqRYLtLn+cRVZCrO7UcCIavlQLJPt2PRMtBbgLdhDVTElUKA==,iv:9v29GXHRtDlrL3PoRCdOqYpBepZrX04+6UjoywRZX0E=,tag:ZvYXBOm9qgB1XA5lkV4LsQ==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA5S3BtTHNLVGVVM2ZpTlZS
+            Y3pGMS9CSkNsdUpPbWtkSTRHK0p6U2lwdEJnClVWdXp4SllyM1hvbTZyTU40SDc2
+            QlVtMDduZWpaVENiYnhMNlFXd01QblEKLS0tIGNzTGRZcmoyaFltUHRDSHBPZE1N
+            OCszUkl1VjQ5V3F2cVI4dXJFcER5YXcK+2Eh1JNLuMiCnpQ3cL/I7XTykkIZ3tqp
+            O3c9UwYs1FAZWlMgElTBTqsmut1ShduIYfDFRKGeS0UxPEM4U+tIGw==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
+    encrypted_regex: ^(data|stringData)$
+    lastmodified: "2026-07-28T19:21:28Z"
+    mac: ENC[AES256_GCM,data:vx6Bs/L0NXKUvvQdu6aYtuur/CYPkIBZzvFLqTyd08Errw0dGMrg73oHQ/imxpe42HgnO2mGwxNdEx2jYYbtc3RBWHE/yPH5m8y/XLoSL3fauzbkGsDwMSWzKiZXyIuGh7SxuB+CFY9qqFMK+dap1Ofno7a1/Gr1qibVDqscwxw=,iv:9AUQQuTCja09OohzVw73URMHE8xCW7iLLtBg7GSDcPA=,tag:g1ibnBIGqokvs2IEVOYq6Q==,type:str]
+    version: 3.13.3

apps/production/turn-secret-rotation.yaml

@@ -0,0 +1,70 @@
+# Automated TURN shared-secret rotation (Issue #38). Generates a new secret, re-encrypts
+# apps/production/coturn-secret.yaml and synapse-turn-secret.yaml using a dedicated,
+# narrowly-scoped age key (see turn-secret-rotation-secret.yaml - it can only decrypt these
+# two files, not the repo's master sops-age key), bumps the checksum/rotated-at annotations
+# so a merge restarts both consumers automatically, and opens a Pull Request rather than
+# pushing straight to main - a human reviews and merges it.
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+  name: turn-secret-rotation
+  namespace: matrix
+spec:
+  schedule: "0 4 1 * *"
+  concurrencyPolicy: Forbid
+  successfulJobsHistoryLimit: 3
+  failedJobsHistoryLimit: 3
+  jobTemplate:
+    spec:
+      backoffLimit: 1
+      template:
+        metadata:
+          labels:
+            app.kubernetes.io/name: turn-secret-rotation
+            app.kubernetes.io/component: rotation
+        spec:
+          restartPolicy: OnFailure
+          # Public-internet reachability to the Gitea host has been flaky (see Issue #41);
+          # both servers share a private Hetzner network. hostAliases (unlike the node-level
+          # /etc/hosts fix used for image pulls) is actually honored by in-pod processes.
+          hostAliases:
+            - ip: "10.0.0.3"
+              hostnames:
+                - "rohana.axion1337.de"
+          containers:
+            - name: rotate
+              image: rohana.axion1337.de/sorb/axion-secret-rotation:v1
+              env:
+                - name: GITEA_HOST
+                  value: "rohana.axion1337.de"
+                - name: GITEA_REPO
+                  value: "sorb/axion1337.chat-gitops"
+                - name: GITEA_TOKEN
+                  valueFrom:
+                    secretKeyRef:
+                      name: turn-secret-rotation-credentials
+                      key: gitea-token
+                - name: SOPS_AGE_KEY_FILE
+                  value: /secrets/age/age-key.txt
+                - name: GIT_AUTHOR_NAME
+                  value: "turn-secret-rotation"
+                - name: GIT_AUTHOR_EMAIL
+                  value: "turn-secret-rotation@axion1337.chat"
+              volumeMounts:
+                - name: age-key
+                  mountPath: /secrets/age
+                  readOnly: true
+              resources:
+                requests:
+                  cpu: 50m
+                  memory: 64Mi
+                limits:
+                  memory: 128Mi
+          volumes:
+            - name: age-key
+              secret:
+                secretName: turn-secret-rotation-credentials
+                items:
+                  - key: age-key.txt
+                    path: age-key.txt
+                    mode: 0400

docs/TASKS.md

@@ -53,10 +53,21 @@ verankert. Was in dieser Session erledigt wurde:
 5. **NetworkPolicies** (Issue #10 ✅ geschlossen) — siehe "Network Security" Abschnitt unten.
    Zwei Live-Incidents beim Rollout (Port-Verwechslungen), beide binnen Minuten live gepatcht
    und danach committed. Nebenbei: `matrixRTC`-Authorisation-Service OOM-Fix (20Mi→128Mi).
+6. **Element Call Qualität nachgeschärft** — 720p-Zwischen-Simulcast-Layer ergänzt (sonst
+   harter Sprung von 1440p auf blockiges 360p bei kleinsten Netzwerkschwankungen), und
+   `video_codec: h264` statt VP8 (klassisches Simulcast wie VP8, kein SVC-Risiko wie bei
+   VP9, oft hardwarebeschleunigt v.a. auf iOS). Live verifiziert: 7/8 Tracks nativ H.264,
+   1 sauberer VP8-Fallback. Deployed als `v0.2.3-elementcall-h264`.
+7. **Backlog nach Gitea migriert** — restlicher offener Backlog (VP9-Retry, ThreadNet-Web-Bug,
+   MAS-Template-Link, WAF und 17 weitere Security-/Infra-Punkte) als Issues #11–#31 angelegt,
+   veraltete erledigte Punkte (Authentik Stage 2/E2E-Test/Invite-Links, Hetzner-Firewall,
+   SSH-Hardening) aus dieser Datei entfernt bzw. als done markiert.
 
 ### Offene Punkte
-- **VP9-Retry**: Root Cause des Call-Ausfalls isolieren (Browser-Konsolen-Repro nötig), bevor
-  wieder versucht wird — siehe Backlog "Element Call Enhancement".
+- **VP9-Retry**: vermutete Ursache jetzt bekannt (LiveKit nutzt SVC für vp9/av1, Fork-Code
+  setzt aber immer Simulcast-Layer) — braucht einen Code-Fix in `buildPublishOptions()`
+  (`src/livekit/options.ts`) bevor erneut versucht wird. Stattdessen H.264 probiert (siehe
+  unten) — läuft gut, kein SVC-Risiko, hardwarebeschleunigt auf mehr Geräten.
 - **`ThreadNet-Web` Build-Bug**: `scripts/docker-link-repos.sh`/`docker-package.sh` nicht
   ausführbar committet + veralteter `matrix-js-sdk#develop`-Pin im Lockfile blockiert
   vollständigen Neu-Build des Web-Forks. Noch nicht gefixt, User hat noch nicht final
@@ -259,31 +270,22 @@ verankert. Was in dieser Session erledigt wurde:
 - [x] **matrix-recovery Flow gebaut** – war komplett leer (0 Stages); Passwort-Reset funktioniert jetzt, verlinkt von der echten Login-Seite
 - [x] **Brand.default_application gesetzt** – behebt mehrere Dead-Ends, wenn eingeloggte User `/` ohne Ziel aufrufen
 - [x] **2FA/Passkey Selbst-Einrichtung** – Links zu `default-authenticator-totp-setup`/`-webauthn-setup` (2FA bleibt optional, `not_configured_action=skip`), dokumentiert unter `axion1337.chat/docs/setup/security.html`
-- [ ] **Backlog**: MAS Custom-Template-Override, um den 2FA/Passkey-Link direkt auf `account.axion1337.chat/account/` anzuzeigen (statt nur über die Doku-Seite) – erfordert Forken der MAS Tera/Askama-Templates via `templates.path`, höherer Aufwand/Risiko, bei MAS-Updates zu pflegen
+- [ ] **Backlog**: → **Issue #13** (MAS Custom-Template-Override für 2FA/Passkey-Link auf `account.axion1337.chat/account/`)
 
 ---
 
 ## 📋 Backlog (Weitere Aufgaben)
 
-### Authentik Completion
-- [ ] **Finish Authentik Stage 2 – MAS Integration**
-  - Prerequisites: Authentik OIDC Provider vollständig konfiguriert
-  - Task: Update `mas-secret.yaml`, enable password login disable
-  - Commit: `enable-authentik-oidc-integration-in-mas`
-  - Est. Effort: 30 min (manual + scripted)
-
-- [ ] **Test End-to-End Login Flow**
-  - Element Web login → MAS → Authentik → Matrix User Creation
-  - Create test users via Authentik
-  - Verify password reset flow
-  - Commit: (implicit in Stage 2)
-  - Est. Effort: 20 min
-
-- [ ] **Create Invite Links für neue User**
-  - Authentik Admin UI → Invitations → Create
-  - Set expiry dates (7d) + use limits
-  - Document procedure
-  - Est. Effort: 15 min
+**Ab 2026-07-28 in Gitea-Issues gepflegt statt hier** (eine Quelle der Wahrheit) — offene Issues:
+[#6](https://rohana.axion1337.de/sorb/axion1337.chat-gitops/issues/6) DB-Backup-Strategie,
+[#9](https://rohana.axion1337.de/sorb/axion1337.chat-gitops/issues/9) Externe PostgreSQL-Migration,
+[#11](https://rohana.axion1337.de/sorb/axion1337.chat-gitops/issues/11)–[#31](https://rohana.axion1337.de/sorb/axion1337.chat-gitops/issues/31)
+(VP9-Retry, ThreadNet-Web-Build-Bug, MAS-Template-Link, WAF, Media-PVC-Backups, Pod Security
+Admission, Federation-Allowlist, Mjolnir/Draupnir, Content-Scanner, External-Secrets,
+Renovate/Trivy, Security-Advisory-Monitoring, automountServiceAccountToken,
+unattended-upgrades, K3s-API-Security, auditd, Kernel-Hardening, Lynis, CrowdSec, Falco).
+Die detaillierten Beschreibungen unten sind das historische Original, aus dem die Issues
+entstanden sind — nicht mehr getrennt pflegen, stattdessen die Issues aktuell halten.
 
 ### Element Call Enhancement
 - [x] **Element Call Fork für Custom Constraints** (2026-07-28, Closes #8)
@@ -448,24 +450,8 @@ verankert. Was in dieser Session erledigt wurde:
 ## 🔒 Security Hardening (Host & Cluster Level)
 
 ### Host OS Layer (Ubuntu/Debian)
-- [ ] **Hetzner Cloud Firewall**
-  - Default-Deny inbound
-  - Allow: 80/443 (HTTP/HTTPS)
-  - Allow: 22 (SSH) from your IP only (or via WireGuard/Tailscale)
-  - Status: ✅ Can be done in Hetzner UI
-  - Est. Effort: 30 min
-  - Priority: **CRITICAL** (immediate, zero config cost)
-
-- [ ] **SSH Hardening**
-  - Disable password auth (key-only)
-  - Disable root login
-  - PermitRootLogin: no
-  - PasswordAuthentication: no
-  - MaxAuthTries: 3
-  - Optional: Change SSH port (cosmetic, reduces log noise)
-  - Optional: SSH hinter WireGuard/Tailscale (eliminates fail2ban für SSH)
-  - Est. Effort: 2 hours
-  - Priority: **HIGH** (immediate)
+- [x] **Hetzner Cloud Firewall** – Default-Deny inbound, siehe "Phase 7" oben. **Done.**
+- [x] **SSH Hardening** – Key-only, Root-Login disabled, Port 2248, siehe "Phase 7" oben. **Done.**
 
 - [ ] **unattended-upgrades**
   - Enable automatic security updates

docs/deployment-guides/04-element-customization.md

@@ -57,9 +57,16 @@
   LiveKits eigener Empfehlung für Mensch-zu-Mensch-Calls. Bewusst **kein** server-seitiges
   ML-Noise-Cancellation (siehe `docs/TASKS.md` Backlog).
 - **Incident (2026-07-28)**: Erster Versuch mit erzwungenem `video_codec: vp9` hat Calls
-  komplett kaputt gemacht (kein Bild/Ton). Sofort zurückgerollt, dann ohne erzwungenen Codec
-  (Standard VP8) mit denselben Auflösungswerten neu deployed — funktioniert, live bestätigt.
-  VP9 vorerst zurückgestellt (Backlog).
+  komplett kaputt gemacht (kein Bild/Ton). Sofort zurückgerollt. Vermutete Ursache: LiveKit
+  nutzt für vp9/av1 SVC statt klassischem Simulcast, `buildPublishOptions()` im Fork setzt
+  aber immer Simulcast-Layer — Code-Fix nötig, bevor vp9 erneut versucht wird (Backlog).
+- **720p-Zwischen-Layer ergänzt** (`simulcast_layers`) — ohne eigene Definition fiel die
+  Übertragung bei kleinsten Netzwerkschwankungen direkt von 1440p auf blockiges 360p, jetzt
+  sanftere Abstufung über 720p.
+- **H.264 statt VP8** (2026-07-28) — nutzt wie VP8 klassisches Simulcast (kein SVC-Risiko wie
+  bei VP9), zusätzlich auf vielen Geräten (v.a. iOS/Safari) hardwarebeschleunigt. Live
+  verifiziert: 7 von 8 Video-Tracks liefen über H.264, 1 fiel sauber auf den VP8-Backup-Codec
+  zurück (kein Ausfall). Deployed als `v0.2.3-elementcall-h264`.
 - Deployt als `rohana.axion1337.de/sorb/threadnet-web:v0.2.1-elementcall-noquotavp9` — nur
   der `/app/widgets/element-call/`-Ordner im bestehenden `v0.1.0`-Image ausgetauscht, da
   `ThreadNet-Web` einen vorbestehenden Build-Bug hat (siehe unten).

docs/install.md

@@ -75,4 +75,59 @@ flux get helmreleases -n matrix --watch
 # Zeigt, wie die Pods hochfahren:
 kubectl get pods -n matrix -w
 ```
-Sobald alle Pods auf `Running` stehen und die Zertifikate über Let's Encrypt validiert wurden (`kubectl get certificate -n matrix`), ist dein Matrix-Stack unter `https://axion1337.chat` erreichbar.
+Sobald alle Pods auf `Running` stehen und die Zertifikate über Let's Encrypt validiert wurden (`kubectl get certificate -n matrix`), ist dein Matrix-Stack unter `https://axion1337.chat` erreichbar.
+
+---
+
+## 🔁 Recovery: lokalen age-Key wiederherstellen (Server läuft bereits)
+
+Anders als Schritt 2 oben (neuen Key **erzeugen**) — falls der Server bereits läuft und nur der
+lokale Rechner den age-Key verloren hat (z.B. nach einer Neuinstallation), lässt sich der
+**bestehende** Private Key direkt aus dem Cluster zurückholen, ohne einen neuen zu generieren
+(das würde `.sops.yaml` und alle bereits verschlüsselten Secrets ungültig machen):
+
+```bash
+mkdir -p ~/.age
+kubectl get secret sops-age -n flux-system -o jsonpath='{.data.age\.agekey}' | base64 -d > ~/.age/keys.txt
+chmod 600 ~/.age/keys.txt
+
+# Public Key zur Kontrolle gegen .sops.yaml abgleichen:
+grep 'public key:' ~/.age/keys.txt
+grep 'age:' .sops.yaml
+```
+
+Voraussetzung: laufender Kubeconfig-Zugriff auf den Cluster (siehe Schritt 1 oben — auch das
+ist reines Zurückkopieren, kein Neu-Erzeugen).
+
+**Bekannte Schwachstelle**: Dieser Key existiert aktuell nur an zwei Orten — im
+`sops-age`-Secret selbst (auf demselben Server) und lokal bei wem auch immer ihn zuletzt
+zurückgeholt hat. Es gibt kein separates, offsite Backup. Fällt der Server komplett aus
+(nicht nur der lokale Rechner), sind alle SOPS-verschlüsselten Secrets im Repo unlesbar.
+Siehe Issue-Backlog für die Entscheidung, ob/wie das abgesichert wird.
+
+---
+
+## 🌐 Node-Konfiguration: `/etc/hosts`-Eintrag für den Gitea-Host
+
+Der K3s-Node und der Gitea-Host (`rohana.axion1337.de`, Container-Registry + Git-Remote)
+teilen sich ein privates Hetzner-Netzwerk (Node `10.0.0.2`, Gitea-Host `10.0.0.3`). Seit
+2026-07-28 hat der Node dafür einen manuellen `/etc/hosts`-Eintrag:
+
+```
+10.0.0.3 rohana.axion1337.de
+```
+
+**Warum**: eine Firewall-Fehlkonfiguration hatte den Node zeitweise komplett von
+`rohana.axion1337.de` über die öffentliche IP abgeschnitten, was Image-Pulls (z.B. für
+Custom-Images wie `sorb/axion-backup`) mit Timeout scheitern ließ. Der Eintrag macht
+Image-Pulls unabhängig vom Zustand der öffentlichen Firewall.
+
+**Wichtig**: Das ist unmanaged Node-Konfiguration (kein GitOps, kein Kubernetes-Objekt) —
+überlebt einen Node-Neuaufbau **nicht** und muss dann erneut gesetzt werden:
+
+```bash
+echo "10.0.0.3 rohana.axion1337.de" | sudo tee -a /etc/hosts
+```
+
+Ein sauberer, clusterweiter Ersatz (z.B. CoreDNS-Rewrite, damit auch Pods selbst intern
+auflösen) ist als Issue #41 nachgehalten.