docs: document node-level /etc/hosts entry for private-network registry access

fix(backup): retry pg_dump connection to survive NetworkPolicy sync lag
Live testing showed pg_dump got "Connection refused" immediately at job start, but a manual pg_isready/psql from an identically-labeled pod a few seconds later succeeded fine every time. The cluster's NetworkPolicy controller needs a brief moment to program a brand-new pod's IP into the target's allowed ingress rules; a job that starts dumping instantly can race that window. backup.sh now waits for pg_isready before each pg_dump (up to 15 tries / ~30s) instead of assuming connectivity is ready at container start. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 12:00:00 +00:00 · 2026-07-28 12:00:00 +00:00 · 2026-07-28 12:00:00 +00:00 · 2026-07-28 12:00:00 +00:00 · 2026-07-28 12:00:00 +00:00
12 changed files with 372 additions and 9 deletions
@@ -0,0 +1,23 @@
+apiVersion: v1
+kind: Secret
+metadata:
+    name: authentik-backup-credentials
+    namespace: authentik
+stringData:
+    borg-passphrase: ENC[AES256_GCM,data:5PApz4TqSNN2vVXeFSuomd051nl+cYk+a+STViwddG/Hj7XWQ099vvTlKSE=,iv:MFG04/66YqtOjZWsLpy236MYwR05z91ngOQ0BmLNxzA=,tag:gxD5prW8Ted3Q2ZY3sYSYQ==,type:str]
+    ssh-private-key: ENC[AES256_GCM,data:87PrkqAIH2xRCu3jlAK3Ts24XKRDoXNCWLomRhizCY43hwLeY5fNzfsGnMTKBt5IxL2bIljx+pbh/Y8IO38ISUUbUAczfi1KeN1gg69wKgqNAlckcmD0dkA2N2F0ZnEFX/rkad68YR8Hw1/A5h3qGUd5FwoVWzIFD/diY86s/LiQBYzwD4bYLUACHqIL7+QyJ6jEsFzA3tLwDySpUpM9OAjFQF0EazObChNYqe/qrc8i1gI8CptWOFkqiZ8Pen1E79gg/zgleOBqSmlcefgZk+JFPAfvORTHkDrHW9X/bLygaTQvskmGNzaKHpe2s/bVafWb7G60ROrpdjYIJfS7c0baRCSrd9kwdM4JQC2gSsMkI4v6oeFSvkTrb684B10zacZdiTKfySDV+Ry4hCZofF2RhAnsTbOpo0XANoOi4InoxUqGPfctVfUCVb6X+HIR5XYxpLrNwWqRpscMP2P3RJsaUZOinmQUKrHX/nMtPZ/ltpwjhQeAXZTtdyPxTCjHAh6fzouuHk66vlCdukXsMQDBdIfdA/3jjGbW,iv:yFNKkdegLLq8jq1Ya6v67urSJTdG3Ge4ZbmKizqQhmk=,tag:6l/qMb1d1oD8sfVEebW6gQ==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBWMllTL2hRcEhCT2MxcWZy
+            dkxndmlZRW5FZ2NiWWdFckxqYTFKS1B6SXlvCjJySXRsZThvczNnWjlyM0N2Snha
+            eitvWUhhdmU4U0V3OG55WjdLbU9KcUEKLS0tIEM3MStzNUJhaGg4M0hKMVF1bnBB
+            RWVqVDRBNmJ0b1Bwd0l3dWxPT3Q0SU0KOEoyejkH4RC0p8ka3FjI7MyzRJg+uu7h
+            j3wf1q+Hgg73djDBSPYJkrB6Bdl4YMwo8SzbtW8O9elDE0qAioR0bQ==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
+    encrypted_regex: ^(data|stringData)$
+    lastmodified: "2026-07-28T18:20:58Z"
+    mac: ENC[AES256_GCM,data:lwwNbxSxtgDTDmaWMU1uf7TBOEw8gFBFKb982VIsGMeM0fIPHvX12Qts90MNgYIJliOWgAWrwyvAgfXWKuE37RNo+BtyfHCWi4IESKSN/RJrd/yMpRKx+02rifH3nl26ZCAQT1Pa0fjI1SfMhbVzfnD9a/AMARXZMhLRc0OqczY=,iv:SgP2iMtENRtZfw6I9EaOsmvecFNYCIZWqVj+cZ+T7EI=,tag:s4oEyxqyUx6ibqsk4g9xpw==,type:str]
+    version: 3.13.3
@@ -0,0 +1,85 @@
+# Nightly Borg backup of the authentik Postgres database to a Hetzner Storage Box
+# (issues #6 + #15). See apps/authentik/authentik-backup-secret.yaml for the SSH key +
+# Borg repo passphrase, and apps/production/synapse-backup.yaml for the matrix-side job
+# (same Storage Box, separate repo/passphrase, offset schedule).
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: authentik-backup-known-hosts
+  namespace: authentik
+data:
+  known_hosts: |
+    [u641795.your-storagebox.de]:23 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIICf9svRenC/PLKIL9nk6K/pxQgoiFC41wTNvoIncOxs
+---
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+  name: authentik-backup
+  namespace: authentik
+spec:
+  schedule: "15 3 * * *"
+  concurrencyPolicy: Forbid
+  successfulJobsHistoryLimit: 3
+  failedJobsHistoryLimit: 3
+  jobTemplate:
+    spec:
+      backoffLimit: 2
+      template:
+        metadata:
+          labels:
+            app.kubernetes.io/name: authentik-backup
+            app.kubernetes.io/component: backup
+        spec:
+          restartPolicy: OnFailure
+          containers:
+            - name: backup
+              image: rohana.axion1337.de/sorb/axion-backup:v2
+              env:
+                - name: BORG_REPO
+                  value: "ssh://u641795@u641795.your-storagebox.de:23/./authentik-backup"
+                - name: BORG_PASSPHRASE
+                  valueFrom:
+                    secretKeyRef:
+                      name: authentik-backup-credentials
+                      key: borg-passphrase
+                - name: SSH_PRIVATE_KEY_FILE
+                  value: /secrets/ssh/ssh-private-key
+                - name: SSH_KNOWN_HOSTS_FILE
+                  value: /secrets/known-hosts/known_hosts
+                - name: DB_HOSTS
+                  value: "authentik:authentik-postgresql"
+                - name: PGUSER
+                  value: authentik
+                - name: PGPASSWORD
+                  valueFrom:
+                    secretKeyRef:
+                      name: authentik-credentials
+                      key: pg-password
+              volumeMounts:
+                - name: ssh-key
+                  mountPath: /secrets/ssh
+                  readOnly: true
+                - name: known-hosts
+                  mountPath: /secrets/known-hosts
+                  readOnly: true
+                - name: scratch
+                  mountPath: /scratch
+              resources:
+                requests:
+                  cpu: 50m
+                  memory: 128Mi
+                limits:
+                  memory: 256Mi
+          volumes:
+            - name: ssh-key
+              secret:
+                secretName: authentik-backup-credentials
+                items:
+                  - key: ssh-private-key
+                    path: ssh-private-key
+                    mode: 0400
+            - name: known-hosts
+              configMap:
+                name: authentik-backup-known-hosts
+            - name: scratch
+              emptyDir: {}
@@ -226,3 +226,58 @@ data:
          domain: authentik-default
        attrs:
          default_application: !Find [authentik_core.application, [slug, matrix]]
+  matrix-oidc-provider.yaml: |
+    # yaml-language-server: $schema=https://goauthentik.io/blueprints/schema.json
+    version: 1
+    metadata:
+      name: matrix-oidc-provider
+      labels:
+        blueprints.goauthentik.io/instantiate: "true"
+    entries:
+      # The OIDC Provider + Application linking Authentik to MAS was originally
+      # clicked together by hand in the UI and existed nowhere as code (issue
+      # #36): losing the Authentik DB would have meant re-creating this from
+      # scratch, including a new client_secret that MAS would then no longer
+      # match. client_secret is read from AUTHENTIK_MAS_OIDC_CLIENT_SECRET
+      # (see authentik.yaml HelmRelease values) rather than inlined here,
+      # since this ConfigMap itself is not SOPS-encrypted - the actual value
+      # lives in the authentik-credentials Secret instead.
+      - model: authentik_providers_oauth2.oauth2provider
+        state: present
+        identifiers:
+          name: Matrix Authentication Service
+        id: matrix_mas_provider
+        attrs:
+          client_type: confidential
+          client_id: dHbTAgAgXvjh3VALh220mB3dxcVXAifiXU2ZO3U6
+          client_secret: !Env AUTHENTIK_MAS_OIDC_CLIENT_SECRET
+          # Path includes MAS's own upstream-provider ID, not Authentik's -
+          # must match MAS's config exactly or the OIDC callback breaks.
+          redirect_uris:
+            - matching_mode: strict
+              url: https://account.axion1337.chat/upstream/callback/01KQDJTR1ZVTG8JQ220F5BNBFZ
+          # Stable across username renames - this is what keeps
+          # upstream_oauth_links rows valid after e.g. the elbojoloco rename.
+          sub_mode: hashed_user_id
+          include_claims_in_id_token: true
+          access_code_validity: minutes=1
+          access_token_validity: minutes=5
+          signing_key: !Find [authentik_crypto.certificatekeypair, [name, "authentik Self-signed Certificate"]]
+          authorization_flow: !Find [authentik_flows.flow, [slug, default-provider-authorization-implicit-consent]]
+          invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
+          property_mappings:
+            - !Find [authentik_core.propertymapping, [managed, "goauthentik.io/providers/oauth2/scope-openid"]]
+            - !Find [authentik_core.propertymapping, [managed, "goauthentik.io/providers/oauth2/scope-email"]]
+            - !Find [authentik_core.propertymapping, [managed, "goauthentik.io/providers/oauth2/scope-profile"]]
+
+      - model: authentik_core.application
+        state: present
+        identifiers:
+          slug: matrix
+        attrs:
+          name: aXion1337.chat Accountverwaltung
+          provider: !KeyOf matrix_mas_provider
+          meta_description: Matrixclient tailored for aXionCommunity
+          meta_publisher: aXionGaming
+          policy_engine_mode: any
+          open_in_new_tab: false
@@ -7,10 +7,10 @@ stringData:
    secret_key: ENC[AES256_GCM,data:yIyQapbFtFM11LynFtkV3ffExhaDfN9QHeFbI1T0xkIhgsV+9sjg3qwMVmeBlAe7xZl8gsAM4kDj2Q6O91OdDg==,iv:+Cl8vOcxG9/mgRheaCO0bLWyCJXN+f1F2DD3oeHbPFY=,tag:711ytyKf6/tmXomBLoffGA==,type:str]
    pg-password: ENC[AES256_GCM,data:3w8R9mRjMXMJDLjrC8QYaXFHsCU3yYZs2PcaFQNp3Z4=,iv:G/aXgoGz3vBOzZ5K3Y+DDJsqer4F5gvcMmtkzRx93CU=,tag:dXPs1pY/APvnMlxdvB1EkA==,type:str]
    smtp-password: ENC[AES256_GCM,data:JpMgaQFPkBzOg5WjvpmhM0kPwvZkH+4tQjT17RJHjG14WjmWtfG9Bg==,iv:zjQRLIlrxKv5hbd4JZowNUEiibiCUMf79acZY0+dYAc=,tag:ORPafTPyOQJvVvHWQGmqhA==,type:str]
+    mas-oidc-client-secret: ENC[AES256_GCM,data:0yx55FroLSxlnuYgfNwczu3PnbPm1kW74JtiU9oFevVqeQDZc385wU6x5X5TN7owXDO7QaOfGTTMvqIpbwQb6Q5Vt1VMToR+0f44oJcktYoTiDFU9Sy6lR/y6nlvBCNqeJg7vIyVpkIqxwqty15EekyqMpkIMp1fT6Pxmek0SO0=,iv:Ey06ljnqbVARDLVt2sLe8R776VEWpTlzI/+Nka5NocA=,tag:I+GNLHz4V8TFa2ijzK5y2Q==,type:str]
 sops:
    age:
-        - recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
-          enc: |
+        - enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBRekJuZythYzliTFJ3RlhS
            R2p6TG9NeFdabFlPRWtpNHJMYVVxTWZEcmlRClk0WUorSzdxNlcyWHYwWFBTMnlq
@@ -18,7 +18,8 @@ sops:
            QXVrY1NTeHZkeTlPRWNlVThzWno3T0kKC0KBoLT64GNqb8Ri9u69G7nqb1KftwwP
            /24aVHrPxKi9d4ij9n3bvCYDF4rhtfexhrE4n7CfuKn2DcSiuTniuw==
            -----END AGE ENCRYPTED FILE-----
-    lastmodified: "2026-04-29T21:43:59Z"
-    mac: ENC[AES256_GCM,data:Y+dJppkaVZ5NOhlvwbbsF5+vDFqGUI1Ps8IcE4J7FIW4HIdMVf6RKM0EInvPUW1LaBlmelCitcE30w0As7ysNRhLY8yUDaKUvuU6mRejlNUIF8wAHzhciL2jTvAQsArHjybJatEig28+wM9VcY8JEa/d/CmuiB9Nq4WbIV+JXlA=,iv:UQj2rIVLNPjtYp3d/jRyNfJyyyUsZ3+NDCgpI4aztzc=,tag:cwiCzG/A+rfRFfLjXVt82w==,type:str]
+          recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
    encrypted_regex: ^(data|stringData)$
+    lastmodified: "2026-07-28T15:54:53Z"
+    mac: ENC[AES256_GCM,data:P6IF+jukwzldK92nHl6s4h6sS4ldXLwpyLpwv2tpI3vFWgTLEnGCnowi2k5lmWUlITEVmLLC0HvsBuduTiGI2sIHHt+r3RdqkV88HGn6oYDVq5a+Ax7ESfqti/4B7ClQCSxl/tU6hBUFe812DiBXJgA03UJQZn8uHY/dP/RgRpc=,iv:V8sqhbJcKglkKsQmJBdgoxDaCYJ3Wt/qRa18jEviH60=,tag:EiNotrYAKIzKndgjU/kTFQ==,type:str]
    version: 3.12.2
@@ -40,6 +40,15 @@ spec:
    global:
      security:
        allowInsecureImages: true
+      # Read by the matrix-oidc-provider blueprint via !Env, so the OAuth2
+      # Provider's client_secret can be captured as code without ever
+      # inlining the live credential into a plain (non-SOPS) ConfigMap.
+      env:
+        - name: AUTHENTIK_MAS_OIDC_CLIENT_SECRET
+          valueFrom:
+            secretKeyRef:
+              name: authentik-credentials
+              key: mas-oidc-client-secret

    authentik:
      log_level: info
@@ -89,3 +98,8 @@ spec:
            memory: 256Mi
          limits:
            memory: 512Mi
+        # Chart's own generated policy allows ANY pod in ANY namespace on 5432
+        # (see issue #37) - disabled in favor of our own scoped policy in
+        # apps/authentik/networkpolicy.yaml.
+        networkPolicy:
+          enabled: false
@@ -9,3 +9,6 @@ resources:
  - authentik.yaml
  - ingress.yaml
  - networkpolicy.yaml
+  # Backup zur Hetzner Storage Box (Issues #6 + #15)
+  - authentik-backup-secret.yaml
+  - authentik-backup.yaml
@@ -4,9 +4,10 @@
 # for upstream OIDC calls. Egress is intentionally untouched (federation-equivalent
 # outbound calls like SMTP aren't restricted here).
 #
-# Note: authentik-postgresql already has its own NetworkPolicy from the Bitnami
-# postgresql subchart (port 5432, no source restriction) - left alone, not duplicated,
-# since it would get reset on the next Helm upgrade anyway.
+# authentik-postgresql: the Bitnami postgresql subchart's own generated NetworkPolicy
+# restricted the port (5432) but not the source - any pod in any namespace could reach
+# it (issue #37). Disabled via postgresql.primary.networkPolicy.enabled: false in
+# authentik.yaml and replaced below with a policy scoped to authentik-server/-worker.
 apiVersion: networking.k8s.io/v1
 kind: NetworkPolicy
 metadata:
@@ -47,6 +48,35 @@ spec:
 ---
 apiVersion: networking.k8s.io/v1
 kind: NetworkPolicy
+metadata:
+  name: allow-ingress-authentik-postgresql
+  namespace: authentik
+spec:
+  podSelector:
+    matchLabels:
+      app.kubernetes.io/name: postgresql
+      app.kubernetes.io/component: primary
+  policyTypes:
+    - Ingress
+  ingress:
+    - from:
+        - podSelector:
+            matchLabels:
+              app.kubernetes.io/name: authentik
+              app.kubernetes.io/component: server
+        - podSelector:
+            matchLabels:
+              app.kubernetes.io/name: authentik
+              app.kubernetes.io/component: worker
+        - podSelector:
+            matchLabels:
+              app.kubernetes.io/name: authentik-backup
+      ports:
+        - protocol: TCP
+          port: 5432
+---
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
 metadata:
  name: allow-ingress-acme-solver
  namespace: authentik
@@ -30,4 +30,7 @@ resources:
  - element-server-suite.yaml
  # Custom Apex Ingress für Element Web + Well-Known auf axion1337.chat
  - apex-ingress.yaml  # Custom Apex Ingress für Element Web + Well-Known auf axion1337.chat
-  - networkpolicy.yaml
+  - networkpolicy.yaml
+  # Backup zur Hetzner Storage Box (Issues #6 + #15)
+  - synapse-backup-secret.yaml
+  - synapse-backup.yaml
@@ -258,6 +258,9 @@ spec:
        - podSelector:
            matchLabels:
              app.kubernetes.io/name: matrix-authentication-service
+        - podSelector:
+            matchLabels:
+              app.kubernetes.io/name: synapse-backup
      ports:
        - protocol: TCP
          port: 5432
@@ -0,0 +1,23 @@
+apiVersion: v1
+kind: Secret
+metadata:
+    name: synapse-backup-credentials
+    namespace: matrix
+stringData:
+    borg-passphrase: ENC[AES256_GCM,data:RRXPwr4UGX30IdozM53abN7ZYztTO3Y1/63dtTh0JSZmU8i8l/ATb4gc3lc=,iv:rIyUr+lOUjo9J53OKZ5ZDmp3d8Nrb9PP2JDK2oCutYU=,tag:MqUmoYgCA03WJQy+RQi04w==,type:str]
+    ssh-private-key: ENC[AES256_GCM,data:320B/lSq7DljCrXZ0BluGv8gLIzYF3KL6VQnR6CM/Vzuf/6qbbZ7lMzrS96XfoubU95OxDeJuOsuNZBQUZNyUGtt4QjQmnl8XHUJzl48xLqh83HLFPtQm2uqU072lscf5OTT7I+JzD2BGks3OIowhrg1q0MdVQdfd4Rhdz8Jphjb+WxvstzNmEX5gxQ9mnBVPmj4DS6ikdXnpe+VDvaCJaVkzD5KgwRmqPy0qbFs9WXziQSo3am5fPeHbwhV3UlRhlok9WrDI40a1T1S5DBhgNhwShq1jAxjr9onuq92REymxAV50oLzsw9ivnH0uimw+3PcplRG1v2xxJ/pimWTCjE17bO7OZ8TyzzwyZ7QA02vSpOMNUdUwVU3N6pSdYQpdTETuVBTVqVc+GKC18Z9fWh2rdwX5eUDTwp0bbDDvPEHdANNxMg3VYD3gwpCgy6/wVnjrO+pVMs8K1CqFn4/H7azhzzeEPkrz27ZBxjlnqvDODsm9tlklr6X4jZDZmTaEmiH+WXuc+1qbNxvsrqtSFfa3CfFGxM1nBOh,iv:sKGsTLsxdQYVUvw7CEARL3YNInSd9LPbFp5Ci5CTgIw=,tag:O1mjNZUsGEqV4uu+LlgD/g==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB3ZUdtN3hyNHlyMkIrbTVS
+            aHQ4OGNUWnA4eTFUVkx4UVp5VWlnMDgxTEU0CkU3M2dMYWgyKytlRVFOVWptZEd2
+            NXlIY0JCd24xcGFzaGpIeks0R2U0U3MKLS0tIHlxYVZ1ZTJsRXNaZ25sVzZtSnp0
+            SEhzQ0tUYzZTRXcwMVNwbG85SHpyb0UKOn3nxy6Y7yQkGargXQ9z6O36vUWW4qJZ
+            D/GbFGmoRi94EtVFdmTGALhjy2D4J9QXy6gHsTapvKyMxF8NEtk+FQ==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
+    encrypted_regex: ^(data|stringData)$
+    lastmodified: "2026-07-28T18:20:58Z"
+    mac: ENC[AES256_GCM,data:Rur32fQdCyM3nr/X+KeSgmPYEi4nKyh8lqTuSW3TPBVDjwTWMDp1I1ZPPyy5syeW6RHbKputFUzBWVnuQmVrfbZaQ6DBBI5kP9InspUAVUjXDRk9XqiWtdg/wYaTMMJ4Nxv/zdwkh6uJQSG2JHQBWce4NZc2hoPokLR0CjcWiZk=,iv:eWTUj48EFjjtuIIuErMltEdDfabLZeolkpInMYtVP5Q=,tag:6DPxkPqLt0ihJ80WnTzHeA==,type:str]
+    version: 3.13.3
@@ -0,0 +1,96 @@
+# Nightly Borg backup of the shared Postgres instance (synapse + MAS databases) and the
+# Synapse media_store PVC to a Hetzner Storage Box (issues #6 + #15). See
+# apps/production/synapse-backup-secret.yaml for the SSH key + Borg repo passphrase, and
+# apps/authentik/authentik-backup.yaml for the equivalent authentik-side job.
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: synapse-backup-known-hosts
+  namespace: matrix
+data:
+  # Pinned via `ssh-keyscan -p 23 u641795.your-storagebox.de` (2026-07-28) rather than
+  # trusting the host key on first connect in an unattended job.
+  known_hosts: |
+    [u641795.your-storagebox.de]:23 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIICf9svRenC/PLKIL9nk6K/pxQgoiFC41wTNvoIncOxs
+---
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+  name: synapse-backup
+  namespace: matrix
+spec:
+  schedule: "0 3 * * *"
+  concurrencyPolicy: Forbid
+  successfulJobsHistoryLimit: 3
+  failedJobsHistoryLimit: 3
+  jobTemplate:
+    spec:
+      backoffLimit: 2
+      template:
+        metadata:
+          labels:
+            app.kubernetes.io/name: synapse-backup
+            app.kubernetes.io/component: backup
+        spec:
+          restartPolicy: OnFailure
+          containers:
+            - name: backup
+              image: rohana.axion1337.de/sorb/axion-backup:v2
+              env:
+                - name: BORG_REPO
+                  value: "ssh://u641795@u641795.your-storagebox.de:23/./synapse-backup"
+                - name: BORG_PASSPHRASE
+                  valueFrom:
+                    secretKeyRef:
+                      name: synapse-backup-credentials
+                      key: borg-passphrase
+                - name: SSH_PRIVATE_KEY_FILE
+                  value: /secrets/ssh/ssh-private-key
+                - name: SSH_KNOWN_HOSTS_FILE
+                  value: /secrets/known-hosts/known_hosts
+                - name: DB_HOSTS
+                  value: "synapse:matrix-stack-postgres matrixauthenticationservice:matrix-stack-postgres"
+                - name: PGUSER
+                  value: postgres
+                - name: PGPASSWORD
+                  valueFrom:
+                    secretKeyRef:
+                      name: matrix-stack-generated
+                      key: POSTGRES_ADMIN_PASSWORD
+                - name: MEDIA_PATH
+                  value: /media/media_store
+              volumeMounts:
+                - name: ssh-key
+                  mountPath: /secrets/ssh
+                  readOnly: true
+                - name: known-hosts
+                  mountPath: /secrets/known-hosts
+                  readOnly: true
+                - name: media
+                  mountPath: /media
+                  readOnly: true
+                - name: scratch
+                  mountPath: /scratch
+              resources:
+                requests:
+                  cpu: 50m
+                  memory: 128Mi
+                limits:
+                  memory: 256Mi
+          volumes:
+            - name: ssh-key
+              secret:
+                secretName: synapse-backup-credentials
+                items:
+                  - key: ssh-private-key
+                    path: ssh-private-key
+                    mode: 0400
+            - name: known-hosts
+              configMap:
+                name: synapse-backup-known-hosts
+            - name: media
+              persistentVolumeClaim:
+                claimName: matrix-stack-synapse-media
+                readOnly: true
+            - name: scratch
+              emptyDir: {}
@@ -103,4 +103,31 @@ ist reines Zurückkopieren, kein Neu-Erzeugen).
 `sops-age`-Secret selbst (auf demselben Server) und lokal bei wem auch immer ihn zuletzt
 zurückgeholt hat. Es gibt kein separates, offsite Backup. Fällt der Server komplett aus
 (nicht nur der lokale Rechner), sind alle SOPS-verschlüsselten Secrets im Repo unlesbar.
-Siehe Issue-Backlog für die Entscheidung, ob/wie das abgesichert wird.
+Siehe Issue-Backlog für die Entscheidung, ob/wie das abgesichert wird.
+
+---
+
+## 🌐 Node-Konfiguration: `/etc/hosts`-Eintrag für den Gitea-Host
+
+Der K3s-Node und der Gitea-Host (`rohana.axion1337.de`, Container-Registry + Git-Remote)
+teilen sich ein privates Hetzner-Netzwerk (Node `10.0.0.2`, Gitea-Host `10.0.0.3`). Seit
+2026-07-28 hat der Node dafür einen manuellen `/etc/hosts`-Eintrag:
+
+```
+10.0.0.3 rohana.axion1337.de
+```
+
+**Warum**: eine Firewall-Fehlkonfiguration hatte den Node zeitweise komplett von
+`rohana.axion1337.de` über die öffentliche IP abgeschnitten, was Image-Pulls (z.B. für
+Custom-Images wie `sorb/axion-backup`) mit Timeout scheitern ließ. Der Eintrag macht
+Image-Pulls unabhängig vom Zustand der öffentlichen Firewall.
+
+**Wichtig**: Das ist unmanaged Node-Konfiguration (kein GitOps, kein Kubernetes-Objekt) —
+überlebt einen Node-Neuaufbau **nicht** und muss dann erneut gesetzt werden:
+
+```bash
+echo "10.0.0.3 rohana.axion1337.de" | sudo tee -a /etc/hosts
+```
+
+Ein sauberer, clusterweiter Ersatz (z.B. CoreDNS-Rewrite, damit auch Pods selbst intern
+auflösen) ist als Issue #41 nachgehalten.
Author	SHA1	Message	Date
Thore Cimbal	9607687e4e	docs: document node-level /etc/hosts entry for private-network registry access	2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5	49005ed325	fix(backup): retry pg_dump connection to survive NetworkPolicy sync lag Live testing showed pg_dump got "Connection refused" immediately at job start, but a manual pg_isready/psql from an identically-labeled pod a few seconds later succeeded fine every time. The cluster's NetworkPolicy controller needs a brief moment to program a brand-new pod's IP into the target's allowed ingress rules; a job that starts dumping instantly can race that window. backup.sh now waits for pg_isready before each pg_dump (up to 15 tries / ~30s) instead of assuming connectivity is ready at container start. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5	ba270bce71	feat(backup): add nightly Borg backups to Hetzner Storage Box Closes issues #6 and #15 (treated jointly, per combined analysis in their comments). Two CronJobs, one per namespace, each pushing to its own Borg repo on the newly booked Storage Box (u641795.your-storagebox.de:23, BX11 1TB) with its own repo passphrase - a leaked passphrase for one doesn't expose the other: - matrix: synapse-backup dumps the synapse + matrixauthenticationservice DBs (shared postgres, existing chart-generated POSTGRES_ADMIN_PASSWORD) and the Synapse media_store PVC (read-only mount) - authentik: authentik-backup dumps the authentik DB (existing authentik-credentials pg-password) Custom image (rohana.axion1337.de/sorb/axion-backup:v1, postgres:17-alpine + borgbackup + openssh-client - matches the live Postgres major version exactly, unlike Alpine's stock postgresql16-client) pushed as a public package, same pattern as the existing threadnet-web image (no imagePullSecret needed). SSH host key pinned via known_hosts ConfigMap (captured via ssh-keyscan ahead of time) rather than trusting on first connect in an unattended job. Retention: 7 daily / 4 weekly / 6 monthly via borg prune. Schedule: 03:00 and 03:15, offset to avoid resource contention. NetworkPolicy: added each backup job's pod as an allowed source to its namespace's existing postgres ingress rule (matrix's allow-ingress-postgres, authentik's allow-ingress-authentik-postgresql from #37). Egress already unrestricted in both namespaces, so no change needed for the outbound SSH connection to the Storage Box. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5	2c5f77686d	fix(authentik): scope authentik-postgresql NetworkPolicy to internal access Closes issue #37. The Bitnami postgresql subchart's auto-generated NetworkPolicy restricted the port (5432) but not the source - any pod in any namespace could reach it, since additive NetworkPolicy rules can't restrict an existing permissive one, only the chart itself controls it (hence not touched during the original #10 rollout). Disabled via postgresql.primary.networkPolicy.enabled: false and replaced with a scoped policy allowing only authentik-server and authentik-worker (same namespace) on 5432, matching the pattern already used for the matrix namespace's postgres access. No metrics exporter is enabled for this instance, so no monitoring-namespace rule is needed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5	414ada956a	feat(authentik): capture MAS OIDC Provider/Application as a Blueprint Closes issue #36. The OAuth2 Provider ("Matrix Authentication Service") and its Application ("matrix" slug) linking Authentik to MAS were originally clicked together by hand in the UI and existed nowhere as code - unlike the flow fixes already captured in authentik-blueprints.yaml. Losing the Authentik DB would have meant re-creating this from scratch, including a fresh client_secret that MAS would no longer match. The client_secret is read via !Env from AUTHENTIK_MAS_OIDC_CLIENT_SECRET, sourced from a new key in the existing SOPS-encrypted authentik-credentials Secret, rather than inlined into the blueprint ConfigMap (which isn't itself encrypted). Value used is the actual live secret already in use, read directly from the running Authentik DB - not a new/rotated one, so this changes nothing about the current MAS<->Authentik pairing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-28 12:00:00 +00:00