Compare commits

...
18 Commits
Author SHA1 Message Date
Thore CimbalandClaude Sonnet 5 2932ffffc3 fix(matrix-stack): shorten HelmRelease reconcile interval to 1m
Live-tested Issue #38's rotation: after merging the automated rotation
PR, coturn restarted quickly (Kustomization-level, 1m interval), but
synapse-main lagged behind since it depends on this separate HelmRelease
with its own 5m interval - a real (self-healing, but avoidable) window
where coturn had the new TURN secret and Synapse still had the old one,
which would reject each other's credentials/relayed media. Matching the
interval to production-apps's 1m tightens that window without needing
any new automation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 12:00:00 +00:00
Thore Cimbal 0f7ff1a896 Merge pull request 'chore(coturn): automated TURN shared-secret rotation' (#42) from turn-secret-rotation-20260728-192656 into main 2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5 debf980d5f fix(turn-rotation): add hostAliases for private-network Gitea access
Job hung indefinitely on git clone - public-internet reachability to
rohana.axion1337.de from inside pods is currently down (related to
Issue #41's firewall/private-network findings), and the node-level
/etc/hosts workaround only helps containerd's image pulls, not processes
running inside pods (they resolve via CoreDNS + their own /etc/hosts, not
the node's). hostAliases routes this job's git/API traffic over the
private Hetzner network directly - verified working with a real pod.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 12:00:00 +00:00
turn-secret-rotation 802e09cd9c chore(coturn): automated TURN shared-secret rotation
Scheduled rotation (Issue #38). New secret generated, re-encrypted with
the scoped rotation age key, checksum/rotated-at annotations bumped so
Flux restarts coturn + synapse-main on merge. Please review and merge.
2026-07-28 12:00:00 +00:00
Thore Cimbal d742c99ba5 chore: set Gitea PAT for TURN secret rotation job 2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5 ac7e65100d feat(coturn): add automated TURN shared-secret rotation CronJob
Closes issue #38's automation half (architecture fix + first rotation
already landed in earlier commits this session). Monthly CronJob
(rohana.axion1337.de/sorb/axion-secret-rotation:v1 - alpine + git/sops/
jq/age) that:

- generates a new secret
- re-encrypts coturn-secret.yaml and synapse-turn-secret.yaml using the
  scoped rotation-only age key (added as an additional recipient in an
  earlier commit) - never touches the repo's master sops-age key
- bumps the turn-secret-checksum (HelmRelease annotation) and rotated-at
  (coturn Deployment annotation) so merging actually restarts both
  consumers, reusing the existing checksum-annotation pattern already in
  this repo rather than inventing a new mechanism
- opens a Pull Request rather than pushing straight to main - a human
  reviews and merges, keeping a checkpoint before production picks up new
  credentials while still automating the tedious coordination work

Needs a Gitea PAT (repo write scope) filled into
turn-secret-rotation-secret.yaml's gitea-token key before first use.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5 be95bd921c feat(coturn): add scoped age recipient for automated rotation job
Adds a second age recipient (dedicated keypair, private key never touches
the repo) to just these two files, so the upcoming rotation CronJob can
decrypt/re-encrypt them without needing the repo's master sops-age key,
which can decrypt every secret in the repo. Verified: master key still
decrypts both files; the new scoped key can decrypt both files but
correctly cannot decrypt an unrelated secret (authentik-secret.yaml).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 12:00:00 +00:00
Thore Cimbal 7795f89fc9 docs(coturn): note the one-time manual rollingUpdate fix instead of an ineffective YAML null 2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5 9fa947cc21 fix(coturn): explicitly null rollingUpdate field alongside Recreate strategy
Server-side apply left the old RollingUpdate defaults (maxSurge/maxUnavailable)
on the live object from before strategy.type was ever set explicitly, which
the API rejects when combined with type: Recreate. Explicit null clears it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5 233b13f972 fix(coturn): use Recreate deployment strategy for hostNetwork pod
Discovered live: the default RollingUpdate strategy tries to schedule the
new pod before removing the old one, which can never succeed for a
hostNetwork Deployment on a single-node cluster (both pods would need the
same node ports 3478/5349 simultaneously). Surfaced by the rotated-at
annotation bump from the previous commit getting stuck in
FailedScheduling. Recreate kills the old pod first, freeing the ports.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5 51ea51352b fix(coturn): remove plaintext TURN shared secret, rotate leaked value
Issue #38 discussion surfaced a real bug: the TURN shared secret was
duplicated - correctly SOPS-encrypted in coturn-secret.yaml, but also
hardcoded in plaintext in synapse-values.yaml (a plain, non-SOPS
ConfigMap), visible in git history. Also found turn_user_lifetime is
86400000ms (24h), not "short-lived" as previously assumed - raising the
stakes of the leak somewhat.

Extracted the turn config block into its own dedicated SOPS-encrypted
Secret (synapse-turn-secret.yaml), wired via a second HelmRelease
valuesFrom entry (same pattern already used for ess-mas-values-secret).
Rotated the value while doing this, so the leaked plaintext secret is no
longer live anywhere.

Added checksum/rotated-at annotations (matrix-stack HelmRelease's
existing element-config-checksum patch gets a sibling turn-secret-checksum;
coturn's Deployment pod template gets a rotated-at annotation) so future
rotations actually restart both consumers - Kubernetes doesn't restart
running pods when a referenced Secret's content changes on its own.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 12:00:00 +00:00
Thore Cimbal 9607687e4e docs: document node-level /etc/hosts entry for private-network registry access 2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5 49005ed325 fix(backup): retry pg_dump connection to survive NetworkPolicy sync lag
Live testing showed pg_dump got "Connection refused" immediately at job
start, but a manual pg_isready/psql from an identically-labeled pod a few
seconds later succeeded fine every time. The cluster's NetworkPolicy
controller needs a brief moment to program a brand-new pod's IP into the
target's allowed ingress rules; a job that starts dumping instantly can
race that window. backup.sh now waits for pg_isready before each pg_dump
(up to 15 tries / ~30s) instead of assuming connectivity is ready at
container start.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5 ba270bce71 feat(backup): add nightly Borg backups to Hetzner Storage Box
Closes issues #6 and #15 (treated jointly, per combined analysis in their
comments). Two CronJobs, one per namespace, each pushing to its own Borg
repo on the newly booked Storage Box (u641795.your-storagebox.de:23, BX11
1TB) with its own repo passphrase - a leaked passphrase for one doesn't
expose the other:

- matrix: synapse-backup dumps the synapse + matrixauthenticationservice
  DBs (shared postgres, existing chart-generated POSTGRES_ADMIN_PASSWORD)
  and the Synapse media_store PVC (read-only mount)
- authentik: authentik-backup dumps the authentik DB (existing
  authentik-credentials pg-password)

Custom image (rohana.axion1337.de/sorb/axion-backup:v1, postgres:17-alpine
+ borgbackup + openssh-client - matches the live Postgres major version
exactly, unlike Alpine's stock postgresql16-client) pushed as a public
package, same pattern as the existing threadnet-web image (no
imagePullSecret needed).

SSH host key pinned via known_hosts ConfigMap (captured via ssh-keyscan
ahead of time) rather than trusting on first connect in an unattended job.
Retention: 7 daily / 4 weekly / 6 monthly via borg prune. Schedule: 03:00
and 03:15, offset to avoid resource contention.

NetworkPolicy: added each backup job's pod as an allowed source to its
namespace's existing postgres ingress rule (matrix's allow-ingress-postgres,
authentik's allow-ingress-authentik-postgresql from #37). Egress already
unrestricted in both namespaces, so no change needed for the outbound SSH
connection to the Storage Box.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5 2c5f77686d fix(authentik): scope authentik-postgresql NetworkPolicy to internal access
Closes issue #37. The Bitnami postgresql subchart's auto-generated
NetworkPolicy restricted the port (5432) but not the source - any pod in
any namespace could reach it, since additive NetworkPolicy rules can't
restrict an existing permissive one, only the chart itself controls it
(hence not touched during the original #10 rollout).

Disabled via postgresql.primary.networkPolicy.enabled: false and replaced
with a scoped policy allowing only authentik-server and authentik-worker
(same namespace) on 5432, matching the pattern already used for the
matrix namespace's postgres access. No metrics exporter is enabled for
this instance, so no monitoring-namespace rule is needed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5 414ada956a feat(authentik): capture MAS OIDC Provider/Application as a Blueprint
Closes issue #36. The OAuth2 Provider ("Matrix Authentication Service")
and its Application ("matrix" slug) linking Authentik to MAS were
originally clicked together by hand in the UI and existed nowhere as
code - unlike the flow fixes already captured in authentik-blueprints.yaml.
Losing the Authentik DB would have meant re-creating this from scratch,
including a fresh client_secret that MAS would no longer match.

The client_secret is read via !Env from AUTHENTIK_MAS_OIDC_CLIENT_SECRET,
sourced from a new key in the existing SOPS-encrypted authentik-credentials
Secret, rather than inlined into the blueprint ConfigMap (which isn't
itself encrypted). Value used is the actual live secret already in use,
read directly from the running Authentik DB - not a new/rotated one, so
this changes nothing about the current MAS<->Authentik pairing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5 4a3d7c70a6 fix: restore correct file permissions, stop tracking .DS_Store
Almost every tracked file in the repo had drifted to mode 777 on disk
(only files created fresh this session were unaffected), and a chunk of
that drift had already been committed as spurious +x bits on plain
YAML/Markdown files (authentik.yaml, kustomization.yaml, coturn.yaml,
element-server-suite.yaml, TASKS.md, install.md, etc.) - none of these
need to be executable. Restored to 644 for regular files, 755 only for
actual scripts (postCreateCommand.sh, docker-init.sh, install-hooks.sh,
pre-commit hook, element-setup-linux.sh).

Also found element-setup-macos.command was missing +x despite having a
shebang and being meant for double-click execution on macOS - fixed.

Added .gitignore for .DS_Store and .claude/ and stopped tracking the five
.DS_Store files that had been committed by accident.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 12:00:00 +00:00
Thore CimbalandClaude Sonnet 5 6973a8dbc7 fix(devcontainer): repair broken build and add docker socket permission fix
The devcontainer could never actually be built successfully - verified by
building it from scratch for the first time in a while. Found and fixed six
issues:

- kubectl: apt.kubernetes.io was deprecated/shut down by Google in 2023,
  switched to the official successor repo pkgs.k8s.io
- docker-ce-cli: apt line hardcoded arch=amd64, breaking the build on
  Apple Silicon; resolved dynamically via dpkg --print-architecture
- useradd -G docker failed because only the Docker CLI (no daemon) is
  installed, so no package ever creates the docker group; added explicit
  groupadd
- oh-my-zsh install had a nested-quoting bug that made the RUN step fail;
  simplified to download-then-run instead of one nested `su -c "sh -c ..."`
- sops binary was hardcoded to linux.amd64, only working on arm64 by luck
  via Docker Desktop's QEMU emulation; resolved dynamically like docker-ce
- docker.sock was mounted but unusable (permission denied) since the
  container's docker group GID never matched the host socket's GID; added
  a root entrypoint (docker-init.sh) that reconciles this at container
  start, then drops to the vscode user via gosu

Also fixed two stale mas-secrets.sops.yaml references (actual filename is
mas-secret.yaml) in README.md and postCreateCommand.sh, set the vscode
user's default shell to zsh (oh-my-zsh was installed but never used by
default), and documented all of the above plus a build+run verification
snippet in README.md so this class of drift is caught before it goes
unnoticed again.

Verified end-to-end: cold `docker build --no-cache`, then a real container
run against the actual mounted kubeconfig, age key, and docker socket -
kubectl reaches the live cluster, sops decrypts a real secret, and docker
ps talks to the real daemon as the vscode user.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 12:00:00 +00:00
35 changed files with 644 additions and 45 deletions
Vendored
BIN
View File
Binary file not shown.
+28 -10
View File
@@ -16,11 +16,14 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
zsh \
sudo \
openssh-client \
gosu \
&& rm -rf /var/lib/apt/lists/*
# Install kubectl
RUN curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && \
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | tee /etc/apt/sources.list.d/kubernetes.list && \
# Install kubectl (apt.kubernetes.io was deprecated/shut down by Google in 2023;
# pkgs.k8s.io is the current community-owned repo, versioned per k8s minor release)
RUN mkdir -p /etc/apt/keyrings && \
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.34/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg && \
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.34/deb/ /" | tee /etc/apt/sources.list.d/kubernetes.list && \
apt-get update && apt-get install -y kubectl && \
rm -rf /var/lib/apt/lists/*
@@ -30,9 +33,10 @@ RUN curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | b
# Install Flux CLI
RUN curl -s https://fluxcd.io/install.sh | bash
# Install sops
RUN SOPS_VERSION=$(curl -s https://api.github.com/repos/getsops/sops/releases/latest | grep tag_name | cut -d '"' -f 4) && \
curl -sL -o /usr/local/bin/sops https://github.com/getsops/sops/releases/download/${SOPS_VERSION}/sops-${SOPS_VERSION}.linux.amd64 && \
# Install sops (arch resolved at build time, same reasoning as the Docker CLI step below)
RUN SOPS_ARCH=$(dpkg --print-architecture) && \
SOPS_VERSION=$(curl -s https://api.github.com/repos/getsops/sops/releases/latest | grep tag_name | cut -d '"' -f 4) && \
curl -sL -o /usr/local/bin/sops https://github.com/getsops/sops/releases/download/${SOPS_VERSION}/sops-${SOPS_VERSION}.linux.${SOPS_ARCH} && \
chmod +x /usr/local/bin/sops
# Install age
@@ -40,17 +44,31 @@ RUN apt-get update && apt-get install -y age && \
rm -rf /var/lib/apt/lists/*
# Install Docker CLI (for interacting with Docker daemon)
# arch is resolved at build time so this works on both amd64 (cloud/CI) and arm64 (Apple Silicon) hosts
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg && \
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null && \
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null && \
apt-get update && apt-get install -y docker-ce-cli && \
rm -rf /var/lib/apt/lists/*
# Create a non-root user 'vscode' for development
RUN useradd -m -s /bin/bash -G docker vscode && \
# groupadd is needed because only the Docker CLI (not the daemon) is installed above,
# so the 'docker' group is never created as a package side effect
RUN groupadd docker && \
useradd -m -s /bin/zsh -G docker vscode && \
echo "vscode ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers.d/vscode
# Install oh-my-zsh for better shell experience
RUN su - vscode -c "sh -c '$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)' '' --unattended"
RUN curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh -o /tmp/install-omz.sh && \
su - vscode -c "sh /tmp/install-omz.sh --unattended" && \
rm /tmp/install-omz.sh
# Entrypoint runs as root to reconcile the docker group's GID against the mounted
# socket (see docker-init.sh), then drops to 'vscode' for the actual session/command.
# Stays root-owned at the PID 1 level; VS Code's own `docker exec -u vscode` sessions
# and the entrypoint's `gosu vscode` both end up correctly grouped either way.
COPY docker-init.sh /usr/local/bin/docker-init.sh
RUN chmod +x /usr/local/bin/docker-init.sh
USER vscode
WORKDIR /workspace
ENTRYPOINT ["/usr/local/bin/docker-init.sh"]
CMD ["/bin/zsh"]
+40 -2
View File
@@ -91,7 +91,7 @@ Der Container mounted `~/.age` automatisch. Setze die Umgebungsvariable:
```bash
# Im Container-Terminal (SOPS_AGE_KEY_FILE ist bereits automatisch gesetzt!)
# Jetzt kannst du Secrets bearbeiten (wird transparent ver-/entschlüsselt):
sops apps/production/custom-configs/mas-secrets.sops.yaml
sops apps/production/custom-configs/mas-secret.yaml
```
### Schritt 3: VSCode Integration (optional)
@@ -130,7 +130,7 @@ kubectl get pods -n matrix
flux get helmreleases -A
# Secrets bearbeiten (mit verschlüsselung)
sops apps/production/custom-configs/mas-secrets.sops.yaml
sops apps/production/custom-configs/mas-secret.yaml
# FluxCD Sync erzwingen
flux reconcile kustomization production-apps --with-source
@@ -193,6 +193,44 @@ Siehe `README.md` → **Issue 3**. Kurz:
- `wellKnownDelegation: enabled: false` setzen
- Oder `.well-known/matrix/server` manuell auf `elementWeb` weiterleiten
## ⚠️ Wartungshinweis: Warum dieser Container regelmäßig getestet werden muss
Der Dockerfile installiert mehrere Tools über externe apt-Repos und Install-Skripte
(`pkgs.k8s.io`, `download.docker.com`, GitHub-Releases, `fluxcd.io`/`ohmyzsh.sh`
Installer). **Diese Quellen sind nicht unter unserer Kontrolle und können jederzeit
brechen** — genau das ist am 2026-07-28 passiert: der Container konnte seit
Fertigstellung nie erfolgreich gebaut werden, ohne dass es jemand bemerkt hat, weil
niemand ihn zwischenzeitlich tatsächlich gebaut hat. Gefundene und behobene Probleme:
| # | Problem | Ursache | Fix |
|---|---------|---------|-----|
| 1 | `apt.kubernetes.io` → `404 Not Found` | Google hat das alte Kubernetes-apt-Repo 2023 abgeschaltet | Umgestellt auf das offizielle Nachfolge-Repo `pkgs.k8s.io` (versioniert pro k8s-Minor-Version, aktuell `v1.34`) |
| 2 | `docker-ce-cli` "has no installation candidate" auf Apple Silicon | Repo-Zeile hatte `arch=amd64` hartkodiert, Build lief aber auf arm64 | `arch=$(dpkg --print-architecture)` zur Build-Zeit ermitteln |
| 3 | `useradd: group 'docker' does not exist` | Nur die Docker-**CLI** wird installiert (kein Daemon), daher legt kein Paket die `docker`-Gruppe automatisch an | `groupadd docker` explizit vor `useradd` |
| 4 | oh-my-zsh-Install schlägt mit Quoting-Fehler fehl | Verschachtelte `sh -c '...'`-Anführungszeichen in einer Zeile | Install-Skript erst in eine Datei laden, dann sauber mit `su - vscode -c "sh /tmp/install-omz.sh --unattended"` ausführen |
| 5 | `sops`-Binary war hart auf `linux.amd64` gepinnt | Lief auf Apple Silicon nur zufällig per QEMU-Emulation von Docker Desktop mit, nicht nativ | Arch dynamisch über `dpkg --print-architecture` auflösen (`linux.arm64` / `linux.amd64`) |
| 6 | `docker.sock`-Zugriff im Container: `permission denied` | Der gemountete Host-Socket gehört (je nach Docker-Setup) einer Gruppe/GID, die im Container nicht existiert oder nicht der `docker`-Gruppe entspricht (auf Docker Desktop für Mac/Windows z.B. GID 0/root statt einer eigenen `docker`-Gruppe) | `docker-init.sh`: Root-Entrypoint gleicht beim Container-Start die GID der `docker`-Gruppe an den tatsächlich gemounteten Socket an (bzw. tritt der GID-Inhaber-Gruppe bei, falls die GID schon vergeben ist), wechselt danach per `gosu` zu `vscode` |
**Konsequenz für die Zukunft:** Vor jeder größeren Änderung an `.devcontainer/` (oder
mindestens vierteljährlich) einmal real bauen und laufen lassen:
```bash
docker build -f .devcontainer/Dockerfile -t ess-gitops-devcontainer-test .devcontainer
docker run --rm \
-v ~/.kube:/home/vscode/.kube \
-v ~/.age:/home/vscode/.age \
-v /var/run/docker.sock:/var/run/docker.sock \
ess-gitops-devcontainer-test bash -c '
kubectl version --client && helm version --short && flux --version && \
sops --version && age --version && docker version --format "{{.Server.Version}}" && \
id vscode
'
```
Wenn `docker version` hier den echten Server, nicht nur die Client-Version zeigt, und
`id vscode` die passende Docker-Gruppe/GID auflistet, funktioniert der Socket-Zugriff
tatsächlich — nicht nur der Build.
## 📚 Weitere Ressourcen
- [Dev Containers Docs](https://containers.dev)
+23
View File
@@ -0,0 +1,23 @@
#!/bin/bash
# Runs as root at container start (before any `docker exec -u vscode` from VS Code).
# The docker.sock's GID is only known once the host socket is actually bind-mounted,
# so it can't be baked in at image build time - it must be reconciled here, at runtime.
set -e
if [ -S /var/run/docker.sock ]; then
SOCK_GID=$(stat -c '%g' /var/run/docker.sock)
CURRENT_GID=$(getent group docker | cut -d: -f3)
if [ -n "$SOCK_GID" ] && [ "$SOCK_GID" != "$CURRENT_GID" ]; then
EXISTING_GROUP=$(getent group "$SOCK_GID" | cut -d: -f1)
if [ -n "$EXISTING_GROUP" ]; then
# GID is already taken by another group (e.g. GID 0/root - Docker Desktop for
# Mac/Windows owns the socket this way inside its VM), so join that group
# instead of trying to reassign it to 'docker'.
usermod -aG "$EXISTING_GROUP" vscode
else
groupmod -g "$SOCK_GID" docker
fi
fi
fi
exec gosu vscode "$@"
+1 -1
View File
@@ -26,7 +26,7 @@ echo ""
echo "📚 Useful commands:"
echo " - kubectl get pods -n matrix (check pod status)"
echo " - flux get helmreleases -A (check helm releases)"
echo " - sops apps/production/custom-configs/mas-secrets.sops.yaml (edit secrets)"
echo " - sops apps/production/custom-configs/mas-secret.yaml (edit secrets)"
echo ""
echo "🔗 For kubeconfig setup:"
echo " - Copy your ~/.kube/config to access the cluster"
+2
View File
@@ -0,0 +1,2 @@
.DS_Store
.claude/
BIN
View File
Binary file not shown.
@@ -0,0 +1,23 @@
apiVersion: v1
kind: Secret
metadata:
name: authentik-backup-credentials
namespace: authentik
stringData:
borg-passphrase: ENC[AES256_GCM,data:5PApz4TqSNN2vVXeFSuomd051nl+cYk+a+STViwddG/Hj7XWQ099vvTlKSE=,iv:MFG04/66YqtOjZWsLpy236MYwR05z91ngOQ0BmLNxzA=,tag:gxD5prW8Ted3Q2ZY3sYSYQ==,type:str]
ssh-private-key: ENC[AES256_GCM,data:87PrkqAIH2xRCu3jlAK3Ts24XKRDoXNCWLomRhizCY43hwLeY5fNzfsGnMTKBt5IxL2bIljx+pbh/Y8IO38ISUUbUAczfi1KeN1gg69wKgqNAlckcmD0dkA2N2F0ZnEFX/rkad68YR8Hw1/A5h3qGUd5FwoVWzIFD/diY86s/LiQBYzwD4bYLUACHqIL7+QyJ6jEsFzA3tLwDySpUpM9OAjFQF0EazObChNYqe/qrc8i1gI8CptWOFkqiZ8Pen1E79gg/zgleOBqSmlcefgZk+JFPAfvORTHkDrHW9X/bLygaTQvskmGNzaKHpe2s/bVafWb7G60ROrpdjYIJfS7c0baRCSrd9kwdM4JQC2gSsMkI4v6oeFSvkTrb684B10zacZdiTKfySDV+Ry4hCZofF2RhAnsTbOpo0XANoOi4InoxUqGPfctVfUCVb6X+HIR5XYxpLrNwWqRpscMP2P3RJsaUZOinmQUKrHX/nMtPZ/ltpwjhQeAXZTtdyPxTCjHAh6fzouuHk66vlCdukXsMQDBdIfdA/3jjGbW,iv:yFNKkdegLLq8jq1Ya6v67urSJTdG3Ge4ZbmKizqQhmk=,tag:6l/qMb1d1oD8sfVEebW6gQ==,type:str]
sops:
age:
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBWMllTL2hRcEhCT2MxcWZy
dkxndmlZRW5FZ2NiWWdFckxqYTFKS1B6SXlvCjJySXRsZThvczNnWjlyM0N2Snha
eitvWUhhdmU4U0V3OG55WjdLbU9KcUEKLS0tIEM3MStzNUJhaGg4M0hKMVF1bnBB
RWVqVDRBNmJ0b1Bwd0l3dWxPT3Q0SU0KOEoyejkH4RC0p8ka3FjI7MyzRJg+uu7h
j3wf1q+Hgg73djDBSPYJkrB6Bdl4YMwo8SzbtW8O9elDE0qAioR0bQ==
-----END AGE ENCRYPTED FILE-----
recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
encrypted_regex: ^(data|stringData)$
lastmodified: "2026-07-28T18:20:58Z"
mac: ENC[AES256_GCM,data:lwwNbxSxtgDTDmaWMU1uf7TBOEw8gFBFKb982VIsGMeM0fIPHvX12Qts90MNgYIJliOWgAWrwyvAgfXWKuE37RNo+BtyfHCWi4IESKSN/RJrd/yMpRKx+02rifH3nl26ZCAQT1Pa0fjI1SfMhbVzfnD9a/AMARXZMhLRc0OqczY=,iv:SgP2iMtENRtZfw6I9EaOsmvecFNYCIZWqVj+cZ+T7EI=,tag:s4oEyxqyUx6ibqsk4g9xpw==,type:str]
version: 3.13.3
+85
View File
@@ -0,0 +1,85 @@
# Nightly Borg backup of the authentik Postgres database to a Hetzner Storage Box
# (issues #6 + #15). See apps/authentik/authentik-backup-secret.yaml for the SSH key +
# Borg repo passphrase, and apps/production/synapse-backup.yaml for the matrix-side job
# (same Storage Box, separate repo/passphrase, offset schedule).
apiVersion: v1
kind: ConfigMap
metadata:
name: authentik-backup-known-hosts
namespace: authentik
data:
known_hosts: |
[u641795.your-storagebox.de]:23 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIICf9svRenC/PLKIL9nk6K/pxQgoiFC41wTNvoIncOxs
---
apiVersion: batch/v1
kind: CronJob
metadata:
name: authentik-backup
namespace: authentik
spec:
schedule: "15 3 * * *"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 3
jobTemplate:
spec:
backoffLimit: 2
template:
metadata:
labels:
app.kubernetes.io/name: authentik-backup
app.kubernetes.io/component: backup
spec:
restartPolicy: OnFailure
containers:
- name: backup
image: rohana.axion1337.de/sorb/axion-backup:v2
env:
- name: BORG_REPO
value: "ssh://u641795@u641795.your-storagebox.de:23/./authentik-backup"
- name: BORG_PASSPHRASE
valueFrom:
secretKeyRef:
name: authentik-backup-credentials
key: borg-passphrase
- name: SSH_PRIVATE_KEY_FILE
value: /secrets/ssh/ssh-private-key
- name: SSH_KNOWN_HOSTS_FILE
value: /secrets/known-hosts/known_hosts
- name: DB_HOSTS
value: "authentik:authentik-postgresql"
- name: PGUSER
value: authentik
- name: PGPASSWORD
valueFrom:
secretKeyRef:
name: authentik-credentials
key: pg-password
volumeMounts:
- name: ssh-key
mountPath: /secrets/ssh
readOnly: true
- name: known-hosts
mountPath: /secrets/known-hosts
readOnly: true
- name: scratch
mountPath: /scratch
resources:
requests:
cpu: 50m
memory: 128Mi
limits:
memory: 256Mi
volumes:
- name: ssh-key
secret:
secretName: authentik-backup-credentials
items:
- key: ssh-private-key
path: ssh-private-key
mode: 0400
- name: known-hosts
configMap:
name: authentik-backup-known-hosts
- name: scratch
emptyDir: {}
+55
View File
@@ -226,3 +226,58 @@ data:
domain: authentik-default
attrs:
default_application: !Find [authentik_core.application, [slug, matrix]]
matrix-oidc-provider.yaml: |
# yaml-language-server: $schema=https://goauthentik.io/blueprints/schema.json
version: 1
metadata:
name: matrix-oidc-provider
labels:
blueprints.goauthentik.io/instantiate: "true"
entries:
# The OIDC Provider + Application linking Authentik to MAS was originally
# clicked together by hand in the UI and existed nowhere as code (issue
# #36): losing the Authentik DB would have meant re-creating this from
# scratch, including a new client_secret that MAS would then no longer
# match. client_secret is read from AUTHENTIK_MAS_OIDC_CLIENT_SECRET
# (see authentik.yaml HelmRelease values) rather than inlined here,
# since this ConfigMap itself is not SOPS-encrypted - the actual value
# lives in the authentik-credentials Secret instead.
- model: authentik_providers_oauth2.oauth2provider
state: present
identifiers:
name: Matrix Authentication Service
id: matrix_mas_provider
attrs:
client_type: confidential
client_id: dHbTAgAgXvjh3VALh220mB3dxcVXAifiXU2ZO3U6
client_secret: !Env AUTHENTIK_MAS_OIDC_CLIENT_SECRET
# Path includes MAS's own upstream-provider ID, not Authentik's -
# must match MAS's config exactly or the OIDC callback breaks.
redirect_uris:
- matching_mode: strict
url: https://account.axion1337.chat/upstream/callback/01KQDJTR1ZVTG8JQ220F5BNBFZ
# Stable across username renames - this is what keeps
# upstream_oauth_links rows valid after e.g. the elbojoloco rename.
sub_mode: hashed_user_id
include_claims_in_id_token: true
access_code_validity: minutes=1
access_token_validity: minutes=5
signing_key: !Find [authentik_crypto.certificatekeypair, [name, "authentik Self-signed Certificate"]]
authorization_flow: !Find [authentik_flows.flow, [slug, default-provider-authorization-implicit-consent]]
invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
property_mappings:
- !Find [authentik_core.propertymapping, [managed, "goauthentik.io/providers/oauth2/scope-openid"]]
- !Find [authentik_core.propertymapping, [managed, "goauthentik.io/providers/oauth2/scope-email"]]
- !Find [authentik_core.propertymapping, [managed, "goauthentik.io/providers/oauth2/scope-profile"]]
- model: authentik_core.application
state: present
identifiers:
slug: matrix
attrs:
name: aXion1337.chat Accountverwaltung
provider: !KeyOf matrix_mas_provider
meta_description: Matrixclient tailored for aXionCommunity
meta_publisher: aXionGaming
policy_engine_mode: any
open_in_new_tab: false
+5 -4
View File
@@ -7,10 +7,10 @@ stringData:
secret_key: ENC[AES256_GCM,data:yIyQapbFtFM11LynFtkV3ffExhaDfN9QHeFbI1T0xkIhgsV+9sjg3qwMVmeBlAe7xZl8gsAM4kDj2Q6O91OdDg==,iv:+Cl8vOcxG9/mgRheaCO0bLWyCJXN+f1F2DD3oeHbPFY=,tag:711ytyKf6/tmXomBLoffGA==,type:str]
pg-password: ENC[AES256_GCM,data:3w8R9mRjMXMJDLjrC8QYaXFHsCU3yYZs2PcaFQNp3Z4=,iv:G/aXgoGz3vBOzZ5K3Y+DDJsqer4F5gvcMmtkzRx93CU=,tag:dXPs1pY/APvnMlxdvB1EkA==,type:str]
smtp-password: ENC[AES256_GCM,data:JpMgaQFPkBzOg5WjvpmhM0kPwvZkH+4tQjT17RJHjG14WjmWtfG9Bg==,iv:zjQRLIlrxKv5hbd4JZowNUEiibiCUMf79acZY0+dYAc=,tag:ORPafTPyOQJvVvHWQGmqhA==,type:str]
mas-oidc-client-secret: ENC[AES256_GCM,data:0yx55FroLSxlnuYgfNwczu3PnbPm1kW74JtiU9oFevVqeQDZc385wU6x5X5TN7owXDO7QaOfGTTMvqIpbwQb6Q5Vt1VMToR+0f44oJcktYoTiDFU9Sy6lR/y6nlvBCNqeJg7vIyVpkIqxwqty15EekyqMpkIMp1fT6Pxmek0SO0=,iv:Ey06ljnqbVARDLVt2sLe8R776VEWpTlzI/+Nka5NocA=,tag:I+GNLHz4V8TFa2ijzK5y2Q==,type:str]
sops:
age:
- recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
enc: |
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBRekJuZythYzliTFJ3RlhS
R2p6TG9NeFdabFlPRWtpNHJMYVVxTWZEcmlRClk0WUorSzdxNlcyWHYwWFBTMnlq
@@ -18,7 +18,8 @@ sops:
QXVrY1NTeHZkeTlPRWNlVThzWno3T0kKC0KBoLT64GNqb8Ri9u69G7nqb1KftwwP
/24aVHrPxKi9d4ij9n3bvCYDF4rhtfexhrE4n7CfuKn2DcSiuTniuw==
-----END AGE ENCRYPTED FILE-----
lastmodified: "2026-04-29T21:43:59Z"
mac: ENC[AES256_GCM,data:Y+dJppkaVZ5NOhlvwbbsF5+vDFqGUI1Ps8IcE4J7FIW4HIdMVf6RKM0EInvPUW1LaBlmelCitcE30w0As7ysNRhLY8yUDaKUvuU6mRejlNUIF8wAHzhciL2jTvAQsArHjybJatEig28+wM9VcY8JEa/d/CmuiB9Nq4WbIV+JXlA=,iv:UQj2rIVLNPjtYp3d/jRyNfJyyyUsZ3+NDCgpI4aztzc=,tag:cwiCzG/A+rfRFfLjXVt82w==,type:str]
recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
encrypted_regex: ^(data|stringData)$
lastmodified: "2026-07-28T15:54:53Z"
mac: ENC[AES256_GCM,data:P6IF+jukwzldK92nHl6s4h6sS4ldXLwpyLpwv2tpI3vFWgTLEnGCnowi2k5lmWUlITEVmLLC0HvsBuduTiGI2sIHHt+r3RdqkV88HGn6oYDVq5a+Ax7ESfqti/4B7ClQCSxl/tU6hBUFe812DiBXJgA03UJQZn8uHY/dP/RgRpc=,iv:V8sqhbJcKglkKsQmJBdgoxDaCYJ3Wt/qRa18jEviH60=,tag:EiNotrYAKIzKndgjU/kTFQ==,type:str]
version: 3.12.2
Executable → Regular
+14
View File
@@ -40,6 +40,15 @@ spec:
global:
security:
allowInsecureImages: true
# Read by the matrix-oidc-provider blueprint via !Env, so the OAuth2
# Provider's client_secret can be captured as code without ever
# inlining the live credential into a plain (non-SOPS) ConfigMap.
env:
- name: AUTHENTIK_MAS_OIDC_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: authentik-credentials
key: mas-oidc-client-secret
authentik:
log_level: info
@@ -89,3 +98,8 @@ spec:
memory: 256Mi
limits:
memory: 512Mi
# Chart's own generated policy allows ANY pod in ANY namespace on 5432
# (see issue #37) - disabled in favor of our own scoped policy in
# apps/authentik/networkpolicy.yaml.
networkPolicy:
enabled: false
+3
View File
@@ -9,3 +9,6 @@ resources:
- authentik.yaml
- ingress.yaml
- networkpolicy.yaml
# Backup zur Hetzner Storage Box (Issues #6 + #15)
- authentik-backup-secret.yaml
- authentik-backup.yaml
+33 -3
View File
@@ -4,9 +4,10 @@
# for upstream OIDC calls. Egress is intentionally untouched (federation-equivalent
# outbound calls like SMTP aren't restricted here).
#
# Note: authentik-postgresql already has its own NetworkPolicy from the Bitnami
# postgresql subchart (port 5432, no source restriction) - left alone, not duplicated,
# since it would get reset on the next Helm upgrade anyway.
# authentik-postgresql: the Bitnami postgresql subchart's own generated NetworkPolicy
# restricted the port (5432) but not the source - any pod in any namespace could reach
# it (issue #37). Disabled via postgresql.primary.networkPolicy.enabled: false in
# authentik.yaml and replaced below with a policy scoped to authentik-server/-worker.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
@@ -47,6 +48,35 @@ spec:
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-ingress-authentik-postgresql
namespace: authentik
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: postgresql
app.kubernetes.io/component: primary
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
app.kubernetes.io/name: authentik
app.kubernetes.io/component: server
- podSelector:
matchLabels:
app.kubernetes.io/name: authentik
app.kubernetes.io/component: worker
- podSelector:
matchLabels:
app.kubernetes.io/name: authentik-backup
ports:
- protocol: TCP
port: 5432
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-ingress-acme-solver
namespace: authentik
BIN
View File
Binary file not shown.
+19 -10
View File
@@ -4,19 +4,28 @@ metadata:
name: coturn-secret
namespace: matrix
stringData:
TURN_SECRET: ENC[AES256_GCM,data:SILIqMB+fmAMFITAL7lG1hOgICec6BJf1mOcK0gdmnCHWYqRuJv7jgjfGylG25xzQKi+zE7Qual9PnkZG2KiOA==,iv:+GZqLGusE4Q0x2jEEtFxj06rryyQmQhXdkTy4eE8ZHw=,tag:OpSZkinPTAi1ZKWyo8OX3A==,type:str]
TURN_SECRET: ENC[AES256_GCM,data:gab4MSNlRANQz/T/Xn0Z45tkUvQ3Uxn5u0MyYfhZTan49P865UbX4rYPf/KmKsekdwFdOGBzvCgcwxhddVbZ1w==,iv:INbz5UUxgA6xdsPBp5W9Mwe+kd9dhHh2Rx5l3yeNRIQ=,tag:dgk/iFjvst7Wul9Lai2QEQ==,type:str]
sops:
age:
- recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
enc: |
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAyRk1mK3NWc1l4T0JCOFpF
S0RuQ3ViZmo3QTNVL2JvZ0hzMy91R2l0TEhzCk01a1VGdk1sdVg4aWswTzRibXI4
ZlJtNFF5MjBONEZOaWVpeU5taHl2bkEKLS0tIGxpUHY3NUFLWFBaWm1QSlZiVFkx
MEJleHFnd3oyT3VPL2dsYkpMUlRkOWMKcKUIgsQ/ff49pGGXMnYwJmwqPVC7woAR
IEzvhcNX97xx746SnrxZe5t2YadsYMkYIl0nvqBPJhSlvqMNafpQbQ==
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBQMncxOEhRZC9jcHpjT3dW
Q1RNaW5pc00zVnJhWHRnZHd3TGhURWNZU2dJCk85bXJkbTEyd1ZybjhDT05pL2c3
ZU9EUSt1eDlSQWJyVGtsak1oS3FSR2sKLS0tIEFUdXVHL0V1ZW5VMVVBZEJaYUIw
U1BrYlJyQVZkZFhBdmdwbDMyK3lTQkkKEaSy1o+IICf2uaT6olapRJa/duXxjOBg
OqRS9axnJ71XxEnHjLTsCbkI5b+8Fux08qKaH9sMsJrWOiSHDdTXXQ==
-----END AGE ENCRYPTED FILE-----
lastmodified: "2026-04-29T21:06:21Z"
mac: ENC[AES256_GCM,data:UhyR5m1HYWrZHwNLW5sg2PxbpaydWbP5cekghGlzSpQg7CYEcvZw3tJ/qB8zA19xZSM7tdSHOXdD+QytRq6qW59M1unqMaumA43B6JxQg1C1NdXAW0mkSc2WiNchvgpVii9P/TVlzSSIRwC3YGCQUsfa3SSfNzI4Z6fMuBnhYLE=,iv:4HYxbrYSRJLe1KcQ6q8bpee8/Lx1m3pPmisb/L2Mu64=,tag:l7n3u+Pg6533OzwtNUZvNw==,type:str]
recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAwQk82eGdWRmdRNFJQR0xF
SWhNRUtNWThZNFM0aXU3V0N4UkhtTy9NTFdnCi9pa3dsVXRja1dTL1pZTnoxQ0JT
UUIxekZnVGUvdFgyblFiS0JLMjU3L2cKLS0tIE1IRTJ0M3kvMFZPWVVDYjJlVkk0
eGJQOTVUc1NsNE5GdmJtODlmdHp3c2cKHTP6YRMTdYE/iBuSZs/Tjt4TwKCxHEIu
f3jTblKIqWwRHKCgOIkC16QDbpMBlNLH3JknJEdIjkB2HIrXrw1MNA==
-----END AGE ENCRYPTED FILE-----
recipient: age1x4jjwc8nuttwr8us924pvdc6dll5npkc6c8f4zf2hx5d2qu75dtqx0fm0d
encrypted_regex: ^(data|stringData)$
lastmodified: "2026-07-28T19:26:57Z"
mac: ENC[AES256_GCM,data:zU3i1WGY1X4igBYzfC1WAWdtMQZWGBSacaJpXnUVt6+BCfucqaH807IRWpWbTR0+xuLteaqcdXuVuAlStqs/VV586Q4cD2FfWDeBq/O09HEwTrfekTjAyyu7zCFyZ2ja+kB4DQ83NoscCiNwNmfTZF+6RzMqx7yphPxK5go1r7c=,iv:6FIYWsNvz78zJscbxyDgy1u6ur20MnOMkMcsA4329OI=,tag:uSDpAWEixGcVmtOloMoKuw==,type:str]
version: 3.12.2
Executable → Regular
+15
View File
@@ -65,6 +65,17 @@ metadata:
namespace: matrix
spec:
replicas: 1
# hostNetwork pods bind directly to the node's ports (3478/5349) - on this single-node
# cluster, RollingUpdate's default "bring up the new pod before removing the old one"
# can never schedule (port conflict). Recreate kills the old pod first.
# Note: switching to Recreate on an existing Deployment that already had the
# RollingUpdate defaults recorded required a one-time manual
# `kubectl patch --type=merge -p '{"spec":{"strategy":{"rollingUpdate":null,"type":"Recreate"}}}'`
# (2026-07-28) - a YAML `rollingUpdate: null` in this file is dropped before reaching the
# API server (client-side omits null keys) rather than sent as an explicit field deletion,
# so it can't clear an already-set field on its own.
strategy:
type: Recreate
selector:
matchLabels:
app: coturn
@@ -74,6 +85,10 @@ spec:
app: coturn
annotations:
prometheus.io/scrape: "false"
# Bumped on every TURN_SECRET rotation (Issue #38) to force a new pod, since
# Kubernetes doesn't restart running pods when a referenced Secret's content
# changes and the initContainer that reads it only runs once at pod start.
rotated-at: "2026-07-28T19:26:57Z"
spec:
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
@@ -58,15 +58,6 @@ data:
room_list_publication_rules:
- user_id: "*"
action: allow
turn:
config: |
turn_uris:
- "turn:turn.axion1337.chat?transport=udp"
- "turn:turn.axion1337.chat?transport=tcp"
- "turns:turn.axion1337.chat?transport=tcp"
turn_shared_secret: "cab3c8408363515d9b4cdc3384a1f76ca17a973242fdfdc72b67ac4d86158527"
turn_user_lifetime: 86400000
turn_allow_guests: false
oembed:
config: |
oembed_enabled: true
+9 -2
View File
@@ -4,7 +4,11 @@ metadata:
name: matrix-stack
namespace: matrix
spec:
interval: 5m
# Shortened from 5m to match production-apps Kustomization's 1m interval - narrows the
# window between coturn (Kustomization-only, no Helm indirection) and synapse-main
# (behind this HelmRelease) picking up a rotated TURN secret after Issue #38's
# automated-rotation PR gets merged. Self-heals either way, just faster now.
interval: 1m
chart:
spec:
chart: matrix-stack
@@ -25,7 +29,10 @@ spec:
- kind: Secret
name: ess-mas-values-secret
valuesKey: values.yaml
- kind: Secret
name: synapse-turn-secret
valuesKey: values.yaml
values:
# Top-Level: serverName das ist dein Matrix-Homeserver-Name
serverName: axion1337.chat
View File
View File
+15 -3
View File
@@ -1,8 +1,10 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
# Patch: Fügt einen Checksum der element-values.yaml zur HelmRelease hinzu
# Damit wird Flux die HelmRelease neu-synced wenn sich die ConfigMap ändert
# Patch: Fügt Checksums der element-values.yaml und des turn_shared_secret zur
# HelmRelease hinzu. Damit wird Flux die HelmRelease neu-synced (und synapse-main neu
# gestartet), wenn sich die jeweilige ConfigMap/Secret ändert - siehe Issue #38's
# Rotations-Mechanismus, der turn-secret-checksum bei jeder Rotation bumpt.
patches:
- target:
kind: HelmRelease
@@ -12,6 +14,9 @@ patches:
- op: add
path: /metadata/annotations/element-config-checksum
value: "401f8a87d0ef5d91d2e5032d4aede42c"
- op: add
path: /metadata/annotations/turn-secret-checksum
value: "d220c0e4ff8f7106328c8827d47e8734"
resources:
- matrix-postgres-auth.yaml
@@ -26,8 +31,15 @@ resources:
# TURN Server für WebRTC
- coturn-secret.yaml
- coturn.yaml
- synapse-turn-secret.yaml
# HelmRelease (muss ganz unten stehen, damit die ConfigMaps vorher da sind!)
- element-server-suite.yaml
# Custom Apex Ingress für Element Web + Well-Known auf axion1337.chat
- apex-ingress.yaml # Custom Apex Ingress für Element Web + Well-Known auf axion1337.chat
- networkpolicy.yaml
- networkpolicy.yaml
# Backup zur Hetzner Storage Box (Issues #6 + #15)
- synapse-backup-secret.yaml
- synapse-backup.yaml
# Automatisierte TURN-Secret-Rotation (Issue #38)
- turn-secret-rotation-secret.yaml
- turn-secret-rotation.yaml
+3
View File
@@ -258,6 +258,9 @@ spec:
- podSelector:
matchLabels:
app.kubernetes.io/name: matrix-authentication-service
- podSelector:
matchLabels:
app.kubernetes.io/name: synapse-backup
ports:
- protocol: TCP
port: 5432
@@ -0,0 +1,23 @@
apiVersion: v1
kind: Secret
metadata:
name: synapse-backup-credentials
namespace: matrix
stringData:
borg-passphrase: ENC[AES256_GCM,data:RRXPwr4UGX30IdozM53abN7ZYztTO3Y1/63dtTh0JSZmU8i8l/ATb4gc3lc=,iv:rIyUr+lOUjo9J53OKZ5ZDmp3d8Nrb9PP2JDK2oCutYU=,tag:MqUmoYgCA03WJQy+RQi04w==,type:str]
ssh-private-key: ENC[AES256_GCM,data:320B/lSq7DljCrXZ0BluGv8gLIzYF3KL6VQnR6CM/Vzuf/6qbbZ7lMzrS96XfoubU95OxDeJuOsuNZBQUZNyUGtt4QjQmnl8XHUJzl48xLqh83HLFPtQm2uqU072lscf5OTT7I+JzD2BGks3OIowhrg1q0MdVQdfd4Rhdz8Jphjb+WxvstzNmEX5gxQ9mnBVPmj4DS6ikdXnpe+VDvaCJaVkzD5KgwRmqPy0qbFs9WXziQSo3am5fPeHbwhV3UlRhlok9WrDI40a1T1S5DBhgNhwShq1jAxjr9onuq92REymxAV50oLzsw9ivnH0uimw+3PcplRG1v2xxJ/pimWTCjE17bO7OZ8TyzzwyZ7QA02vSpOMNUdUwVU3N6pSdYQpdTETuVBTVqVc+GKC18Z9fWh2rdwX5eUDTwp0bbDDvPEHdANNxMg3VYD3gwpCgy6/wVnjrO+pVMs8K1CqFn4/H7azhzzeEPkrz27ZBxjlnqvDODsm9tlklr6X4jZDZmTaEmiH+WXuc+1qbNxvsrqtSFfa3CfFGxM1nBOh,iv:sKGsTLsxdQYVUvw7CEARL3YNInSd9LPbFp5Ci5CTgIw=,tag:O1mjNZUsGEqV4uu+LlgD/g==,type:str]
sops:
age:
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB3ZUdtN3hyNHlyMkIrbTVS
aHQ4OGNUWnA4eTFUVkx4UVp5VWlnMDgxTEU0CkU3M2dMYWgyKytlRVFOVWptZEd2
NXlIY0JCd24xcGFzaGpIeks0R2U0U3MKLS0tIHlxYVZ1ZTJsRXNaZ25sVzZtSnp0
SEhzQ0tUYzZTRXcwMVNwbG85SHpyb0UKOn3nxy6Y7yQkGargXQ9z6O36vUWW4qJZ
D/GbFGmoRi94EtVFdmTGALhjy2D4J9QXy6gHsTapvKyMxF8NEtk+FQ==
-----END AGE ENCRYPTED FILE-----
recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
encrypted_regex: ^(data|stringData)$
lastmodified: "2026-07-28T18:20:58Z"
mac: ENC[AES256_GCM,data:Rur32fQdCyM3nr/X+KeSgmPYEi4nKyh8lqTuSW3TPBVDjwTWMDp1I1ZPPyy5syeW6RHbKputFUzBWVnuQmVrfbZaQ6DBBI5kP9InspUAVUjXDRk9XqiWtdg/wYaTMMJ4Nxv/zdwkh6uJQSG2JHQBWce4NZc2hoPokLR0CjcWiZk=,iv:eWTUj48EFjjtuIIuErMltEdDfabLZeolkpInMYtVP5Q=,tag:6DPxkPqLt0ihJ80WnTzHeA==,type:str]
version: 3.13.3
+96
View File
@@ -0,0 +1,96 @@
# Nightly Borg backup of the shared Postgres instance (synapse + MAS databases) and the
# Synapse media_store PVC to a Hetzner Storage Box (issues #6 + #15). See
# apps/production/synapse-backup-secret.yaml for the SSH key + Borg repo passphrase, and
# apps/authentik/authentik-backup.yaml for the equivalent authentik-side job.
apiVersion: v1
kind: ConfigMap
metadata:
name: synapse-backup-known-hosts
namespace: matrix
data:
# Pinned via `ssh-keyscan -p 23 u641795.your-storagebox.de` (2026-07-28) rather than
# trusting the host key on first connect in an unattended job.
known_hosts: |
[u641795.your-storagebox.de]:23 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIICf9svRenC/PLKIL9nk6K/pxQgoiFC41wTNvoIncOxs
---
apiVersion: batch/v1
kind: CronJob
metadata:
name: synapse-backup
namespace: matrix
spec:
schedule: "0 3 * * *"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 3
jobTemplate:
spec:
backoffLimit: 2
template:
metadata:
labels:
app.kubernetes.io/name: synapse-backup
app.kubernetes.io/component: backup
spec:
restartPolicy: OnFailure
containers:
- name: backup
image: rohana.axion1337.de/sorb/axion-backup:v2
env:
- name: BORG_REPO
value: "ssh://u641795@u641795.your-storagebox.de:23/./synapse-backup"
- name: BORG_PASSPHRASE
valueFrom:
secretKeyRef:
name: synapse-backup-credentials
key: borg-passphrase
- name: SSH_PRIVATE_KEY_FILE
value: /secrets/ssh/ssh-private-key
- name: SSH_KNOWN_HOSTS_FILE
value: /secrets/known-hosts/known_hosts
- name: DB_HOSTS
value: "synapse:matrix-stack-postgres matrixauthenticationservice:matrix-stack-postgres"
- name: PGUSER
value: postgres
- name: PGPASSWORD
valueFrom:
secretKeyRef:
name: matrix-stack-generated
key: POSTGRES_ADMIN_PASSWORD
- name: MEDIA_PATH
value: /media/media_store
volumeMounts:
- name: ssh-key
mountPath: /secrets/ssh
readOnly: true
- name: known-hosts
mountPath: /secrets/known-hosts
readOnly: true
- name: media
mountPath: /media
readOnly: true
- name: scratch
mountPath: /scratch
resources:
requests:
cpu: 50m
memory: 128Mi
limits:
memory: 256Mi
volumes:
- name: ssh-key
secret:
secretName: synapse-backup-credentials
items:
- key: ssh-private-key
path: ssh-private-key
mode: 0400
- name: known-hosts
configMap:
name: synapse-backup-known-hosts
- name: media
persistentVolumeClaim:
claimName: matrix-stack-synapse-media
readOnly: true
- name: scratch
emptyDir: {}
+31
View File
@@ -0,0 +1,31 @@
apiVersion: v1
kind: Secret
metadata:
name: synapse-turn-secret
namespace: matrix
stringData:
values.yaml: ENC[AES256_GCM,data:An+GypE/btGmfnvL1rPzvbN1sAnZKQnHV20GxXkZiRb+Wi8BymAUA6JRBo5mdtvK1dgDMF6H1wN+Y7wjzwduteaoskVfgd7Xw+lL1SrZ/6pchGxlUIWSJlLTmp6IHZaruleDUsCKdrWfN9TQDE+WYdiECBktlGGXs8ZZMTzjHmHKJ3tNwSimaxuG662Amb0LiHLX+zBUKOuG0L4cWrp6JI3NKQFgNNiF6xwXjHDX550bGOQ2pi5UVPL9+HlH2tPeTzoCCZdJLWF+2kZg31bZ7dzRp8M23eDrzfGC9e9m0NrA2z1ljagbnuT0tixp6J8205qfBOIOVOWbXHmd/LQ8nphA64En6JXqU8XeZ+mgMUmFuGwyewLq4oilLoQfdZujEyGhc9Hd/yonRlc/RDBlONlI8d+IWqZxf0AaUYEAe9uE6MulqRs0CowPUpCmWtZjz68qQlIvJEAqxNdWEnp+SkExDNNe9y8tBOCXy726TRX0MLIlZ+CjPDCHhwDeKw5oQU0wXr9QezY=,iv:bPK1+PViBg046N3q2Eb1ZOJOYv7X/HgpCEU/MLp7o6k=,tag:/2Kyv7aB4baOL+3KdAeQ9w==,type:str]
sops:
age:
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBuK2xIT3M0cGtVOHdKYklF
OHl1VHFRTkVSaHVQNUUyRlhZUjRWM3JUN0hBCnpUS2YzdG1mSjlRaEVvTHdKVkR4
L1hrR05IRUdqdGp2aU95aWxRRXdsQWcKLS0tICtLbnFsK3Zza053VXFWdVErRy92
WUUzZTBIRzUyWnp5a1ZScUVqb0NyencKvnFyJCR6j1/aH4gJvFmLPNlk5XpC08wF
mTmL981uGfz4NULc+O3sDkonJ827glpefgWPgPW2HmKT88d4A9vyJw==
-----END AGE ENCRYPTED FILE-----
recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBVY0NXa1B5c2RzbzRReU1h
RHBSbk9aK3d6VjRWZ1hzL3FDZFZJZGFqTVQ0Ck9ib0p3bkR6cU8yc0VFNjEzSFFi
T3dWY1luQW1VZ1NjZFNoZFFLSCsvelkKLS0tIHIxV0d6TWhpSEc3d3c0L1VvOWxk
eDhTM2pDbTNXZXlWRVYxR2tPQU5iLzgKrLLLSBU/g5ebeRNi7hWYbcuJ/2JOfiUn
0DBnzMVJPBfqq/u8THiRYaMajx3k4D9+FN7qc5nBgTd85iGBo+OowA==
-----END AGE ENCRYPTED FILE-----
recipient: age1x4jjwc8nuttwr8us924pvdc6dll5npkc6c8f4zf2hx5d2qu75dtqx0fm0d
encrypted_regex: ^(data|stringData)$
lastmodified: "2026-07-28T19:26:57Z"
mac: ENC[AES256_GCM,data:UfD+QJel9FygoKPM02KdOLvP1id/21ghQ0uWAQJxEs/w+B2vVkpe/dXSJ7QnI0e528ns1HW81GAxkxyAupsUDq8/pBdEQgItwDs5DIgHvH+sTPAtj2Kyq2wMOeEt+XDGLRdOlAwbcKDS/LyfeZ5b9QnAbY2w/tTinTkzEvbudT8=,iv:WmmwjAmZmM0W8dvqBcv9XYZkYtan3lg0Sb8ypVuGoJk=,tag:3yzZhSFtbSbLUXSdDWGAtw==,type:str]
version: 3.13.3
@@ -0,0 +1,23 @@
apiVersion: v1
kind: Secret
metadata:
name: turn-secret-rotation-credentials
namespace: matrix
stringData:
age-key.txt: ENC[AES256_GCM,data:4LAs9LLFo38UMHXCo4lun9RHxGnDyp7GWlaNdIqqkSL9lNv7+ILdlc03CxFVobCYxK65xMOn1xdEty+887JBMlawST04am/5MkAnUivKwXCw8OHmbZhCwKHFqSYH/NsgVNf+btZKIIny8XPVQAPj/vQIi+Ity+BQyPkEZ1WUcsqjDoaK9IFhQTePJtHgWivhOY2WpUt/TP7vTfub4TOrgVpzNd9LIpBkwq+zhpVXKwnYUWOuLMXlPe08kazy,iv:9A24HbTl24slj+qTCfyI01+dGqRFVPDUA0wp2kSUHpc=,tag:iQxU0dWD4Noo1m0HXhV6vQ==,type:str]
gitea-token: ENC[AES256_GCM,data:cmZ1GCrqRYLtLn+cRVZCrO7UcCIavlQLJPt2PRMtBbgLdhDVTElUKA==,iv:9v29GXHRtDlrL3PoRCdOqYpBepZrX04+6UjoywRZX0E=,tag:ZvYXBOm9qgB1XA5lkV4LsQ==,type:str]
sops:
age:
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA5S3BtTHNLVGVVM2ZpTlZS
Y3pGMS9CSkNsdUpPbWtkSTRHK0p6U2lwdEJnClVWdXp4SllyM1hvbTZyTU40SDc2
QlVtMDduZWpaVENiYnhMNlFXd01QblEKLS0tIGNzTGRZcmoyaFltUHRDSHBPZE1N
OCszUkl1VjQ5V3F2cVI4dXJFcER5YXcK+2Eh1JNLuMiCnpQ3cL/I7XTykkIZ3tqp
O3c9UwYs1FAZWlMgElTBTqsmut1ShduIYfDFRKGeS0UxPEM4U+tIGw==
-----END AGE ENCRYPTED FILE-----
recipient: age14l0hwfqylwpemz5y2ghh2yxk0phszlnj3qlejhue0fw0kz3tmfgqdsjzdh
encrypted_regex: ^(data|stringData)$
lastmodified: "2026-07-28T19:21:28Z"
mac: ENC[AES256_GCM,data:vx6Bs/L0NXKUvvQdu6aYtuur/CYPkIBZzvFLqTyd08Errw0dGMrg73oHQ/imxpe42HgnO2mGwxNdEx2jYYbtc3RBWHE/yPH5m8y/XLoSL3fauzbkGsDwMSWzKiZXyIuGh7SxuB+CFY9qqFMK+dap1Ofno7a1/Gr1qibVDqscwxw=,iv:9AUQQuTCja09OohzVw73URMHE8xCW7iLLtBg7GSDcPA=,tag:g1ibnBIGqokvs2IEVOYq6Q==,type:str]
version: 3.13.3
+70
View File
@@ -0,0 +1,70 @@
# Automated TURN shared-secret rotation (Issue #38). Generates a new secret, re-encrypts
# apps/production/coturn-secret.yaml and synapse-turn-secret.yaml using a dedicated,
# narrowly-scoped age key (see turn-secret-rotation-secret.yaml - it can only decrypt these
# two files, not the repo's master sops-age key), bumps the checksum/rotated-at annotations
# so a merge restarts both consumers automatically, and opens a Pull Request rather than
# pushing straight to main - a human reviews and merges it.
apiVersion: batch/v1
kind: CronJob
metadata:
name: turn-secret-rotation
namespace: matrix
spec:
schedule: "0 4 1 * *"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 3
jobTemplate:
spec:
backoffLimit: 1
template:
metadata:
labels:
app.kubernetes.io/name: turn-secret-rotation
app.kubernetes.io/component: rotation
spec:
restartPolicy: OnFailure
# Public-internet reachability to the Gitea host has been flaky (see Issue #41);
# both servers share a private Hetzner network. hostAliases (unlike the node-level
# /etc/hosts fix used for image pulls) is actually honored by in-pod processes.
hostAliases:
- ip: "10.0.0.3"
hostnames:
- "rohana.axion1337.de"
containers:
- name: rotate
image: rohana.axion1337.de/sorb/axion-secret-rotation:v1
env:
- name: GITEA_HOST
value: "rohana.axion1337.de"
- name: GITEA_REPO
value: "sorb/axion1337.chat-gitops"
- name: GITEA_TOKEN
valueFrom:
secretKeyRef:
name: turn-secret-rotation-credentials
key: gitea-token
- name: SOPS_AGE_KEY_FILE
value: /secrets/age/age-key.txt
- name: GIT_AUTHOR_NAME
value: "turn-secret-rotation"
- name: GIT_AUTHOR_EMAIL
value: "turn-secret-rotation@axion1337.chat"
volumeMounts:
- name: age-key
mountPath: /secrets/age
readOnly: true
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
memory: 128Mi
volumes:
- name: age-key
secret:
secretName: turn-secret-rotation-credentials
items:
- key: age-key.txt
path: age-key.txt
mode: 0400
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
Executable → Regular
View File
View File
Executable → Regular
+28 -1
View File
@@ -103,4 +103,31 @@ ist reines Zurückkopieren, kein Neu-Erzeugen).
`sops-age`-Secret selbst (auf demselben Server) und lokal bei wem auch immer ihn zuletzt
zurückgeholt hat. Es gibt kein separates, offsite Backup. Fällt der Server komplett aus
(nicht nur der lokale Rechner), sind alle SOPS-verschlüsselten Secrets im Repo unlesbar.
Siehe Issue-Backlog für die Entscheidung, ob/wie das abgesichert wird.
Siehe Issue-Backlog für die Entscheidung, ob/wie das abgesichert wird.
---
## 🌐 Node-Konfiguration: `/etc/hosts`-Eintrag für den Gitea-Host
Der K3s-Node und der Gitea-Host (`rohana.axion1337.de`, Container-Registry + Git-Remote)
teilen sich ein privates Hetzner-Netzwerk (Node `10.0.0.2`, Gitea-Host `10.0.0.3`). Seit
2026-07-28 hat der Node dafür einen manuellen `/etc/hosts`-Eintrag:
```
10.0.0.3 rohana.axion1337.de
```
**Warum**: eine Firewall-Fehlkonfiguration hatte den Node zeitweise komplett von
`rohana.axion1337.de` über die öffentliche IP abgeschnitten, was Image-Pulls (z.B. für
Custom-Images wie `sorb/axion-backup`) mit Timeout scheitern ließ. Der Eintrag macht
Image-Pulls unabhängig vom Zustand der öffentlichen Firewall.
**Wichtig**: Das ist unmanaged Node-Konfiguration (kein GitOps, kein Kubernetes-Objekt) —
überlebt einen Node-Neuaufbau **nicht** und muss dann erneut gesetzt werden:
```bash
echo "10.0.0.3 rohana.axion1337.de" | sudo tee -a /etc/hosts
```
Ein sauberer, clusterweiter Ersatz (z.B. CoreDNS-Rewrite, damit auch Pods selbst intern
auflösen) ist als Issue #41 nachgehalten.
View File
Executable → Regular
View File