First egress rule in matrix, authentik and monitoring. It allows DNS, the cluster
ranges and the whole internet, and denies only 169.254.0.0/16 - link-local, where
Hetzner serves instance metadata unauthenticated to any pod.
Deliberately narrow. The textbook cut, 0.0.0.0/0 except RFC1918, would have severed
two things here, both over 10.0.0.3 on the private Hetzner network: Alloy writes
metrics and logs there, and the TURN rotation reaches Gitea through a hostAlias to
that address. Private ranges therefore stay open.
The payoff is modest and should be stated as such: measured from a pod, the service
answers with instance-id, hostname, region, MAC and network config, while userdata
and public-keys are empty. No credentials are exposed here, unlike the AWS case this
hardening usually targets. It costs nothing though, and it closes the class.
Two preconditions checked rather than assumed, because both are the usual way this
breaks: kube-system carries kubernetes.io/metadata.name so the DNS rule actually
matches, and the cluster is IPv4-only so 0.0.0.0/0 really does cover everything.
Rollback is deleting the one policy per namespace.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add wikijs-backup, a nightly Borg CronJob (03:30) that dumps the Wiki.js postgres
DB to the Storage Box — the content lives in git-storage, but comments, local
accounts and the search index do not, so the runtime state gets its own backup.
Mirrors authentik-backup (DB-only), reusing synapse-backup-credentials and
-known-hosts with a separate repo path; NetworkPolicy lets the backup pod reach
wikijs-postgres. Also flip tocPosition to right so the page TOC sits beside the
content instead of in the left rail.
Idempotent GraphQL bootstrap job (verified live): /finalize with the random SOPS
admin, then the OIDC strategy and the groups 'authentik Admins' (full) and
'wiki-anwender' (read /anwender), Guests locked. Replaces the manual setup wizard
entirely. OIDC client_id/secret in a SOPS secret; NetworkPolicy lets the job reach
wikijs. Script as a fixed-name ConfigMap; re-run = delete the Job.
Wiki.js + a dedicated Postgres + public ingress for wiki.axion1337.chat, added to
the production kustomization so Flux applies it. Both pods intentionally wait on
the SOPS secret `wikijs-postgres-secret` (username/password) until sorb creates it
— same loud-but-visible pattern as the concierge bot. NetworkPolicy: Traefik ->
wikijs:http, wikijs -> wikijs-postgres:postgres (ingress-only, named container
ports). Ingress/Cert mirror the authentik pattern (letsencrypt-prod). Native OIDC
login (#0049, guide 10) and git-storage are configured post-start. All manifests
validated server-side (kubectl --dry-run=server).
Synapse's own media-scanning module (Issue #19) can never see E2EE
attachment content - a structural limit, not a bug. This adds a small
standalone HTTP wrapper around the same ClamAV instance, reachable
from browser JS at /_scan, so the ThreadNet-Web client fork can scan
plaintext both before encrypting/uploading and after downloading/
decrypting - covering both directions regardless of room encryption.
Auth via Synapse's own /whoami endpoint, no separate auth system.
Deploys ClamAV and a small stdlib-only Synapse spam-checker module
implementing check_media_file_for_spam over clamd's INSTREAM protocol.
Unlike the originally-considered matrix-content-scanner proxy (which
needs client-side cooperation neither Element Web nor Element X
provide), this hooks Synapse's own module API directly - transparent
to every client for unencrypted media. No custom Synapse image needed:
the module is mounted via a ConfigMap onto PYTHONPATH using the ESS
chart's extraVolumes/extraVolumeMounts/extraEnv support. Fails open on
scanner errors so a ClamAV outage can't block all uploads.
Bot account registered via mas-cli, compatibility token issued and
stored via sops. Adds an explicit NetworkPolicy allow-rule since
Draupnir needs to reach Synapse's client-server API in-namespace,
which the existing default-deny policy would otherwise silently block.
Closes issues #6 and #15 (treated jointly, per combined analysis in their
comments). Two CronJobs, one per namespace, each pushing to its own Borg
repo on the newly booked Storage Box (u641795.your-storagebox.de:23, BX11
1TB) with its own repo passphrase - a leaked passphrase for one doesn't
expose the other:
- matrix: synapse-backup dumps the synapse + matrixauthenticationservice
DBs (shared postgres, existing chart-generated POSTGRES_ADMIN_PASSWORD)
and the Synapse media_store PVC (read-only mount)
- authentik: authentik-backup dumps the authentik DB (existing
authentik-credentials pg-password)
Custom image (rohana.axion1337.de/sorb/axion-backup:v1, postgres:17-alpine
+ borgbackup + openssh-client - matches the live Postgres major version
exactly, unlike Alpine's stock postgresql16-client) pushed as a public
package, same pattern as the existing threadnet-web image (no
imagePullSecret needed).
SSH host key pinned via known_hosts ConfigMap (captured via ssh-keyscan
ahead of time) rather than trusting on first connect in an unattended job.
Retention: 7 daily / 4 weekly / 6 monthly via borg prune. Schedule: 03:00
and 03:15, offset to avoid resource contention.
NetworkPolicy: added each backup job's pod as an allowed source to its
namespace's existing postgres ingress rule (matrix's allow-ingress-postgres,
authentik's allow-ingress-authentik-postgresql from #37). Egress already
unrestricted in both namespaces, so no change needed for the outbound SSH
connection to the Storage Box.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Second port-config incident from this rollout: synapse calls MAS's
oauth2/introspect endpoint on port 8080 (not 8081 as assumed) for
every single authenticated request (sync, pushrules, capabilities,
etc). The rule only allowed kube-system (Traefik) on 8080, so every
authenticated API call failed with 503 "Unable to introspect the
access token" - all clients showed a lost connection. Patched live
immediately, this commit brings Git back in sync with that patch.
Ingress-only default-deny plus per-component allow rules, using named
container ports throughout (not Service ports) after the authentik
port-mismatch incident. Traefik -> element-web/element-admin/haproxy
(fronts both matrix.axion1337.chat and well-known)/MAS; synapse <-
haproxy+MAS; postgres <- synapse+MAS; RTC signalling via Traefik, RTC
media NodePorts left open to the internet by design; monitoring scrape
allows for synapse/postgres/rtc-sfu metrics; ACME solver allow. coturn
(hostNetwork) needs nothing - NetworkPolicy doesn't apply to it. Part
of issue #10.
Thore CimbalandClaude Opus 5