chore(coturn): automated TURN shared-secret rotation #46
@@ -0,0 +1,331 @@
|
|||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This is a **GitOps-based Kubernetes deployment** of **Element Server Suite (ESS Community v26.4.0)**, a complete Matrix homeserver stack. The repository contains Infrastructure-as-Code using **FluxCD** for GitOps synchronization, with encryption (SOPS/age), service mesh (Traefik), certificate management (Cert-Manager), and auxiliary services like Authentik, TURN/coturn, Draupnir (moderation), ClamAV (content scanning), and Grafana monitoring.
|
||||||
|
|
||||||
|
**Key Stack Components:**
|
||||||
|
- **K3s**: Lightweight Kubernetes distribution running on Hetzner Cloud
|
||||||
|
- **FluxCD**: GitOps controller that watches this repository and auto-syncs changes
|
||||||
|
- **ESS (Matrix Umbrella Chart v26.4.0)**: Synapse, MAS (Matrix Authentication Service), ElementWeb, MatrixRTC
|
||||||
|
- **Authentik**: OIDC-based identity provider for centralized authentication, deployed both via HelmRelease and declarative Blueprints (`apps/authentik/authentik-blueprints.yaml`) for flows/OIDC-provider config that would otherwise only exist as manual admin-UI clicks
|
||||||
|
- **Traefik**: Ingress controller (built into K3s) for routing HTTP/HTTPS traffic
|
||||||
|
- **Cert-Manager**: Automatic TLS certificate provisioning from Let's Encrypt
|
||||||
|
- **SOPS + age**: Transparent encryption/decryption of secrets in Git
|
||||||
|
- **Monitoring**: Grafana Alloy (agent), Prometheus (metrics), Loki (logs)
|
||||||
|
- **coturn**: TURN/STUN server for WebRTC audio/video calls, with monthly automated shared-secret rotation via CronJob + PR workflow
|
||||||
|
- **Draupnir**: Matrix moderation bot (community successor to Mjolnir), ban lists/policy rooms
|
||||||
|
- **ClamAV**: Content scanning — a Synapse module for unencrypted-room uploads, plus a standalone `clamav-http-scanner` service that a patched Element Web client (ThreadNet-Web) calls both on send and on receive, extending coverage to encrypted rooms/DMs
|
||||||
|
- **NetworkPolicies**: default-deny-with-explicit-allow across `matrix` and `authentik` namespaces
|
||||||
|
- **`host-config/`**: the one part of this repo that is deliberately **not** managed by Flux/GitOps — see "Host-Level (non-GitOps) Changes" below
|
||||||
|
|
||||||
|
## Repository Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
gitops/
|
||||||
|
├── clusters/matrix/ # Flux GitRepository definition; entry point for reconciliation
|
||||||
|
├── apps/
|
||||||
|
│ ├── base/
|
||||||
|
│ │ ├── infra/ # Core infrastructure (Cert-Manager, Namespaces, etc.)
|
||||||
|
│ │ └── matrix/ # HelmRepository definition for ESS OCI chart
|
||||||
|
│ ├── production/ # Main ESS deployment
|
||||||
|
│ │ ├── element-server-suite.yaml # HelmRelease (ESS chart v26.4.0)
|
||||||
|
│ │ ├── custom-configs/ # Overrides & custom configurations
|
||||||
|
│ │ │ ├── synapse-values.yaml # Synapse customizations (ConfigMap)
|
||||||
|
│ │ │ ├── element-values.yaml # ElementWeb customizations (ConfigMap)
|
||||||
|
│ │ │ └── mas-secret.yaml # MAS secrets (encrypted with SOPS)
|
||||||
|
│ │ ├── cert-issuer.yaml # Let's Encrypt ClusterIssuer
|
||||||
|
│ │ ├── apex-ingress.yaml # Apex-domain IngressRoutes (Element Web, /_scan, etc.)
|
||||||
|
│ │ ├── matrix-postgres-auth.yaml # PostgreSQL credentials
|
||||||
|
│ │ ├── coturn.yaml / coturn-secret.yaml / synapse-turn-secret.yaml
|
||||||
|
│ │ ├── turn-secret-rotation.yaml # Monthly CronJob, rotates coturn shared secret via PR
|
||||||
|
│ │ ├── draupnir.yaml / draupnir-pvc.yaml / draupnir-secret.yaml
|
||||||
|
│ │ ├── clamav.yaml / clamav-pvc.yaml / clamav_spam_checker.py # Synapse-side scan module
|
||||||
|
│ │ ├── clamav-http-scanner.py / -Dockerfile / .yaml # Client-side scan service
|
||||||
|
│ │ ├── synapse-backup.yaml / synapse-backup-secret.yaml
|
||||||
|
│ │ └── networkpolicy.yaml # Default-deny + explicit allow rules
|
||||||
|
│ ├── authentik/ # Identity Provider (separate namespace)
|
||||||
|
│ │ ├── authentik.yaml # HelmRelease
|
||||||
|
│ │ ├── authentik-blueprints.yaml # Flows/OIDC-provider as declarative code
|
||||||
|
│ │ ├── helm-repo.yaml # HelmRepository source
|
||||||
|
│ │ ├── ingress.yaml # Ingress route
|
||||||
|
│ │ ├── networkpolicy.yaml
|
||||||
|
│ │ └── authentik-secret.yaml # Secrets (admin password, OIDC client secret, etc.)
|
||||||
|
│ └── monitoring/ # Observability (Alloy, kube-state-metrics, node-exporter)
|
||||||
|
│ ├── alloy-config.yaml # Grafana Alloy configuration
|
||||||
|
│ └── kube-state-metrics.yaml # K8s metrics exporter
|
||||||
|
├── host-config/ # Host-level (non-GitOps) config, see below
|
||||||
|
│ └── maintenance-notify/ # systemd timer: pre-update mail/Matrix notifications (Issue #24)
|
||||||
|
├── .sops.yaml # SOPS encryption rules (age key definition)
|
||||||
|
├── scripts/
|
||||||
|
│ ├── install-hooks.sh # Installs git hooks for ConfigMap auto-tracking
|
||||||
|
│ └── hooks/ # Git hooks (pre-commit, post-commit, etc.)
|
||||||
|
└── docs/
|
||||||
|
├── README.md # Main deployment guide
|
||||||
|
├── TASKS.md # Task list & milestones (backlog itself lives in Gitea issues)
|
||||||
|
├── install.md # Installation instructions
|
||||||
|
├── ops-configmap-sync.md # ConfigMap syncing with git hooks
|
||||||
|
└── deployment-guides/ # Detailed guides for specific components (01-07)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Host-Level (non-GitOps) Changes
|
||||||
|
|
||||||
|
Almost everything in this repo is reconciled by Flux. `host-config/` is the deliberate
|
||||||
|
exception: it holds scripts/systemd units meant to run **on the bare Hetzner host itself**
|
||||||
|
(not as a Kubernetes pod), for things Flux structurally can't reach — e.g. host package
|
||||||
|
management. There is no SOPS-on-host or Ansible-equivalent mechanism yet; deployment to the
|
||||||
|
host is manual (`scp`/SSH), and instance-specific values live in a config file on the host
|
||||||
|
(`/etc/<name>/config`), not hardcoded in the versioned script, so the pattern is reusable
|
||||||
|
across forks/other communities running this same stack. See
|
||||||
|
`docs/deployment-guides/07-host-maintenance-notifications.md` for the first (and so far only)
|
||||||
|
example of this pattern.
|
||||||
|
|
||||||
|
## Common Development Commands
|
||||||
|
|
||||||
|
### Flux / GitOps Synchronization
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Force immediate reconciliation (don't wait for 10-min auto-sync)
|
||||||
|
flux reconcile kustomization flux-system --with-source
|
||||||
|
flux reconcile kustomization production-apps --with-source
|
||||||
|
|
||||||
|
# Check reconciliation status
|
||||||
|
flux get kustomizations -A
|
||||||
|
flux get helmreleases -A
|
||||||
|
|
||||||
|
# View Flux logs
|
||||||
|
kubectl logs -n flux-system deployment/source-controller -f
|
||||||
|
kubectl logs -n flux-system deployment/helm-controller -f
|
||||||
|
```
|
||||||
|
|
||||||
|
### Kubernetes Cluster Status
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check pod health in Matrix namespace
|
||||||
|
kubectl get pods -n matrix
|
||||||
|
kubectl get pods -n authentik
|
||||||
|
kubectl get pods -n monitoring
|
||||||
|
|
||||||
|
# Detailed pod inspection
|
||||||
|
kubectl describe pod <pod-name> -n matrix
|
||||||
|
kubectl logs <pod-name> -n matrix -f
|
||||||
|
|
||||||
|
# Check all services and ingresses
|
||||||
|
kubectl get svc -n matrix
|
||||||
|
kubectl get ingress -n matrix
|
||||||
|
```
|
||||||
|
|
||||||
|
### Certificate Management (Let's Encrypt / Cert-Manager)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# View certificate status
|
||||||
|
kubectl get certificate -n matrix
|
||||||
|
kubectl get certificaterequest -n matrix
|
||||||
|
kubectl get challenges -n matrix
|
||||||
|
|
||||||
|
# Debug failed certificate issuance
|
||||||
|
kubectl describe challenge <challenge-name> -n matrix
|
||||||
|
kubectl logs -n cert-manager deployment/cert-manager -f
|
||||||
|
|
||||||
|
# Inspect the issued certificate
|
||||||
|
kubectl get secret <cert-secret-name> -n matrix -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -text -noout
|
||||||
|
```
|
||||||
|
|
||||||
|
### SOPS Secret Editing
|
||||||
|
|
||||||
|
SOPS transparently encrypts/decrypts secrets using the `age` key specified in `.sops.yaml`. The environment variable `SOPS_AGE_KEY_FILE` must point to your age private key.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Edit an encrypted secret (decrypted for editing, re-encrypted on save)
|
||||||
|
sops apps/production/custom-configs/mas-secret.yaml
|
||||||
|
sops apps/authentik/authentik-secret.yaml
|
||||||
|
|
||||||
|
# Create a new secret file
|
||||||
|
sops -i --encrypted-regex '^(data|stringData)$' --input-type yaml --output-type yaml new-secret.yaml
|
||||||
|
|
||||||
|
# Decrypt to view
|
||||||
|
sops -d apps/production/custom-configs/mas-secret.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
Ensure `~/.age/keys.txt` exists and contains your age private key. See `.devcontainer/devcontainer.json` for setup details.
|
||||||
|
|
||||||
|
### Helm Chart Inspection
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# List installed charts
|
||||||
|
helm list -n matrix
|
||||||
|
helm list -n authentik
|
||||||
|
|
||||||
|
# View rendered chart values
|
||||||
|
helm get values matrix-stack -n matrix
|
||||||
|
helm get manifest matrix-stack -n matrix | less
|
||||||
|
```
|
||||||
|
|
||||||
|
### Useful kubectl Shortcuts
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Port-forward to access services locally
|
||||||
|
kubectl port-forward -n matrix svc/synapse 8008:8008
|
||||||
|
|
||||||
|
# Execute command inside pod (for debugging)
|
||||||
|
kubectl exec -it <pod-name> -n matrix -- bash
|
||||||
|
|
||||||
|
# Stream logs from multiple pods
|
||||||
|
kubectl logs -n matrix -l app=synapse -f
|
||||||
|
|
||||||
|
# Bootstrap a service/bot account via MAS (no registration_shared_secret in this stack)
|
||||||
|
kubectl exec -it -n matrix deploy/matrix-stack-matrix-authentication-service -- \
|
||||||
|
mas-cli manage register-user <name> --yes
|
||||||
|
kubectl exec -it -n matrix deploy/matrix-stack-matrix-authentication-service -- \
|
||||||
|
mas-cli manage issue-compatibility-token <name>
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture & Key Concepts
|
||||||
|
|
||||||
|
### FluxCD Reconciliation Flow
|
||||||
|
|
||||||
|
1. **Flux watches** `clusters/matrix/` for a FluxRepository resource pointing to this Git repo
|
||||||
|
2. **Kustomization stages** pull in configurations in order:
|
||||||
|
- `flux-system` (FluxCD itself)
|
||||||
|
- `infra-apps` (Namespaces, RBAC, Cert-Manager, HelmRepository sources)
|
||||||
|
- `production-apps` (Main ESS deployment and related services)
|
||||||
|
3. **HelmReleases** specify which charts to install and what values to use
|
||||||
|
4. **ConfigMaps/Secrets** provide values from files in the repo (e.g., custom Synapse config)
|
||||||
|
5. **Flux auto-reconciles** every 10 minutes, or immediately if Git changes are detected
|
||||||
|
|
||||||
|
### Element Server Suite (ESS) Chart Constraints
|
||||||
|
|
||||||
|
The ESS Helm chart (v26.4.0) has strict validation and specific quirks:
|
||||||
|
|
||||||
|
- **No `config:` blocks for core components** — use ConfigMap overrides instead
|
||||||
|
- **`serverName` must be at root level**, not nested under `synapse`
|
||||||
|
- **TLS in Ingress blocks is forbidden** — use `certManager: true` at root to auto-manage certificates
|
||||||
|
- **`camelCase` for component names**: `elementWeb`, `synapseAdmin`, `matrixAuthenticationService`, etc.
|
||||||
|
- **OCI HelmRepository only** — the chart is distributed via `oci://ghcr.io/element-hq/ess-helm`, not HTTP
|
||||||
|
- **Values must pass JSON schema validation** — invalid configs will cause reconciliation failures with cryptic schema errors
|
||||||
|
|
||||||
|
### NetworkPolicy Convention
|
||||||
|
|
||||||
|
Default-deny-with-explicit-allow across `matrix` and `authentik` namespaces
|
||||||
|
(`apps/production/networkpolicy.yaml`, `apps/authentik/networkpolicy.yaml`). Every new pod
|
||||||
|
needs its own explicit ingress-allow rule; NetworkPolicy matches on named **container ports**,
|
||||||
|
not Service ports — a frequent source of live incidents when a new component is added (wrong
|
||||||
|
port number/name silently blocks all traffic to it).
|
||||||
|
|
||||||
|
### Known Issues & Workarounds
|
||||||
|
|
||||||
|
**Issue: Let's Encrypt ACME Race Condition (Error 403 Order's status is processing)**
|
||||||
|
- Symptom: Certificate provisioning hangs when `elementWeb` and `wellKnownDelegation` are both enabled on the same domain
|
||||||
|
- Cause: Both request certificates for the same domain simultaneously; Let's Encrypt rejects concurrent requests
|
||||||
|
- Fix: Set `wellKnownDelegation: enabled: false` and serve `.well-known/matrix/server` via a separate Ingress route or static file
|
||||||
|
|
||||||
|
**Issue: HelmChart not ready / stat no such file or directory**
|
||||||
|
- Cause: Attempting to use a GitRepository source for the ESS chart (it has sub-charts that don't render correctly)
|
||||||
|
- Fix: Use the OCI HelmRepository source (`oci://ghcr.io/element-hq/ess-helm`) instead
|
||||||
|
|
||||||
|
**Issue: Certificate validation failures (No resources found)**
|
||||||
|
- Cause: Manual Kustomize patches conflict with the Helm chart's built-in certificate management
|
||||||
|
- Fix: Remove manual patches; rely on `certManager: true` at the root level of HelmRelease values
|
||||||
|
|
||||||
|
**Issue: Synapse module can't use asyncio**
|
||||||
|
- Cause: Synapse runs on Twisted's reactor, not a running asyncio event loop — `asyncio.open_connection`/`asyncio.wait_for` inside a Synapse module (e.g. `clamav_spam_checker.py`) fail immediately with `RuntimeError: no running event loop`, and can silently trigger a fail-open path instead of an obvious crash
|
||||||
|
- Fix: use `twisted.internet.reactor`/`HostnameEndpoint`/`connectProtocol` + a custom `Protocol` subclass; Twisted `Deferred`s are natively awaitable from `async def` inside Synapse. Standalone processes outside Synapse (e.g. `clamav-http-scanner.py`) don't have this constraint and can use plain sockets/asyncio.
|
||||||
|
|
||||||
|
### SOPS Encryption & Key Management
|
||||||
|
|
||||||
|
- `.sops.yaml` defines encryption rules (currently using `age` keys)
|
||||||
|
- Secrets matching the regex in `.sops.yaml` are automatically encrypted when committed
|
||||||
|
- The age private key (`~/.age/keys.txt`) must be available in your environment for decryption
|
||||||
|
- In the cluster, Flux decrypts secrets "on the fly" using a secret stored in `flux-system` namespace
|
||||||
|
|
||||||
|
To rotate SOPS keys:
|
||||||
|
```bash
|
||||||
|
# Regenerate and re-encrypt all secrets
|
||||||
|
sops updatekeys -y apps/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Development Workflow
|
||||||
|
|
||||||
|
### Before Making Changes
|
||||||
|
|
||||||
|
1. **Understand dependencies** — check `kustomization.yaml` files to see the order of resource creation
|
||||||
|
2. **Verify chart schema** — review ESS chart documentation for constraints on the version being used
|
||||||
|
3. **Test locally if possible** — use `kubectl` port-forwards to verify connectivity before pushing changes
|
||||||
|
|
||||||
|
### Making Changes
|
||||||
|
|
||||||
|
1. **Edit ConfigMap files directly** — for non-secret customizations (Synapse config, Element Web themes, etc.)
|
||||||
|
- Changes are auto-tracked by git hooks installed via `./scripts/install-hooks.sh`
|
||||||
|
2. **Edit secrets with SOPS** — `sops` transparently decrypts/re-encrypts on save
|
||||||
|
3. **Update HelmRelease values** — modify the `values` section in `element-server-suite.yaml` or reference ConfigMap sources
|
||||||
|
|
||||||
|
### After Committing
|
||||||
|
|
||||||
|
1. **Flux auto-detects changes** within ~1 minute (or manually trigger with `flux reconcile kustomization production-apps`)
|
||||||
|
2. **Monitor reconciliation** — watch pod logs and Flux status for errors
|
||||||
|
3. **Test functionality** — verify services are accessible and functioning as expected
|
||||||
|
|
||||||
|
### Git Hooks
|
||||||
|
|
||||||
|
After cloning, run:
|
||||||
|
```bash
|
||||||
|
./scripts/install-hooks.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
This installs hooks that automatically commit ConfigMap changes to `.gitignore`-like tracking. See `docs/ops-configmap-sync.md` for details.
|
||||||
|
|
||||||
|
## Environment Setup
|
||||||
|
|
||||||
|
### Local Machine Prerequisites
|
||||||
|
|
||||||
|
- `kubectl` — cluster communication
|
||||||
|
- `flux` — GitOps CLI
|
||||||
|
- `helm` — chart inspection & debugging
|
||||||
|
- `sops` & `age` — secret management
|
||||||
|
- `git` — version control
|
||||||
|
- age key file at `~/.age/keys.txt` (request from team)
|
||||||
|
- kubeconfig at `~/.kube/config` (request from team)
|
||||||
|
|
||||||
|
### DevContainer (Recommended)
|
||||||
|
|
||||||
|
The `.devcontainer/` configuration provides a pre-configured environment:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# In VS Code: "Reopen in Container"
|
||||||
|
# Or manually:
|
||||||
|
docker build -t ess-devcontainer .devcontainer
|
||||||
|
docker run -it --rm \
|
||||||
|
-v ~/.kube:/home/vscode/.kube \
|
||||||
|
-v ~/.age:/home/vscode/.age \
|
||||||
|
-v ~/.ssh:/home/vscode/.ssh \
|
||||||
|
-v /var/run/docker.sock:/var/run/docker.sock \
|
||||||
|
ess-devcontainer
|
||||||
|
```
|
||||||
|
|
||||||
|
DevContainer includes:
|
||||||
|
- All required CLI tools (kubectl, flux, helm, sops, age, git, docker)
|
||||||
|
- VS Code extensions for YAML, Kubernetes, Helm
|
||||||
|
- Proper environment variables (`KUBECONFIG`, `SOPS_AGE_KEY_FILE`)
|
||||||
|
- Git hooks pre-installed
|
||||||
|
|
||||||
|
## Troubleshooting Checklist
|
||||||
|
|
||||||
|
- **Pod not starting?** → `kubectl describe pod <name> -n matrix` (check events)
|
||||||
|
- **Image pull failures?** → Check HelmRelease status: `kubectl get helmrelease -n matrix`
|
||||||
|
- **Secret not found?** → Verify SOPS decryption: `sops -d <secret.sops.yaml>` (must output valid YAML)
|
||||||
|
- **Certificate stuck?** → `kubectl describe certificate <name> -n matrix` (check for ACME errors)
|
||||||
|
- **Config validation error?** → Inspect HelmRelease status: `kubectl describe helmrelease <name> -n matrix` (JSON schema error message)
|
||||||
|
- **Cluster unreachable?** → Verify kubeconfig: `kubectl get nodes` (must connect to K3s)
|
||||||
|
- **NetworkPolicy blocking a new pod?** → Check it matches on container port name, not Service port
|
||||||
|
|
||||||
|
## Resources & References
|
||||||
|
|
||||||
|
- **README.md** — High-level overview and architecture
|
||||||
|
- **docs/TASKS.md** — Task backlog, milestones, and priority list (open backlog lives in [Gitea issues](https://rohana.axion1337.de/sorb/axion1337.chat-gitops/issues))
|
||||||
|
- **docs/deployment-guides/** — Detailed setup guides for specific components (01-07)
|
||||||
|
- **docs/ops-configmap-sync.md** — Git hook configuration and auto-sync behavior
|
||||||
|
- **ESS Chart Docs** — `https://github.com/element-hq/ess-helm` (official Helm chart repository)
|
||||||
|
- **FluxCD Docs** — `https://fluxcd.io/docs/` (GitOps reconciliation & Kustomization)
|
||||||
|
- **Matrix Spec** — `https://spec.matrix.org/` (Matrix protocol specification)
|
||||||
@@ -97,6 +97,15 @@ Die Suite ist ein "Umbrella Chart", das aus mehreren Microservices besteht:
|
|||||||
* **Monitoring**: Grafana Alloy sammelt Metriken/Logs, Remote-Write zu einem externen Prometheus/Loki-Stack.
|
* **Monitoring**: Grafana Alloy sammelt Metriken/Logs, Remote-Write zu einem externen Prometheus/Loki-Stack.
|
||||||
* **Backups**: Nächtliche, verschlüsselte & deduplizierte Borg-Backups (Postgres-Dumps + Synapse-`media_store`) zu einer Hetzner Storage Box, getrennt nach Namespace, mit eigenen Repos/Passphrasen.
|
* **Backups**: Nächtliche, verschlüsselte & deduplizierte Borg-Backups (Postgres-Dumps + Synapse-`media_store`) zu einer Hetzner Storage Box, getrennt nach Namespace, mit eigenen Repos/Passphrasen.
|
||||||
|
|
||||||
|
### Moderation & Content Scanning
|
||||||
|
|
||||||
|
* **Draupnir**: Moderationsbot (Community-Nachfolger von Mjolnir) für Ban-Listen/Policy-Rooms.
|
||||||
|
* **ClamAV**: Zwei Bausteine für unterschiedliche Räume - ein eigenes Synapse-Modul (`clamav_spam_checker.py`) scannt Uploads in unverschlüsselten Räumen; ein zusätzlicher, eigenständiger `clamav-http-scanner`-Dienst wird vom gepatchten Element-Web-Client (`sorb/threadnet-web`) sowohl beim Senden als auch beim Empfangen aufgerufen und deckt damit auch verschlüsselte Räume/DMs ab. Details: `docs/deployment-guides/06-moderation-content-scanning.md`.
|
||||||
|
|
||||||
|
### Host-Level (nicht-GitOps) Änderungen
|
||||||
|
|
||||||
|
* `host-config/` ist bewusst der einzige Teil dieses Repos, den Flux **nicht** verwaltet - Skripte/systemd-Units, die direkt auf dem nackten Hetzner-Host laufen (z.B. `unattended-upgrades`-Vorab-Benachrichtigungen), für Dinge, die strukturell außerhalb der Reichweite von Flux liegen. Deployment erfolgt manuell per SSH, instanzspezifische Werte liegen in einer Config-Datei auf dem Host, nicht im versionierten Skript. Details: `docs/deployment-guides/07-host-maintenance-notifications.md`.
|
||||||
|
|
||||||
-----
|
-----
|
||||||
|
|
||||||
## 3\. Aufbau des Repositories
|
## 3\. Aufbau des Repositories
|
||||||
@@ -271,8 +280,8 @@ sops apps/production/custom-configs/mas-secret.yaml
|
|||||||
|
|
||||||
## 7\. Weitere Ressourcen
|
## 7\. Weitere Ressourcen
|
||||||
|
|
||||||
* **`CLAUDE.md`** (`prod/`): Technische Referenz für KI-gestützte Arbeit an diesem Repo - Architektur, bekannte Chart-Quirks, Troubleshooting-Checkliste.
|
* **`CLAUDE.md`** (Repo-Root): Technische Referenz für KI-gestützte Arbeit an diesem Repo - Architektur, bekannte Chart-Quirks, Troubleshooting-Checkliste.
|
||||||
* **`docs/TASKS.md`**: Backlog-Pointer zu den [Gitea Issues](https://rohana.axion1337.de/sorb/axion1337.chat-gitops/issues) - Details werden nicht mehr doppelt gepflegt.
|
* **`docs/TASKS.md`**: Backlog-Pointer zu den [Gitea Issues](https://rohana.axion1337.de/sorb/axion1337.chat-gitops/issues) - Details werden nicht mehr doppelt gepflegt.
|
||||||
* **[Gitea Releases](https://rohana.axion1337.de/sorb/axion1337.chat-gitops/releases)**: Versionshistorie (SemVer, `vMAJOR.MINOR.PATCH` als Änderungsgrößen-Konvention, kein Kompatibilitätsvertrag - siehe [[00-TASKS]] Wiki für die Konvention).
|
* **[Gitea Releases](https://rohana.axion1337.de/sorb/axion1337.chat-gitops/releases)**: Versionshistorie (SemVer, `vMAJOR.MINOR.PATCH` als Änderungsgrößen-Konvention, kein Kompatibilitätsvertrag - siehe [[00-TASKS]] Wiki für die Konvention).
|
||||||
* **[Wiki](https://rohana.axion1337.de/sorb/axion1337.chat-gitops/wiki)**: Ausführliche Historie, Incident-Notizen, Setup-Guides pro Komponente.
|
* **[Wiki](https://rohana.axion1337.de/sorb/axion1337.chat-gitops/wiki)**: Ausführliche Historie, Incident-Notizen, Setup-Guides pro Komponente.
|
||||||
* **`docs/deployment-guides/`**: Detaillierte Guides für TURN-Server, Authentik, Monitoring, Element-Customization, Room-Policies.
|
* **`docs/deployment-guides/`**: Detaillierte Guides für TURN-Server, Authentik, Monitoring, Element-Customization, Room-Policies, Moderation & Content-Scanning, Host-Wartungsbenachrichtigungen.
|
||||||
+19
-41
@@ -115,30 +115,21 @@ verankert. Was in dieser Session erledigt wurde:
|
|||||||
- **Status**: NEXT
|
- **Status**: NEXT
|
||||||
|
|
||||||
### 🟠 **NEXT 1–2 WEEKS – HIGH**
|
### 🟠 **NEXT 1–2 WEEKS – HIGH**
|
||||||
1. **Authentik End-to-End Test**
|
1. ✅ **Authentik End-to-End Test** — erledigt als Teil von Issue #7 (Enrollment/Recovery/2FA,
|
||||||
- Test: Login flow Element → MAS → Authentik → Matrix User
|
2026-07-27), mit echten Test-Usern verifiziert. **Status**: COMPLETE
|
||||||
- Test: Password reset
|
|
||||||
- Create: Test invite links
|
|
||||||
- Est. Time: 2 hours
|
|
||||||
|
|
||||||
2. **Element Call Fork**
|
2. ✅ **Element Call Fork** — erledigt, Closes Issue #8 (2026-07-28), siehe
|
||||||
- Fork: element-hq/element-call
|
`docs/deployment-guides/04-element-customization.md` Kapitel 4. **Status**: COMPLETE
|
||||||
- Feature: Video/audio constraints parameters
|
|
||||||
- Integration: Synapse well-known config
|
|
||||||
- Est. Time: 2–3 days
|
|
||||||
|
|
||||||
3. **External PostgreSQL Migration**
|
3. **External PostgreSQL Migration** → [Issue #9](https://rohana.axion1337.de/sorb/axion1337.chat-gitops/issues/9)
|
||||||
- Decision: CloudNativePG vs. Hetzner Postgres
|
- Decision: CloudNativePG vs. Hetzner Postgres
|
||||||
- Setup: HA + Replication
|
- Setup: HA + Replication
|
||||||
- Migration: Move data from ESS embedded Postgres
|
- Migration: Move data from ESS embedded Postgres
|
||||||
- Testing: Verify all services work
|
- Testing: Verify all services work
|
||||||
- Est. Time: 1–2 days
|
- Est. Time: 1–2 days
|
||||||
|
|
||||||
4. **NetworkPolicies Deployment**
|
4. ✅ **NetworkPolicies Deployment** — erledigt, Closes Issue #10 (2026-07-28), Default-Deny
|
||||||
- Create: Default-Deny for `matrix` namespace
|
für `matrix`+`authentik` Namespaces. **Status**: COMPLETE
|
||||||
- Create: Allow rules (Synapse↔Postgres, MAS↔Postgres, Ingress→Web, etc.)
|
|
||||||
- Test: Ensure no service breakage
|
|
||||||
- Est. Time: 1 day
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -419,39 +410,25 @@ detaillierte Historie (aktuell bis v0.17.0) und [[00-TASKS]] im Wiki für die Ko
|
|||||||
|
|
||||||
## 📊 Prioritäts-Kategorien
|
## 📊 Prioritäts-Kategorien
|
||||||
|
|
||||||
### 🔴 CRITICAL (do immediately)
|
Alle Punkte hier sind als Gitea-Issues nachgehalten (Nummern siehe oben/Backlog-Verweis) - diese
|
||||||
- Hetzner Cloud Firewall setup
|
Kategorisierung ist nur eine grobe Einordnung, keine zweite Tracking-Quelle.
|
||||||
- Database backup strategy
|
|
||||||
- SSH hardening
|
|
||||||
|
|
||||||
### 🟠 HIGH (do within 1–2 weeks)
|
### 🟠 HIGH
|
||||||
- Authentik Stage 2 completion
|
- External PostgreSQL migration (#9)
|
||||||
- External PostgreSQL migration
|
|
||||||
- NetworkPolicies
|
|
||||||
- Element Call fork
|
|
||||||
|
|
||||||
### 🟡 MEDIUM (do within 1 month)
|
### 🟡 MEDIUM (do within 1 month)
|
||||||
- CrowdSec + Falco
|
- CrowdSec + Falco (#29, #30)
|
||||||
- Mjolnir bot
|
- Renovate/Trivy (#31, #32)
|
||||||
- Renovate/Trivy
|
- K3s API Hardening, auditd, Kernel Hardening, Lynis (#25-#28)
|
||||||
- PSA restricted mode
|
|
||||||
- Kernel hardening
|
|
||||||
|
|
||||||
### 🟢 LOW (nice-to-have, do if time allows)
|
### 🟢 LOW (nice-to-have, do if time allows)
|
||||||
- Content scanner (ClamAV)
|
|
||||||
- External-Secrets upgrade
|
- External-Secrets upgrade
|
||||||
- SSH port relocation
|
|
||||||
- Advanced federation rules
|
- Advanced federation rules
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📝 Notes & Decision Points
|
## 📝 Notes & Decision Points
|
||||||
|
|
||||||
### Authentik Stage 2 Blocker
|
|
||||||
⏳ **Waiting for**: User to manually configure Authentik OIDC Provider in Authentik Admin UI.
|
|
||||||
- Once done, provide Client ID + Secret
|
|
||||||
- Then: Commit Stage 2 MAS config
|
|
||||||
|
|
||||||
### Database: CloudNativePG vs. Hetzner Postgres
|
### Database: CloudNativePG vs. Hetzner Postgres
|
||||||
- **CloudNativePG**: Open-source, runs on K3S, full control
|
- **CloudNativePG**: Open-source, runs on K3S, full control
|
||||||
- **Hetzner Postgres**: Managed, backups included, less ops overhead
|
- **Hetzner Postgres**: Managed, backups included, less ops overhead
|
||||||
@@ -472,12 +449,13 @@ detaillierte Historie (aktuell bis v0.17.0) und [[00-TASKS]] im Wiki für die Ko
|
|||||||
|
|
||||||
- `docs/deployment-guides/README.md` – Overview
|
- `docs/deployment-guides/README.md` – Overview
|
||||||
- `docs/deployment-guides/01-turn-server-setup.md` – TURN
|
- `docs/deployment-guides/01-turn-server-setup.md` – TURN
|
||||||
- `docs/deployment-guides/02-authentik-identity-provider.md` – Authentik (Stage 1 + Stage 2 plan)
|
- `docs/deployment-guides/02-authentik-identity-provider.md` – Authentik (Stage 1+2 + Enrollment/Recovery/2FA)
|
||||||
- `docs/deployment-guides/03-monitoring-integration.md` – Monitoring
|
- `docs/deployment-guides/03-monitoring-integration.md` – Monitoring
|
||||||
- `docs/deployment-guides/04-element-customization.md` – Themes, Desktop
|
- `docs/deployment-guides/04-element-customization.md` – Themes, Desktop, Element Call Fork
|
||||||
- `docs/deployment-guides/05-room-policies.md` – Policies
|
- `docs/deployment-guides/05-room-policies.md` – Policies
|
||||||
|
- `docs/deployment-guides/06-moderation-content-scanning.md` – Draupnir, ClamAV Content Scanning
|
||||||
|
- `docs/deployment-guides/07-host-maintenance-notifications.md` – Host-Wartungsbenachrichtigungen
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**Last Updated**: 2026-05-14
|
**Last Updated**: 2026-07-30
|
||||||
**Next Review**: 2026-05-21
|
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
# Authentik als Identity Provider für Matrix
|
# Authentik als Identity Provider für Matrix
|
||||||
|
|
||||||
**Status**: ✅ Stage 1 Deployed (Authentik läuft)
|
**Status**: ✅ Deployed (Stage 1 + Stage 2 + Enrollment/Recovery/2FA, Closes Issue #7)
|
||||||
**Pending**: Stage 2 (MAS Integration)
|
|
||||||
**Domain**: `auth.axion1337.chat`
|
**Domain**: `auth.axion1337.chat`
|
||||||
|
|
||||||
## Überblick
|
## Überblick
|
||||||
@@ -41,5 +40,22 @@ Authentik = OIDC Provider für MAS → Zentrales Login + Einladungs-basierte Reg
|
|||||||
|
|
||||||
Authentik Admin → Flows & Stages → Invitations → Create
|
Authentik Admin → Flows & Stages → Invitations → Create
|
||||||
|
|
||||||
|
## Enrollment/Recovery/2FA Fix (2026-07-27, Issue #7)
|
||||||
|
|
||||||
|
Der `matrix-invitation`-Flow hatte nur 2 von 5 nötigen Stages (kein Write/Password/Login) -
|
||||||
|
Nutzer wurden nie in Synapse angelegt. Behoben und als Authentik Blueprint
|
||||||
|
(`apps/authentik/authentik-blueprints.yaml`) deklarativ ins Repo übernommen: vollständiger
|
||||||
|
`matrix-invitation`-Flow (Invite → Prompt → Write → Password → Login → Redirect), leerer
|
||||||
|
`matrix-recovery`-Flow ergänzt, `Brand.default_application` gesetzt. 2FA/Passkey-Selbst-
|
||||||
|
Einrichtung optional (`not_configured_action=skip`), auffindbar über
|
||||||
|
`axion1337.chat/docs/setup/security.html`. Details: siehe Wiki
|
||||||
|
[Authentik-OIDC.md](https://rohana.axion1337.de/sorb/axion1337.chat-gitops/wiki/Authentik-OIDC).
|
||||||
|
|
||||||
|
**Issue #13 geschlossen (2026-07-29)**: der ursprünglich hier vorgesehene direkte 2FA-Link auf
|
||||||
|
`account.axion1337.chat/account/` (per MAS Custom-Template-Override) wurde verworfen - live
|
||||||
|
geprüfte OIDC-Discovery zeigt, dass MAS keine 2FA/Passkey-Deep-Link-Action unterstützt. Jetzt
|
||||||
|
als Client-seitige Änderung nachgehalten:
|
||||||
|
[ThreadNet-Web#4](https://rohana.axion1337.de/sorb/ThreadNet-Web/issues/4).
|
||||||
|
|
||||||
---
|
---
|
||||||
**Weitere Details**: Siehe Kapitel 2 in diesem Projekt.
|
**Weitere Details**: Siehe Kapitel 2 in diesem Projekt.
|
||||||
|
|||||||
@@ -9,11 +9,12 @@ Die Implementierungen wurden in dieser Reihenfolge durchgeführt. Für neue Setu
|
|||||||
| # | Titel | Datei | Status | Zieldomäne |
|
| # | Titel | Datei | Status | Zieldomäne |
|
||||||
|---|-------|-------|--------|-----------|
|
|---|-------|-------|--------|-----------|
|
||||||
| 1 | TURN Server für WebRTC Video-Calls | `01-turn-server-setup.md` | ✅ Deployed | `turn.axion1337.chat` |
|
| 1 | TURN Server für WebRTC Video-Calls | `01-turn-server-setup.md` | ✅ Deployed | `turn.axion1337.chat` |
|
||||||
| 2 | Authentik als Identity Provider | `02-authentik-identity-provider.md` | ✅ Stage 1 Deployed | `auth.axion1337.chat` |
|
| 2 | Authentik als Identity Provider | `02-authentik-identity-provider.md` | ✅ Deployed | `auth.axion1337.chat` |
|
||||||
| 3 | Monitoring mit Alloy/Prometheus/Loki | `03-monitoring-integration.md` | ✅ Deployed | lokal (10.0.0.3) |
|
| 3 | Monitoring mit Alloy/Prometheus/Loki | `03-monitoring-integration.md` | ✅ Deployed | lokal (10.0.0.3) |
|
||||||
| 4 | Element Web Anpassung & Desktop-Apps | `04-element-customization.md` | ✅ Deployed | `axion1337.chat` |
|
| 4 | Element Web Anpassung & Desktop-Apps | `04-element-customization.md` | ✅ Deployed | `axion1337.chat` |
|
||||||
| 5 | Room Policies (Retention, Publication, Auto-Join) | `05-room-policies.md` | ✅ Deployed | Matrix Synapse |
|
| 5 | Room Policies (Retention, Publication, Auto-Join) | `05-room-policies.md` | ✅ Deployed | Matrix Synapse |
|
||||||
| 6 | Moderationsbot (Draupnir) & Content Scanning | `06-moderation-content-scanning.md` | ✅ Deployed | Matrix Synapse |
|
| 6 | Moderationsbot (Draupnir) & Content Scanning | `06-moderation-content-scanning.md` | ✅ Deployed | Matrix Synapse |
|
||||||
|
| 7 | Host-Wartungsbenachrichtigungen (unattended-upgrades) | `07-host-maintenance-notifications.md` | ✅ Deployed | Host-Ebene (kein K8s) |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -87,7 +88,13 @@ Custom Themes, Desktop-Setup-Scripts, Element Admin.
|
|||||||
Message Retention, Room Publication, Auto-Join Policies.
|
Message Retention, Room Publication, Auto-Join Policies.
|
||||||
|
|
||||||
### [06-moderation-content-scanning.md](06-moderation-content-scanning.md)
|
### [06-moderation-content-scanning.md](06-moderation-content-scanning.md)
|
||||||
Draupnir Moderationsbot (Bans, Policy-Listen), Content Scanner via eigenes Synapse-Modul (Issue #19) - beide live getestet.
|
Draupnir Moderationsbot (Bans, Policy-Listen), Content Scanner via eigenes Synapse-Modul für
|
||||||
|
unverschlüsselte Räume UND client-seitiges Scanning für verschlüsselte Räume/DMs (Issue #19 +
|
||||||
|
Erweiterung) - inkl. Electron/Desktop-Deckungslücke (Issue #44). Beide live getestet.
|
||||||
|
|
||||||
|
### [07-host-maintenance-notifications.md](07-host-maintenance-notifications.md)
|
||||||
|
Erster nicht-GitOps-verwalteter Mechanismus im Repo: systemd-Timer auf dem nackten Host meldet
|
||||||
|
per Mail + Matrix-Thread-Reply anstehende `unattended-upgrades`, bevor sie laufen (Issue #24).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -85,7 +85,7 @@ Dieser Ordner enthält detaillierte Troubleshooting- und Reparaturanleitungen f
|
|||||||
| Problem | Nutzer | Guide | Status |
|
| Problem | Nutzer | Guide | Status |
|
||||||
|---------|--------|-------|--------|
|
|---------|--------|-------|--------|
|
||||||
| Nur Standard Enrollment funktioniert | akadmin ✅ | - | Resolved |
|
| Nur Standard Enrollment funktioniert | akadmin ✅ | - | Resolved |
|
||||||
| User nur in Authentik, nicht in Synapse | Boje | `DIAGNOSTIK-AUTHENTIK-FLOW.md` | In Progress |
|
| User nur in Authentik, nicht in Synapse | Boje | `DIAGNOSTIK-AUTHENTIK-FLOW.md` | **Resolved (2026-07-27)** — identischer Root Cause wie bei Klaus (fehlende Write/Password/Login-Stages im `matrix-invitation`-Flow), behoben durch denselben Issue-#7-Fix. Nicht erneut mit Boje selbst nachgetestet, aber mit anderen Test-Usern (`clark`, `lucky`) end-to-end verifiziert - der zugrundeliegende Flow ist jetzt für jeden Nutzer korrekt. |
|
||||||
| Einladungslink-Fehler: "kein ausstehender benutzer" | Klaus | `AUTHENTIK-CREATE-INVITATION-FLOW.md` | **Fixed (2026-07-27)** — `matrix-invitation` Flow hatte nur Invite+Prompt Stage-Bindings, beide auf `order=0`. Write/Password/Login-Stages fehlten komplett. Live gefixt + als Blueprint (`apps/authentik/authentik-blueprints.yaml`) reproduzierbar gemacht. |
|
| Einladungslink-Fehler: "kein ausstehender benutzer" | Klaus | `AUTHENTIK-CREATE-INVITATION-FLOW.md` | **Fixed (2026-07-27)** — `matrix-invitation` Flow hatte nur Invite+Prompt Stage-Bindings, beide auf `order=0`. Write/Password/Login-Stages fehlten komplett. Live gefixt + als Blueprint (`apps/authentik/authentik-blueprints.yaml`) reproduzierbar gemacht. |
|
||||||
| OIDC-Integration unklar | General | `AUTHENTIK-FIX-TEMPLATE.md` | Reference |
|
| OIDC-Integration unklar | General | `AUTHENTIK-FIX-TEMPLATE.md` | Reference |
|
||||||
|
|
||||||
@@ -108,5 +108,5 @@ Dieser Ordner enthält detaillierte Troubleshooting- und Reparaturanleitungen f
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**Zuletzt aktualisiert**: 2026-05-18
|
**Zuletzt aktualisiert**: 2026-07-30
|
||||||
**Verfasser**: Claude Code + Thore
|
**Verfasser**: Claude Code + Thore
|
||||||
|
|||||||
Reference in New Issue
Block a user