sorb/axion1337.chat-gitops
1
0
Fork 0
Code Issues Pull Requests Packages Projects 1 Releases 24 Wiki Activity

fix(devcontainer): repair broken build and add docker socket permission fix

Browse Source 
The devcontainer could never actually be built successfully - verified by
building it from scratch for the first time in a while. Found and fixed six
issues:

- kubectl: apt.kubernetes.io was deprecated/shut down by Google in 2023,
  switched to the official successor repo pkgs.k8s.io
- docker-ce-cli: apt line hardcoded arch=amd64, breaking the build on
  Apple Silicon; resolved dynamically via dpkg --print-architecture
- useradd -G docker failed because only the Docker CLI (no daemon) is
  installed, so no package ever creates the docker group; added explicit
  groupadd
- oh-my-zsh install had a nested-quoting bug that made the RUN step fail;
  simplified to download-then-run instead of one nested `su -c "sh -c ..."`
- sops binary was hardcoded to linux.amd64, only working on arm64 by luck
  via Docker Desktop's QEMU emulation; resolved dynamically like docker-ce
- docker.sock was mounted but unusable (permission denied) since the
  container's docker group GID never matched the host socket's GID; added
  a root entrypoint (docker-init.sh) that reconciles this at container
  start, then drops to the vscode user via gosu

Also fixed two stale mas-secrets.sops.yaml references (actual filename is
mas-secret.yaml) in README.md and postCreateCommand.sh, set the vscode
user's default shell to zsh (oh-my-zsh was installed but never used by
default), and documented all of the above plus a build+run verification
snippet in README.md so this class of drift is caught before it goes
unnoticed again.

Verified end-to-end: cold `docker build --no-cache`, then a real container
run against the actual mounted kubeconfig, age key, and docker socket -
kubectl reaches the live cluster, sops decrypts a real secret, and docker
ps talks to the real daemon as the vscode user.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Thore Cimbal
2026-07-28 12:00:00 +00:00
co-authored by Claude Sonnet 5
parent 09aaf1b2b5
commit 6973a8dbc7
4 changed files with 92 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.devcontainer/Dockerfile
Regular → Executable
+28 -10
View File
@@ -16,11 +16,14 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
zsh \
sudo \
openssh-client \
gosu \
&& rm -rf /var/lib/apt/lists/*
# Install kubectl
RUN curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && \
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | tee /etc/apt/sources.list.d/kubernetes.list && \
# Install kubectl (apt.kubernetes.io was deprecated/shut down by Google in 2023;
# pkgs.k8s.io is the current community-owned repo, versioned per k8s minor release)
RUN mkdir -p /etc/apt/keyrings && \
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.34/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg && \
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.34/deb/ /" | tee /etc/apt/sources.list.d/kubernetes.list && \
apt-get update && apt-get install -y kubectl && \
rm -rf /var/lib/apt/lists/*
@@ -30,9 +33,10 @@ RUN curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | b
# Install Flux CLI
RUN curl -s https://fluxcd.io/install.sh | bash
# Install sops
RUN SOPS_VERSION=$(curl -s https://api.github.com/repos/getsops/sops/releases/latest | grep tag_name | cut -d '"' -f 4) && \
curl -sL -o /usr/local/bin/sops https://github.com/getsops/sops/releases/download/${SOPS_VERSION}/sops-${SOPS_VERSION}.linux.amd64 && \
# Install sops (arch resolved at build time, same reasoning as the Docker CLI step below)
RUN SOPS_ARCH=$(dpkg --print-architecture) && \
SOPS_VERSION=$(curl -s https://api.github.com/repos/getsops/sops/releases/latest | grep tag_name | cut -d '"' -f 4) && \
curl -sL -o /usr/local/bin/sops https://github.com/getsops/sops/releases/download/${SOPS_VERSION}/sops-${SOPS_VERSION}.linux.${SOPS_ARCH} && \
chmod +x /usr/local/bin/sops
# Install age
@@ -40,17 +44,31 @@ RUN apt-get update && apt-get install -y age && \
rm -rf /var/lib/apt/lists/*
# Install Docker CLI (for interacting with Docker daemon)
# arch is resolved at build time so this works on both amd64 (cloud/CI) and arm64 (Apple Silicon) hosts
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg && \
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null && \
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null && \
apt-get update && apt-get install -y docker-ce-cli && \
rm -rf /var/lib/apt/lists/*
# Create a non-root user 'vscode' for development
RUN useradd -m -s /bin/bash -G docker vscode && \
# groupadd is needed because only the Docker CLI (not the daemon) is installed above,
# so the 'docker' group is never created as a package side effect
RUN groupadd docker && \
useradd -m -s /bin/zsh -G docker vscode && \
echo "vscode ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers.d/vscode
# Install oh-my-zsh for better shell experience
RUN su - vscode -c "sh -c '$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)' '' --unattended"
RUN curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh -o /tmp/install-omz.sh && \
su - vscode -c "sh /tmp/install-omz.sh --unattended" && \
rm /tmp/install-omz.sh
# Entrypoint runs as root to reconcile the docker group's GID against the mounted
# socket (see docker-init.sh), then drops to 'vscode' for the actual session/command.
# Stays root-owned at the PID 1 level; VS Code's own `docker exec -u vscode` sessions
# and the entrypoint's `gosu vscode` both end up correctly grouped either way.
COPY docker-init.sh /usr/local/bin/docker-init.sh
RUN chmod +x /usr/local/bin/docker-init.sh
USER vscode
WORKDIR /workspace
ENTRYPOINT ["/usr/local/bin/docker-init.sh"]
CMD ["/bin/zsh"]
.devcontainer/README.md
Regular → Executable
+40 -2
View File
@@ -91,7 +91,7 @@ Der Container mounted `~/.age` automatisch. Setze die Umgebungsvariable:
```bash
# Im Container-Terminal (SOPS_AGE_KEY_FILE ist bereits automatisch gesetzt!)
# Jetzt kannst du Secrets bearbeiten (wird transparent ver-/entschlüsselt):
sops apps/production/custom-configs/mas-secrets.sops.yaml
sops apps/production/custom-configs/mas-secret.yaml
```
### Schritt 3: VSCode Integration (optional)
@@ -130,7 +130,7 @@ kubectl get pods -n matrix
flux get helmreleases -A
# Secrets bearbeiten (mit verschlüsselung)
sops apps/production/custom-configs/mas-secrets.sops.yaml
sops apps/production/custom-configs/mas-secret.yaml
# FluxCD Sync erzwingen
flux reconcile kustomization production-apps --with-source
@@ -193,6 +193,44 @@ Siehe `README.md` → **Issue 3**. Kurz:
- `wellKnownDelegation: enabled: false` setzen
- Oder `.well-known/matrix/server` manuell auf `elementWeb` weiterleiten
## ⚠️ Wartungshinweis: Warum dieser Container regelmäßig getestet werden muss
Der Dockerfile installiert mehrere Tools über externe apt-Repos und Install-Skripte
(`pkgs.k8s.io`, `download.docker.com`, GitHub-Releases, `fluxcd.io`/`ohmyzsh.sh`
Installer). **Diese Quellen sind nicht unter unserer Kontrolle und können jederzeit
brechen** — genau das ist am 2026-07-28 passiert: der Container konnte seit
Fertigstellung nie erfolgreich gebaut werden, ohne dass es jemand bemerkt hat, weil
niemand ihn zwischenzeitlich tatsächlich gebaut hat. Gefundene und behobene Probleme:
| # | Problem | Ursache | Fix |
|---|---------|---------|-----|
| 1 | `apt.kubernetes.io` → `404 Not Found` | Google hat das alte Kubernetes-apt-Repo 2023 abgeschaltet | Umgestellt auf das offizielle Nachfolge-Repo `pkgs.k8s.io` (versioniert pro k8s-Minor-Version, aktuell `v1.34`) |
| 2 | `docker-ce-cli` "has no installation candidate" auf Apple Silicon | Repo-Zeile hatte `arch=amd64` hartkodiert, Build lief aber auf arm64 | `arch=$(dpkg --print-architecture)` zur Build-Zeit ermitteln |
| 3 | `useradd: group 'docker' does not exist` | Nur die Docker-**CLI** wird installiert (kein Daemon), daher legt kein Paket die `docker`-Gruppe automatisch an | `groupadd docker` explizit vor `useradd` |
| 4 | oh-my-zsh-Install schlägt mit Quoting-Fehler fehl | Verschachtelte `sh -c '...'`-Anführungszeichen in einer Zeile | Install-Skript erst in eine Datei laden, dann sauber mit `su - vscode -c "sh /tmp/install-omz.sh --unattended"` ausführen |
| 5 | `sops`-Binary war hart auf `linux.amd64` gepinnt | Lief auf Apple Silicon nur zufällig per QEMU-Emulation von Docker Desktop mit, nicht nativ | Arch dynamisch über `dpkg --print-architecture` auflösen (`linux.arm64` / `linux.amd64`) |
| 6 | `docker.sock`-Zugriff im Container: `permission denied` | Der gemountete Host-Socket gehört (je nach Docker-Setup) einer Gruppe/GID, die im Container nicht existiert oder nicht der `docker`-Gruppe entspricht (auf Docker Desktop für Mac/Windows z.B. GID 0/root statt einer eigenen `docker`-Gruppe) | `docker-init.sh`: Root-Entrypoint gleicht beim Container-Start die GID der `docker`-Gruppe an den tatsächlich gemounteten Socket an (bzw. tritt der GID-Inhaber-Gruppe bei, falls die GID schon vergeben ist), wechselt danach per `gosu` zu `vscode` |
**Konsequenz für die Zukunft:** Vor jeder größeren Änderung an `.devcontainer/` (oder
mindestens vierteljährlich) einmal real bauen und laufen lassen:
```bash
docker build -f .devcontainer/Dockerfile -t ess-gitops-devcontainer-test .devcontainer
docker run --rm \
-v ~/.kube:/home/vscode/.kube \
-v ~/.age:/home/vscode/.age \
-v /var/run/docker.sock:/var/run/docker.sock \
ess-gitops-devcontainer-test bash -c '
kubectl version --client && helm version --short && flux --version && \
sops --version && age --version && docker version --format "{{.Server.Version}}" && \
id vscode
'
```
Wenn `docker version` hier den echten Server, nicht nur die Client-Version zeigt, und
`id vscode` die passende Docker-Gruppe/GID auflistet, funktioniert der Socket-Zugriff
tatsächlich — nicht nur der Build.
## 📚 Weitere Ressourcen
- [Dev Containers Docs](https://containers.dev)
.devcontainer/docker-init.sh
+23
View File
@@ -0,0 +1,23 @@
#!/bin/bash
# Runs as root at container start (before any `docker exec -u vscode` from VS Code).
# The docker.sock's GID is only known once the host socket is actually bind-mounted,
# so it can't be baked in at image build time - it must be reconciled here, at runtime.
set -e
if [ -S /var/run/docker.sock ]; then
SOCK_GID=$(stat -c '%g' /var/run/docker.sock)
CURRENT_GID=$(getent group docker | cut -d: -f3)
if [ -n "$SOCK_GID" ] && [ "$SOCK_GID" != "$CURRENT_GID" ]; then
EXISTING_GROUP=$(getent group "$SOCK_GID" | cut -d: -f1)
if [ -n "$EXISTING_GROUP" ]; then
# GID is already taken by another group (e.g. GID 0/root - Docker Desktop for
# Mac/Windows owns the socket this way inside its VM), so join that group
# instead of trying to reassign it to 'docker'.
usermod -aG "$EXISTING_GROUP" vscode
else
groupmod -g "$SOCK_GID" docker
fi
fi
fi
exec gosu vscode "$@"
.devcontainer/postCreateCommand.sh
+1 -1
View File
@@ -26,7 +26,7 @@ echo ""
echo "📚 Useful commands:"
echo " - kubectl get pods -n matrix (check pod status)"
echo " - flux get helmreleases -A (check helm releases)"
echo " - sops apps/production/custom-configs/mas-secrets.sops.yaml (edit secrets)"
echo " - sops apps/production/custom-configs/mas-secret.yaml (edit secrets)"
echo ""
echo "🔗 For kubeconfig setup:"
echo " - Copy your ~/.kube/config to access the cluster"