2026-10-20 16:35:56 +02:00
2026-10-18 19:13:26 +02:00
2026-10-18 19:21:12 +02:00
2026-10-19 20:09:53 +02:00
2026-10-18 19:11:52 +02:00
2026-11-13 15:33:33 +01:00
2026-10-20 16:03:32 +02:00
2026-10-20 15:39:50 +02:00
2026-10-18 17:58:31 +02:00
2026-10-20 15:45:18 +02:00
2026-10-20 15:47:57 +02:00
2026-10-20 15:50:34 +02:00
2026-10-18 18:39:59 +02:00
2026-10-20 15:51:03 +02:00
2026-10-20 15:53:22 +02:00
2026-10-20 15:54:07 +02:00
2026-10-14 08:47:43 +02:00
2026-11-11 17:14:41 +01:00
2026-10-20 16:29:20 +02:00
2026-10-20 16:22:39 +02:00
2026-10-18 19:06:40 +02:00
2026-10-20 16:29:20 +02:00
2026-10-20 16:35:56 +02:00
2026-10-18 19:14:07 +02:00

TapTap Homelab — Kubernetes on Proxmox

A self-hosted homelab built around a single-node K3s cluster running on NixOS inside a Proxmox VE VM. Routing is handled by an OPNsense VM, storage and backups by a TrueNAS VM, and the Kubernetes layer is delivered declaratively with ArgoCD, Kustomize and Helm.

Kubernetes Proxmox NixOS OPNsense TrueNAS ArgoCD Traefik Vault Cloudflare

Last verified: 2026-11-11 against the live cluster and host.


Contents


Overview

This repository contains the Kubernetes manifests for a personal homelab. The goal is to run useful self-hosted services while treating the whole stack as code: the host is provisioned and configured with a declarative NixOS flake, and the cluster is reconciled continuously by ArgoCD from Git.

The platform runs as a set of virtual machines on a single Proxmox VE host:

  • OPNsense — routing, firewall, DHCP, NAT and port forwarding.
  • TrueNAS — mirrored storage, NFS exports and the primary data copy.
  • NixOS / K3s — the Kubernetes node, host-level automation and Borg backups.

The cluster is intentionally small and single-node, but the delivery model is the same one used in larger environments: declarative configuration, Git as the source of truth, automated reconciliation, runtime secret injection, TLS automation and tested backups.

At a glance

Layer Technology Role
Virtualisation Proxmox VE Hosts the OPNsense, TrueNAS and K3s VMs
Routing / firewall OPNsense VM LAN gateway, DHCP, NAT, port forwarding
Storage TrueNAS VM Mirrored storage, NFS export
Host OS NixOS 26.05 Declarative, reproducible host configuration
Kubernetes K3s v1.35.4+k3s1 Single-node cluster with embedded etcd
Ingress Traefik 3.6.13 via K3s ServiceLB HTTP/HTTPS entry point on 192.168.1.119
GitOps ArgoCD v3.5.4 Continuous reconciliation from Gitea
Packaging Kustomize + Helm Plain YAML, Kustomize composition, rendered Helm charts
Secrets HashiCorp Vault + External Secrets Operator 2.12.0 Runtime secret injection
TLS cert-manager 1.13.2 + Let's Encrypt Automated certificates via Cloudflare DNS-01
Source control / CI Gitea + Gitea Actions runner Git hosting and CI job execution
Remote access Twingate Zero-trust access to private resources
Backups BorgBackup + rclone Weekly local backups plus offsite copy

Architecture

flowchart TB
    Internet((Internet))

    subgraph CF["Cloudflare"]
        DNS["Authoritative DNS + DDNS<br/>*.tap-tap.win"]
        PROXY["Reverse proxy<br/>homeassistant / vaultwarden / nginx2"]
    end

    subgraph PVE["Proxmox VE host"]
        OPN["OPNsense VM<br/>Router / Firewall / DHCP / NAT"]
        NAS["TrueNAS VM<br/>Mirrored storage<br/>192.168.1.110"]

        subgraph K3S["K3s VM — NixOS 26.05 — 192.168.1.119"]
            SVC["Traefik Ingress<br/>via K3s ServiceLB"]
            ARGO["ArgoCD"]
            APPS["Application workloads"]
            VAULT["HashiCorp Vault"]
            ESO["External Secrets Operator"]
            CERT["cert-manager"]

            ARGO -->|"reconciles"| APPS
            ESO -->|"reads secrets"| VAULT
            CERT -->|"issues TLS"| SVC
            SVC --> APPS
        end
    end

    TW["Twingate connector"]

    Internet -.->|"DNS lookup"| DNS
    Internet ==>|"direct HTTP / HTTPS<br/>all other services"| OPN
    Internet -->|"proxied HTTP / HTTPS<br/>homeassistant / vaultwarden / nginx2"| PROXY
    PROXY -->|"HTTP / HTTPS"| OPN
    OPN -->|"HTTP / HTTPS"| SVC
    NAS -->|"NFS"| K3S
    Internet --> TW
    TW --> K3S

Cloudflare is the authoritative DNS provider for every public hostname. It also reverse-proxies three specific services: Home Assistant, Vaultwarden and nginx2. All other public services are DNS-only: clients resolve the public A record and connect directly to the public IP, where OPNsense handles the inbound connection.

Traffic flow:

  1. Public DNS records live in Cloudflare and are kept current by cloudflare-ddns.
  2. For most services, the client connects directly to the public IP; OPNsense forwards HTTP/HTTPS to the K3s node at 192.168.1.119.
  3. Home Assistant, Vaultwarden and nginx2 are proxied through Cloudflare before reaching OPNsense.
  4. Traefik routes requests to the correct Kubernetes Service based on the ingress host.
  5. Twingate provides private, identity-aware access without exposing services publicly.
  6. TrueNAS exports storage over NFS to the Kubernetes node for Nextcloud and media workloads.

Infrastructure layers

Component Platform Address Responsibility
Proxmox VE host Host — Virtualisation for all infrastructure VMs
OPNsense VM on Proxmox LAN gateway Routing, firewall, DHCP, NAT, port forwarding
TrueNAS VM on Proxmox 192.168.1.110 Mirrored storage pool and NFS export
K3s node NixOS VM on Proxmox 192.168.1.119 Single-node Kubernetes, host automation, Borg repository

Proxmox VE

The Proxmox host is the virtualisation layer. Keeping routing, storage and compute in separate VMs makes each role independently configurable and snapshot-able, while still fitting the whole platform on one physical machine.

OPNsense

OPNsense is the LAN router and firewall. It provides:

  • Routing between the LAN and the internet.
  • DHCP for the local network.
  • NAT and port forwarding for HTTP/HTTPS to the Kubernetes ingress.
  • A natural segmentation point between the public edge and internal services.

TrueNAS

TrueNAS provides the mirrored storage pool and exports the Nextcloud data set over NFS:

192.168.1.110:/mnt/MainMirroredStorage/nextcloud

The K3s node mounts it at /root/kubernetes/nextcloud/data/mount with systemd automount and a 10-minute idle timeout, so the host can boot even when the NAS is unavailable.

NixOS / K3s

The Kubernetes node is a NixOS VM managed by a separate NixOS flake configuration. The server profile configures:

  • K3s in server mode with clusterInit = true.
  • Node IP and TLS SAN 192.168.1.119.
  • Firewall rules for Kubernetes, HTTP/HTTPS, Matter, Thread and mDNS.
  • The TrueNAS NFS mount and the external backup disk.
  • Weekly Borg backups and the Nextcloud cron job.

The live node reports NixOS 26.05 and K3s v1.35.4+k3s1 with embedded etcd.

Network and edge

Item Value
LAN 192.168.1.0/24
Kubernetes node 192.168.1.119
TrueNAS 192.168.1.110
Public domain *.tap-tap.win
Ingress controller Traefik 3.6.13
LoadBalancer implementation K3s ServiceLB (no MetalLB)
Ingress address 192.168.1.119 on ports 80/443
Traefik NodePorts 30123 (HTTP), 32467 (HTTPS)
Cloudflare-proxied hostnames homeassistant.tap-tap.win, vaultwarden.tap-tap.win, nginx2.tap-tap.win

Load balancing: MetalLB was previously used and has been removed. K3s built-in ServiceLB now implements LoadBalancer services by binding the node IP. MetalLB CRDs remain in the cluster as leftovers from the previous installation, but there is no MetalLB namespace, controller or address pool.

DNS and exposure: Cloudflare is the authoritative DNS provider for all public names, and cloudflare-ddns keeps the A records pointed at the current public address. It also reverse-proxies three hostnames: Home Assistant, Vaultwarden and nginx2. All other services are DNS-only — clients resolve the public address and connect directly to the public IP, where OPNsense forwards HTTP/HTTPS to Traefik. Twingate is used for zero-trust access to private services. A cloudflared manifest exists in the repository, but the connector is not deployed in the cluster.

TLS: cert-manager issues certificates from Let's Encrypt using the Cloudflare DNS-01 challenge. Certificates are currently ready for ArgoCD, Gitea, Vault, Home Assistant, Jellyfin, Nextcloud, nginx2, OnlyOffice and Plex. The nginx demo (nginx.tap-tap.win) and Vaultwarden remain HTTP-only at the ingress level; Vaultwarden, Home Assistant and nginx2 are the hostnames proxied through Cloudflare.

Internal DNS: Kubernetes uses CoreDNS for cluster service discovery. Public names resolve through Cloudflare, while TrueNAS provides persistent storage over NFS.

Platform components

Component Namespace Version / image Purpose
ArgoCD argocd v3.5.4 GitOps reconciliation and application management
Traefik kube-system 3.6.13 Ingress controller and TLS termination
K3s ServiceLB kube-system K3s built-in LoadBalancer implementation on the node IP
cert-manager cert-manager 1.13.2 Automated Let's Encrypt certificates
HashiCorp Vault hashicorp-vault chart 0.34.1 Central secret storage (KV v2)
External Secrets Operator external-secrets 2.12.0 Syncs Vault secrets into Kubernetes Secrets
cloudflare-ddns cloudflare-ddns favonia/cloudflare-ddns Keeps public DNS records current
Gitea gitea gitea/gitea Git hosting for this repository
Gitea runner gitea gitea/runner + Docker-in-Docker CI job execution
metrics-server kube-system 0.8.1 Resource metrics for the Kubernetes API
local-path provisioner kube-system v0.0.35 Local persistent volumes
CoreDNS kube-system 1.14.2 Cluster DNS

local-path is the default StorageClass. local-persistent-storage is a retained local-path class used by Vault. Most stateful services use hostPath volumes on the K3s VM or the TrueNAS NFS mount.

Infrastructure as Code

The infrastructure is split into two declarative layers.

Host layer — NixOS flake

The NixOS host configuration is a declarative flake with separate profiles for laptops, an AI lab machine and the server node. Inputs target specific Nixpkgs release channels (26.05, 25.11 and unstable) plus nixos-hardware, nixos-wsl and nixos-generators.

The server profile is the source of truth for:

  • K3s server configuration and firewall rules.
  • NFS mounts and backup disk mounts.
  • Borg backup jobs.
  • Kernel modules, timezone, SSH and base packages.
  • The Nextcloud OCC cron job.

Typical host change:

sudo nixos-rebuild switch --flake .#server

Provisioning is reproducible too: the flake can build a custom NixOS install ISO, and an automated install script partitions the disks, clones the configuration repositories and runs nixos-install. Rebuilding the node does not rely on hand-edited files on the machine.

Cluster layer — GitOps

Kubernetes resources use plain YAML, Kustomize and Helm charts. ArgoCD runs an ApplicationSet that watches a fixed list of directories in this repository and creates one ArgoCD Application per directory.

Key properties:

  • Automated sync from the Gitea repository.
  • CreateNamespace=true for new applications.
  • ServerSideApply=true to avoid apply conflicts.
  • Helm charts rendered through Kustomize with --enable-helm.
  • A root application also tracks the repository root.

Adding a service follows a predictable flow:

  1. Create a directory with a kustomization.yaml and the required resources.
  2. Add the directory to the ApplicationSet path list.
  3. Commit and push to Gitea.
  4. ArgoCD detects the change and reconciles the cluster.

A manual render/apply for debugging looks like:

kustomize build <directory> --enable-helm | kubectl apply --server-side -f -

Separation of concerns

Layer Managed by Examples
Host NixOS flake K3s, NFS mounts, firewall, Borg
Platform ArgoCD / Helm / Kustomize Traefik, cert-manager, Vault, ESO, ArgoCD
Applications Kustomize Nextcloud, Plex, Jellyfin, OnlyOffice, Home Assistant

Repository layout

Path Purpose
argocd/ ArgoCD installation, root application and ApplicationSet
cert-manager/ cert-manager, Let's Encrypt ClusterIssuer, Cloudflare token sync
external-secrets/ External Secrets Operator and Vault ClusterSecretStore
hashicorp-vault/ Vault Helm chart, UI/ingress and storage configuration
cloudflare-ddns/ Dynamic DNS updater for public hostnames
gitea/ Gitea server and CI runner
homeassistant/ Home Assistant, Mosquitto, Zigbee2MQTT, Matter and OTBR
nextcloud/ Nextcloud, MariaDB, Redis, ingress and secrets
onlyoffice/ OnlyOffice Document Server
plex/ Plex Media Server
jellyfin/ Jellyfin media server
vaultwarden/ Vaultwarden password manager
twingate/ Twingate connector
nginx/ nginx demo workload and an alternative Terraform/Helm deployment
local-persistent-storage-class.yaml Retained local-path StorageClass

data/ and charts/ are ignored by Git. Application data lives on the host, on TrueNAS storage, or on the external backup disk and is not tracked in this repository.

Several directories are kept for reference or future use but are not currently deployed. These are listed in the service catalogue.

Service catalogue

Running now

Service Namespace Public hostname Notes
ArgoCD argocd argocd.tap-tap.win GitOps UI and API
HashiCorp Vault hashicorp-vault hashicorp-vault.tap-tap.win KV v2 secret store and UI
Gitea gitea gitea.tap-tap.win Git hosting; includes an Actions runner with Docker-in-Docker
Nextcloud nextcloud nextcloud.tap-tap.win MariaDB 11.4.4 and Redis 7 backends
OnlyOffice onlyoffice onlyoffice.tap-tap.win OnlyOffice Document Server
Plex plex plex.tap-tap.win Media server using the Nextcloud media library
Jellyfin jellyfin jellyfin.tap-tap.win Media server using the Nextcloud media library
Vaultwarden vaultwarden vaultwarden.tap-tap.win Password manager (HTTP ingress; proxied through Cloudflare)
Home Assistant homeassistant homeassistant.tap-tap.win Smart-home hub (proxied through Cloudflare)
Mosquitto homeassistant internal MQTT broker for Zigbee2MQTT and Home Assistant
Zigbee2MQTT homeassistant NodePort 30080 / 30081 Zigbee bridge with USB stick passthrough
Matter Server homeassistant internal Matter device integration
OpenThread Border Router homeassistant host network Thread border router with USB radio passthrough
Twingate twingate — Zero-trust connector
Cloudflare DDNS cloudflare-ddns — Updates public DNS records
nginx demo nginx nginx.tap-tap.win, nginx2.tap-tap.win Simple ingress demo; nginx.tap-tap.win is HTTP-only, while nginx2 uses TLS and is proxied through Cloudflare

ArgoCD currently reports 14 application directories plus the root application, all Synced and Healthy.

Present in the repository but not deployed

These directories contain manifests that are not currently running in the cluster:

Path What it contains
monitoring/ kube-prometheus-stack (Prometheus, Grafana, Alertmanager)
grafana/ Standalone Grafana deployment, superseded by the monitoring chart
pihole/ Pi-hole DNS and ad blocking
papertrader/ Custom PaperTrader server and browser images
homarr/ Homarr dashboard
audiobookshelf/ Audiobookshelf server
navidrome/ Navidrome music server
gitlab/ GitLab CE
cloudflared/ Cloudflare Tunnel connector
metallb/ Previous MetalLB manifests; replaced by K3s ServiceLB

Legacy and local working directories

Path Status
idrive/ Tracked but unused. Backups are handled by Borg, not IDrive.
oracle/ Local, untracked Terraform experiment for an Oracle Cloud free-tier VM; not part of the live platform.
old/ Local, untracked early prototype scripts and migration experiments.
downloader/ Local, untracked placeholder directory with no active manifests.

Backup and recovery

The backup design keeps multiple independent copies of the data:

Copy Location Method Frequency
Primary TrueNAS mirrored storage NFS export to the K3s node Automount, 10-minute idle timeout
Local backup /mnt/backup/borg on a 3.6 TB external USB disk BorgBackup Weekly
Offsite Google Drive Backup folder rclone sync of the Borg repository Weekly, Thursday 04:30

Borg

The NixOS host runs a weekly Borg job:

  • Paths: /root and /mnt/data.
  • Repository: /mnt/backup/borg.
  • Compression: auto,zstd.
  • Encryption: keyfile mode.
  • Checkpoint interval: 300 seconds.
  • Retention: 1 daily, 2 weekly, 2 monthly.
  • Failure handling: restart on failure after 30 seconds.

The latest run completed successfully. Deduplication keeps incremental snapshots small: a run representing roughly 3 TB of logical data produced about 387 MB of deduplicated archive data.

Offsite copy

A systemd timer runs every Thursday at 04:30 and synchronises the entire Borg repository to Google Drive:

rclone sync /mnt/backup/borg gdrive:Backup

The timer is persistent, so a missed run is executed after the next boot.

What is not backed up

There is currently no active Proxmox VM-level backup and no active Proxmox Backup Server datastore. The host configuration contains a prepared NFS export for a PBS datastore, but it is not in use.

Recovery outline

  1. Reinstall or rebuild the NixOS host from the flake (or restore the VM).
  2. Mount the TrueNAS NFS export and the external backup disk.
  3. Restore application data and configuration from the Borg repository.
  4. Let ArgoCD reconcile the Kubernetes manifests from Git.

Observability

The live cluster currently runs only metrics-server for resource metrics. The monitoring/ directory contains a complete kube-prometheus-stack configuration (Prometheus, Grafana, Alertmanager, node-exporter and kube-state-metrics), but it is not currently deployed.

Observability is therefore a known gap and a roadmap item rather than a running feature.

Security

  • Automated TLS: cert-manager obtains Let's Encrypt certificates through the Cloudflare DNS-01 challenge. Nine certificates are currently issued and ready.
  • Runtime secrets: Vault stores KV v2 secrets. External Secrets Operator authenticates using Kubernetes service-account auth and syncs only the secrets each workload needs.
  • No credentials in this README: tokens, passwords, keys and private endpoints are intentionally excluded.
  • Zero-trust remote access: Twingate provides identity-aware access to private services without exposing them directly.
  • Network separation: Public ingress passes through OPNsense to Traefik; Twingate provides a separate identity-aware path for private access. The host firewall also permits cluster, Matter/Thread and NodePort ports, so isolation relies on OPNsense and the private LAN.
  • Minimal public surface: services such as Vault, ArgoCD, Gitea and the media servers are protected by ingress and TLS; platform internals do not have public ingresses. Home Assistant, Vaultwarden and nginx2 additionally sit behind Cloudflare's reverse proxy.
  • Data protection: Borg repositories are encrypted, and the offsite copy is synchronised to Google Drive.

Engineering highlights

  • Two-layer Infrastructure as Code: NixOS owns the host, ArgoCD owns the cluster. Both are declarative and managed as code.
  • Immutable host configuration: packages, services, filesystems, firewall rules and backup jobs are declared in the NixOS flake, not configured by hand.
  • GitOps delivery: every application change is reconciled automatically; manual kubectl apply is only used for debugging.
  • Versioned platform components: K3s, ArgoCD, Traefik, cert-manager, Vault and External Secrets use explicit versions.
  • Secretless application manifests: workloads consume Kubernetes Secrets produced by External Secrets rather than embedding credentials in Git.
  • Layered storage: TrueNAS provides mirrored network storage, local-path classes provide fast cluster-local volumes, and an external disk holds the Borg repository.
  • 3-copy backup chain: mirrored primary storage, a local Borg repository and an offsite Google Drive copy.
  • Practical home automation: Home Assistant, MQTT, Zigbee, Matter and Thread run alongside the Kubernetes platform with USB device passthrough and host networking where required.

Known limitations and roadmap

This is a personal homelab, not a production platform. Known limitations:

  • Single-node cluster: no high availability; the node and its embedded etcd are a single point of failure.
  • hostPath coupling: many applications bind directly to paths on the K3s VM, which makes rescheduling or multi-node operation harder.
  • Repo/live drift: several manifests are kept in the repository but are not deployed. This README distinguishes live services from repo-only services.
  • Unpinned images: some workloads use :latest tags; pinning digests or versions would make rollbacks deterministic.
  • Monitoring gap: Prometheus, Grafana and Alertmanager are defined but not deployed.
  • No VM-level backup: recovery relies on the NixOS flake plus data-level Borg backups.
  • Host-level secrets: remaining host secrets should move to a declarative secret manager such as sops-nix or agenix.
  • Flake lock not committed: the NixOS flake lock file is ignored by Git, so host builds should commit it for fully reproducible package versions.
  • MetalLB leftovers: CRDs remain from the previous load-balancer setup and can be cleaned up.
  • No automated restore drills: backups complete successfully, but restores should be tested on a schedule.

Planned improvements:

  • Deploy the monitoring stack and configure alert routing.
  • Pin all container images and add automated manifest validation.
  • Move host secrets to sops-nix or agenix.
  • Commit the NixOS flake lock file for reproducible host builds.
  • Replace selected hostPath volumes with PVCs backed by network storage.
  • Add restore tests and document recovery time.

This repository documents a personal homelab. It is shared for learning and portfolio purposes and is not intended as a production reference architecture.

S
Description
No description provided
Readme
1.4 MiB
0 Stars 1 Watchers 0 Forks
Languages
Dockerfile 59.6%
HCL 21.6%
Shell 18.8%