Environments

Verified 2026-08-23

The state below was checked live on 2026-08-23 by SSH/HTTP against the systems and with gh against the repository myfoxit/northplane. Where a detail comes from the lab notes of 2026-08-19/21 instead, it is marked as such. Re-verify before acting on anything that looks stale.

Inventory

Environment	Role	Where	URL	Data mode	Running build	State (2026-08-23)
np-01	production	VM101 saas1 (10.10.10.11) on Proxmox 51.83.96.40, behind CT100 Caddy + Cloudflare	https://doktrace.com	real (since 2026-08-20)	ghcr.io/myfoxit/northplane:main-daa6dc518a2b (= main HEAD)	healthy; deploy job green
np-02	second production instance (standalone, bundled Caddy)	Hetzner 91.98.92.10	https://91.98.92.10	real (by workflow)	—	decommissioned — box reclaimed, IP reassigned; the deploy-hetzner job was removed, Deploy runs are green again
np-staging	federation edge of production (tenant MyFoxIT, site vm104-edge); lab mirror	VM104 saas4 (10.10.10.14) on the same Proxmox host	https://10.10.10.14:8443 (private bridge; self-signed)	real (lab data)	not re-verified on 2026-08-23	up; edge connected to prod (lab notes 2026-08-21)
CT110 targets	lab check targets	10.10.10.20	—	—	—	up
VM102 saas2 “netlab”	network lab (containerlab, 2× Cisco IOSv, SR Linux) + np-agent → prod	10.10.10.12	—	—	—	up
VM103 saas3 “alarmlab”	alarm sinks (Mosquitto, Mailpit, GreenMail, ntfy, echo sink) + np-agent → prod	10.10.10.13	—	—	—	up
CT100 caddy	TLS edge for all slots	10.10.10.10	—	—	—	up; one site (doktrace.com)
VM9000 / VM9001	templates (debian13-base, rocky9-base)	—	—	—	—	stopped
developer loop	make dev on a workstation	local	http://localhost:5173 (UI), http://127.0.0.1:8443 (API)	demo by default	source	—

np-01 — doktrace.com (production)

Item	Verified value
Host	VM101 saas1, Rocky Linux 9.8, 4 vCPU / 8 GB, 10.10.10.11 on the hypervisor ns3147660 (Proxmox VE 9.2.11)
Stack	deploy/docker-compose.vm.yml (edge-proxied, no bundled Caddy) in /opt/northplane; container northplane-northplane-1; volume northplane_northplane-data; secret.key 65 bytes, uid 65532, 0600
Image	ghcr.io/myfoxit/northplane:main-daa6dc518a2b, started 2026-08-23T08:50:57Z; /api/v1/system/info → version: main-daa6dc518a2b, storage: sqlite, goVersion: go1.25.14, aiEnabled: false
Health	/healthz → ok; /readyz → ready: true (storage sqlite, eventbus, scheduler all ok)
Data mode	real — NORTHPLANE_DEMO=false, NORTHPLANE_DATA_DIR=/var/lib/northplane/real, switched on 2026-08-20 (repo variable NORTHPLANE_DEMO=false set the same day); /var/lib/northplane/demo still present in the volume
Env keys	NORTHPLANE_IMAGE, NORTHPLANE_BASE_URL=https://doktrace.com, NORTHPLANE_DEMO=false, NORTHPLANE_ALLOW_SIGNUP (now from the repo variable of the same name; default false), NORTHPLANE_DATA_DIR, [email protected], NP_DEFAULT_ADMIN_PASSWORD (secret); compose-set NORTHPLANE_LISTEN=:8443, NORTHPLANE_TRUST_PROXY=true, NORTHPLANE_SECRET_KEY_FILE=/etc/northplane/secret.key
Signup	was on in the build verified at 08:50 UTC (hard-coded); since the signup-flag merge it follows the repo variable NORTHPLANE_ALLOW_SIGNUP — unset, so the next deploy turns /register off. Set the variable to true to re-enable (self-registered accounts get viewer)
Break-glass admin	[email protected] (password = repo secret NP_DEFAULT_ADMIN_PASSWORD; not printed here)
Tenants	default tenant plus tenant MyFoxIT (own users and a tenant-admin role; site vm104-edge lives here) — see Tenants and sites
TLS / ingress	Cloudflare-proxied A record → 51.83.96.40 → DNAT 443 → CT100 Caddy (/etc/caddy/sites/saas1.caddy, Let’s Encrypt HTTP-01) → 10.10.10.11:8443
Published ports	8443 plus 9162/udp, 2023, 8123, 4573 (private bridge only; merged 2026-08-23, live with the next deploy)
Agents reporting here	np-prod (VM101), pve-host (hypervisor), netlab (VM102), alarmlab (VM103)
Last deploy	Deploy run 32629242562, 2026-08-23 08:48 UTC, for merge daa6dc5: publish and deploy succeeded (the since-removed deploy-hetzner job failed)
Docs	/docs/ is served by every image built from the commit that introduced the embedded documentation onwards; the build verified at 08:50 UTC (main-daa6dc518a2b) predates it, the next Deploy run after the merge carries it (CI/CD)
Backups	none scheduled (no vzdump job on the host, no periodic app backup) — Operations → Backups

Topology, access and the compose file: Proxmox VM. Day-2 handling: Operations.

np-02 — Hetzner standalone (decommissioned, unreachable)

Item	Verified value
Address	91.98.92.10 (repo variable HETZNER_HOST)
2026-08-23	TCP/22 times out; https://91.98.92.10/ answers 404 with a parking page (earlier observation: a Yahoo parking page). The Hetzner box has been reclaimed and the IP reassigned to a stranger
Pipeline effect	none since the deploy-hetzner job was removed from deploy.yml; before that it failed at “Ship compose stack” on every run (observed 2026-08-14 … 2026-08-23) while np-01’s deploy job kept succeeding
Design	the standalone recipe: deploy/docker-compose.yml + deploy/Caddyfile (bundled Caddy, bare-IP internal certificate, Let’s Encrypt once DOMAIN is set), .env rendered by CI with DOMAIN=localhost, SERVER_IP=91.98.92.10, NORTHPLANE_BASE_URL=https://91.98.92.10, NORTHPLANE_DEMO=false, NORTHPLANE_DATA_DIR=/var/lib/northplane/real, NP_DEFAULT_ADMIN_EMAIL=root@localhost; provisioned with deploy/provision-server.sh as root on Rocky 10
Stale configuration	variables HETZNER_HOST, HETZNER_KNOWN_HOSTS; secrets HETZNER_SSH_KEY, HETZNER_ADMIN_PASSWORD — unused since the job removal, still referring to the lost box
Data	whatever lived in its northplane-data volume is gone with the box; no backup existed

Done: the deploy-hetzner job was removed, so Deploy runs are green again. To bring a standalone box back, follow the np-02 recreation checklist, restore the job from git history and rotate the four HETZNER_* entries.

np-staging — VM104 (federation edge of production)

Item	Value (lab notes 2026-08-21 unless marked)
Host	VM104 saas4, 10.10.10.14, Rocky 9 (guest verified 2026-08-23)
Role	second Northplane instance: real data, self-signed TLS, a local admin; federation edge of production for tenant MyFoxIT, site vm104-edge (role verified 2026-08-23)
Edge config	federation: mode: edge, mainUrl: https://doktrace.com, site: vm104-edge, interval: 60s, token minted on main with scope sites:connect (tenant-scoped) — in /opt/northplane/config.yaml, mounted read-only at /etc/northplane/config.yaml, owned by uid 65532
What flows	the edge pulls the site bundle from main (ETag-conditional) and applies it locally (hosts np-staging, lab-web, passive agent services, an ntfy channel, a policy and a contact); it heartbeats status and counters back — GET /api/v1/sites:overview on main (with the tenant header) shows connected plus stats
Agent	its own np-agent pushes to the local edge instance with a token minted on the edge — not to production (verified 2026-08-23)
Ports	publishes 8443 plus 9162/udp, 2023, 8123, 4573 (lab notes)
Caveats	its mqtt-in event source is disabled (two instances sharing an MQTT client id on one broker evict each other); mounted config files must be owned by uid 65532
Reach it	ssh -N -L 18443:10.10.10.14:8443 [email protected] → https://localhost:18443; shell via ssh -J [email protected] [email protected]

The federation mechanics are described under Federation and Tenants and sites.

Lab guests

Guest list verified 2026-08-23; service details from the lab notes (see the operations appendix for the full picture).

Guest	Address	Purpose
CT110 targets	10.10.10.20	one live target per builtin check: nginx variants (ok / slow / 500), valid / expiring / expired certificates, snmpd, NRPE, BIND lab.local, chrony, an np-agent in listener mode
VM102 saas2 “netlab”	10.10.10.12	containerlab with 2× Cisco IOSv (real OSPF link, SNMP polling + traps into prod), SR Linux, Alpine + snmpd; np-agent → prod
VM103 saas3 “alarmlab”	10.10.10.13	Mosquitto, Mailpit, GreenMail, ntfy, echo sink :9000 for Slack/Teams/ticket channels; ESPA drivers; np-agent → prod
VM104 saas4	10.10.10.14	np-staging — see above
VM9000 / VM9001	—	debian13-base / rocky9-base templates, stopped

DNS and Cloudflare

• doktrace.com — Cloudflare-proxied A record → 51.83.96.40. Cloudflare terminates the visitor’s TLS and connects to the origin on 443; the origin (CT100 Caddy, Let’s Encrypt certificate via HTTP-01) also answers direct connections (curl --resolve doktrace.com:443:51.83.96.40 …).
• Cloudflare blocks Python user agents (403 / error 1010) and replaces 5xx bodies with its own error page — script with a curl-like User-Agent, and read API error details from the origin.
• CT100’s Caddyfile lists the Cloudflare IPv4/IPv6 ranges as trusted_proxies; Northplane only consumes X-Forwarded-Proto (NORTHPLANE_TRUST_PROXY=true), never X-Forwarded-For.
• 91.98.92.10 (np-02) has no DNS name and is no longer ours.
• No DNS name exists for the staging/lab guests; they are reached through the hypervisor.

Who has access

Path	Who / how
Hypervisor 51.83.96.40	root by SSH key (the operator’s key); the Proxmox web UI (8006) is not exposed — use ssh -N -L 8006:127.0.0.1:8006 [email protected] → https://localhost:8006
VMs (saas1–saas4)	user rocky with sudo, via the jump host: ssh -J [email protected] [email protected] (the VMs trust a different key than the hypervisor)
CT100 Caddy	no SSH key inside; ssh [email protected] -t 'pct enter 100'
CI → VM101	[email protected] -p 2201 (DNAT → 10.10.10.11:22) with the DEPLOY_SSH_KEY secret; deploy is in the docker group (root-equivalent on that VM)
GitHub repository	public; whoever administers myfoxit/northplane controls variables/secrets and therefore production (.env is rendered from them). NORTHPLANE_DEMO is the only variable expected to change
GHCR image	public package; the pipeline pushes with its GITHUB_TOKEN
Application admin	break-glass [email protected] on np-01 (instance-wide admin); tenant MyFoxIT has its own users; /register creates viewer accounts for anyone
Agents	API tokens with scope objects:write in /etc/northplane/agent.yaml on each host (root-readable files)

Open items

1. np-02 is gone — the deploy-hetzner job was removed (Deploy runs green again); the HETZNER_* variables/secrets remain stale until a new box is provisioned. (Provisioning)
2. Schedule production backups — there is no automated job yet; secret.key and the data volume need regular off-host copies. (Operations → Backups)
3. Confirm the alarm-input ports after the next deploy — 9162/udp, 2023, 8123, 4573 were merged into deploy/docker-compose.vm.yml on 2026-08-23; check with docker compose ps on the VM that they are mapped.
4. Confirm /docs/ on doktrace.com after the next deploy — the image verified on 2026-08-23 08:50 UTC predates the docs embedding; the first Deploy run after the docs merge serves it (check https://doktrace.com/docs/).
5. Public signup is now governed by the repo variable NORTHPLANE_ALLOW_SIGNUP (default off). Decide: leave /register closed, or gh variable set NORTHPLANE_ALLOW_SIGNUP --body true for the showcase. (Security)
6. DEPLOY_DOMAIN variable is unused by any workflow (informational only).
7. Known CI flake — the postgres job’s TestAuditChain failure (jsonb normalisation) is non-blocking; re-run failed jobs with gh run rerun --failed <id> when other jobs flake.

Related: Deployment overview, Proxmox VM, CI/CD, Provisioning, Operations.