vanlink/docs/superpowers/specs/2026-09-23-nm-wan-provisioning-design.md

NetworkManager-owned WANs, provisioned by deploy.sh

Date: 2026-09-23. Status: design approved in chat; spec for review. Builds on 2026-09-23-host-config-design.md (host files, ENABLE_* flags, --check/--diff/--post-check).

Problem

van-failover steers multi-WAN through NetworkManager: it reads active connections and gateways with nmcli, sets route metrics per NM profile, and the cellular WAN is a NM gsm profile (Koodo). The Pi 5 runs that model. The R5C has no NetworkManager (networkd only; Armbian's 10-dhcp-all-interfaces.yaml does DHCP on every e* port), so on the R5C van-failover sees no connections at all (present: false, connection: null); it "works" only because networkd's single DHCP default route happens to be on the Starlink port. A GSM modem (Quectel EC25-AF, MBIM via ModemManager) will be attached later, and the home LAN on enP2p33s0 is a stand-in for the field's Starlink connection, so the R5C needs the same NM-owned WAN model as the Pi, and it must be provisioned by deploy.sh rather than by hand.

Decisions (approved)

• NetworkManager owns the WAN side on every host that sets ENABLE_NM_WAN=1; systemd-networkd keeps br0, the AP radios and the wired LAN port(s), as today. One code path on both hosts.
• A routine deploy.sh never flaps uplinks. Changing the WAN netplan configuration is an explicit step, ./deploy.sh --apply-network, with an automatic rollback.
• deploy.sh installs the packages it needs.

Design

1. Host variables (add to REQUIRED_VARS / OPTIONAL_VARS; hosts/vanlink.conf.example and both host files)

Variable	Kind	Meaning
ENABLE_NM_WAN	required flag 0/1	NetworkManager owns the WAN ports and the modem. Both hosts/van.conf and hosts/wan.conf set 1.
CELLULAR_CONN_NAME	optional	NM connection name of the cellular WAN; failover/config.json uses @CELLULAR_CONN_NAME@ instead of the hardcoded Koodo. Default in both host files: Koodo.
CELLULAR_APN	optional	APN for the cellular profile (sp.koodo.com). Empty: no profile is created.
WIFI_WAN_IFACE	now optional	Empty means the host has no Wi-Fi uplink (the R5C: both radios are APs); the wifis: stanza of the netplan template is then omitted.

ENABLE_NM_WAN=0 means none of the below applies (current R5C behaviour).

2. Package installation

• Missing REQUIRED packages are installed by deploy.sh itself (DEBIAN_FRONTEND=noninteractive apt-get install -y) instead of aborting with "apt install X". If apt cannot install one, abort with the apt error (today's outcome). --check/--diff/--post-check never install anything.
• network-manager is the exception in ordering: it is installed after the files that keep NM off the AP radios and LAN ports (van-ap-unmanaged.conf) are in place, because the package starts NM on install and NM manages Wi-Fi and gsm devices by default. After installing it, deploy.sh checks nmcli -t -f DEVICE,STATE device and warns if WIFI_5G_IFACE, WIFI_2G_IFACE, LAN_IFACE or ONBOARD_LAN_IFACE is not unmanaged.
• With ENABLE_NM_WAN=1: network-manager is required; NetworkManager and ModemManager are enabled.

3. Routine deploy behaviour (ENABLE_NM_WAN=1)

• The cellular NM profile is created if absent, matching the Pi's working Koodo: type gsm, no interface binding, apn=$CELLULAR_APN, no credentials, autoconnect yes, ipv4.method auto, ipv4.route-metric 300, ipv4.never-default yes, ipv6.method auto. An existing profile is never modified. Creation needs no modem to be present.
• /etc/netplan/50-van-wan.yaml, the Armbian netplan file and the netplan state are not touched. If /etc/netplan/50-van-wan.yaml is missing or differs from the rendered template, deploy warns and names ./deploy.sh --apply-network (this replaces today's "run netplan by hand" text). The Pi 5's live uplinks are therefore never flapped by a routine deploy.

4. ./deploy.sh --apply-network (root, explicit)

1. Refuse unless ENABLE_NM_WAN=1 and network-manager is installed.
2. Back up /etc/netplan to /var/backups/vanlink-netplan-<timestamp>/.
3. Install the rendered 50-van-wan.yaml. If /etc/netplan/10-dhcp-all-interfaces.yaml exists (Armbian), move it aside as 10-dhcp-all-interfaces.yaml.vanlink-disabled so networkd stops DHCP-ing the WAN port behind NM's back.
4. Apply from a detached transient unit (systemd-run), so the sequence survives the SSH session dropping: netplan generate && netplan apply, then wait up to 60 s for: NM reports an active connection on STARLINK_IFACE (or the WAN is otherwise up), a default route exists, and ping -c1 1.1.1.1 works.
5. On success: log, keep the backup. On failure: restore the backup (including the Armbian file), re-apply, and record a warning with the reason. The exit status is non-zero on failure.
6. The detached unit writes its outcome to /var/log/vanlink-apply-network.log.

5. Template changes

• ap/50-van-wan.yaml is split: the ethernet (Starlink) part stays; the wifis: stanza (with the Wapana access point) moves to a fragment ap/50-van-wan-wifi.yaml that --apply-network appends only when WIFI_WAN_IFACE is non-empty. The rendered result for the Pi 5 must be byte-identical to today's /etc/netplan/50-van-wan.yaml there (checked with --diff before the Pi is ever touched). The Wapana PSK stays where it is (known, out of scope).
• failover/config.json: cellular connection becomes @CELLULAR_CONN_NAME@.

6. Post-check additions (only when ENABLE_NM_WAN=1)

NetworkManager active; ModemManager active; the cellular profile exists (if CELLULAR_APN set); AP/LAN devices are unmanaged in NM; van-failover's /run/van-failover/state.json shows each WAN whose device exists as present: true.

Testing

• tests/check-templates.sh: new variables in the example (existing sync check); host files pass --check; the awk flag-guard check gains ENABLE_NM_WAN for network-manager, nmcli, 50-van-wan and the netplan-moving code (only inside guards); rendered failover/config.json has Koodo for both hosts; the rendered netplan for wan.conf contains the wifis stanza and for van.conf does not; a --wan-plan sub-command prints (no changes, no root) the actions --apply-network would take, asserted for both hosts; the rollback decision logic is a small function tested through a seam like --classify-unit.
• Not testable without root/live hosts: the apply and rollback themselves. Those are verified on the R5C (Rollout) with a console or alternate access path available.

Rollout

1. Build, review, push. Sync to the R5C (excluding hosts/van.conf). Set ENABLE_NM_WAN=1, CELLULAR_CONN_NAME, CELLULAR_APN in its hosts/van.conf (currently an uncommitted local edit).
2. R5C: routine sudo ./deploy.sh (installs network-manager, creates the cellular profile, warns about the pending WAN netplan), then sudo ./deploy.sh --apply-network once. Uplink today is the home LAN on enP2p33s0; the port changes from networkd to NM, a short blip.
3. Pi 5: routine deploy only; --diff must show no change to 50-van-wan.yaml, and no apply is run.
4. When the modem is attached: mmcli -L, the profile connects, cat /run/van-failover/state.json shows cellular present, route metrics 200 (Starlink) / 300 (cellular).

Out of scope

Moving the Wapana PSK out of git; choosing the modem's USB port; a Wi-Fi WAN on the R5C (needs a spare radio); replacing NetworkManager on either host; cellular credentials/PIN (the Koodo profile has none).

Risks

• The cutover changes the manager of the uplink port on a live host: mitigated by the detached apply, the connectivity gate and automatic rollback; a console or ZeroTier path should exist anyway.
• NM installation must not touch AP/LAN ports: mitigated by the install ordering and the unmanaged check.
• Auto-installing packages changes deploy's contract: only missing required packages are installed and a failure aborts with the apt error.

Amendments made during implementation (review-driven; the code and tests are authoritative)

• §3/§4 cellular profile is written as a NetworkManager keyfile from ap/cellular.nmconnection (uuid derived in pure bash from the connection name), only if no connection with that id exists (keyfile scan by id= plus nmcli); ap/nm-no-auto-default.conf (no-auto-default=*) is installed before the package so NM never auto-creates DHCP "Wired connection" profiles on ports not yet handed to it.
• §4 --apply-network changes nothing under /etc/netplan itself. It validates (NM installed and running, van-failover unit present, no apply already running, last status not running/rollback-failed unless --force), no-ops when the netplan already matches and no Armbian file exists (unless --force), takes a per-run backup plus a sticky pristine backup /var/backups/vanlink-netplan-orig, renders to /run, and launches a detached transient unit. The helper installs the file, moves the Armbian file aside, applies, and gates: stage 1 (≤60 s): apply succeeded, NM device state 100 (connected) with a netplan-* connection, IPv4 address, gateway (IP4.GATEWAY, else the DHCP lease's routers option, because a never-default profile leaves IP4.GATEWAY empty) reachable on-link; stage 2: restart van-failover, then default route + ping 1.1.1.1 (≥120 s). Failure → checked rollback (restore, re-apply, connectivity re-check, ROLLBACK FAILED if not restored; a last-resort reboot only when the rollback itself failed). Timeouts, a TERM/INT/HUP trap that restores exactly once, budgets derived in one block, log /var/log/vanlink-apply-network.log, status file /var/lib/vanlink/apply-network.status (running|ok|rolled-back|rollback-failed <ts> …). Not built: a boot-time recovery unit (power loss mid-apply boots the new config ungated and leaves status running), an arping fallback for gateways that do not answer ICMP.
• §6 post-check: the failover-state rule is refined: present in van-failover's state means "has an active NM connection", so an unplugged or out-of-range uplink is normal in a van. Post-check warns only when the WAN's device has link (carrier 1) but present is false; it also reports the last --apply-network status (non-ok warns) and checks NM/ModemManager, NM-unmanaged AP/LAN ports, the cellular profile and the netplan state.
• Routine deploys also install missing REQUIRED packages via apt (spec §2) and never touch /etc/netplan.