vanlink/docs/superpowers/specs/2026-09-23-nm-wan-provisioning-design.md

# NetworkManager-owned WANs, provisioned by deploy.sh

Date: 2026-09-23. Status: design approved in chat; spec for review. Builds on
`2026-09-23-host-config-design.md` (host files, `ENABLE_*` flags, `--check/--diff/--post-check`).

## Problem

`van-failover` steers multi-WAN through NetworkManager: it reads active connections and
gateways with `nmcli`, sets route metrics per NM profile, and the cellular WAN is a NM `gsm`
profile (`Koodo`). The Pi 5 runs that model. The R5C has no NetworkManager (networkd only;
Armbian's `10-dhcp-all-interfaces.yaml` does DHCP on every `e*` port), so on the R5C
`van-failover` sees no connections at all (`present: false`, `connection: null`); it "works"
only because networkd's single DHCP default route happens to be on the Starlink port. A GSM
modem (Quectel EC25-AF, MBIM via ModemManager) will be attached later, and the home LAN on
`enP2p33s0` is a stand-in for the field's Starlink connection, so the R5C needs the same
NM-owned WAN model as the Pi, and it must be provisioned by `deploy.sh` rather than by hand.

## Decisions (approved)

- NetworkManager owns the WAN side on every host that sets `ENABLE_NM_WAN=1`; `systemd-networkd`
  keeps br0, the AP radios and the wired LAN port(s), as today. One code path on both hosts.
- A routine `deploy.sh` never flaps uplinks. Changing the WAN netplan configuration is an explicit
  step, `./deploy.sh --apply-network`, with an automatic rollback.
- `deploy.sh` installs the packages it needs.

## Design

### 1. Host variables (add to `REQUIRED_VARS` / `OPTIONAL_VARS`; `hosts/vanlink.conf.example` and both host files)

| Variable | Kind | Meaning |
|---|---|---|
| `ENABLE_NM_WAN` | required flag 0/1 | NetworkManager owns the WAN ports and the modem. Both `hosts/van.conf` and `hosts/wan.conf` set 1. |
| `CELLULAR_CONN_NAME` | optional | NM connection name of the cellular WAN; `failover/config.json` uses `@CELLULAR_CONN_NAME@` instead of the hardcoded `Koodo`. Default in both host files: `Koodo`. |
| `CELLULAR_APN` | optional | APN for the cellular profile (`sp.koodo.com`). Empty: no profile is created. |
| `WIFI_WAN_IFACE` | now optional | Empty means the host has no Wi-Fi uplink (the R5C: both radios are APs); the `wifis:` stanza of the netplan template is then omitted. |

`ENABLE_NM_WAN=0` means none of the below applies (current R5C behaviour).

### 2. Package installation

- Missing REQUIRED packages are installed by `deploy.sh` itself (`DEBIAN_FRONTEND=noninteractive
  apt-get install -y`) instead of aborting with "apt install X". If apt cannot install one, abort
  with the apt error (today's outcome). `--check/--diff/--post-check` never install anything.
- `network-manager` is the exception in ordering: it is installed **after** the files that keep
  NM off the AP radios and LAN ports (`van-ap-unmanaged.conf`) are in place, because the package
  starts NM on install and NM manages Wi-Fi and gsm devices by default. After installing it,
  `deploy.sh` checks `nmcli -t -f DEVICE,STATE device` and warns if `WIFI_5G_IFACE`,
  `WIFI_2G_IFACE`, `LAN_IFACE` or `ONBOARD_LAN_IFACE` is not `unmanaged`.
- With `ENABLE_NM_WAN=1`: `network-manager` is required; NetworkManager and ModemManager are enabled.

### 3. Routine deploy behaviour (`ENABLE_NM_WAN=1`)

- The cellular NM profile is created if absent, matching the Pi's working `Koodo`: type `gsm`,
  no interface binding, `apn=$CELLULAR_APN`, no credentials, autoconnect yes, `ipv4.method auto`,
  `ipv4.route-metric 300`, `ipv4.never-default yes`, `ipv6.method auto`. An existing profile is
  never modified. Creation needs no modem to be present.
- `/etc/netplan/50-van-wan.yaml`, the Armbian netplan file and the netplan state are **not**
  touched. If `/etc/netplan/50-van-wan.yaml` is missing or differs from the rendered template,
  deploy warns and names `./deploy.sh --apply-network` (this replaces today's "run netplan by
  hand" text). The Pi 5's live uplinks are therefore never flapped by a routine deploy.

### 4. `./deploy.sh --apply-network` (root, explicit)

1. Refuse unless `ENABLE_NM_WAN=1` and `network-manager` is installed.
2. Back up `/etc/netplan` to `/var/backups/vanlink-netplan-<timestamp>/`.
3. Install the rendered `50-van-wan.yaml`. If `/etc/netplan/10-dhcp-all-interfaces.yaml` exists
   (Armbian), move it aside as `10-dhcp-all-interfaces.yaml.vanlink-disabled` so networkd stops
   DHCP-ing the WAN port behind NM's back.
4. Apply from a detached transient unit (`systemd-run`), so the sequence survives the SSH session
   dropping: `netplan generate && netplan apply`, then wait up to 60 s for: NM reports an active
   connection on `STARLINK_IFACE` (or the WAN is otherwise up), a default route exists, and
   `ping -c1 1.1.1.1` works.
5. On success: log, keep the backup. On failure: restore the backup (including the Armbian file),
   re-apply, and record a warning with the reason. The exit status is non-zero on failure.
6. The detached unit writes its outcome to `/var/log/vanlink-apply-network.log`.

### 5. Template changes

- `ap/50-van-wan.yaml` is split: the ethernet (Starlink) part stays; the `wifis:` stanza (with
  the Wapana access point) moves to a fragment `ap/50-van-wan-wifi.yaml` that `--apply-network`
  appends only when `WIFI_WAN_IFACE` is non-empty. The rendered result for the Pi 5 must be
  byte-identical to today's `/etc/netplan/50-van-wan.yaml` there (checked with `--diff` before
  the Pi is ever touched). The Wapana PSK stays where it is (known, out of scope).
- `failover/config.json`: cellular `connection` becomes `@CELLULAR_CONN_NAME@`.

### 6. Post-check additions (only when `ENABLE_NM_WAN=1`)

NetworkManager active; ModemManager active; the cellular profile exists (if `CELLULAR_APN` set);
AP/LAN devices are `unmanaged` in NM; `van-failover`'s `/run/van-failover/state.json` shows each
WAN whose device exists as `present: true`.

## Testing

- `tests/check-templates.sh`: new variables in the example (existing sync check); host files pass
  `--check`; the awk flag-guard check gains `ENABLE_NM_WAN` for `network-manager`, `nmcli`,
  `50-van-wan` and the netplan-moving code (only inside guards); rendered `failover/config.json`
  has `Koodo` for both hosts; the rendered netplan for `wan.conf` contains the wifis stanza and for
  `van.conf` does not; a `--wan-plan` sub-command prints (no changes, no root) the actions
  `--apply-network` would take, asserted for both hosts; the rollback decision logic is a small
  function tested through a seam like `--classify-unit`.
- Not testable without root/live hosts: the apply and rollback themselves. Those are verified on the
  R5C (Rollout) with a console or alternate access path available.

## Rollout

1. Build, review, push. Sync to the R5C (excluding `hosts/van.conf`). Set `ENABLE_NM_WAN=1`,
   `CELLULAR_CONN_NAME`, `CELLULAR_APN` in its `hosts/van.conf` (currently an uncommitted local edit).
2. R5C: routine `sudo ./deploy.sh` (installs `network-manager`, creates the cellular profile, warns
   about the pending WAN netplan), then `sudo ./deploy.sh --apply-network` once. Uplink today is the
   home LAN on `enP2p33s0`; the port changes from networkd to NM, a short blip.
3. Pi 5: routine deploy only; `--diff` must show no change to `50-van-wan.yaml`, and no apply is run.
4. When the modem is attached: `mmcli -L`, the profile connects, `cat /run/van-failover/state.json`
   shows cellular present, route metrics 200 (Starlink) / 300 (cellular).

## Out of scope

Moving the Wapana PSK out of git; choosing the modem's USB port; a Wi-Fi WAN on the R5C (needs a
spare radio); replacing NetworkManager on either host; cellular credentials/PIN (the Koodo profile
has none).

## Risks

- The cutover changes the manager of the uplink port on a live host: mitigated by the detached apply,
  the connectivity gate and automatic rollback; a console or ZeroTier path should exist anyway.
- NM installation must not touch AP/LAN ports: mitigated by the install ordering and the unmanaged check.
- Auto-installing packages changes deploy's contract: only missing required packages are installed and
  a failure aborts with the apt error.