vanlink/README.md

vanlink — campervan router on wayback

Turns wayback (Asus ZenBook UX391U, Ubuntu 24.04, zabbly kernel) into a self-contained campervan hub/router/AP:

• WiFi-6 access point for client devices (hand-rolled hostapd, not NetworkManager).
• Multi-WAN failover across WiFi (neighbour AP) → Ethernet (Starlink) → 4G/5G cellular.
• ZeroTier-managed DNS so *.wrede.pvt resolves over the overlay when away from home.
• Cockpit dashboard ("Van Router") for status + manual control.

This directory is the source of truth. The live system files live under /etc, /usr/...; deploy.sh copies from here to there. Edit here, run sudo ./deploy.sh, done.

---

1. Architecture at a glance

            clients ──WiFi(VanLink, 5GHz ch149, 80MHz WPA2)
                          │
        wlxc83a35a4ee55  (Realtek RTL8852BU, rtw89)  10.42.0.1/24   ← AP, hostapd
                          │  dnsmasq DHCP .10–.254 + DNS
                          │  nftables masquerade (oifname != AP)
                          │  ip_forward=1
        ┌─────────────────┼──────────────────────────────────────┐
   wlp1s0 (m100)     enxd8ec5eeb3512 (m200)        Koodo gsm (m300)
   neighbour WiFi    USB ethernet → Starlink       4G/5G modem (when present)
        └──────── van-failover picks lowest-metric HEALTHY WAN ────┘

   NetworkManager owns the WAN side.  systemd-networkd owns the AP IP + the ZeroTier DNS link.

Roles / responsibilities

Concern	Owner
WAN interfaces (eth / wifi / gsm), DHCP-client, metrics	NetworkManager
AP interface IP (10.42.0.1)	systemd-networkd (10-van-ap.network)
AP beaconing / WPA	hostapd (AP iface is NM-unmanaged)
AP DHCP + DNS	dnsmasq (dedicated instance, bound to AP only)
NAT + forwarding	nftables + sysctl
WAN health + failover	van-failover daemon
ZeroTier DNS → resolved	zerotier-systemd-manager + systemd-networkd
Web UI	Cockpit + vanrouter plugin

Key network facts

• AP LAN: 10.42.0.0/24, gateway/AP 10.42.0.1, DHCP .10–.254.
• WAN priority (metrics): wifi 100 → eth/Starlink 200 → 4G 300 (lower = preferred).
• Management / recovery: ethernet 192.168.10.251, wifi 192.168.10.27, ZeroTier 192.168.196.22.
• Cockpit: https://192.168.10.251:9090 (or .27, or ZeroTier). Log in with a Unix account; enable Administrative access for action buttons.
• Regulatory domain CA (unlocks 5GHz ch149–161 @30dBm, no DFS).
• ZeroTier network d3ecf5726d041b2a, pushed DNS domain wrede.pvt via 192.168.196.115 + 192.168.10.5.

---

2. Directory layout → system destinations

deploy.sh performs exactly this mapping.

ap/ — access point

file	→ installs to	purpose
hostapd.conf	/etc/hostapd/hostapd.conf	AP: SSID VanLink, ch149, VHT80/HE, WPA2-PSK, country CA. WPA passphrase lives here.
default-hostapd	/etc/default/hostapd	DAEMON_CONF=...
van-ap-dnsmasq.conf	/etc/van-ap/dnsmasq.conf	DHCP/DNS bound to AP iface (bind-dynamic, so it does not clash with systemd-resolved)
van-ap-dnsmasq.service	/etc/systemd/system/van-ap-dnsmasq.service	dedicated dnsmasq unit (uses the dnsmasq-base binary; the distro dnsmasq service is NOT used)
10-van-ap.network	/etc/systemd/network/10-van-ap.network	static 10.42.0.1/24 on the AP iface (ConfigureWithoutCarrier)
van-ap-unmanaged.conf	/etc/NetworkManager/conf.d/van-ap-unmanaged.conf	tells NM to leave the AP iface alone
nftables.conf	/etc/nftables.conf	NAT: masquerade ip saddr 10.42.0.0/24 oifname != <AP> → follows whatever WAN is active
99-van-router.conf	/etc/sysctl.d/99-van-router.conf	net.ipv4.ip_forward=1
regdomain.service	/etc/systemd/system/regdomain.service	iw reg set CA at boot, before NetworkManager
rtw89.conf	/etc/modprobe.d/rtw89.conf	options rtw89_core disable_ps_mode=Y (else AP drops beacon when idle)

failover/ — multi-WAN

file	→ installs to	purpose
van-failover	/usr/local/sbin/van-failover	the daemon (Python)
config.json	/etc/van-failover/config.json	WAN list + priorities + probe settings
van-failover.service	/etc/systemd/system/van-failover.service	Restart=always
50-disable-eee	/etc/NetworkManager/dispatcher.d/50-disable-eee	disables EEE on en*/eth* at connect (r8152 idle parking)
99-van-arp.conf	/etc/sysctl.d/99-van-arp.conf	arp_ignore=1 / arp_announce=2 (multi-NIC ARP hygiene)

dns/ — ZeroTier managed DNS

file	→ installs to	purpose
zt-search.conf	/etc/systemd/network/99-ztuga7c2kh.network.d/search.conf	makes wrede.pvt a search domain (bare-hostname completion)
zt-network.local.conf	/var/lib/zerotier-one/networks.d/d3ecf5726d041b2a.local.conf	contains allowDNS=1 (prerequisite)
99-ztuga7c2kh.network.generated	(reference only — generated by the manager)	what zerotier-systemd-manager writes
zerotier-systemd-manager.{service,timer}	/usr/lib/systemd/system/	the manager (binary installed separately, see §4)

cockpit/vanrouter/ — web UI

manifest.json, index.html, vanrouter.css, vanrouter.js → /usr/share/cockpit/vanrouter/.

---

3. Components & how to operate

Access Point

• Change SSID / channel / password: edit ap/hostapd.conf, then sudo cp it to /etc/hostapd/hostapd.conf (or ./deploy.sh) and sudo systemctl restart hostapd.
• Width is 80MHz VHT + HE (WiFi-6). Channel 149 (UNII-3, non-DFS).
• DHCP range / DNS options: ap/van-ap-dnsmasq.conf + restart van-ap-dnsmasq.
• Services: hostapd, van-ap-dnsmasq, systemd-networkd, nftables, regdomain.
• Do not add noscan to hostapd.conf — Ubuntu's hostapd rejects it and fails to start.
• Verify it's actually beaconing with iw dev wlxc83a35a4ee55 info (ssid+channel+width present), not just systemctl is-active hostapd.

Multi-WAN failover (van-failover)

• Probes each WAN's real internet every few seconds (HTTP-204 check, captive-portal-aware), demotes a failed WAN by raising its route-metric, fails back on recovery (hysteresis).
• Status: systemctl status van-failover, journalctl -u van-failover -f, or the Cockpit "WAN Failover" card. Live state JSON at /run/van-failover/state.json.
• Tune priorities/timing in failover/config.json (wans[].metric, probe_interval, probe_timeout, fail_threshold, ok_threshold, probe_urls) then restart.
• WAN identity: eth/wifi keyed by device, cellular by NM connection name (Koodo).
• The daemon changes metrics with ip route, never nmcli device reapply — see Gotchas.

Manual preference (Cockpit "Prefer" button)

• The daemon is the single writer of route-metrics. To force a specific WAN to the top it reads an optional file /run/van-failover/prefer containing one device name (e.g. enxd8ec5eeb3512). That WAN gets base metric 50 (below every config metric) so it wins while healthy; the health PENALTY still stacks on top, so failover/fail-back are unchanged. Empty/absent file = follow config priorities.
• The Cockpit "Prefer" button just writes that file (root via polkit) — it does not set metrics itself. Doing so directly would fight the daemon (reverted within one probe loop) and reapply-flap the USB carrier. The daemon applies the change on its next loop (~probe_interval); state.json carries a preferred flag the UI shows as "(preferred)".
• Single-WAN, sticky override with no explicit "clear": to return to automatic priority, prefer your top WAN (wifi) or rm /run/van-failover/prefer. Ephemeral by design — /run is wiped on reboot, so a reboot returns to config priorities. (To persist it, point PREFER at /etc/van-failover/prefer in the daemon and the matching path in vanrouter.js.)
• Swapping the preference between two WANs is collision-safe: a second default route can't take the new WAN's target metric while the old preferred WAN still holds it, so enforce_route keeps the existing route and retries next loop (settles in ~2 loops) instead of stranding the interface. enforce_route also restores a default route that went missing while the carrier is up (gateway from NM via device_gateway), not just rebases an existing one.

Adding the 4G/5G modem

1. Plug the USB modem in. ModemManager + the existing Koodo gsm NM connection (autoconnect) bring it up.
2. It auto-joins as the cellular WAN at metric 300 (last resort). Nothing else to configure.
3. Confirm with mmcli -L and the Cockpit failover card (cellular flips from absent to up).

ZeroTier managed DNS

• wrede.pvt resolves over ZeroTier when off the home LAN. Mechanism: allowDNS=1 (prereq) + zerotier-systemd-manager writes 99-ztuga7c2kh.network, networkd applies it to resolved.
• Verify: resolvectl status ztuga7c2kh shows DNS scope + the two servers + wrede.pvt.
• NOTE: ZeroTier's native DNS push is a no-op on Linux (setDns ... not implemented), which is why the manager is required.
• systemd-networkd-wait-online is masked so the ZT link does not gate network-online.target — see Gotchas.

Cockpit dashboard

• https://<wayback>:9090 → "Van Router". Reload the browser after deploying plugin changes (Cockpit caches packages per session).
• Read-only status works as any user; Prefer/Up/Down/Restart need Administrative access (polkit).
• Prefer sets the manual WAN preference (see Manual preference above); Up/Down connect/disconnect the NM device; Restart restarts hostapd.

---

4. Deploy / rebuild

cd ~/vanlink
sudo ./deploy.sh          # copies all files to their system locations, reloads + enables services

Two things deploy.sh does not do (one-time, manual):

1. zerotier-systemd-manager binary (v0.4.0, hand-installed — not in the ZeroTier apt repo):

   # copy /usr/bin/zerotier-systemd-manager from a host that has it (e.g. wertvoll), then:
   sudo systemctl enable --now zerotier-systemd-manager.timer
   sudo zerotier-cli set d3ecf5726d041b2a allowDNS=1
   

2. hostapd unmask (Ubuntu ships it masked): sudo systemctl unmask hostapd.

---

5. Key design decisions & hard-won gotchas

• AP on hostapd, not NetworkManager. NM's hotspot caps the rtw89 radio at HT20/20MHz; hostapd gives the full VHT80/HE (WiFi-6). The AP iface is therefore NM-unmanaged; networkd gives it its static IP.
• rtw89 power-save must be off (disable_ps_mode=Y) or the AP stops beaconing when idle and the SSID vanishes.
• RTL8852BU is USB-2.0 and hangs under load if it shares a USB hub. Keep the AP dongle on its own USB controller, separate from the WAN ethernet. Symptom of a shared bus: c2h reg timeout + Polling beacon packet empty fail under throughput, SSID drops. (timed out to flush queues alone is benign.)
• van-failover changes metrics via ip route, never nmcli device reapply. reapply resets the r8152 USB-ethernet carrier, which caused a ~10s-ping-drop flapping feedback loop. Pure ip route changes are carrier-safe.
• Disable EEE on USB ethernet (50-disable-eee dispatcher) — its idle power-save parks the backup WAN link and breaks health probes.
• Per-WAN probing needs curl --interface if!<dev> (forces SO_BINDTODEVICE); plain --interface <name> only sets the source IP and still routes via the default WAN. rp_filter is loose (2), required for this.
• arp_ignore/arp_announce matter only when two WANs share a subnet (a home-LAN test artifact; Starlink + neighbour-wifi will be on different subnets in the van). Harmless to keep.
• dnsmasq uses bind-dynamic bound to the AP iface so it coexists with systemd-resolved (no port-53 fight) — resolved stays intact for the host.
• systemd-networkd-wait-online must be masked or it deadlocks boot. networkd here manages only two links — the AP (wlxc…, local-only) and the ZT overlay ztuga7c2kh — and neither is a real uplink. The ZT link only appears once zerotier-one starts, but zerotier-one is ordered After=network-online.target, so wait-online would block on the not-yet-existing link until its 120s timeout. Marking the links RequiredForOnline=no does not help: with zero candidate links, wait-online can never satisfy "online" and times out anyway (systemd 255 — verified: Timeout occurred while waiting for network connectivity at exactly +120s). Both add ~2min to every boot and delay ZeroTier (the recovery path). The fix is to mask the service entirely (deploy.sh does this); real uplink readiness is covered by NetworkManager-wait-online, which owns the actual WANs.

---

6. Recovery / safety

• Three independent management paths: ethernet .251, wifi .27, ZeroTier 192.168.196.22. Changing default-route metrics does not affect the local-subnet (SSH) routes.
• The AP fan/EC, hostapd, networkd, nftables, sysctl, failover, regdomain are all enabled and reboot-persistent.

7. Pending / future ideas

• Real-world soak once the actual Starlink terminal + USB 4G/5G modem are installed (current eth is the home LAN; subnets/probe targets may want tuning).
• Cockpit plugin: add an AP-config form (SSID/channel/PSK editing hostapd.conf). (Manual failover override via the "Prefer" button is done — see §3.)
• Consider pinning a fixed interface name for the WAN ethernet (USB MAC-derived names change if the adapter is swapped).