Problem: R5C spontaneously rebooted due to hardware watchdog timeout
Cause 1: van-gps-owntracks in infinite restart loop (MQTT connect failed)
Cause 2: mosquitto only listening on localhost, so clients couldn't connect
Solution:
- Add restart limits to van-gps-owntracks and van-li3-battery services
(max 3 restarts per 5 minutes prevents infinite restart loops)
- Configure mosquitto to listen on all interfaces (0.0.0.0:1883)
- Deploy mosquitto listener config via deploy.sh
- Restart mosquitto during deploy to apply config changes
This prevents the system from hitting the 11-second hardware watchdog timeout
when services crash and systemd auto-restarts them indefinitely.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
| LAN bridge `br0` + IP (10.42.0.1) + wired LAN member | **systemd-networkd** (`2x-van-br0/lan`) |
| LAN bridge `br0` + IP (10.42.1.1) + wired LAN member | **systemd-networkd** (`2x-van-br0/lan`) |
| AP beaconing / WPA + adding the wlan to `br0` | **hostapd** (`bridge=br0`; AP iface + wired port are NM-*unmanaged*) |
| Keep the AP beaconing across USB re-enumeration (Starlink flap) | **hostapd**`Restart=always`/no start-limit drop-in + **van-ap-watchdog** daemon (recovers a wedged radio) |
| AP DHCP + DNS | **dnsmasq** (dedicated instance, bound to AP only) |
@@ -56,13 +56,13 @@ This directory is the source of truth. The live system files live under `/etc`,
| Web UI | **Cockpit** + `vanrouter` plugin |
### Key network facts
- LAN: `10.42.0.0/24`, gateway `10.42.0.1`, DHCP `.10–.254` — shared by Wi-Fi (`VanLink`, dual-band: 5GHz `wlxc83a35a4ee55` + 2.4GHz `wlxd8ec5e2faa8c`) and the wired port `enx00e04c331140`, all bridged into `br0`.
- LAN: `10.42.1.0/24`, gateway `10.42.0.1`, DHCP `.10–.254` — shared by Wi-Fi (`VanLink`, dual-band: 5GHz `wlxc83a35a4ee55` + 2.4GHz `wlxd8ec5e2faa8c`) and the wired port `enx00e04c331140`, all bridged into `br0`.
- Cockpit: `https://192.168.10.251:9090` (or `.27`, or ZeroTier). Log in with a Unix account; enable *Administrative access* for action buttons.
- Regulatory domain **CA** (unlocks 5GHz ch149–161 @30dBm, no DFS).
- ZeroTier network `d3ecf5726d041b2a`, pushed DNS domain `wrede.pvt` via `192.168.196.115` + `192.168.10.5`.
- Home Assistant runs **natively on this host** (Podman Quadlet, host network) at **`http://10.42.0.1:8123`** (name `homeassistant`). The old HAOS VM `ha_van` (`10.42.0.50`) was retired 2026-07-07.
- Home Assistant runs **natively on this host** (Podman Quadlet, host network) at **`http://10.42.1.1:8123`** (name `homeassistant`). The old HAOS VM `ha_van` (`10.42.0.50`) was retired 2026-07-07.
---
@@ -86,10 +86,10 @@ This directory is the source of truth. The live system files live under `/etc`,
| `10-van-ap.network` | `/etc/systemd/network/10-van-ap.network` | brings the AP wlan up with **no** IP (it's a `br0` member; hostapd enslaves it) |
| `11-van-ap-2g.network` | `/etc/systemd/network/11-van-ap-2g.network` | same, for the 2.4GHz wlan |
| `20-van-br0.netdev` | `/etc/systemd/network/20-van-br0.netdev` | creates the LAN bridge `br0` (STP off) |
| `homeassistant.container` | `/etc/containers/systemd/homeassistant.container` | Podman Quadlet: HA Container on the host network (`:8123`), config in `/srv/homeassistant`, host D-Bus mounted for onboard Bluetooth. |
The `homeassistant` LAN name comes from `ap/van-ap-dnsmasq.conf` (`host-record` →
`10.42.0.1`). The retired HAOS VM's domain XML (`ha_van.xml`) lives in git history.
`10.42.1.1`). The retired HAOS VM's domain XML (`ha_van.xml`) lives in git history.
`custom_components/` and `dashboards/` are **not deployed by `deploy.sh`** — HA's
`/srv/homeassistant` is hand-managed live config (it has other integrations installed
@@ -348,7 +348,7 @@ script's docstring for the full field layout.
### Home Assistant (native Podman container)
- HA Container runs on the host via a **Podman Quadlet** (`ha/homeassistant.container` →
| LAN bridge `br0` + IP (10.42.0.1) + wired LAN member | **systemd-networkd** (`2x-van-br0/lan`) |
| AP beaconing / WPA + adding the wlan to `br0` | **hostapd** (`bridge=br0`; AP iface + wired port are NM-*unmanaged*) |
| Keep the AP beaconing across USB re-enumeration (Starlink flap) | **hostapd** `Restart=always`/no start-limit drop-in + **van-ap-watchdog** daemon (recovers a wedged radio) |
| AP DHCP + DNS | **dnsmasq** (dedicated instance, bound to AP only) |
| NAT + forwarding | **nftables** + sysctl |
| WAN health + failover | **van-failover** daemon |
| ZeroTier DNS → resolved | **zerotier-systemd-manager** + systemd-networkd |
| Web UI | **Cockpit** + `vanrouter` plugin |
### Key network facts
- LAN: `10.42.0.0/24`, gateway `10.42.0.1`, DHCP `.10–.254` — shared by Wi-Fi (`VanLink`, dual-band: 5GHz `wlxc83a35a4ee55` + 2.4GHz `wlxd8ec5e2faa8c`) and the wired port `enx00e04c331140`, all bridged into `br0`.
- Cockpit: `https://192.168.10.251:9090` (or `.27`, or ZeroTier). Log in with a Unix account; enable *Administrative access* for action buttons.
- Regulatory domain **CA** (unlocks 5GHz ch149–161 @30dBm, no DFS).
- ZeroTier network `d3ecf5726d041b2a`, pushed DNS domain `wrede.pvt` via `192.168.196.115` + `192.168.10.5`.
- Home Assistant runs **natively on this host** (Podman Quadlet, host network) at **`http://10.42.0.1:8123`** (name `homeassistant`). The old HAOS VM `ha_van` (`10.42.0.50`) was retired 2026-07-07.
---
## 2. Directory layout → system destinations
`deploy.sh` performs exactly this mapping.
### `ap/` — access point
| file | → installs to | purpose |
|---|---|---|
| `hostapd.conf` | `/etc/hostapd/hostapd.conf` | AP: SSID `VanLink`, ch149, VHT80/HE, WPA2-PSK, country CA, `ctrl_interface` for the watchdog. **WPA passphrase lives here.** |
| `hostapd-restart.conf` | `/etc/systemd/system/hostapd.service.d/restart.conf` | `Restart=always` + `StartLimitIntervalSec=0` so hostapd never gives up after a USB re-enumeration bounces the radio |
| `van-ap-watchdog` | `/usr/local/sbin/van-ap-watchdog` | watches `hostapd_cli status`; restarts hostapd if the radio wedges (running but not `ENABLED`) |
| `van-ap-watchdog.service` | `/etc/systemd/system/van-ap-watchdog.service` | runs the watchdog |
| `van-ap-dnsmasq.conf` | `/etc/van-ap/dnsmasq.conf` | DHCP/DNS bound to AP iface (`bind-dynamic`, so it does not clash with systemd-resolved) |
| `van-ap-dnsmasq.service` | `/etc/systemd/system/van-ap-dnsmasq.service` | dedicated dnsmasq unit (uses the `dnsmasq-base` binary; the distro dnsmasq service is NOT used) |
| `hostapd-2g.conf` | `/etc/hostapd/hostapd-2g.conf` | 2.4GHz AP (Linksys WUSB6300v2 / RTL8822BU): same SSID+PSK, ch6, HT20. Own hostapd instance so one radio wedging never kills the other. |
| `hostapd-2g.service` | `/etc/systemd/system/hostapd-2g.service` | dedicated unit for the 2.4GHz hostapd (same Restart=always / no-start-limit semantics as the 5GHz drop-in) |
Being superseded by `router-status/` below (Cockpit's PAM+TLS login was unreliable
— rejected Firefox sometimes, failed outright from Android — and it was a resource
hog); left in place until the Home Assistant dashboard fully covers its actions.
### `router-status/` — same status/actions, via MQTT -> Home Assistant
| file | → installs to | purpose |
|---|---|---|
| `van-router-mqtt` | `/usr/local/sbin/van-router-mqtt` | status+control daemon (Python, paho-mqtt): AP/clients/Starlink/WAN/modem, HA MQTT discovery; republishes van-failover/van-thermal/van-battery's own state.json verbatim rather than recomputing them |
| `config.json` | `/etc/van-router-mqtt/config.json` | broker + interface identifiers (`@TOKEN@` templated like `failover/config.json` — no secrets, rendered fresh every deploy) |
| `pushover.json.example` | → `/etc/van-battery/pushover.json` (seeded if absent) | Pushover token+user **template**; real file is 0600, **not** in the repo |
The `hbc` binary itself (venv at `~/venvs/hbd`, symlink `~/bin/hbc`) is installed once via the
heartbeat project's own installer — see §4. `deploy.sh` only starts the service once it exists.
### `li3/` — RV house battery (BLE BMS → MQTT/HA)
| file | → installs to | purpose |
|---|---|---|
| `van-li3-battery` | `/usr/local/sbin/van-li3-battery` | BLE→MQTT daemon (Python, bleak + paho-mqtt): reads the Lithionics Li3 BMS, publishes HA MQTT discovery + state |
| `config.json.example` | → `/etc/van-li3/config.json` (seeded if absent) | BLE address, broker, MQTT topic/discovery **template**; real file is 0600, **not** in the repo |
Not to be confused with `power/van-battery` — that's the host's own AC/battery power
supply (laptop UPS-style monitor), unrelated hardware and purpose.
### `ha/` — Home Assistant (native container)
| file | → installs to | purpose |
|---|---|---|
| `homeassistant.container` | `/etc/containers/systemd/homeassistant.container` | Podman Quadlet: HA Container on the host network (`:8123`), config in `/srv/homeassistant`, host D-Bus mounted for onboard Bluetooth. |
The `homeassistant` LAN name comes from `ap/van-ap-dnsmasq.conf` (`host-record` →
`10.42.0.1`). The retired HAOS VM's domain XML (`ha_van.xml`) lives in git history.
`custom_components/` and `dashboards/` are **not deployed by `deploy.sh`** — HA's
`/srv/homeassistant` is hand-managed live config (it has other integrations installed
via HACS that were never in this repo, e.g. `mbapi2020`); these two are kept here purely
as a reference/backup copy of the hand-authored pieces. `dashboards/van-router.yaml` is
the Lovelace dashboard for `van-router-mqtt`'s entities — the live copy is
`/srv/homeassistant/dashboards/van-router.yaml`, wired in via a `lovelace: dashboards:`
block manually added to `/srv/homeassistant/configuration.yaml` (also not repo-tracked;
see that file's own comment). After editing the dashboard, copy both places and
`systemctl restart homeassistant` (or use HA's own "reload" for lovelace-only changes).
---
## 3. Components & how to operate
### Access Point
- **Dual band, one SSID**: 5GHz (`hostapd.conf` → unit `hostapd`, ch149 80MHz VHT+HE) and 2.4GHz (`hostapd-2g.conf` → unit `hostapd-2g`, ch6 HT20, for range + 2.4-only IoT). Same SSID/PSK — clients pick their band. Deliberately two hostapd processes: a USB wedge on one radio never takes the other down, and each has its own watchdog (`van-ap-watchdog`, `van-ap-watchdog-2g`).
- Change SSID / channel / password: edit `ap/hostapd.conf` and/or `ap/hostapd-2g.conf`, then `sudo cp` to `/etc/hostapd/` (or `./deploy.sh`) and restart the matching unit.
- DHCP range / DNS options: `ap/van-ap-dnsmasq.conf` + restart `van-ap-dnsmasq`.
- **Do not** add `noscan` to hostapd.conf — Ubuntu's hostapd rejects it and fails to start.
- Verify each is actually beaconing with `iw dev wlxc83a35a4ee55 info` / `iw dev wlxd8ec5e2faa8c info` (ssid+channel+width present), not just `systemctl is-active`.
### Multi-WAN failover (`van-failover`)
- Probes each WAN's real internet every few seconds (HTTP-204 check, captive-portal-aware), demotes a failed WAN by raising its route-metric, fails back on recovery (hysteresis).
- Status: `systemctl status van-failover`, `journalctl -u van-failover -f`, or the Cockpit "WAN Failover" card. Live state JSON at `/run/van-failover/state.json`.
- Tune priorities/timing in `failover/config.json` (`wans[].metric`, `probe_interval`, `probe_timeout`, `fail_threshold`, `ok_threshold`, `probe_urls`) then restart.
- WAN identity: eth/wifi keyed by `device`, cellular by NM `connection` name (`Koodo`).
- The daemon changes metrics with **`ip route`**, never `nmcli device reapply` — see Gotchas.
#### Manual preference (Cockpit "Prefer" button)
- The daemon is the **single writer** of route-metrics. To force a specific WAN to the top it reads an optional file `/run/van-failover/prefer` containing one **device name** (e.g. `enxd8ec5eeb3512`). That WAN gets base metric **50** (below every config metric) so it wins while healthy; the health PENALTY still stacks on top, so failover/fail-back are unchanged. Empty/absent file = follow config priorities.
- The Cockpit "Prefer" button just writes that file (root via polkit) — it does **not** set metrics itself. Doing so directly would fight the daemon (reverted within one probe loop) and `reapply`-flap the USB carrier. The daemon applies the change on its next loop (~`probe_interval`); `state.json` carries a `preferred` flag the UI shows as "(preferred)".
- Single-WAN, sticky override with no explicit "clear": to return to automatic priority, prefer your top WAN (wifi) or `rm /run/van-failover/prefer`. **Ephemeral by design** — `/run` is wiped on reboot, so a reboot returns to config priorities. (To persist it, point `PREFER` at `/etc/van-failover/prefer` in the daemon and the matching path in `vanrouter.js`.)
- Swapping the preference between two WANs is collision-safe: a second default route can't take the new WAN's target metric while the old preferred WAN still holds it, so `enforce_route` keeps the existing route and retries next loop (settles in ~2 loops) instead of stranding the interface. `enforce_route` also restores a default route that went missing while the carrier is up (gateway from NM via `device_gateway`), not just rebases an existing one.
### Thermal monitor (`van-thermal`)
- One daemon off a single sysfs sample loop does all three jobs: live Cockpit readout, threshold alerting, history.
- **Live state**: `/run/van-thermal/state.json` (atomic-swapped each sample). The Cockpit "Temperatures" card reads this — no shelling out to `sensors` per refresh. Sensors are resolved by hwmon **name** + **label** (`coretemp`/`Package id 0`, `nvme`/`Composite`), never by `hwmonN` index (not stable across boots).
- **Alerts**: level changes (ok ↔ warn ↔ crit) are logged to the journal with hysteresis (`clear_margin`, default 5 °C) so a sensor sitting on the line doesn't spam. Watch with `journalctl -u van-thermal -f`; warn/crit carry sd-daemon severity (`-p warning`/`-p err`).
- **History**: throttled CSV at `/var/log/van-thermal.csv` (one row per `log_interval`, default 60 s), self-rotating to `.csv.1` past `log_max_bytes` (5 MB).
- **Tuning**: edit `/etc/van-thermal/config.json` (thresholds, intervals, sensor list), then `systemctl restart van-thermal`. Defaults: CPU warn 80 / crit 95 °C (silicon crit is 100), NVMe warn 65 / crit 70 °C (drive crit ~71).
- Status: `systemctl status van-thermal` or `cat /run/van-thermal/state.json`.
### NVMe watchdog (`van-nvme-watch`)
- Event-driven, not polled: tails `journalctl -kf` for the `nvme nvmeN: I/O tag ... timeout, reset controller` signature that crashed and corrupted the root fs on 2026-08-02 (recurred 2026-08-04, self-healed — see the memory notes for that investigation). There's no sensor to sample, only a log line to catch.
- On a match it watches a `grace_period` (default 20 s) for either a clean re-init (self-healed) or a second timeout / an ext4 error following it (escalated) before alerting, so the Pushover message already says which outcome happened — no need to SSH in during a scare.
- The alert body includes a live `smartctl`/`tune2fs` snapshot (SMART health, critical-warning flag, media error count, filesystem state).
- **Live state**: `/run/van-nvme-watch/state.json` (last event, same convention as van-failover/van-thermal).
- Shares Pushover credentials with van-thermal/van-battery (`/etc/van-battery/pushover.json`).
- Status: `systemctl status van-nvme-watch`, `journalctl -u van-nvme-watch -f`, or `cat /run/van-nvme-watch/state.json`.
### Battery monitor (`van-battery`)
- Watches mains vs battery via `/sys/class/power_supply/AC0/online` (0 = on battery) and charge via `BAT0/capacity`. Both resolve by `type` (Mains/Battery) if those names ever differ.
- **Only while on battery**, it sends escalating **Pushover** alerts at **25 / 20 / 15 %**, and at **10 %** sends a final alert and runs `systemctl poweroff` (after `shutdown_grace`, default 8 s, so the alert flushes first).
- Edge-triggered per discharge episode: each severity fires once; the sequence **re-arms when mains returns**. Unplugging already below a warn level fires a single alert for the current severity (no burst), then shutdown at 10 %.
- **Credentials** live in `/etc/van-battery/pushover.json` (mode 0600), seeded from `pushover.json.example` on first deploy and **never committed**. Missing/placeholder creds disable *sending* but **not** the shutdown — running flat must always power down safely (the skip is logged to the journal).
- Tune in `/etc/van-battery/config.json` (`warn_levels`, `shutdown_level`, `poll_interval`), then `systemctl restart van-battery`.
- Status: `systemctl status van-battery`, `journalctl -u van-battery -f`, or `cat /run/van-battery/state.json`.
- **Caveat — the 10 % shutdown is one-way.** A laptop won't power itself back on when mains returns: "restore on AC loss" is a BIOS/firmware feature (not OS-controllable — `/proc/acpi/wakeup` only covers wake-from-suspend). If your BIOS exposes an "AC power-on / restore on AC loss" option, enable it so the router reboots itself once shore/solar power is back; this ZenBook likely doesn't have it, in which case a low-battery shutdown needs a manual power-on.
### Adding the 4G/5G modem
1. Plug the USB modem in. ModemManager + the `Koodo` gsm NM connection (autoconnect) bring it up.
On a fresh host create the profile once:
`nmcli con add type gsm ifname "*" con-name Koodo apn sp.koodo.com connection.autoconnect yes`
2. It auto-joins as the `cellular` WAN at metric 300 (last resort). Nothing else to configure.
3. Confirm with `mmcli -L` and the Cockpit failover card (cellular flips from `absent` to `up`).
4. Gotcha (handled in `van-failover` since the Quectel EC25-AF landed): for MBIM/QMI modems
NM's device is the control port (`cdc-wdm0`) while IP + routes live on the wwan netdev
(e.g. `wwu1u2i4`). The daemon maps via `GENERAL.IP-IFACE` — probing or `ip route`
against `cdc-wdm0` silently sees nothing. The Cockpit WAN table does the same mapping
(shows the netdev + its IP; nmcli actions still target `cdc-wdm0`).
5. The Cockpit WAN row shows signal % / tech / operator (from `mmcli`) and has a
or `mosquitto_sub -h localhost -u homeassistant -P <pw> -t van/li3_battery/state`.
### ZeroTier managed DNS
- `wrede.pvt` resolves over ZeroTier when off the home LAN. Mechanism: `allowDNS=1` (prereq) + `zerotier-systemd-manager` writes `99-ztuga7c2kh.network`, networkd applies it to resolved.
- Verify: `resolvectl status ztuga7c2kh` shows `DNS` scope + the two servers + `wrede.pvt`.
- NOTE: ZeroTier's *native* DNS push is a no-op on Linux (`setDns ... not implemented`), which is why the manager is required.
- `systemd-networkd-wait-online` is **masked** so the ZT link does **not** gate `network-online.target` — see Gotchas.
### Cockpit dashboard
- `https://<wayback>:9090` → "Van Router". Reload the browser after deploying plugin changes (Cockpit caches packages per session).
- Read-only status works as any user; Prefer/Up/Down/Restart need *Administrative access* (polkit).
- The Access Points card shows both bands (5GHz `hostapd`, 2.4GHz `hostapd-2g`) with per-band client lists and **Restart** buttons; the radio list lives in `vanrouter.js` (`const APS`).
- **Prefer** sets the manual WAN preference (see *Manual preference* above); **Up/Down** connect/disconnect the NM device.
### Router status via MQTT (`van-router-mqtt`) — replacing Cockpit above
- Publishes the same status Cockpit's plugin showed, and accepts the same actions, over
MQTT with Home Assistant discovery — no web server/login of its own; HA (already running
on this Pi) owns the UI. Broker is the local Mosquitto on `127.0.0.1:1883` (already used by
HA itself and by `van-gps-owntracks`/`li3`); it currently allows anonymous connections, so
`config.json` carries no credentials.
- State topics (`van/router/...`): `ap/5g`, `ap/2g`, `clients`, `starlink`, `wan`,
`deploywarn`, plus verbatim republishes of `van-failover`'s and `van-thermal`'s own
`/run/van-*/state.json` (those two daemons were **not** modified — `van-thermal` is
deliberately stdlib-only for boot resilience, and `van-failover`'s control loop wasn't
touched). `van/router/battery` only appears if `power/van-battery` (documented in this
README, not actually deployed on this host) ever starts writing its state file.
- History: HA started as a HAOS VM on wayback (libvirt NAT, then bridged onto `br0` at
`10.42.0.50` with a legacy-URL DNAT), moved to this Pi as an aarch64 VM 2026-07-06, and
went native 2026-07-07 — the fixed 2 GiB VM allocation starved the 4 GB Pi. The domain
XML (`ha/ha_van.xml`) and the DNAT/hairpin nftables rules are in git history.
### Never sleep (lid-closed operation)
- wayback lives lid-closed in the van and must stay up. Stock logind `HandleLidSwitch=suspend` would sleep it on lid close (even on AC). The `power/10-vanlink-nolid.conf` drop-in sets all three lid actions to `ignore`; `deploy.sh` also masks every sleep target so idle / GUI / a stray `systemctl suspend` can't suspend it.
3. **Pushover credentials** for `van-battery` — deploy seeds `/etc/van-battery/pushover.json` (0600) with placeholders; fill in your app token + user key, then `sudo systemctl restart van-battery`. Until then low-battery alerts are skipped (logged), but the 10 % auto-shutdown still works.
4. **Heartbeat client `hbc`** (dead-man's switch) — install the binary once into andreas' venv:
```bash
git clone https://git.wrede.ca/andreas/heartbeat.git ~/git/heartbeat # if not already cloned
sh ~/git/heartbeat/scripts/hb_install.sh client # -> ~/venvs/hbd + ~/bin/hbc
```
`deploy.sh` installs `/etc/hbc.yaml` + the `hbc.service` unit and starts it once `~/bin/hbc` exists. The service reports to **hbd.wrede.pvt** (UDP 50003) as user `andreas`. Upgrade later with `~/bin/hb_install.sh client`.
---
## 5. Key design decisions & hard-won gotchas
- **AP on hostapd, not NetworkManager.** NM's hotspot caps the rtw89 radio at HT20/20MHz; hostapd gives the full VHT80/HE (WiFi-6). The AP iface is therefore NM-*unmanaged*; networkd gives it its static IP.
- **`rtw89` power-save must be off** (`disable_ps_mode=Y`) or the AP stops beaconing when idle and the SSID vanishes.
- **RTL8852BU is USB-2.0 and hangs under load if it shares a USB hub.** Keep the AP dongle on its **own** USB controller, separate from the WAN ethernet. Symptom of a shared bus: `c2h reg timeout` + `Polling beacon packet empty fail` under throughput, SSID drops. (`timed out to flush queues` alone is benign.) True regardless of which driver (in-kernel or out-of-tree) is bound — it's a physical bus-bandwidth issue, not a driver bug.
- **5GHz dongle runs the out-of-tree `morrownr/rtw89` driver (since 2026-08-23), not the in-kernel one.** This Ubuntu kernel base (`7.0.0-1017-raspi`) predates mainline's rtw89 USB2→3 auto-switch, so the in-kernel driver permanently caps the 8852BU at USB2/480M. The out-of-tree driver has that switch (`switch_usb_mode=y` in `ap/rtw89.conf`) and needs a real USB3-capable port to actually benefit from it — a USB2-only port still caps it regardless of driver (confirmed live: 480M on a USB2-only controller, 5000M/SuperSpeed after moving to a dedicated USB3 controller). Driver itself logs `"2.4 GHz performance may be better in a USB 2 port"` on load — watch the 2.4GHz radio for new interference now that 5GHz runs SuperSpeed signalling nearby. Installed/updated via `sudo ./ap/install-rtw89-driver.sh` (pins a specific upstream commit, `dkms install`, `make install_fw`) — **not** run automatically by `deploy.sh`, since a dkms rebuild is slow and shouldn't fire on every routine deploy; run it manually after a fresh Pi provisioning or to bump the pin. DKMS itself auto-rebuilds across kernel upgrades. Module names get a `_git` suffix (`rtw89_8852bu_git` etc.) — `ap/rtw89.conf`'s blacklist stops the in-kernel modules from claiming the device instead. **Gotcha hit switching a live system over**: the in-kernel `rtw89_core` won't unload while its own in-kernel dependents (`rtw89_8852b`, `rtw89_8852b_common`) are still loaded — remove all of them first or the out-of-tree module fails to load (`exports duplicate symbol`). The install script's own output has the exact recovery steps.
- **van-failover changes metrics via `ip route`, never `nmcli device reapply`.** `reapply` **resets the r8152 USB-ethernet carrier**, which caused a ~10s-ping-drop flapping feedback loop. Pure `ip route` changes are carrier-safe.
- **Disable EEE on USB ethernet** (`50-disable-eee` dispatcher) — its idle power-save parks the *backup* WAN link and breaks health probes.
- **Per-WAN probing needs `curl --interface if!<dev>`** (forces `SO_BINDTODEVICE`); plain `--interface <name>` only sets the source IP and still routes via the default WAN. `rp_filter` is loose (`2`), required for this.
- **`arp_ignore`/`arp_announce`** matter only when two WANs share a subnet (a home-LAN test artifact; Starlink + neighbour-wifi will be on different subnets in the van). Harmless to keep.
- **dnsmasq uses `bind-dynamic` bound to the AP iface** so it coexists with systemd-resolved (no port-53 fight) — resolved stays intact for the host.
- **`systemd-networkd-wait-online` must be masked** or it deadlocks boot. networkd here manages only two links — the AP (`wlxc…`, local-only) and the ZT overlay `ztuga7c2kh` — and **neither is a real uplink**. The ZT link only appears once `zerotier-one` starts, but `zerotier-one` is ordered `After=network-online.target`, so wait-online would block on the not-yet-existing link until its 120s timeout. Marking the links `RequiredForOnline=no` does **not** help: with *zero* candidate links, wait-online can never satisfy "online" and times out anyway (systemd 255 — verified: `Timeout occurred while waiting for network connectivity` at exactly +120s). Both add ~2min to every boot and delay ZeroTier (the recovery path). The fix is to mask the service entirely (`deploy.sh` does this); real uplink readiness is covered by `NetworkManager-wait-online`, which owns the actual WANs.
---
## 6. Recovery / safety
- Three independent management paths: ethernet `.251`, wifi `.27`, ZeroTier `192.168.196.22`. Changing default-route metrics does **not** affect the local-subnet (SSH) routes.
- The AP fan/EC, hostapd, networkd, nftables, sysctl, failover, regdomain are all enabled and reboot-persistent.
## 7. Pending / future ideas
- Real-world soak once the actual Starlink terminal + USB 4G/5G modem are installed (current eth is the home LAN; subnets/probe targets may want tuning).
- Cockpit plugin: add an AP-config form (SSID/channel/PSK editing `hostapd.conf`). *(Manual failover override via the "Prefer" button is done — see §3.)*
- Consider pinning a fixed interface name for the WAN ethernet (USB MAC-derived names change if the adapter is swapped).
# Restart limit for van-gps-owntracks to prevent watchdog timeout
# Allows max 3 restarts per 5 minutes; after that, the service stays failed
# (systemd will not keep restarting it indefinitely)
[Service]
StartLimitIntervalSec=300
StartLimitBurst=3
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Claude Haiku 4.5