ap: switch 5GHz driver to out-of-tree morrownr/rtw89 for USB3 support

The in-kernel rtw89_8852bu on this Ubuntu kernel base predates mainline's
USB2->3 auto-switch for this chip, permanently capping the AP dongle at
USB2/480M. morrownr/rtw89 (dkms) has that switch. ap/rtw89.conf now
blacklists the in-kernel rtw89 modules and tunes the replacement
(disable_ps_mode + switch_usb_mode); ap/install-rtw89-driver.sh builds
and installs it, pinned to a specific upstream commit, kept separate
from deploy.sh since a dkms rebuild is too slow to run on every deploy.

Verified live: negotiates USB3/5000M on a dedicated USB3 controller
(480M on a USB2-only one, as expected), AP recovered via hostapd's
Restart=always + van-ap-watchdog with no manual intervention. Hit and
documented one real gotcha along the way: in-kernel rtw89_core refused
to unload while its own dependents (rtw89_8852b, rtw89_8852b_common)
were still loaded, which blocked the new module with a duplicate-symbol
error until all in-kernel modules were removed first.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJfEELeh3ercpRBp8yYrYS

README.md

@@ -92,7 +92,7 @@ This directory is the source of truth. The live system files live under `/etc`,
 | `nftables.conf` | `/etc/nftables.conf` | NAT: `masquerade ip saddr 10.42.0.0/24 oifname != br0` → follows whatever WAN is active |
 | `99-van-router.conf` | `/etc/sysctl.d/99-van-router.conf` | `net.ipv4.ip_forward=1` |
 | `regdomain.service` | `/etc/systemd/system/regdomain.service` | `iw reg set CA` at boot, before NetworkManager |
-| `rtw89.conf` | `/etc/modprobe.d/rtw89.conf` | `options rtw89_core disable_ps_mode=Y` (else AP drops beacon when idle) |
+| `rtw89.conf` | `/etc/modprobe.d/rtw89.conf` | 5GHz dongle: `disable_ps_mode=Y` (else AP drops beacon when idle) + blacklists in-kernel rtw89 so the out-of-tree morrownr/rtw89 dkms driver (installed separately, not by deploy.sh) loads instead — see gotchas below |
 | `rtw88.conf` | `/etc/modprobe.d/rtw88.conf` | 2.4GHz dongle: `disable_lps_deep=Y` (same PS reasoning) + `switch_usb_mode=N` (don't self-upgrade to USB3 — it radiates into 2.4GHz) |
 
 ### `failover/` — multi-WAN
@@ -366,7 +366,8 @@ Four things `deploy.sh` does **not** do (one-time, manual):
 
 - **AP on hostapd, not NetworkManager.** NM's hotspot caps the rtw89 radio at HT20/20MHz; hostapd gives the full VHT80/HE (WiFi-6). The AP iface is therefore NM-*unmanaged*; networkd gives it its static IP.
 - **`rtw89` power-save must be off** (`disable_ps_mode=Y`) or the AP stops beaconing when idle and the SSID vanishes.
-- **RTL8852BU is USB-2.0 and hangs under load if it shares a USB hub.** Keep the AP dongle on its **own** USB controller, separate from the WAN ethernet. Symptom of a shared bus: `c2h reg timeout` + `Polling beacon packet empty fail` under throughput, SSID drops. (`timed out to flush queues` alone is benign.)
+- **RTL8852BU is USB-2.0 and hangs under load if it shares a USB hub.** Keep the AP dongle on its **own** USB controller, separate from the WAN ethernet. Symptom of a shared bus: `c2h reg timeout` + `Polling beacon packet empty fail` under throughput, SSID drops. (`timed out to flush queues` alone is benign.) True regardless of which driver (in-kernel or out-of-tree) is bound — it's a physical bus-bandwidth issue, not a driver bug.
+- **5GHz dongle runs the out-of-tree `morrownr/rtw89` driver (since 2026-08-23), not the in-kernel one.** This Ubuntu kernel base (`7.0.0-1017-raspi`) predates mainline's rtw89 USB2→3 auto-switch, so the in-kernel driver permanently caps the 8852BU at USB2/480M. The out-of-tree driver has that switch (`switch_usb_mode=y` in `ap/rtw89.conf`) and needs a real USB3-capable port to actually benefit from it — a USB2-only port still caps it regardless of driver (confirmed live: 480M on a USB2-only controller, 5000M/SuperSpeed after moving to a dedicated USB3 controller). Driver itself logs `"2.4 GHz performance may be better in a USB 2 port"` on load — watch the 2.4GHz radio for new interference now that 5GHz runs SuperSpeed signalling nearby. Installed/updated via `sudo ./ap/install-rtw89-driver.sh` (pins a specific upstream commit, `dkms install`, `make install_fw`) — **not** run automatically by `deploy.sh`, since a dkms rebuild is slow and shouldn't fire on every routine deploy; run it manually after a fresh Pi provisioning or to bump the pin. DKMS itself auto-rebuilds across kernel upgrades. Module names get a `_git` suffix (`rtw89_8852bu_git` etc.) — `ap/rtw89.conf`'s blacklist stops the in-kernel modules from claiming the device instead. **Gotcha hit switching a live system over**: the in-kernel `rtw89_core` won't unload while its own in-kernel dependents (`rtw89_8852b`, `rtw89_8852b_common`) are still loaded — remove all of them first or the out-of-tree module fails to load (`exports duplicate symbol`). The install script's own output has the exact recovery steps.
 - **van-failover changes metrics via `ip route`, never `nmcli device reapply`.** `reapply` **resets the r8152 USB-ethernet carrier**, which caused a ~10s-ping-drop flapping feedback loop. Pure `ip route` changes are carrier-safe.
 - **Disable EEE on USB ethernet** (`50-disable-eee` dispatcher) — its idle power-save parks the *backup* WAN link and breaks health probes.
 - **Per-WAN probing needs `curl --interface if!<dev>`** (forces `SO_BINDTODEVICE`); plain `--interface <name>` only sets the source IP and still routes via the default WAN. `rp_filter` is loose (`2`), required for this.

ap/install-rtw89-driver.sh

@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+# ap/install-rtw89-driver.sh — install/update the out-of-tree morrownr/rtw89 driver
+# for the 5GHz AP dongle (RTL8852BU, USB).
+#
+# Why: this Ubuntu kernel base predates mainline rtw89's USB2->3 auto-switch for this
+# chip, so the in-kernel rtw89_8852bu permanently caps the dongle at USB2/480M. The
+# morrownr/rtw89 driver has that switch. ap/rtw89.conf blacklists the in-kernel rtw89
+# modules so udev loads this one instead — that file is deploy.sh's job to install;
+# this script is the (heavier, slower, not idempotent-per-deploy) driver build/install
+# itself, kept separate so a routine `deploy.sh` run doesn't trigger a dkms rebuild.
+#
+# Run manually after a fresh Pi provisioning, or to bump PIN_COMMIT to a newer upstream
+# revision: sudo ./ap/install-rtw89-driver.sh
+#
+# Safe to re-run — no-ops if the pinned commit is already built+installed for the
+# running kernel (e.g. after a kernel upgrade where DKMS's own auto-rebuild already
+# handled it).
+set -euo pipefail
+
+[ "$(id -u)" -eq 0 ] || { echo "Run as root (sudo)." >&2; exit 1; }
+
+REPO_URL=https://github.com/morrownr/rtw89
+# Pinned for reproducibility — bump deliberately, not by tracking a moving branch.
+PIN_COMMIT=8b2b78deb357d01fd5808164046e31239139ed9f
+PKG_NAME=rtw89
+PKG_VERSION=7.3   # from upstream's dkms.conf; independent of PIN_COMMIT
+SRC="/usr/src/${PKG_NAME}-${PKG_VERSION}"
+KVER=$(uname -r)
+
+if [ -d "$SRC/.git" ]; then
+    git -C "$SRC" fetch origin
+else
+    rm -rf "$SRC"
+    git clone "$REPO_URL" "$SRC"
+fi
+git -C "$SRC" checkout "$PIN_COMMIT"
+
+if dkms status "${PKG_NAME}/${PKG_VERSION}" 2>/dev/null | grep -q "${KVER}.*: installed" \
+   && [ "$(git -C "$SRC" rev-parse HEAD)" = "$PIN_COMMIT" ]; then
+    echo "rtw89 driver already installed at pinned commit ${PIN_COMMIT:0:12} for ${KVER}; nothing to do."
+    exit 0
+fi
+
+dkms remove "${PKG_NAME}/${PKG_VERSION}" --all 2>/dev/null || true
+dkms add "$SRC"
+dkms install "${PKG_NAME}/${PKG_VERSION}"
+make -C "$SRC" install_fw
+
+cat <<EOF
+
+rtw89 driver installed (commit ${PIN_COMMIT:0:12}) for kernel ${KVER}.
+
+If in-kernel rtw89 modules are currently loaded, they won't unload themselves —
+loading this driver alongside them fails with "exports duplicate symbol" (hit this
+2026-08-23: rtw89_core stayed resident because rtw89_8852b/rtw89_8852b_common, its own
+in-kernel dependents, were still loaded too). To switch a live system over:
+
+  sudo rmmod rtw89_8852bu rtw89_8852b rtw89_8852b_common rtw89_core rtw89_usb 2>/dev/null
+  # then force the AP dongle to re-enumerate, e.g.:
+  echo 0 | sudo tee /sys/bus/usb/devices/<bus-port>/authorized
+  echo 1 | sudo tee /sys/bus/usb/devices/<bus-port>/authorized
+
+(or just reboot). hostapd's Restart=always + van-ap-watchdog recover the AP
+automatically once the new driver claims the interface — no manual hostapd restart
+needed. ap/rtw89.conf must already be deployed (sudo ./deploy.sh) so the blacklist is
+in place before the device re-enumerates, or the in-kernel driver will just reclaim it.
+EOF

ap/rtw89.conf

@@ -1 +1,48 @@
-options rtw89_core disable_ps_mode=Y
+# 5GHz AP dongle (Realtek RTL8852BU). Since 2026-08-23 this uses the out-of-tree
+# morrownr/rtw89 driver (dkms package "rtw89", modules suffixed _git) instead of the
+# in-kernel rtw89_8852bu — the in-kernel driver on this Ubuntu kernel base predates
+# mainline's USB2->3 auto-switch, so the dongle was permanently capped at USB2/480M.
+# Install: https://github.com/morrownr/rtw89 (`sudo dkms install`, `sudo make
+# install_fw`). This file both tunes the driver and blacklists the in-kernel modules
+# so udev picks the out-of-tree ones on (re)enumeration.
+options rtw89_core_git disable_ps_mode=Y
+options rtw89_usb_git switch_usb_mode=y
+
+# Blacklist the in-kernel rtw89 drivers.
+blacklist rtw89_8851bu
+blacklist rtw89_8851be
+blacklist rtw89_8851b
+blacklist rtw89_8852au
+blacklist rtw89_8852ae
+blacklist rtw89_8852a
+blacklist rtw89_8852b_common
+blacklist rtw89_8852bu
+blacklist rtw89_8852be
+blacklist rtw89_8852b
+blacklist rtw89_8852bte
+blacklist rtw89_8852bt
+blacklist rtw89_8852cu
+blacklist rtw89_8852ce
+blacklist rtw89_8852c
+blacklist rtw89_8922au
+blacklist rtw89_8922ae
+blacklist rtw89_8922a
+blacklist rtw89_8922de
+blacklist rtw89_8922d
+blacklist rtw89_core
+blacklist rtw89_usb
+blacklist rtw89_pci
+
+# Blacklist Larry Finger's out-of-tree rtw89 driver too, in case it's ever installed.
+blacklist rtw89core
+blacklist rtw89pci
+blacklist rtw_8851b
+blacklist rtw_8851be
+blacklist rtw_8852a
+blacklist rtw_8852ae
+blacklist rtw_8852b
+blacklist rtw_8852be
+blacklist rtw_8852c
+blacklist rtw_8852ce
+blacklist rtw_8922a
+blacklist rtw_8922ae