ap: keep AP beaconing across USB re-enumeration
Starlink's USB ethernet shares a hub with the rtw89 Wi-Fi adapter; when Starlink flaps, the hub re-enumerates and tears the radio down, so hostapd loses its interface and exits. Two stock defaults left the AP dark: Restart=on-failure misses clean exits, and the 5-in-10s start limit makes systemd give up during a re-enumeration storm. - hostapd-restart.conf drop-in: Restart=always, RestartSec=5, StartLimitIntervalSec=0 so hostapd retries forever until the interface returns (verified: SIGKILL -> back to state=ENABLED in ~7s). - van-ap-watchdog daemon: backstop for the case systemd can't see (hostapd running but radio wedged). Polls hostapd_cli status; restarts hostapd if the iface is present but not ENABLED, waits it out if the iface is mid-re-enumeration. - hostapd.conf: add ctrl_interface so the watchdog can read the real AP state. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Andreas Wrede
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
830ef52641
commit
877fa14d0c
@@ -37,6 +37,7 @@ This directory is the source of truth. The live system files live under `/etc`,
|
||||
| WAN interfaces (eth / wifi / gsm), DHCP-client, metrics | **NetworkManager** |
|
||||
| LAN bridge `br0` + IP (10.42.0.1) + wired LAN member | **systemd-networkd** (`2x-van-br0/lan`) |
|
||||
| AP beaconing / WPA + adding the wlan to `br0` | **hostapd** (`bridge=br0`; AP iface + wired port are NM-*unmanaged*) |
|
||||
| Keep the AP beaconing across USB re-enumeration (Starlink flap) | **hostapd** `Restart=always`/no start-limit drop-in + **van-ap-watchdog** daemon (recovers a wedged radio) |
|
||||
| AP DHCP + DNS | **dnsmasq** (dedicated instance, bound to AP only) |
|
||||
| NAT + forwarding | **nftables** + sysctl |
|
||||
| WAN health + failover | **van-failover** daemon |
|
||||
@@ -64,7 +65,10 @@ This directory is the source of truth. The live system files live under `/etc`,
|
||||
### `ap/` — access point
|
||||
| file | → installs to | purpose |
|
||||
|---|---|---|
|
||||
| `hostapd.conf` | `/etc/hostapd/hostapd.conf` | AP: SSID `VanLink`, ch149, VHT80/HE, WPA2-PSK, country CA. **WPA passphrase lives here.** |
|
||||
| `hostapd.conf` | `/etc/hostapd/hostapd.conf` | AP: SSID `VanLink`, ch149, VHT80/HE, WPA2-PSK, country CA, `ctrl_interface` for the watchdog. **WPA passphrase lives here.** |
|
||||
| `hostapd-restart.conf` | `/etc/systemd/system/hostapd.service.d/restart.conf` | `Restart=always` + `StartLimitIntervalSec=0` so hostapd never gives up after a USB re-enumeration bounces the radio |
|
||||
| `van-ap-watchdog` | `/usr/local/sbin/van-ap-watchdog` | watches `hostapd_cli status`; restarts hostapd if the radio wedges (running but not `ENABLED`) |
|
||||
| `van-ap-watchdog.service` | `/etc/systemd/system/van-ap-watchdog.service` | runs the watchdog |
|
||||
| `default-hostapd` | `/etc/default/hostapd` | `DAEMON_CONF=...` |
|
||||
| `van-ap-dnsmasq.conf` | `/etc/van-ap/dnsmasq.conf` | DHCP/DNS bound to AP iface (`bind-dynamic`, so it does not clash with systemd-resolved) |
|
||||
| `van-ap-dnsmasq.service` | `/etc/systemd/system/van-ap-dnsmasq.service` | dedicated dnsmasq unit (uses the `dnsmasq-base` binary; the distro dnsmasq service is NOT used) |
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
# Drop-in: /etc/systemd/system/hostapd.service.d/restart.conf
|
||||
#
|
||||
# The AP wlan (rtw89, USB) shares a USB hub with the Starlink ethernet (r8152, USB).
|
||||
# When Starlink flaps, the hub re-enumerates and the Wi-Fi radio is torn down with it:
|
||||
# hostapd's interface vanishes ("No such device") and it exits. Two stock defaults then
|
||||
# leave the AP dark until someone intervenes — this drop-in closes both:
|
||||
#
|
||||
# 1. Restart=on-failure ignores a *clean* exit, but hostapd exits 0 in some of these
|
||||
# cases. Restart=always covers every exit.
|
||||
# 2. The default start limit (5 starts / 10s) is exhausted by a re-enumeration storm,
|
||||
# after which systemd gives up permanently. StartLimitIntervalSec=0 disables the
|
||||
# limit so hostapd keeps retrying until the interface comes back.
|
||||
[Unit]
|
||||
StartLimitIntervalSec=0
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
# A few seconds' backoff so the USB device has a chance to re-enumerate before we retry
|
||||
# (retrying instantly just fails on "No such device" and spins).
|
||||
RestartSec=5
|
||||
@@ -3,6 +3,10 @@ interface=wlxc83a35a4ee55
|
||||
# hostapd adds the wlan to the bridge after setting AP mode; the bridge itself + its
|
||||
# wired member + the gateway IP are defined under /etc/systemd/network (2x-van-br0/lan).
|
||||
bridge=br0
|
||||
# Control socket so van-ap-watchdog (and hostapd_cli) can read the AP's real state
|
||||
# (state=ENABLED means it's actually beaconing) — used to detect a wedged radio that
|
||||
# stays "running" but stops serving after a USB glitch.
|
||||
ctrl_interface=/var/run/hostapd
|
||||
driver=nl80211
|
||||
ssid=VanLink
|
||||
country_code=CA
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
#!/usr/bin/env python3
|
||||
"""van-ap-watchdog — keep the VanLink AP beaconing.
|
||||
|
||||
The AP wlan (rtw89, USB) shares a USB hub with the Starlink ethernet (r8152, USB).
|
||||
When Starlink's link flaps the hub can re-enumerate and take the Wi-Fi radio down
|
||||
with it. The hostapd.service drop-in (restart.conf) handles the common case — hostapd
|
||||
exits and systemd restarts it (Restart=always, no start limit) until the interface
|
||||
returns. This daemon is the backstop for the case systemd *can't* see: hostapd stays
|
||||
running but the radio has wedged and stopped serving (dmesg "timed out to flush queues").
|
||||
|
||||
Every `interval` seconds it asks hostapd for its real state via the control socket
|
||||
(hostapd_cli status -> state=ENABLED). If the interface is present but the AP is not
|
||||
ENABLED for `fail_threshold` checks in a row, it clears any failed state and restarts
|
||||
hostapd. If the interface is simply gone (mid re-enumeration) it waits — there is
|
||||
nothing to restart onto, and Restart=always reclaims it when it reappears. Stdlib only.
|
||||
"""
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
HOSTAPD_CONF = "/etc/hostapd/hostapd.conf"
|
||||
CTRL_DIR = "/var/run/hostapd"
|
||||
INTERVAL = 15 # seconds between health checks (backstop cadence, not first response)
|
||||
FAIL_THRESHOLD = 2 # consecutive bad checks before restarting (debounces re-enum blips)
|
||||
|
||||
|
||||
def log(msg, level="info"):
|
||||
# systemd journal severity prefixes (sd-daemon), same convention as van-thermal.
|
||||
pri = {"info": "<6>", "warn": "<4>", "crit": "<2>"}.get(level, "<6>")
|
||||
print(pri + msg, flush=True)
|
||||
|
||||
|
||||
def ap_ifname():
|
||||
"""The AP interface name, read from hostapd.conf so there's one source of truth."""
|
||||
try:
|
||||
for line in Path(HOSTAPD_CONF).read_text().splitlines():
|
||||
m = re.match(r"\s*interface=(\S+)", line)
|
||||
if m:
|
||||
return m.group(1)
|
||||
except OSError as e:
|
||||
log(f"cannot read {HOSTAPD_CONF} ({e})", "crit")
|
||||
return None
|
||||
|
||||
|
||||
def iface_present(ifname):
|
||||
return Path(f"/sys/class/net/{ifname}").exists()
|
||||
|
||||
|
||||
def ap_enabled(ifname):
|
||||
"""True if hostapd reports the AP as beaconing (state=ENABLED). False if it's
|
||||
running but not enabled; None if the control socket is unreachable (hostapd down)."""
|
||||
try:
|
||||
out = subprocess.run(
|
||||
["hostapd_cli", "-p", CTRL_DIR, "-i", ifname, "status"],
|
||||
capture_output=True, text=True, timeout=10,
|
||||
).stdout
|
||||
except (OSError, subprocess.SubprocessError) as e:
|
||||
log(f"hostapd_cli failed: {e}", "warn")
|
||||
return None
|
||||
for line in out.splitlines():
|
||||
if line.startswith("state="):
|
||||
return line.strip() == "state=ENABLED"
|
||||
return None # no state line -> couldn't talk to hostapd
|
||||
|
||||
|
||||
def recover(ifname):
|
||||
log(f"AP {ifname} not beaconing — clearing failed state and restarting hostapd", "warn")
|
||||
# reset-failed first in case some other path parked the unit in a failed state;
|
||||
# harmless when it isn't.
|
||||
subprocess.run(["systemctl", "reset-failed", "hostapd"],
|
||||
capture_output=True, text=True)
|
||||
r = subprocess.run(["systemctl", "restart", "hostapd"],
|
||||
capture_output=True, text=True)
|
||||
if r.returncode != 0:
|
||||
log(f"hostapd restart returned {r.returncode}: {r.stderr.strip()}", "warn")
|
||||
|
||||
|
||||
def main():
|
||||
ifname = ap_ifname()
|
||||
if not ifname:
|
||||
log("no interface= in hostapd.conf; nothing to watch", "crit")
|
||||
sys.exit(1)
|
||||
log(f"van-ap-watchdog up: watching {ifname} every {INTERVAL}s "
|
||||
f"(restart after {FAIL_THRESHOLD} bad checks)")
|
||||
|
||||
bad = 0
|
||||
waiting = False # latch so "interface absent" logs once, not every tick
|
||||
while True:
|
||||
if not iface_present(ifname):
|
||||
if not waiting:
|
||||
log(f"{ifname} absent — USB re-enumeration in progress; "
|
||||
f"waiting for it to return (hostapd Restart=always will reclaim it)", "warn")
|
||||
waiting = True
|
||||
bad = 0
|
||||
else:
|
||||
waiting = False
|
||||
if ap_enabled(ifname):
|
||||
if bad:
|
||||
log(f"AP {ifname} beaconing again")
|
||||
bad = 0
|
||||
else:
|
||||
bad += 1
|
||||
if bad >= FAIL_THRESHOLD:
|
||||
recover(ifname)
|
||||
bad = 0
|
||||
time.sleep(INTERVAL)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
main()
|
||||
except KeyboardInterrupt:
|
||||
sys.exit(0)
|
||||
@@ -0,0 +1,13 @@
|
||||
[Unit]
|
||||
Description=VanLink AP watchdog — recover hostapd if the radio wedges after a USB glitch
|
||||
After=hostapd.service
|
||||
Wants=hostapd.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/usr/local/sbin/van-ap-watchdog
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -9,7 +9,10 @@ cd "$(dirname "$(readlink -f "$0")")"
|
||||
|
||||
echo "== access point =="
|
||||
install -D -m0644 ap/hostapd.conf /etc/hostapd/hostapd.conf
|
||||
install -D -m0644 ap/hostapd-restart.conf /etc/systemd/system/hostapd.service.d/restart.conf
|
||||
install -D -m0644 ap/default-hostapd /etc/default/hostapd
|
||||
install -D -m0755 ap/van-ap-watchdog /usr/local/sbin/van-ap-watchdog
|
||||
install -D -m0644 ap/van-ap-watchdog.service /etc/systemd/system/van-ap-watchdog.service
|
||||
install -D -m0644 ap/van-ap-dnsmasq.conf /etc/van-ap/dnsmasq.conf
|
||||
install -D -m0644 ap/van-ap-dnsmasq.service /etc/systemd/system/van-ap-dnsmasq.service
|
||||
install -D -m0644 ap/10-van-ap.network /etc/systemd/network/10-van-ap.network
|
||||
@@ -82,7 +85,7 @@ systemctl mask systemd-networkd-wait-online.service >/dev/null 2>&1 || true
|
||||
# pick up the lid drop-in (re-execs logind; does NOT drop the network)
|
||||
systemctl restart systemd-logind >/dev/null 2>&1 || true
|
||||
systemctl unmask hostapd >/dev/null 2>&1 || true
|
||||
systemctl enable regdomain.service hostapd van-ap-dnsmasq nftables systemd-networkd van-failover van-thermal van-battery >/dev/null 2>&1 || true
|
||||
systemctl enable regdomain.service hostapd van-ap-dnsmasq nftables systemd-networkd van-failover van-thermal van-battery van-ap-watchdog >/dev/null 2>&1 || true
|
||||
systemctl restart van-thermal van-battery
|
||||
# Heartbeat: only enable/start once the client binary is installed (README §4).
|
||||
if [ -x /home/andreas/bin/hbc ]; then
|
||||
@@ -96,6 +99,8 @@ nmcli general reload 2>/dev/null || systemctl reload NetworkManager 2>/dev/null
|
||||
# the wlan to br0, then dnsmasq binds br0, then NAT/failover
|
||||
systemctl restart systemd-networkd
|
||||
systemctl restart hostapd van-ap-dnsmasq nftables van-failover
|
||||
# AP watchdog last, after hostapd is back up (it only ever restarts a wedged hostapd)
|
||||
systemctl restart van-ap-watchdog
|
||||
networkctl reload 2>/dev/null || true
|
||||
|
||||
echo
|
||||
|
||||
Reference in New Issue
Block a user