diff --git a/README.md b/README.md index 16a820f..a47ccb8 100644 --- a/README.md +++ b/README.md @@ -16,11 +16,12 @@ This directory is the source of truth. The live system files live under `/etc`, ## 1. Architecture at a glance ``` - clients ──WiFi(VanLink, 5GHz ch149, 80MHz WPA2) - │ - wlxc83a35a4ee55 (Realtek RTL8852BU, rtw89) 10.42.0.1/24 ← AP, hostapd + WiFi clients (VanLink, 5GHz ch149, 80MHz WPA2) wired LAN clients + wlxc83a35a4ee55 (RTL8852BU, rtw89, hostapd) enx00e04c331140 + └───────────┬───────────────────┘ + br0 10.42.0.1/24 ← LAN bridge │ dnsmasq DHCP .10–.254 + DNS - │ nftables masquerade (oifname != AP) + │ nftables masquerade (oifname != br0) │ ip_forward=1 ┌─────────────────┼──────────────────────────────────────┐ wlp1s0 (m100) enxd8ec5eeb3512 (m200) Koodo gsm (m300) @@ -34,8 +35,8 @@ This directory is the source of truth. The live system files live under `/etc`, | Concern | Owner | |---|---| | WAN interfaces (eth / wifi / gsm), DHCP-client, metrics | **NetworkManager** | -| AP interface IP (10.42.0.1) | **systemd-networkd** (`10-van-ap.network`) | -| AP beaconing / WPA | **hostapd** (AP iface is NM-*unmanaged*) | +| LAN bridge `br0` + IP (10.42.0.1) + wired LAN member | **systemd-networkd** (`2x-van-br0/lan`) | +| AP beaconing / WPA + adding the wlan to `br0` | **hostapd** (`bridge=br0`; AP iface + wired port are NM-*unmanaged*) | | AP DHCP + DNS | **dnsmasq** (dedicated instance, bound to AP only) | | NAT + forwarding | **nftables** + sysctl | | WAN health + failover | **van-failover** daemon | @@ -47,7 +48,7 @@ This directory is the source of truth. The live system files live under `/etc`, | Web UI | **Cockpit** + `vanrouter` plugin | ### Key network facts -- AP LAN: `10.42.0.0/24`, gateway/AP `10.42.0.1`, DHCP `.10–.254`. +- LAN: `10.42.0.0/24`, gateway `10.42.0.1`, DHCP `.10–.254` — shared by Wi-Fi (`VanLink`) and the wired port `enx00e04c331140`, both bridged into `br0`. - WAN priority (metrics): **wifi 100 → eth/Starlink 200 → 4G 300** (lower = preferred). - Management / recovery: ethernet `192.168.10.251`, wifi `192.168.10.27`, ZeroTier `192.168.196.22`. - Cockpit: `https://192.168.10.251:9090` (or `.27`, or ZeroTier). Log in with a Unix account; enable *Administrative access* for action buttons. @@ -67,9 +68,12 @@ This directory is the source of truth. The live system files live under `/etc`, | `default-hostapd` | `/etc/default/hostapd` | `DAEMON_CONF=...` | | `van-ap-dnsmasq.conf` | `/etc/van-ap/dnsmasq.conf` | DHCP/DNS bound to AP iface (`bind-dynamic`, so it does not clash with systemd-resolved) | | `van-ap-dnsmasq.service` | `/etc/systemd/system/van-ap-dnsmasq.service` | dedicated dnsmasq unit (uses the `dnsmasq-base` binary; the distro dnsmasq service is NOT used) | -| `10-van-ap.network` | `/etc/systemd/network/10-van-ap.network` | static `10.42.0.1/24` on the AP iface (`ConfigureWithoutCarrier`) | -| `van-ap-unmanaged.conf` | `/etc/NetworkManager/conf.d/van-ap-unmanaged.conf` | tells NM to leave the AP iface alone | -| `nftables.conf` | `/etc/nftables.conf` | NAT: `masquerade ip saddr 10.42.0.0/24 oifname != ` → follows whatever WAN is active | +| `10-van-ap.network` | `/etc/systemd/network/10-van-ap.network` | brings the AP wlan up with **no** IP (it's a `br0` member; hostapd enslaves it) | +| `20-van-br0.netdev` | `/etc/systemd/network/20-van-br0.netdev` | creates the LAN bridge `br0` (STP off) | +| `21-van-br0.network` | `/etc/systemd/network/21-van-br0.network` | static `10.42.0.1/24` on `br0` (`ConfigureWithoutCarrier`) | +| `22-van-lan.network` | `/etc/systemd/network/22-van-lan.network` | enslaves the wired LAN port `enx00e04c331140` to `br0` | +| `van-ap-unmanaged.conf` | `/etc/NetworkManager/conf.d/van-ap-unmanaged.conf` | tells NM to leave the AP wlan **and** the wired LAN port alone | +| `nftables.conf` | `/etc/nftables.conf` | NAT: `masquerade ip saddr 10.42.0.0/24 oifname != br0` → follows whatever WAN is active | | `99-van-router.conf` | `/etc/sysctl.d/99-van-router.conf` | `net.ipv4.ip_forward=1` | | `regdomain.service` | `/etc/systemd/system/regdomain.service` | `iw reg set CA` at boot, before NetworkManager | | `rtw89.conf` | `/etc/modprobe.d/rtw89.conf` | `options rtw89_core disable_ps_mode=Y` (else AP drops beacon when idle) | diff --git a/ap/10-van-ap.network b/ap/10-van-ap.network index eabb9c2..976b150 100644 --- a/ap/10-van-ap.network +++ b/ap/10-van-ap.network @@ -4,6 +4,9 @@ Name=wlxc83a35a4ee55 [Link] RequiredForOnline=no +# No IP here: the AP wlan is a member of br0, which carries the 10.42.0.1 gateway. +# hostapd adds this interface to br0 (bridge=br0) after it puts the radio into AP +# mode — networkd must NOT enslave it (a station-mode wlan can't be bridged), so +# this file only brings the link up. [Network] -Address=10.42.0.1/24 ConfigureWithoutCarrier=yes diff --git a/ap/20-van-br0.netdev b/ap/20-van-br0.netdev new file mode 100644 index 0000000..9625f90 --- /dev/null +++ b/ap/20-van-br0.netdev @@ -0,0 +1,10 @@ +# LAN bridge: joins the AP Wi-Fi (added by hostapd, once it's in AP mode) and the +# wired LAN port (enx00e04c331140, added by networkd) into one 10.42.0.0/24 segment. +# STP off: only two member ports, no loops, and it avoids the forwarding delay that +# would otherwise stall the first DHCP handshake on a freshly-plugged client. +[NetDev] +Name=br0 +Kind=bridge + +[Bridge] +STP=no diff --git a/ap/21-van-br0.network b/ap/21-van-br0.network new file mode 100644 index 0000000..fbff32d --- /dev/null +++ b/ap/21-van-br0.network @@ -0,0 +1,12 @@ +# The LAN gateway IP now lives on the bridge, not on the AP wlan (which is just a +# bridge member). ConfigureWithoutCarrier keeps 10.42.0.1 up even before any wired +# device is plugged, exactly as it was on the wlan-only setup. +[Match] +Name=br0 + +[Link] +RequiredForOnline=no + +[Network] +Address=10.42.0.1/24 +ConfigureWithoutCarrier=yes diff --git a/ap/22-van-lan.network b/ap/22-van-lan.network new file mode 100644 index 0000000..029bf91 --- /dev/null +++ b/ap/22-van-lan.network @@ -0,0 +1,11 @@ +# Wired LAN port: enslaved to br0 so anything plugged in lands on 10.42.0.0/24 and +# gets DHCP/DNS from the same dnsmasq as Wi-Fi clients. NM must leave this device +# alone (see van-ap-unmanaged.conf) for networkd to own it here. +[Match] +Name=enx00e04c331140 + +[Link] +RequiredForOnline=no + +[Network] +Bridge=br0 diff --git a/ap/hostapd.conf b/ap/hostapd.conf index c8b9fdd..04e27e5 100644 --- a/ap/hostapd.conf +++ b/ap/hostapd.conf @@ -1,4 +1,8 @@ interface=wlxc83a35a4ee55 +# Put the AP into br0 so Wi-Fi and the wired LAN port share one 10.42.0.0/24 segment. +# hostapd adds the wlan to the bridge after setting AP mode; the bridge itself + its +# wired member + the gateway IP are defined under /etc/systemd/network (2x-van-br0/lan). +bridge=br0 driver=nl80211 ssid=VanLink country_code=CA diff --git a/ap/nftables.conf b/ap/nftables.conf index a41bb89..95766fd 100755 --- a/ap/nftables.conf +++ b/ap/nftables.conf @@ -1,11 +1,26 @@ #!/usr/sbin/nft -f # van-router NAT — masquerade LAN out whatever the WAN of the moment is -# (anything that is NOT the AP interface: ethernet/Starlink, wifi, future 4G) +# (anything that is NOT the LAN bridge br0: ethernet/Starlink, wifi, future 4G) table ip van_router_nat delete table ip van_router_nat table ip van_router_nat { + chain prerouting { + type nat hook prerouting priority dstnat; policy accept; + # Expose the Home Assistant VM (on libvirt's NAT net, 192.168.122.50) on the + # router's own LAN IP, so VanLink clients reach it at http://10.42.0.1:8123. + # The matching FORWARD accept lives in iptables (re-applied by the libvirt + # /etc/libvirt/hooks/network hook), since libvirt's reject is in that chain. + ip daddr 10.42.0.1 tcp dport 8123 dnat to 192.168.122.50:8123 + } chain postrouting { type nat hook postrouting priority srcnat; policy accept; - ip saddr 10.42.0.0/24 oifname != "wlxc83a35a4ee55" masquerade + ip saddr 10.42.0.0/24 oifname != "br0" masquerade + # ZeroTier clients reaching the HA VM via the 10.42.0.1:8123 DNAT: masquerade + # them onto virbr0 so HA sees the router (192.168.122.1) and the reply path is + # symmetric, exactly like VanLink clients. Matched by ZT ingress interface so it + # covers direct members (192.168.196.x) AND the LAN subnets routed in behind other + # ZT nodes (192.168.0/6/8/10/13.x). Scoped to virbr0 so ZT<->ZT and ZT->WAN are + # untouched. + iifname "ztuga7c2kh" oifname "virbr0" masquerade } } diff --git a/ap/van-ap-dnsmasq.conf b/ap/van-ap-dnsmasq.conf index 684d135..1639585 100644 --- a/ap/van-ap-dnsmasq.conf +++ b/ap/van-ap-dnsmasq.conf @@ -1,4 +1,4 @@ -interface=wlxc83a35a4ee55 +interface=br0 bind-dynamic except-interface=lo domain-needed diff --git a/ap/van-ap-unmanaged.conf b/ap/van-ap-unmanaged.conf index ed8ef8b..9901a1c 100644 --- a/ap/van-ap-unmanaged.conf +++ b/ap/van-ap-unmanaged.conf @@ -1,2 +1,4 @@ [keyfile] -unmanaged-devices=interface-name:wlxc83a35a4ee55 +# The AP wlan and the wired LAN port are both owned by systemd-networkd/hostapd +# (bridged into br0), so NetworkManager must not touch either of them. +unmanaged-devices=interface-name:wlxc83a35a4ee55;interface-name:enx00e04c331140 diff --git a/deploy.sh b/deploy.sh index 8a93a72..62da8f8 100755 --- a/deploy.sh +++ b/deploy.sh @@ -13,6 +13,9 @@ install -D -m0644 ap/default-hostapd /etc/default/hostapd install -D -m0644 ap/van-ap-dnsmasq.conf /etc/van-ap/dnsmasq.conf install -D -m0644 ap/van-ap-dnsmasq.service /etc/systemd/system/van-ap-dnsmasq.service install -D -m0644 ap/10-van-ap.network /etc/systemd/network/10-van-ap.network +install -D -m0644 ap/20-van-br0.netdev /etc/systemd/network/20-van-br0.netdev +install -D -m0644 ap/21-van-br0.network /etc/systemd/network/21-van-br0.network +install -D -m0644 ap/22-van-lan.network /etc/systemd/network/22-van-lan.network install -D -m0644 ap/van-ap-unmanaged.conf /etc/NetworkManager/conf.d/van-ap-unmanaged.conf install -D -m0644 ap/nftables.conf /etc/nftables.conf install -D -m0644 ap/regdomain.service /etc/systemd/system/regdomain.service @@ -86,7 +89,11 @@ if [ -x /home/andreas/bin/hbc ]; then systemctl enable hbc >/dev/null 2>&1 || true systemctl restart hbc fi -# restart in dependency order; AP iface IP first, then hostapd/dnsmasq, then NAT/failover +# Pick up the unmanaged-devices change so NM releases the wired LAN port (drops its +# old 192.168.10.x lease); networkd then enslaves it to br0 on the restart below. +nmcli general reload 2>/dev/null || systemctl reload NetworkManager 2>/dev/null || true +# restart in dependency order: br0 + AP iface + LAN member first, then hostapd adds +# the wlan to br0, then dnsmasq binds br0, then NAT/failover systemctl restart systemd-networkd systemctl restart hostapd van-ap-dnsmasq nftables van-failover networkctl reload 2>/dev/null || true