From 387375de1e96cde5e63bc1fd1810ee2714a108ee Mon Sep 17 00:00:00 2001 From: Andreas Wrede Date: Sun, 28 Jun 2026 20:57:21 -0400 Subject: [PATCH] failover/boot: mask networkd-wait-online instead of RequiredForOnline=no MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous fix (RequiredForOnline=no on the ZT link) could not work and the ~2min boot stall returned. systemd-networkd-wait-online (systemd 255) only declares the network online when at least one managed, online-eligible link goes routable. networkd here manages only the AP (wlxc…, local-only) and the ZT overlay — and with the ZT link also marked RequiredForOnline=no (on top of the AP link that already was), wait-online had zero candidate links and could never satisfy "online", burning its full 120s timeout every boot. Journal confirmed: "Timeout occurred while waiting for network connectivity" at exactly +120s, gating network-online.target → zerotier-one (the recovery path). Mask systemd-networkd-wait-online entirely — neither networkd link is a real uplink, and real uplink readiness is already covered by NetworkManager-wait-online. Wire the mask into deploy.sh, delete the now-dead RequiredForOnline=no drop-in, and correct README §3/§5. Co-Authored-By: Claude Opus 4.8 --- README.md | 4 ++-- deploy.sh | 7 +++++-- dns/zt-required-for-online.conf | 7 ------- 3 files changed, 7 insertions(+), 11 deletions(-) delete mode 100644 dns/zt-required-for-online.conf diff --git a/README.md b/README.md index 59ff6de..9c7e41a 100644 --- a/README.md +++ b/README.md @@ -124,7 +124,7 @@ This directory is the source of truth. The live system files live under `/etc`, - `wrede.pvt` resolves over ZeroTier when off the home LAN. Mechanism: `allowDNS=1` (prereq) + `zerotier-systemd-manager` writes `99-ztuga7c2kh.network`, networkd applies it to resolved. - Verify: `resolvectl status ztuga7c2kh` shows `DNS` scope + the two servers + `wrede.pvt`. - NOTE: ZeroTier's *native* DNS push is a no-op on Linux (`setDns ... not implemented`), which is why the manager is required. -- The ZT link carries `RequiredForOnline=no` (drop-in `…network.d/required-for-online.conf`) so it does **not** gate `network-online.target` — see Gotchas. +- `systemd-networkd-wait-online` is **masked** so the ZT link does **not** gate `network-online.target` — see Gotchas. ### Cockpit dashboard - `https://:9090` → "Van Router". Reload the browser after deploying plugin changes (Cockpit caches packages per session). @@ -162,7 +162,7 @@ Two things `deploy.sh` does **not** do (one-time, manual): - **Per-WAN probing needs `curl --interface if!`** (forces `SO_BINDTODEVICE`); plain `--interface ` only sets the source IP and still routes via the default WAN. `rp_filter` is loose (`2`), required for this. - **`arp_ignore`/`arp_announce`** matter only when two WANs share a subnet (a home-LAN test artifact; Starlink + neighbour-wifi will be on different subnets in the van). Harmless to keep. - **dnsmasq uses `bind-dynamic` bound to the AP iface** so it coexists with systemd-resolved (no port-53 fight) — resolved stays intact for the host. -- **ZeroTier must be `RequiredForOnline=no`** or it deadlocks boot. `systemd-networkd-wait-online` waits for every *managed* networkd link to go routable; the ZT link `ztuga7c2kh` only appears once `zerotier-one` starts, but `zerotier-one` is ordered `After=network-online.target`. The loop breaks only via wait-online's 120s timeout — adding ~2min to every boot and delaying ZeroTier (the recovery path). The drop-in excludes the ZT link from the gate; real uplink readiness is covered by `NetworkManager-wait-online` (networkd here owns only the AP + ZT links, neither a true uplink). The AP link is already `RequiredForOnline=no`. +- **`systemd-networkd-wait-online` must be masked** or it deadlocks boot. networkd here manages only two links — the AP (`wlxc…`, local-only) and the ZT overlay `ztuga7c2kh` — and **neither is a real uplink**. The ZT link only appears once `zerotier-one` starts, but `zerotier-one` is ordered `After=network-online.target`, so wait-online would block on the not-yet-existing link until its 120s timeout. Marking the links `RequiredForOnline=no` does **not** help: with *zero* candidate links, wait-online can never satisfy "online" and times out anyway (systemd 255 — verified: `Timeout occurred while waiting for network connectivity` at exactly +120s). Both add ~2min to every boot and delay ZeroTier (the recovery path). The fix is to mask the service entirely (`deploy.sh` does this); real uplink readiness is covered by `NetworkManager-wait-online`, which owns the actual WANs. --- diff --git a/deploy.sh b/deploy.sh index 009c750..969c1c5 100755 --- a/deploy.sh +++ b/deploy.sh @@ -27,8 +27,7 @@ install -D -m0755 failover/50-disable-eee /etc/NetworkManager/dispatcher.d install -D -m0644 failover/99-van-arp.conf /etc/sysctl.d/99-van-arp.conf echo "== zerotier managed dns ==" -install -D -m0644 dns/zt-search.conf /etc/systemd/network/99-ztuga7c2kh.network.d/search.conf -install -D -m0644 dns/zt-required-for-online.conf /etc/systemd/network/99-ztuga7c2kh.network.d/required-for-online.conf +install -D -m0644 dns/zt-search.conf /etc/systemd/network/99-ztuga7c2kh.network.d/search.conf echo "== cockpit plugin ==" install -d /usr/share/cockpit/vanrouter @@ -37,6 +36,10 @@ install -m0644 cockpit/vanrouter/* /usr/share/cockpit/vanrouter/ echo "== apply ==" sysctl --system >/dev/null systemctl daemon-reload +# networkd here owns only the AP + ZT overlay (neither a real uplink), so its wait-online +# can never satisfy "online" and just burns its 120s timeout, stalling network-online.target +# and ZeroTier by ~2min every boot. Real uplink readiness is covered by NetworkManager-wait-online. +systemctl mask systemd-networkd-wait-online.service >/dev/null 2>&1 || true systemctl unmask hostapd >/dev/null 2>&1 || true systemctl enable regdomain.service hostapd van-ap-dnsmasq nftables systemd-networkd van-failover >/dev/null 2>&1 || true # restart in dependency order; AP iface IP first, then hostapd/dnsmasq, then NAT/failover diff --git a/dns/zt-required-for-online.conf b/dns/zt-required-for-online.conf deleted file mode 100644 index 0fff87a..0000000 --- a/dns/zt-required-for-online.conf +++ /dev/null @@ -1,7 +0,0 @@ -[Link] -# The ZeroTier overlay must NOT gate network-online.target. The ztuga7c2kh interface -# only exists once zerotier-one starts, but zerotier-one is ordered After=network-online.target. -# Without this, systemd-networkd-wait-online blocks on the (not-yet-existing) ZT link until its -# 120s timeout, delaying boot and ZeroTier by ~2min every reboot. Real uplink readiness is -# covered by NetworkManager-wait-online, not networkd. Drop-in (manager regenerates the main file). -RequiredForOnline=no