Files
Andreas WredeandClaude Opus 4.8 8fcc959551 robustness: hardware watchdog + hbc heartbeat client
Two TODO Tier-1 items:

- Hardware watchdog: power/10-vanlink-watchdog.conf drop-in sets
  RuntimeWatchdogSec=20s + RebootWatchdogSec=5min; deploy.sh applies it via
  daemon-reexec. PID1 pets intel_oc_wdt; a >20s systemd hang hard-resets the box.

- Heartbeat / dead-man's switch: vendored hbc.yaml + hbc.service report to
  hbd.wrede.pvt (UDP 50003) every 15s as user andreas, shipping cpu/mem/disk/
  network/zfs metrics. The hbc binary is installed once via the heartbeat
  project's installer (README §4); deploy.sh starts the service when present.

Auto-power-on-when-mains-returns: documented as a BIOS-only option (not
OS-controllable on this ZenBook) in README §3 + TODO, per decision.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 22:20:09 -04:00

71 lines
3.7 KiB
Markdown

# TODO — robustness backlog for the van router (`wayback`)
Candidate improvements toward a robust unattended mobile router, tiered by
value-for-effort. Generated 2026-06-29 from a robustness review of the live box.
## Already in place (for reference — don't redo)
- [x] Multi-WAN failover (wifi → Starlink → 4G) — `van-failover`
- [x] Thermal monitoring + alerts + history — `van-thermal`
- [x] Battery low-charge Pushover alerts + safe auto-shutdown — `van-battery`
- [x] Mains ↔ battery transition alerts
- [x] Cockpit dashboard (AP, temps, battery, failover, WAN)
- [x] Never-sleep / lid-closed operation
- [x] `unattended-upgrades` enabled
- [x] `smartd` active (SMART being read) — but see Tier 2 (alerts go nowhere)
- [x] Time sync healthy (NTP synced)
- [x] journald within sane ZFS limits
---
## Tier 1 — high value, pieces already on hand
- [x] **systemd hardware watchdog** — DONE. `power/10-vanlink-watchdog.conf` drop-in sets
`RuntimeWatchdogSec=20s` + `RebootWatchdogSec=5min`; PID1 pets `intel_oc_wdt`, chip
hard-resets the box if systemd hangs >20s. Deployed via `daemon-reexec`.
- [x] **Auto power-on when mains returns** — RESOLVED as "document only". Not
OS-controllable on this ZenBook: `/proc/acpi/wakeup` is wake-from-suspend (S3/S4) only,
and "restore on AC loss" is a BIOS/firmware feature this laptop likely doesn't expose.
The 10% auto-shutdown is therefore a one-way trip until manual power-on. **Action:**
check BIOS setup for an "AC power-on / restore on AC loss" option and enable it if
present (documented in README §3, Battery monitor).
- [x] **Heartbeat / dead-man's switch** — DONE. `hbc` (heartbeat client, from
git.wrede.ca/andreas/heartbeat) installed to `~/venvs/hbd` + `~/bin/hbc`; `hbc.service`
reports to **hbd.wrede.pvt** (UDP 50003) every 15s as user `andreas`, shipping cpu/mem/
disk/network/zfs metrics. Config `/etc/hbc.yaml`, vendored in `heartbeat/`.
- [ ] **Connectivity alerts → Pushover**`van-failover` already knows when it fails
over or when ALL WANs are down; it just logs it. Pipe those events to the Pushover
plumbing so internet loss is alerted, not just battery.
## Tier 2 — worth doing soon
- [ ] **smartd → Pushover** — smartd alerts default to emailing root (a black hole on a
headless box). Wire `-M exec` to a Pushover script so NVMe SMART/wear warnings
(reallocated sectors, wear-leveling) actually reach a phone.
- [ ] **ZFS hygiene** (root is `zroot`) — schedule a periodic scrub (with zfs-zed
alerting), and auto-snapshot before `deploy.sh` for one-command rollback. Cap ARC if
RAM ever gets tight.
- [ ] **Security hardening**:
- [ ] Cockpit binds `*:9090` (all interfaces incl. WANs unless nftables drops inbound —
confirm). Restrict to LAN/ZeroTier.
- [ ] Add `fail2ban` for Cockpit/SSH.
- [ ] WPA3-transition on the AP.
- [ ] Guest SSID isolated from trusted devices.
- [ ] **Cellular data-cap tracking**`vnstat` or nftables counters on the gsm iface +
a Pushover alert near the monthly cap, to avoid bill shock when parked on 4G.
## Tier 3 — situational
- [ ] **Captive-portal handling** for public-WiFi WANs (campsites / neighbour APs) —
detection at minimum, auto-auth for frequently-used ones.
- [ ] **GPS** — location logging / geofencing / offline time source.
- [ ] **Secrets backup** — git covers configs, but `/etc/van-battery/pushover.json` and
other secrets are intentionally out of git. Document a restore path so a disk swap
doesn't lose them.
---
**Suggested next:** Tier 1. The watchdog and connectivity-alerts are quick and reuse
existing plumbing; the AC-restore BIOS setting is what makes the battery auto-shutdown
genuinely safe rather than a trap.