diff --git a/README.md b/README.md index 87b95ef..72ca0e9 100644 --- a/README.md +++ b/README.md @@ -39,6 +39,7 @@ This directory is the source of truth. The live system files live under `/etc`, | AP DHCP + DNS | **dnsmasq** (dedicated instance, bound to AP only) | | NAT + forwarding | **nftables** + sysctl | | WAN health + failover | **van-failover** daemon | +| Temperature monitor / alert / log | **van-thermal** daemon | | ZeroTier DNS → resolved | **zerotier-systemd-manager** + systemd-networkd | | Web UI | **Cockpit** + `vanrouter` plugin | @@ -90,10 +91,13 @@ This directory is the source of truth. The live system files live under `/etc`, ### `cockpit/vanrouter/` — web UI `manifest.json`, `index.html`, `vanrouter.css`, `vanrouter.js` → `/usr/share/cockpit/vanrouter/`. -### `power/` — never sleep +### `power/` — never sleep + thermal monitor | file | → installs to | purpose | |---|---|---| | `10-vanlink-nolid.conf` | `/etc/systemd/logind.conf.d/10-vanlink-nolid.conf` | logind ignores the lid in all states (closed / on AC / docked) | +| `van-thermal` | `/usr/local/sbin/van-thermal` | temperature daemon (Python): publishes state, alerts, logs history | +| `thermal-config.json` | `/etc/van-thermal/config.json` | sensors + warn/crit thresholds + sample/log intervals | +| `van-thermal.service` | `/etc/systemd/system/van-thermal.service` | `Restart=always` | `deploy.sh` additionally **masks** `sleep.target suspend.target hibernate.target hybrid-sleep.target` (no repo file — symlinks to `/dev/null` under `/etc/systemd/system/`) so nothing else can suspend either. @@ -123,6 +127,14 @@ This directory is the source of truth. The live system files live under `/etc`, - Single-WAN, sticky override with no explicit "clear": to return to automatic priority, prefer your top WAN (wifi) or `rm /run/van-failover/prefer`. **Ephemeral by design** — `/run` is wiped on reboot, so a reboot returns to config priorities. (To persist it, point `PREFER` at `/etc/van-failover/prefer` in the daemon and the matching path in `vanrouter.js`.) - Swapping the preference between two WANs is collision-safe: a second default route can't take the new WAN's target metric while the old preferred WAN still holds it, so `enforce_route` keeps the existing route and retries next loop (settles in ~2 loops) instead of stranding the interface. `enforce_route` also restores a default route that went missing while the carrier is up (gateway from NM via `device_gateway`), not just rebases an existing one. +### Thermal monitor (`van-thermal`) +- One daemon off a single sysfs sample loop does all three jobs: live Cockpit readout, threshold alerting, history. +- **Live state**: `/run/van-thermal/state.json` (atomic-swapped each sample). The Cockpit "Temperatures" card reads this — no shelling out to `sensors` per refresh. Sensors are resolved by hwmon **name** + **label** (`coretemp`/`Package id 0`, `nvme`/`Composite`), never by `hwmonN` index (not stable across boots). +- **Alerts**: level changes (ok ↔ warn ↔ crit) are logged to the journal with hysteresis (`clear_margin`, default 5 °C) so a sensor sitting on the line doesn't spam. Watch with `journalctl -u van-thermal -f`; warn/crit carry sd-daemon severity (`-p warning`/`-p err`). +- **History**: throttled CSV at `/var/log/van-thermal.csv` (one row per `log_interval`, default 60 s), self-rotating to `.csv.1` past `log_max_bytes` (5 MB). +- **Tuning**: edit `/etc/van-thermal/config.json` (thresholds, intervals, sensor list), then `systemctl restart van-thermal`. Defaults: CPU warn 80 / crit 95 °C (silicon crit is 100), NVMe warn 65 / crit 70 °C (drive crit ~71). +- Status: `systemctl status van-thermal` or `cat /run/van-thermal/state.json`. + ### Adding the 4G/5G modem 1. Plug the USB modem in. ModemManager + the existing `Koodo` gsm NM connection (autoconnect) bring it up. 2. It auto-joins as the `cellular` WAN at metric 300 (last resort). Nothing else to configure. diff --git a/cockpit/vanrouter/index.html b/cockpit/vanrouter/index.html index cc395f9..1994314 100644 --- a/cockpit/vanrouter/index.html +++ b/cockpit/vanrouter/index.html @@ -23,6 +23,11 @@
+
+

Temperatures

+
+
+

WAN Failover

diff --git a/cockpit/vanrouter/vanrouter.css b/cockpit/vanrouter/vanrouter.css index feb8452..04a733c 100644 --- a/cockpit/vanrouter/vanrouter.css +++ b/cockpit/vanrouter/vanrouter.css @@ -12,6 +12,7 @@ th { color: #6a6e73; font-weight: 600; } .pill { padding: 2px 8px; border-radius: 10px; font-size: 12px; white-space: nowrap; } .ok { background: #bde5b8; color: #1e4f18; } .bad { background: #f0b8b8; color: #5f1414; } +.warn { background: #f5d9a8; color: #5f4414; } .muted { color: #6a6e73; font-size: 12px; } .active-wan { font-weight: 700; color: #0066cc; } .btn { margin-left: 6px; padding: 3px 10px; cursor: pointer; } diff --git a/cockpit/vanrouter/vanrouter.js b/cockpit/vanrouter/vanrouter.js index f19675c..4bc07fe 100644 --- a/cockpit/vanrouter/vanrouter.js +++ b/cockpit/vanrouter/vanrouter.js @@ -66,6 +66,37 @@ function renderAP(ap) { document.getElementById("ap").innerHTML = html; } +/* ---------- Temperatures (van-thermal daemon state) ---------- */ + +async function readThermal() { + try { + return JSON.parse(await run(["cat", "/run/van-thermal/state.json"])); + } catch (e) { + return null; + } +} + +function renderThermal(th) { + const el = document.getElementById("thermal"); + if (!th || !th.sensors) { + el.innerHTML = `

van-thermal daemon not running (no state file).

`; + return; + } + const pillClass = { ok: "ok", warn: "warn", crit: "bad" }; + let html = ``; + Object.keys(th.sensors).forEach(name => { + const s = th.sensors[name]; + const temp = s.temp == null ? "—" : `${esc(s.temp)} °C`; + const lvl = s.level || "ok"; + html += `` + + `` + + ``; + }); + html += `
SensorTempStatusWarn / Crit
${esc(name.toUpperCase())}${temp}${esc(lvl)}${esc(s.warn)} / ${esc(s.crit)} °C
`; + html += `

History: /var/log/van-thermal.csv · alerts: journalctl -u van-thermal

`; + el.innerHTML = html; +} + /* ---------- WAN failover (van-failover daemon state) ---------- */ async function readFailover() { @@ -194,8 +225,9 @@ async function restartAP() { async function refresh() { try { - const [ap, fo, wan] = await Promise.all([readAP(), readFailover(), readWAN()]); + const [ap, th, fo, wan] = await Promise.all([readAP(), readThermal(), readFailover(), readWAN()]); renderAP(ap); + renderThermal(th); renderFailover(fo); renderWAN(wan); document.getElementById("updated").textContent = "updated " + new Date().toLocaleTimeString(); diff --git a/deploy.sh b/deploy.sh index 7300a71..fece603 100755 --- a/deploy.sh +++ b/deploy.sh @@ -33,6 +33,11 @@ echo "== cockpit plugin ==" install -d /usr/share/cockpit/vanrouter install -m0644 cockpit/vanrouter/* /usr/share/cockpit/vanrouter/ +echo "== thermal monitor ==" +install -D -m0755 power/van-thermal /usr/local/sbin/van-thermal +install -D -m0644 power/thermal-config.json /etc/van-thermal/config.json +install -D -m0644 power/van-thermal.service /etc/systemd/system/van-thermal.service + echo "== power / never-sleep ==" install -D -m0644 power/10-vanlink-nolid.conf /etc/systemd/logind.conf.d/10-vanlink-nolid.conf # Belt-and-suspenders: a router must never suspend from idle, GUI, or a stray `systemctl suspend`. @@ -48,7 +53,8 @@ systemctl mask systemd-networkd-wait-online.service >/dev/null 2>&1 || true # pick up the lid drop-in (re-execs logind; does NOT drop the network) systemctl restart systemd-logind >/dev/null 2>&1 || true systemctl unmask hostapd >/dev/null 2>&1 || true -systemctl enable regdomain.service hostapd van-ap-dnsmasq nftables systemd-networkd van-failover >/dev/null 2>&1 || true +systemctl enable regdomain.service hostapd van-ap-dnsmasq nftables systemd-networkd van-failover van-thermal >/dev/null 2>&1 || true +systemctl restart van-thermal # restart in dependency order; AP iface IP first, then hostapd/dnsmasq, then NAT/failover systemctl restart systemd-networkd systemctl restart hostapd van-ap-dnsmasq nftables van-failover @@ -58,4 +64,5 @@ echo echo "Deployed. Verify:" echo " iw dev wlxc83a35a4ee55 info | grep -E 'ssid|channel|width'" echo " cat /run/van-failover/state.json" +echo " cat /run/van-thermal/state.json # CPU + NVMe temps" echo "Manual one-time steps (see README §4): zerotier-systemd-manager binary + 'zerotier-cli set allowDNS=1'." diff --git a/power/thermal-config.json b/power/thermal-config.json new file mode 100644 index 0000000..4af9a53 --- /dev/null +++ b/power/thermal-config.json @@ -0,0 +1,10 @@ +{ + "sample_interval": 10, + "log_interval": 60, + "log_path": "/var/log/van-thermal.csv", + "log_max_bytes": 5242880, + "sensors": [ + { "name": "cpu", "hwmon": "coretemp", "label": "Package id 0", "warn": 80, "crit": 95, "clear_margin": 5 }, + { "name": "nvme", "hwmon": "nvme", "label": "Composite", "warn": 65, "crit": 70, "clear_margin": 5 } + ] +} diff --git a/power/van-thermal b/power/van-thermal new file mode 100644 index 0000000..6dc3ac9 --- /dev/null +++ b/power/van-thermal @@ -0,0 +1,207 @@ +#!/usr/bin/env python3 +"""van-thermal — temperature monitor for the campervan router (wayback). + +One small daemon that does three jobs off a single sysfs sample loop: + 1. publishes /run/van-thermal/state.json (the Cockpit "Temps" card reads this, + exactly like van-failover's state.json — no shelling out to `sensors` per refresh) + 2. warns to the journal on threshold crossings, with hysteresis so a sensor hovering + on the line doesn't spam (journalctl -u van-thermal) + 3. appends throttled CSV history to /var/log/van-thermal.csv with self-rotation + +Sensors are resolved by hwmon *name* + *label* at runtime, never by hwmonN index +(that number is assigned at boot and is not stable). Stdlib only. +""" + +import json +import os +import sys +import time +from datetime import datetime, timezone +from pathlib import Path + +CONFIG_PATH = os.environ.get("VAN_THERMAL_CONFIG", "/etc/van-thermal/config.json") +STATE_DIR = Path("/run/van-thermal") +STATE_PATH = STATE_DIR / "state.json" +HWMON = Path("/sys/class/hwmon") + +DEFAULTS = { + "sample_interval": 10, # seconds between sysfs reads (alerting cadence) + "log_interval": 60, # seconds between CSV rows (history cadence) + "log_path": "/var/log/van-thermal.csv", + "log_max_bytes": 5 * 1024 * 1024, # rotate to .1 past this, keep one old file + "sensors": [ + {"name": "cpu", "hwmon": "coretemp", "label": "Package id 0", + "warn": 85, "crit": 95, "clear_margin": 5}, + {"name": "nvme", "hwmon": "nvme", "label": "Composite", + "warn": 65, "crit": 70, "clear_margin": 5}, + ], +} + +LEVELS = ("ok", "warn", "crit") + + +def log(msg, level="info"): + # systemd journal severity prefixes (sd-daemon); shows up via journalctl -p. + pri = {"info": "<6>", "warn": "<4>", "crit": "<2>"}.get(level, "<6>") + print(pri + msg, flush=True) + + +def load_config(): + cfg = dict(DEFAULTS) + try: + with open(CONFIG_PATH) as f: + cfg.update(json.load(f)) + except FileNotFoundError: + log(f"config {CONFIG_PATH} not found, using built-in defaults") + except Exception as e: + log(f"config {CONFIG_PATH} unreadable ({e}), using built-in defaults", "warn") + return cfg + + +def find_hwmon(name): + """Return the hwmon dir whose name matches, or None. Re-resolved on demand + because the hwmonN index can shift across boots / module reloads.""" + for d in sorted(HWMON.glob("hwmon*")): + try: + if (d / "name").read_text().strip() == name: + return d + except OSError: + continue + return None + + +def read_temp(spec, cache): + """Read one sensor's temperature in °C, or None if unavailable. + + spec: {hwmon, label}. Resolves hwmon dir + the tempN whose *_label matches, + falling back to temp1 when the chip exposes no labels.""" + d = cache.get(spec["hwmon"]) + if d is None or not d.exists(): + d = find_hwmon(spec["hwmon"]) + cache[spec["hwmon"]] = d + if d is None: + return None + + input_file = None + want = spec.get("label") + if want: + for lbl in sorted(d.glob("temp*_label")): + try: + if lbl.read_text().strip() == want: + input_file = lbl.with_name(lbl.name.replace("_label", "_input")) + break + except OSError: + continue + if input_file is None: + input_file = d / "temp1_input" # label not found / chip is label-less + + try: + return int(input_file.read_text().strip()) / 1000.0 + except (OSError, ValueError): + # hwmon may have re-enumerated; drop the cache so next pass re-resolves. + cache[spec["hwmon"]] = None + return None + + +def classify(temp, spec, prev_level): + """Level with hysteresis: step up at the threshold, step down only after + dropping clear_margin below it, so a sensor on the line doesn't oscillate.""" + if temp is None: + return prev_level + warn, crit = spec["warn"], spec["crit"] + margin = spec.get("clear_margin", 5) + if temp >= crit: + return "crit" + if temp >= warn: + return "crit" if prev_level == "crit" and temp > crit - margin else "warn" + if temp >= warn - margin: + return prev_level if prev_level in ("warn", "crit") else "ok" + return "ok" + + +def announce(name, temp, old, new): + if old == new: + return + rising = LEVELS.index(new) > LEVELS.index(old) + sev = {"crit": "crit", "warn": "warn", "ok": "info"}[new] + arrow = "rose to" if rising else "fell back to" + log(f"{name} {arrow} {new.upper()} ({temp:.1f}°C)", sev if rising else "info") + + +def rotate_log(path, max_bytes): + try: + if path.exists() and path.stat().st_size > max_bytes: + path.replace(path.with_suffix(path.suffix + ".1")) + except OSError as e: + log(f"log rotate failed: {e}", "warn") + + +def write_csv(cfg, readings): + path = Path(cfg["log_path"]) + rotate_log(path, cfg["log_max_bytes"]) + new = not path.exists() + try: + path.parent.mkdir(parents=True, exist_ok=True) + with open(path, "a") as f: + if new: + f.write(",".join(["time"] + sum( + [[f"{s['name']}_c", f"{s['name']}_lvl"] for s in cfg["sensors"]], [])) + "\n") + row = [datetime.now(timezone.utc).isoformat(timespec="seconds")] + for s in cfg["sensors"]: + r = readings[s["name"]] + row += ["" if r["temp"] is None else f"{r['temp']:.1f}", r["level"]] + f.write(",".join(row) + "\n") + except OSError as e: + log(f"csv write failed: {e}", "warn") + + +def write_state(readings): + STATE_DIR.mkdir(parents=True, exist_ok=True) + payload = { + "updated": datetime.now(timezone.utc).isoformat(timespec="seconds"), + "sensors": readings, + } + tmp = STATE_PATH.with_suffix(".tmp") + tmp.write_text(json.dumps(payload)) + tmp.replace(STATE_PATH) # atomic swap so the Cockpit reader never sees a partial file + + +def main(): + cfg = load_config() + cache = {} + levels = {s["name"]: "ok" for s in cfg["sensors"]} + last_log = 0.0 + log(f"van-thermal up: sampling every {cfg['sample_interval']}s, " + f"logging every {cfg['log_interval']}s to {cfg['log_path']}") + + while True: + readings = {} + for s in cfg["sensors"]: + temp = read_temp(s, cache) + new = classify(temp, s, levels[s["name"]]) + if temp is not None: + announce(s["name"], temp, levels[s["name"]], new) + levels[s["name"]] = new + readings[s["name"]] = { + "temp": None if temp is None else round(temp, 1), + "level": new, "warn": s["warn"], "crit": s["crit"], + } + + try: + write_state(readings) + except OSError as e: + log(f"state write failed: {e}", "warn") + + now = time.monotonic() + if now - last_log >= cfg["log_interval"]: + write_csv(cfg, readings) + last_log = now + + time.sleep(cfg["sample_interval"]) + + +if __name__ == "__main__": + try: + main() + except KeyboardInterrupt: + sys.exit(0) diff --git a/power/van-thermal.service b/power/van-thermal.service new file mode 100644 index 0000000..a5b287a --- /dev/null +++ b/power/van-thermal.service @@ -0,0 +1,12 @@ +[Unit] +Description=Temperature monitor (CPU + NVMe) for the campervan router +After=local-fs.target + +[Service] +Type=simple +ExecStart=/usr/local/sbin/van-thermal +Restart=always +RestartSec=5 + +[Install] +WantedBy=multi-user.target