thermal: van-thermal daemon (CPU+NVMe monitor, alert, log)

Single sysfs sample loop drives three jobs off one daemon (modeled on
van-failover):

- live state at /run/van-thermal/state.json; new Cockpit "Temperatures"
  card reads it in the existing 5s loop (no per-refresh `sensors` shell-out)
- journal alerts on ok<->warn<->crit crossings, with clear_margin hysteresis
- throttled, self-rotating CSV history at /var/log/van-thermal.csv

Sensors resolved by hwmon name+label (coretemp/Package id 0, nvme/Composite),
never by hwmonN index (not stable across boots). Defaults: CPU warn 80 /
crit 95, NVMe warn 65 / crit 70. deploy.sh installs + enables the service.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

README.md

@@ -39,6 +39,7 @@ This directory is the source of truth. The live system files live under `/etc`,
 | AP DHCP + DNS | **dnsmasq** (dedicated instance, bound to AP only) |
 | NAT + forwarding | **nftables** + sysctl |
 | WAN health + failover | **van-failover** daemon |
+| Temperature monitor / alert / log | **van-thermal** daemon |
 | ZeroTier DNS → resolved | **zerotier-systemd-manager** + systemd-networkd |
 | Web UI | **Cockpit** + `vanrouter` plugin |
 
@@ -90,10 +91,13 @@ This directory is the source of truth. The live system files live under `/etc`,
 ### `cockpit/vanrouter/` — web UI
 `manifest.json`, `index.html`, `vanrouter.css`, `vanrouter.js` → `/usr/share/cockpit/vanrouter/`.
 
-### `power/` — never sleep
+### `power/` — never sleep + thermal monitor
 | file | → installs to | purpose |
 |---|---|---|
 | `10-vanlink-nolid.conf` | `/etc/systemd/logind.conf.d/10-vanlink-nolid.conf` | logind ignores the lid in all states (closed / on AC / docked) |
+| `van-thermal` | `/usr/local/sbin/van-thermal` | temperature daemon (Python): publishes state, alerts, logs history |
+| `thermal-config.json` | `/etc/van-thermal/config.json` | sensors + warn/crit thresholds + sample/log intervals |
+| `van-thermal.service` | `/etc/systemd/system/van-thermal.service` | `Restart=always` |
 
 `deploy.sh` additionally **masks** `sleep.target suspend.target hibernate.target hybrid-sleep.target`
 (no repo file — symlinks to `/dev/null` under `/etc/systemd/system/`) so nothing else can suspend either.
@@ -123,6 +127,14 @@ This directory is the source of truth. The live system files live under `/etc`,
 - Single-WAN, sticky override with no explicit "clear": to return to automatic priority, prefer your top WAN (wifi) or `rm /run/van-failover/prefer`. **Ephemeral by design** — `/run` is wiped on reboot, so a reboot returns to config priorities. (To persist it, point `PREFER` at `/etc/van-failover/prefer` in the daemon and the matching path in `vanrouter.js`.)
 - Swapping the preference between two WANs is collision-safe: a second default route can't take the new WAN's target metric while the old preferred WAN still holds it, so `enforce_route` keeps the existing route and retries next loop (settles in ~2 loops) instead of stranding the interface. `enforce_route` also restores a default route that went missing while the carrier is up (gateway from NM via `device_gateway`), not just rebases an existing one.
 
+### Thermal monitor (`van-thermal`)
+- One daemon off a single sysfs sample loop does all three jobs: live Cockpit readout, threshold alerting, history.
+- **Live state**: `/run/van-thermal/state.json` (atomic-swapped each sample). The Cockpit "Temperatures" card reads this — no shelling out to `sensors` per refresh. Sensors are resolved by hwmon **name** + **label** (`coretemp`/`Package id 0`, `nvme`/`Composite`), never by `hwmonN` index (not stable across boots).
+- **Alerts**: level changes (ok ↔ warn ↔ crit) are logged to the journal with hysteresis (`clear_margin`, default 5 °C) so a sensor sitting on the line doesn't spam. Watch with `journalctl -u van-thermal -f`; warn/crit carry sd-daemon severity (`-p warning`/`-p err`).
+- **History**: throttled CSV at `/var/log/van-thermal.csv` (one row per `log_interval`, default 60 s), self-rotating to `.csv.1` past `log_max_bytes` (5 MB).
+- **Tuning**: edit `/etc/van-thermal/config.json` (thresholds, intervals, sensor list), then `systemctl restart van-thermal`. Defaults: CPU warn 80 / crit 95 °C (silicon crit is 100), NVMe warn 65 / crit 70 °C (drive crit ~71).
+- Status: `systemctl status van-thermal` or `cat /run/van-thermal/state.json`.
+
 ### Adding the 4G/5G modem
 1. Plug the USB modem in. ModemManager + the existing `Koodo` gsm NM connection (autoconnect) bring it up.
 2. It auto-joins as the `cellular` WAN at metric 300 (last resort). Nothing else to configure.

cockpit/vanrouter/index.html

@@ -23,6 +23,11 @@
             <div id="ap"></div>
         </div>
 
+        <div class="card">
+            <h3>Temperatures</h3>
+            <div id="thermal"></div>
+        </div>
+
         <div class="card">
             <h3>WAN Failover</h3>
             <div id="failover"></div>

cockpit/vanrouter/vanrouter.css

@@ -12,6 +12,7 @@ th { color: #6a6e73; font-weight: 600; }
 .pill { padding: 2px 8px; border-radius: 10px; font-size: 12px; white-space: nowrap; }
 .ok { background: #bde5b8; color: #1e4f18; }
 .bad { background: #f0b8b8; color: #5f1414; }
+.warn { background: #f5d9a8; color: #5f4414; }
 .muted { color: #6a6e73; font-size: 12px; }
 .active-wan { font-weight: 700; color: #0066cc; }
 .btn { margin-left: 6px; padding: 3px 10px; cursor: pointer; }

cockpit/vanrouter/vanrouter.js

@@ -66,6 +66,37 @@ function renderAP(ap) {
     document.getElementById("ap").innerHTML = html;
 }
 
+/* ---------- Temperatures (van-thermal daemon state) ---------- */
+
+async function readThermal() {
+    try {
+        return JSON.parse(await run(["cat", "/run/van-thermal/state.json"]));
+    } catch (e) {
+        return null;
+    }
+}
+
+function renderThermal(th) {
+    const el = document.getElementById("thermal");
+    if (!th || !th.sensors) {
+        el.innerHTML = `<p class="muted">van-thermal daemon not running (no state file).</p>`;
+        return;
+    }
+    const pillClass = { ok: "ok", warn: "warn", crit: "bad" };
+    let html = `<table><thead><tr><th>Sensor</th><th>Temp</th><th>Status</th><th>Warn / Crit</th></tr></thead><tbody>`;
+    Object.keys(th.sensors).forEach(name => {
+        const s = th.sensors[name];
+        const temp = s.temp == null ? "—" : `${esc(s.temp)} °C`;
+        const lvl = s.level || "ok";
+        html += `<tr><td>${esc(name.toUpperCase())}</td><td>${temp}</td>` +
+                `<td><span class="pill ${pillClass[lvl] || ""}">${esc(lvl)}</span></td>` +
+                `<td class="muted">${esc(s.warn)} / ${esc(s.crit)} °C</td></tr>`;
+    });
+    html += `</tbody></table>`;
+    html += `<p class="muted">History: <code>/var/log/van-thermal.csv</code> · alerts: <code>journalctl -u van-thermal</code></p>`;
+    el.innerHTML = html;
+}
+
 /* ---------- WAN failover (van-failover daemon state) ---------- */
 
 async function readFailover() {
@@ -194,8 +225,9 @@ async function restartAP() {
 
 async function refresh() {
     try {
-        const [ap, fo, wan] = await Promise.all([readAP(), readFailover(), readWAN()]);
+        const [ap, th, fo, wan] = await Promise.all([readAP(), readThermal(), readFailover(), readWAN()]);
         renderAP(ap);
+        renderThermal(th);
         renderFailover(fo);
         renderWAN(wan);
         document.getElementById("updated").textContent = "updated " + new Date().toLocaleTimeString();

deploy.sh

@@ -33,6 +33,11 @@ echo "== cockpit plugin =="
 install -d /usr/share/cockpit/vanrouter
 install -m0644 cockpit/vanrouter/* /usr/share/cockpit/vanrouter/
 
+echo "== thermal monitor =="
+install -D -m0755 power/van-thermal         /usr/local/sbin/van-thermal
+install -D -m0644 power/thermal-config.json /etc/van-thermal/config.json
+install -D -m0644 power/van-thermal.service /etc/systemd/system/van-thermal.service
+
 echo "== power / never-sleep =="
 install -D -m0644 power/10-vanlink-nolid.conf /etc/systemd/logind.conf.d/10-vanlink-nolid.conf
 # Belt-and-suspenders: a router must never suspend from idle, GUI, or a stray `systemctl suspend`.
@@ -48,7 +53,8 @@ systemctl mask systemd-networkd-wait-online.service >/dev/null 2>&1 || true
 # pick up the lid drop-in (re-execs logind; does NOT drop the network)
 systemctl restart systemd-logind >/dev/null 2>&1 || true
 systemctl unmask hostapd >/dev/null 2>&1 || true
-systemctl enable regdomain.service hostapd van-ap-dnsmasq nftables systemd-networkd van-failover >/dev/null 2>&1 || true
+systemctl enable regdomain.service hostapd van-ap-dnsmasq nftables systemd-networkd van-failover van-thermal >/dev/null 2>&1 || true
+systemctl restart van-thermal
 # restart in dependency order; AP iface IP first, then hostapd/dnsmasq, then NAT/failover
 systemctl restart systemd-networkd
 systemctl restart hostapd van-ap-dnsmasq nftables van-failover
@@ -58,4 +64,5 @@ echo
 echo "Deployed. Verify:"
 echo "  iw dev wlxc83a35a4ee55 info | grep -E 'ssid|channel|width'"
 echo "  cat /run/van-failover/state.json"
+echo "  cat /run/van-thermal/state.json   # CPU + NVMe temps"
 echo "Manual one-time steps (see README §4): zerotier-systemd-manager binary + 'zerotier-cli set <nwid> allowDNS=1'."

power/thermal-config.json

@@ -0,0 +1,10 @@
+{
+  "sample_interval": 10,
+  "log_interval": 60,
+  "log_path": "/var/log/van-thermal.csv",
+  "log_max_bytes": 5242880,
+  "sensors": [
+    { "name": "cpu",  "hwmon": "coretemp", "label": "Package id 0", "warn": 80, "crit": 95, "clear_margin": 5 },
+    { "name": "nvme", "hwmon": "nvme",     "label": "Composite",    "warn": 65, "crit": 70, "clear_margin": 5 }
+  ]
+}

power/van-thermal

@@ -0,0 +1,207 @@
+#!/usr/bin/env python3
+"""van-thermal — temperature monitor for the campervan router (wayback).
+
+One small daemon that does three jobs off a single sysfs sample loop:
+  1. publishes /run/van-thermal/state.json  (the Cockpit "Temps" card reads this,
+     exactly like van-failover's state.json — no shelling out to `sensors` per refresh)
+  2. warns to the journal on threshold crossings, with hysteresis so a sensor hovering
+     on the line doesn't spam (journalctl -u van-thermal)
+  3. appends throttled CSV history to /var/log/van-thermal.csv with self-rotation
+
+Sensors are resolved by hwmon *name* + *label* at runtime, never by hwmonN index
+(that number is assigned at boot and is not stable). Stdlib only.
+"""
+
+import json
+import os
+import sys
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+
+CONFIG_PATH = os.environ.get("VAN_THERMAL_CONFIG", "/etc/van-thermal/config.json")
+STATE_DIR = Path("/run/van-thermal")
+STATE_PATH = STATE_DIR / "state.json"
+HWMON = Path("/sys/class/hwmon")
+
+DEFAULTS = {
+    "sample_interval": 10,      # seconds between sysfs reads (alerting cadence)
+    "log_interval": 60,         # seconds between CSV rows (history cadence)
+    "log_path": "/var/log/van-thermal.csv",
+    "log_max_bytes": 5 * 1024 * 1024,   # rotate to .1 past this, keep one old file
+    "sensors": [
+        {"name": "cpu",  "hwmon": "coretemp", "label": "Package id 0",
+         "warn": 85, "crit": 95, "clear_margin": 5},
+        {"name": "nvme", "hwmon": "nvme",     "label": "Composite",
+         "warn": 65, "crit": 70, "clear_margin": 5},
+    ],
+}
+
+LEVELS = ("ok", "warn", "crit")
+
+
+def log(msg, level="info"):
+    # systemd journal severity prefixes (sd-daemon); shows up via journalctl -p.
+    pri = {"info": "<6>", "warn": "<4>", "crit": "<2>"}.get(level, "<6>")
+    print(pri + msg, flush=True)
+
+
+def load_config():
+    cfg = dict(DEFAULTS)
+    try:
+        with open(CONFIG_PATH) as f:
+            cfg.update(json.load(f))
+    except FileNotFoundError:
+        log(f"config {CONFIG_PATH} not found, using built-in defaults")
+    except Exception as e:
+        log(f"config {CONFIG_PATH} unreadable ({e}), using built-in defaults", "warn")
+    return cfg
+
+
+def find_hwmon(name):
+    """Return the hwmon dir whose name matches, or None. Re-resolved on demand
+    because the hwmonN index can shift across boots / module reloads."""
+    for d in sorted(HWMON.glob("hwmon*")):
+        try:
+            if (d / "name").read_text().strip() == name:
+                return d
+        except OSError:
+            continue
+    return None
+
+
+def read_temp(spec, cache):
+    """Read one sensor's temperature in °C, or None if unavailable.
+
+    spec: {hwmon, label}. Resolves hwmon dir + the tempN whose *_label matches,
+    falling back to temp1 when the chip exposes no labels."""
+    d = cache.get(spec["hwmon"])
+    if d is None or not d.exists():
+        d = find_hwmon(spec["hwmon"])
+        cache[spec["hwmon"]] = d
+    if d is None:
+        return None
+
+    input_file = None
+    want = spec.get("label")
+    if want:
+        for lbl in sorted(d.glob("temp*_label")):
+            try:
+                if lbl.read_text().strip() == want:
+                    input_file = lbl.with_name(lbl.name.replace("_label", "_input"))
+                    break
+            except OSError:
+                continue
+    if input_file is None:
+        input_file = d / "temp1_input"   # label not found / chip is label-less
+
+    try:
+        return int(input_file.read_text().strip()) / 1000.0
+    except (OSError, ValueError):
+        # hwmon may have re-enumerated; drop the cache so next pass re-resolves.
+        cache[spec["hwmon"]] = None
+        return None
+
+
+def classify(temp, spec, prev_level):
+    """Level with hysteresis: step up at the threshold, step down only after
+    dropping clear_margin below it, so a sensor on the line doesn't oscillate."""
+    if temp is None:
+        return prev_level
+    warn, crit = spec["warn"], spec["crit"]
+    margin = spec.get("clear_margin", 5)
+    if temp >= crit:
+        return "crit"
+    if temp >= warn:
+        return "crit" if prev_level == "crit" and temp > crit - margin else "warn"
+    if temp >= warn - margin:
+        return prev_level if prev_level in ("warn", "crit") else "ok"
+    return "ok"
+
+
+def announce(name, temp, old, new):
+    if old == new:
+        return
+    rising = LEVELS.index(new) > LEVELS.index(old)
+    sev = {"crit": "crit", "warn": "warn", "ok": "info"}[new]
+    arrow = "rose to" if rising else "fell back to"
+    log(f"{name} {arrow} {new.upper()} ({temp:.1f}°C)", sev if rising else "info")
+
+
+def rotate_log(path, max_bytes):
+    try:
+        if path.exists() and path.stat().st_size > max_bytes:
+            path.replace(path.with_suffix(path.suffix + ".1"))
+    except OSError as e:
+        log(f"log rotate failed: {e}", "warn")
+
+
+def write_csv(cfg, readings):
+    path = Path(cfg["log_path"])
+    rotate_log(path, cfg["log_max_bytes"])
+    new = not path.exists()
+    try:
+        path.parent.mkdir(parents=True, exist_ok=True)
+        with open(path, "a") as f:
+            if new:
+                f.write(",".join(["time"] + sum(
+                    [[f"{s['name']}_c", f"{s['name']}_lvl"] for s in cfg["sensors"]], [])) + "\n")
+            row = [datetime.now(timezone.utc).isoformat(timespec="seconds")]
+            for s in cfg["sensors"]:
+                r = readings[s["name"]]
+                row += ["" if r["temp"] is None else f"{r['temp']:.1f}", r["level"]]
+            f.write(",".join(row) + "\n")
+    except OSError as e:
+        log(f"csv write failed: {e}", "warn")
+
+
+def write_state(readings):
+    STATE_DIR.mkdir(parents=True, exist_ok=True)
+    payload = {
+        "updated": datetime.now(timezone.utc).isoformat(timespec="seconds"),
+        "sensors": readings,
+    }
+    tmp = STATE_PATH.with_suffix(".tmp")
+    tmp.write_text(json.dumps(payload))
+    tmp.replace(STATE_PATH)   # atomic swap so the Cockpit reader never sees a partial file
+
+
+def main():
+    cfg = load_config()
+    cache = {}
+    levels = {s["name"]: "ok" for s in cfg["sensors"]}
+    last_log = 0.0
+    log(f"van-thermal up: sampling every {cfg['sample_interval']}s, "
+        f"logging every {cfg['log_interval']}s to {cfg['log_path']}")
+
+    while True:
+        readings = {}
+        for s in cfg["sensors"]:
+            temp = read_temp(s, cache)
+            new = classify(temp, s, levels[s["name"]])
+            if temp is not None:
+                announce(s["name"], temp, levels[s["name"]], new)
+            levels[s["name"]] = new
+            readings[s["name"]] = {
+                "temp": None if temp is None else round(temp, 1),
+                "level": new, "warn": s["warn"], "crit": s["crit"],
+            }
+
+        try:
+            write_state(readings)
+        except OSError as e:
+            log(f"state write failed: {e}", "warn")
+
+        now = time.monotonic()
+        if now - last_log >= cfg["log_interval"]:
+            write_csv(cfg, readings)
+            last_log = now
+
+        time.sleep(cfg["sample_interval"])
+
+
+if __name__ == "__main__":
+    try:
+        main()
+    except KeyboardInterrupt:
+        sys.exit(0)

power/van-thermal.service

@@ -0,0 +1,12 @@
+[Unit]
+Description=Temperature monitor (CPU + NVMe) for the campervan router
+After=local-fs.target
+
+[Service]
+Type=simple
+ExecStart=/usr/local/sbin/van-thermal
+Restart=always
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target