docs: complete the 5.4.5 changelog entry

The release script dropped every scoped feat:/fix: commit from the
generated entry (fixed in the prior commit); this backfills the mini
client presence feature and both C-client fixes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uc9V1v7iXruubpoLiSE24M

CHANGELOG.md

@@ -2,6 +2,52 @@
 
 All notable changes to this project are documented here, organized by release.
 
+## [5.4.5]
+
+### Added
+- `hbc_mini.py` and `hbc_mini.c` (the dependency-free clients) get the same per-protocol connectivity probe as `hbc`: `p4`/`p6` flags on every heartbeat and a `connectivity` metric (presence, own address, gateway, RTT) via `network_monitor`, with gateway lookup on Linux, FreeBSD/NetBSD/DragonFly (both clients) and macOS/Windows (`hbc_mini.py` only)
+- `scripts/c/test_probe.c` and a `make test` target — the C client's first automated test suite
+
+### Fixed
+- `hbc_mini.c`: the clean-shutdown heartbeat could be delayed or lost behind an in-flight connectivity probe on SIGTERM; it is now sent before the probe thread is joined
+- `hbc_mini.c`: a connectivity probe that fails to start, or is configured with a non-IP-literal target, now logs a warning instead of failing silently
+- `hbc_mini.py`: removed an unused import
+
+---
+
+## [5.4.4]
+
+### Added
+- client reports per-protocol (IPv4/IPv6) internet presence: a connectivity probe (route check every ~5s, ICMP echo every ~60s, debounced) sends `p4`/`p6` flags on every heartbeat and a `connectivity` metric (presence, own address, gateway, RTT) via `network_monitor`
+- server monitors only the protocols a client reports as present; a protocol reported absent is shown as `absent` and raises no alert
+- host status (`up`, `simplex`, `overdue`, `down`, `unknown`) computed on the server: a host is overdue only when every present protocol is overdue; `simplex` (both present, one silent) has the same severity as `up` and is logged without notification
+- live and plugins pages show the host status, `simplex`/`absent` protocols and a Connectivity table
+- `hbc` config keys under `network_monitor`: `probe_enabled`, `probe_v4`, `probe_v6`, `probe_interval`
+
+### Changed
+- overdue and recovery alerts are raised per host, not per protocol: a single CRITICAL when the last present protocol goes overdue; a protocol coming back while another stayed up is logged as INFO instead of a RECOVER notification (also applies to clients that do not report presence)
+
+### Fixed
+- never park the last present family as absent, so a host that reports no protocol as present still alerts when it dies
+- keep the connectivity alert for a host overdue past the drop window across an `hbd` restart
+- `default_gateway` never raises; a protocol whose probe cannot run stays unknown rather than absent
+
+---
+
+## [5.4.3]
+
+### Fixed
+- fix RTT y-axis label clipping, add x-axis tickmarks, enlarge charts
+
+---
+
+## [5.4.2]
+
+### Fixed
+- preserve chart history across reconnects, show gaps for missing data
+
+---
+
 ## [5.4.1]
 
 ### Added

README.md

@@ -20,7 +20,7 @@ A lightweight UDP-based host monitoring system. Monitored hosts run a client (`h
   └────────────────────┘              └────────────────────────────┘
 ```
 
-**Package:** `hbd` v5.4.1
+**Package:** `hbd` v5.4.5
 **Python:** 3.11+
 
 ### Subpackages
@@ -659,7 +659,28 @@ Config: `~/.hbc.json` (JSON format, same keys as `~/.hbc.yaml`).
 | `cpu_monitor` | Linux (`/proc/stat`; no per-core, no frequency) |
 | `memory_monitor` | Linux (`/proc/meminfo`) |
 | `disk_monitor` | Linux, macOS, BSD (`df -P`) |
-| `network_monitor` | Linux (`/proc/net/dev`) |
+| `network_monitor` | Linux (`/proc/net/dev`); per-protocol connectivity probe (all platforms) |
+
+`network_monitor` also reports per-protocol (IPv4/IPv6) internet presence — whether the host
+has a working route and reply to a well-known address, plus its own address, the local
+gateway, and RTT — as `p4`/`p6` flags on every heartbeat and a `connectivity` object in the
+plugin data. Configurable, defaults shown:
+
+```json
+{
+  "plugins": {
+    "network_monitor": {
+      "probe_enabled": true,
+      "probe_v4": "1.1.1.1",
+      "probe_v6": "2606:4700:4700::1111",
+      "probe_interval": 60
+    }
+  }
+}
+```
+
+`probe_v4`/`probe_v6` must be IP literals (no hostnames). Gateway lookup works on Linux,
+FreeBSD/NetBSD/DragonFly/macOS (`route -n get`), and Windows (`route print`).
 
 Not available vs full `hbc`: no YAML config, no `filesystem_info`, no `zfs_monitor`, no IPv6 early-fail protection.
 
@@ -712,6 +733,7 @@ SIGHUP causes the process to restart itself via `execv()`. SIGTERM/SIGINT trigge
 | `disk_monitor` | All | `df -P` subprocess |
 | `network_monitor` | Linux | `/proc/net/dev` |
 | `network_monitor` | FreeBSD, NetBSD, DragonFly | `getifaddrs()` + `AF_LINK` |
+| `network_monitor` (connectivity probe) | Linux, FreeBSD, NetBSD, DragonFly | UDP `connect()` route check + one ICMP echo per protocol |
 | `ping_monitor` | All | `ping` subprocess |
 | `nagios_runner` | All | `popen()` subprocess |
 
@@ -743,12 +765,33 @@ SIGHUP causes the process to restart itself via `execv()`. SIGTERM/SIGINT trigge
 }
 ```
 
+`network_monitor` also runs the per-protocol connectivity probe described above, in a
+dedicated thread (its ICMP echo can block for a few seconds). Same config keys and defaults
+as `hbc_mini.py`:
+
+```json
+{
+  "plugins": {
+    "network_monitor": {
+      "probe_enabled": true,
+      "probe_v4": "1.1.1.1",
+      "probe_v6": "2606:4700:4700::1111",
+      "probe_interval": 60
+    }
+  }
+}
+```
+
+Gateway lookup on the C client covers Linux and FreeBSD/NetBSD/DragonFly only (no macOS or
+Windows build of `hbc_mini.c` exists).
+
 ### Differences from `hbc_mini.py`
 
 - No `filesystem_info` or `zfs_monitor` plugins
 - `UPD` (self-update) messages are logged but not acted on
 - No IPv6 early-fail protection
 - Config is JSON only (`~/.hbc.json`), no YAML
+- No macOS or Windows build (Linux and the BSDs only)
 
 ---
 

hbd/__init__.py

@@ -14,4 +14,4 @@ Install options:
 """
 
 __all__ = ["__version__"]
-__version__ = "5.4.1"
+__version__ = "5.4.5"

hbd/client/connectivity.py

@@ -0,0 +1,204 @@
+"""Per-protocol internet presence probe.
+
+Tier 1 (every tier1_interval seconds, sends no packets): a UDP connect() to the probe
+address reveals whether a route exists and which source address would be used.
+Tier 2 (every probe_interval seconds, sooner after a failure or a change of address or
+gateway): one ICMP echo to the probe address. Absence needs fail_limit consecutive
+failures (or no route at all); presence needs one success.
+
+The result is read by the heartbeat sender (p4/p6 flags on every HTB) and by the
+network_monitor plugin (connectivity metric) via the shared instance.
+"""
+
+import asyncio
+import logging
+import socket
+import sys
+import time
+from dataclasses import asdict, dataclass, replace
+from typing import Any, Awaitable, Callable, Dict, Optional, Set, Tuple
+
+from hbd.client import gateway as gateway_mod
+from hbd.client.pingutil import PingUnavailable, run_ping
+
+logger = logging.getLogger("hbc.connectivity")
+
+FAMILIES = ("ipv4", "ipv6")
+FLAG_KEYS = {"ipv4": "p4", "ipv6": "p6"}
+DEFAULT_PROBES = {"ipv4": "1.1.1.1", "ipv6": "2606:4700:4700::1111"}
+
+
+class ProbeUnavailable(Exception):
+    """The address family cannot be probed on this host (e.g. no IPv6 support)."""
+
+
+@dataclass
+class FamilyState:
+    present: Optional[bool] = None      # None = unknown (probe cannot run yet/at all)
+    addr: Optional[str] = None
+    gateway: Optional[str] = None
+    rtt: Optional[float] = None
+
+
+def local_address(family: str, target: str) -> Optional[str]:
+    """Source address the kernel would use to reach target, or None if there is no route.
+
+    A UDP connect() sends nothing. Raises ProbeUnavailable if the family is unsupported.
+    """
+    af = socket.AF_INET if family == "ipv4" else socket.AF_INET6
+    try:
+        sock = socket.socket(af, socket.SOCK_DGRAM)
+    except OSError as e:
+        raise ProbeUnavailable(str(e)) from e
+    try:
+        sock.connect((target, 9))
+        return str(sock.getsockname()[0])
+    except OSError:
+        return None
+    finally:
+        sock.close()
+
+
+async def ping_rtt(target: str, family: str) -> Optional[float]:
+    """One ICMP echo; returns RTT in ms or None if no reply."""
+    result = await run_ping(target, 1, 2, 4 if family == "ipv4" else 6)
+    rtt = result["rtt_avg"]
+    return None if rtt == float("inf") else rtt
+
+
+class Connectivity:
+    def __init__(
+        self,
+        probes: Optional[Dict[str, str]] = None,
+        probe_interval: float = 60.0,
+        retry_interval: float = 10.0,
+        tier1_interval: float = 5.0,
+        fail_limit: int = 2,
+        local_address_fn: Callable[[str, str], Optional[str]] = local_address,
+        ping_fn: Callable[[str, str], Awaitable[Optional[float]]] = ping_rtt,
+        gateway_fn: Callable[[str], Awaitable[Optional[str]]] = gateway_mod.default_gateway,
+        clock: Callable[[], float] = time.monotonic,
+    ):
+        self.probes = {**DEFAULT_PROBES, **(probes or {})}
+        self.probe_interval = probe_interval
+        self.retry_interval = retry_interval
+        self.tier1_interval = tier1_interval
+        self.fail_limit = fail_limit
+        self._local_address = local_address_fn
+        self._ping = ping_fn
+        self._gateway_fn = gateway_fn
+        self._clock = clock
+        # Reading the gateway is a cheap /proc read on Linux but a subprocess elsewhere,
+        # so elsewhere it is refreshed only when a tier-2 probe is due.
+        self._cheap_gateway = sys.platform.startswith("linux")
+        self.states: Dict[str, FamilyState] = {f: FamilyState() for f in FAMILIES}
+        self._fails: Dict[str, int] = {f: 0 for f in FAMILIES}
+        self._next_ping: Dict[str, float] = {f: 0.0 for f in FAMILIES}
+        self._last_key: Dict[str, Optional[Tuple[Optional[str], Optional[str]]]] = {
+            f: None for f in FAMILIES}
+        self._ping_disabled: Set[str] = set()
+
+    @classmethod
+    def from_config(cls, cfg: Optional[Dict[str, Any]]) -> "Connectivity":
+        cfg = cfg or {}
+        probes = {}
+        if cfg.get("probe_v4"):
+            probes["ipv4"] = str(cfg["probe_v4"])
+        if cfg.get("probe_v6"):
+            probes["ipv6"] = str(cfg["probe_v6"])
+        return cls(probes=probes, probe_interval=float(cfg.get("probe_interval", 60)))
+
+    # ---- readers (cheap; safe to call on every heartbeat) --------------------------
+    def snapshot(self) -> Dict[str, FamilyState]:
+        return {f: replace(s) for f, s in self.states.items()}
+
+    def flags(self) -> Dict[str, bool]:
+        """HTB flags for families whose presence is known."""
+        return {FLAG_KEYS[f]: s.present for f, s in self.states.items() if s.present is not None}
+
+    def as_metric(self) -> Dict[str, Dict[str, Any]]:
+        return {f: asdict(s) for f, s in self.states.items()}
+
+    # ---- probing -------------------------------------------------------------------
+    async def refresh(self) -> None:
+        for fam in FAMILIES:
+            try:
+                await self._refresh_family(fam)
+            except Exception:
+                logger.exception("connectivity probe failed for %s", fam)
+
+    async def run(self, stop: asyncio.Event) -> None:
+        while not stop.is_set():
+            await self.refresh()
+            try:
+                await asyncio.wait_for(stop.wait(), timeout=self.tier1_interval)
+            except asyncio.TimeoutError:
+                pass
+
+    async def _gateway(self, fam: str) -> Optional[str]:
+        try:
+            return await self._gateway_fn(fam)
+        except Exception:
+            return None
+
+    async def _refresh_family(self, fam: str) -> None:
+        st = self.states[fam]
+        try:
+            addr = self._local_address(fam, self.probes[fam])
+        except ProbeUnavailable:
+            st.present = None
+            return
+
+        if addr is None:                       # tier 1: no route => absent immediately
+            self._fails[fam] = 0
+            self._next_ping[fam] = 0.0
+            self._last_key[fam] = None
+            st.present, st.addr, st.gateway, st.rtt = False, None, None, None
+            return
+
+        now = self._clock()
+        gw = await self._gateway(fam) if self._cheap_gateway else st.gateway
+        key = (addr, gw)
+        due = key != self._last_key[fam] or now >= self._next_ping[fam]
+        if due and not self._cheap_gateway:
+            gw = await self._gateway(fam)
+            key = (addr, gw)
+        st.addr, st.gateway = addr, gw
+        self._last_key[fam] = key
+        if fam in self._ping_disabled:
+            st.present = None          # cannot probe: unknown, never absent, while a route exists
+        elif due:
+            await self._tier2(fam, st, now)
+
+    async def _tier2(self, fam: str, st: FamilyState, now: float) -> None:
+        try:
+            rtt = await self._ping(self.probes[fam], fam)
+        except PingUnavailable:
+            logger.warning("ping unavailable; %s presence stays unknown", fam)
+            self._ping_disabled.add(fam)
+            st.present = None
+            return
+        except (asyncio.TimeoutError, OSError):
+            rtt = None
+        if rtt is None:
+            self._fails[fam] += 1
+            self._next_ping[fam] = now + self.retry_interval
+            if self._fails[fam] >= self.fail_limit:
+                st.present, st.rtt = False, None
+        else:
+            self._fails[fam] = 0
+            self._next_ping[fam] = now + self.probe_interval
+            st.present, st.rtt = True, rtt
+
+
+# ---- shared instance (heartbeat sender and network_monitor plugin) -----------------
+_shared: Optional[Connectivity] = None
+
+
+def set_shared(conn: Optional[Connectivity]) -> None:
+    global _shared
+    _shared = conn
+
+
+def get_shared() -> Optional[Connectivity]:
+    return _shared

hbd/client/gateway.py

@@ -0,0 +1,103 @@
+"""Best-effort default-gateway lookup (informational only; never affects presence)."""
+
+import asyncio
+import re
+import socket
+import struct
+import sys
+from typing import List, Optional, Tuple
+
+_RTF_GATEWAY = 0x2
+_RTF_REJECT = 0x200
+
+
+def parse_proc_route_v4(text: str) -> Optional[str]:
+    """Default gateway from /proc/net/route (lowest metric wins)."""
+    best: Optional[Tuple[int, str]] = None
+    for line in text.splitlines()[1:]:
+        f = line.split()
+        if len(f) < 8:
+            continue
+        _iface, dest, gw, flags = f[0], f[1], f[2], f[3]  # noqa: F841
+        if dest != "00000000" or not int(flags, 16) & _RTF_GATEWAY:
+            continue
+        metric = int(f[6])
+        if best is None or metric < best[0]:
+            best = (metric, socket.inet_ntoa(struct.pack("<L", int(gw, 16))))
+    return best[1] if best else None
+
+
+def parse_proc_route_v6(text: str) -> Optional[str]:
+    """Default gateway from /proc/net/ipv6_route (lowest metric wins)."""
+    best: Optional[Tuple[int, str]] = None
+    for line in text.splitlines():
+        f = line.split()
+        if len(f) < 10:
+            continue
+        dest, plen, nexthop, metric, flags, iface = f[0], f[1], f[4], f[5], f[8], f[9]
+        if dest != "0" * 32 or plen != "00" or nexthop == "0" * 32 or iface == "lo":
+            continue
+        fl = int(flags, 16)
+        if not fl & _RTF_GATEWAY or fl & _RTF_REJECT:
+            continue
+        m = int(metric, 16)
+        if best is None or m < best[0]:
+            best = (m, socket.inet_ntop(socket.AF_INET6, bytes.fromhex(nexthop)))
+    return best[1] if best else None
+
+
+def parse_macos_route(text: str) -> Optional[str]:
+    """Gateway from `route -n get [-inet6] default`."""
+    m = re.search(r"^\s*gateway:\s*(\S+)", text, re.MULTILINE)
+    return m.group(1) if m else None
+
+
+def parse_windows_route(text: str, family: str) -> Optional[str]:
+    """Default gateway from `route print -4` / `route print -6` (lowest metric wins)."""
+    best: Optional[Tuple[int, str]] = None
+    for line in text.splitlines():
+        if family == "ipv4":
+            f = line.split()
+            if len(f) >= 5 and f[0] == "0.0.0.0" and f[1] == "0.0.0.0" and f[2] != "On-link":
+                cand = (int(f[4]), f[2])
+            else:
+                continue
+        else:
+            m = re.match(r"^\s*\d+\s+(\d+)\s+::/0\s+(\S+)\s*$", line)
+            if not m or m.group(2) == "On-link":
+                continue
+            cand = (int(m.group(1)), m.group(2))
+        if best is None or cand[0] < best[0]:
+            best = cand
+    return best[1] if best else None
+
+
+async def _run(cmd: List[str]) -> str:
+    proc = await asyncio.create_subprocess_exec(
+        *cmd, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.DEVNULL,
+    )
+    try:
+        out, _ = await asyncio.wait_for(proc.communicate(), timeout=5)
+    except asyncio.TimeoutError:
+        proc.kill()
+        await proc.wait()
+        raise
+    return out.decode(errors="replace")
+
+
+async def default_gateway(family: str) -> Optional[str]:
+    """Return the default gateway address for "ipv4"/"ipv6", or None on any failure."""
+    v4 = family == "ipv4"
+    try:
+        if sys.platform.startswith("linux"):
+            with open("/proc/net/route" if v4 else "/proc/net/ipv6_route") as fh:
+                text = fh.read()
+            return parse_proc_route_v4(text) if v4 else parse_proc_route_v6(text)
+        if sys.platform == "darwin":
+            cmd = ["route", "-n", "get"] + ([] if v4 else ["-inet6"]) + ["default"]
+            return parse_macos_route(await _run(cmd))
+        if sys.platform == "win32":
+            return parse_windows_route(await _run(["route", "print", "-4" if v4 else "-6"]), family)
+    except Exception:
+        return None
+    return None

hbd/client/main.py

@@ -25,6 +25,7 @@ from .. import __version__
 
 # Import plugin system
 from .plugin import PluginRegistry, PluginLoader, InfoPlugin, MonitorPlugin
+from .connectivity import Connectivity, set_shared
 
 # Constants
 PORT = 50003
@@ -281,7 +282,16 @@ async def handle_update(conn: AsyncConnection, _msg: dict):  # pyright: ignore[r
     stop()
 
 
-async def heartbeat_sender(conn: AsyncConnection, interval: int):
+def build_heartbeat_msg(conn, interval: int, connectivity: Optional[Connectivity] = None) -> dict:
+    """HTB payload; carries p4/p6 presence flags when the probe knows them."""
+    msg = {"acks": conn.ackcount, "rtt": conn.rtts[-1], "interval": interval}
+    if connectivity is not None:
+        msg.update(connectivity.flags())
+    return msg
+
+
+async def heartbeat_sender(conn: AsyncConnection, interval: int,
+                           connectivity: Optional[Connectivity] = None):
     """Send periodic heartbeats, retrying the connection if it is not open.
 
     IPv6 connections that fail to open before their first successful send are
@@ -291,6 +301,7 @@ async def heartbeat_sender(conn: AsyncConnection, interval: int):
     Args:
         conn: Connection to send on
         interval: Heartbeat interval in seconds
+        connectivity: Optional probe whose presence flags are attached to every heartbeat
     """
     logger = logging.getLogger("hbc.heartbeat")
     IPV6_EARLY_FAIL_LIMIT = 3
@@ -328,11 +339,7 @@ async def heartbeat_sender(conn: AsyncConnection, interval: int):
                 continue
 
         try:
-            msg = {
+            msg = build_heartbeat_msg(conn, interval, connectivity)
-                "acks": conn.ackcount,
-                "rtt": conn.rtts[-1],
-                "interval": interval
-            }
             await conn.sendto(msg, "HTB")
 
         except asyncio.CancelledError:
@@ -619,9 +626,18 @@ async def async_main(args, config):
     loop.add_signal_handler(signal.SIGHUP, _sighup)
     
     # Start async tasks
+    # Per-protocol internet presence probe (shared with the network_monitor plugin)
+    net_cfg = dict((config.get("plugins") or {}).get("network_monitor")
+                   or config.get("network_monitor") or {})
+    connectivity = None
+    if net_cfg.get("probe_enabled", True):
+        connectivity = Connectivity.from_config(net_cfg)
+        set_shared(connectivity)
+        active_tasks.append(asyncio.create_task(connectivity.run(shutdown_event)))
+
     # Heartbeat senders (one per connection)
     for conn in connections:
-        task = asyncio.create_task(heartbeat_sender(conn, interval))
+        task = asyncio.create_task(heartbeat_sender(conn, interval, connectivity))
         active_tasks.append(task)
     
     # Plugin collector (uses all connections, but we'll use first one)

hbd/client/pingutil.py

@@ -0,0 +1,73 @@
+"""System ping helpers shared by ping_monitor and the connectivity probe."""
+
+import asyncio
+import re
+import sys
+from typing import Dict, List, Optional
+
+_INF = float("inf")
+
+
+class PingUnavailable(Exception):
+    """The system ping binary could not be executed."""
+
+
+def ping_command(host: str, count: int, timeout: int, family: Optional[int] = None) -> List[str]:
+    """Build the platform ping command. family is 4, 6 or None (let ping decide)."""
+    if sys.platform == "win32":
+        cmd = ["ping", "-n", str(count), "-w", str(timeout * 1000)]
+        if family:
+            cmd.append(f"-{family}")
+        return cmd + [host]
+    if sys.platform == "darwin" and family:
+        if family == 6:
+            return ["ping6", "-c", str(count), host]
+        return ["ping", "-c", str(count), "-W", str(timeout * 1000), host]
+    cmd = ["ping", "-c", str(count), "-W", str(timeout)]
+    if family:
+        cmd.append(f"-{family}")
+    return cmd + [host]
+
+
+def parse_ping_output(output: str) -> Dict[str, float]:
+    """Parse rtt_min/avg/max/loss from ping output (Linux, macOS, BSD)."""
+    loss = 100.0
+    loss_match = re.search(r"(\d+(?:\.\d+)?)\s*%\s*packet\s*loss", output)
+    if loss_match:
+        loss = float(loss_match.group(1))
+
+    # Linux: "rtt min/avg/max/mdev = x/x/x/x ms"; macOS: "round-trip min/avg/max/stddev = ..."
+    rtt_match = re.search(
+        r"(?:rtt|round-trip)\s+min/avg/max/\S+\s*=\s*([\d.]+)/([\d.]+)/([\d.]+)",
+        output,
+    )
+    if rtt_match:
+        return {
+            "rtt_min": float(rtt_match.group(1)),
+            "rtt_avg": float(rtt_match.group(2)),
+            "rtt_max": float(rtt_match.group(3)),
+            "loss": loss,
+        }
+    return {"rtt_min": _INF, "rtt_avg": _INF, "rtt_max": _INF, "loss": loss}
+
+
+async def run_ping(host: str, count: int, timeout: int, family: Optional[int] = None) -> Dict[str, float]:
+    """Run ping and return parsed metrics.
+
+    Raises PingUnavailable if the ping binary is missing and asyncio.TimeoutError
+    (after killing the process) if it hangs.
+    """
+    cmd = ping_command(host, count, timeout, family)
+    try:
+        proc = await asyncio.create_subprocess_exec(
+            *cmd, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE,
+        )
+    except FileNotFoundError as e:
+        raise PingUnavailable(str(e)) from e
+    try:
+        stdout, _ = await asyncio.wait_for(proc.communicate(), timeout=timeout * count + 2)
+    except asyncio.TimeoutError:
+        proc.kill()
+        await proc.wait()
+        raise
+    return parse_ping_output(stdout.decode(errors="replace"))

hbd/client/plugins/network_monitor.py

@@ -13,6 +13,7 @@ except ImportError:
     psutil = None
 
 from hbd.client.plugin import MonitorPlugin
+from hbd.client.connectivity import get_shared
 
 logger = logging.getLogger(__name__)
 
@@ -32,6 +33,10 @@ class NetworkMonitorPlugin(MonitorPlugin):
         interfaces: List of interfaces to monitor (default: all)
         include_connections: Include connection statistics (default: True)
         include_addresses: Include interface addresses (default: False)
+        probe_enabled: Run the per-protocol internet presence probe (default: True)
+        probe_v4: IPv4 address to ping (default: 1.1.1.1)
+        probe_v6: IPv6 address to ping (default: 2606:4700:4700::1111)
+        probe_interval: Seconds between ICMP probes while healthy (default: 60)
     """
     
     name = "network_monitor"
@@ -103,6 +108,7 @@ class NetworkMonitorPlugin(MonitorPlugin):
               - etc.
             - addresses: Interface address information (if include_addresses)
               - Dict keyed by interface name with address details
+            - connectivity: per-protocol presence {ipv4, ipv6} -> {present, addr, gateway, rtt}
         """
         if psutil is None:
             logger.error("psutil not available")
@@ -228,6 +234,11 @@ class NetworkMonitorPlugin(MonitorPlugin):
             
         except Exception as e:
             logger.warning(f"Could not collect interface stats: {e}")
+
+        # Per-protocol internet presence, address and gateway (from the shared probe)
+        shared = get_shared()
+        if shared is not None:
+            metrics['connectivity'] = shared.as_metric()
         
         return metrics
     

hbd/client/plugins/ping_monitor.py

@@ -40,9 +40,9 @@ threshold_configs:
 
 import asyncio
 import re
-import sys
 from typing import Any, Dict, Optional
 
+from hbd.client.pingutil import PingUnavailable, run_ping
 from hbd.client.plugin import MonitorPlugin
 
 
@@ -84,51 +84,13 @@ class PingMonitorPlugin(MonitorPlugin):
 
     async def _ping(self, host: str) -> Dict[str, float]:
         """Run a system ping command and return rtt_min/avg/max/loss."""
-        if sys.platform == "win32":
-            cmd = ["ping", "-n", str(self.count), "-w", str(self.timeout * 1000), host]
-        else:
-            cmd = ["ping", "-c", str(self.count), "-W", str(self.timeout), host]
-
         try:
-            proc = await asyncio.create_subprocess_exec(
+            return await run_ping(host, self.count, self.timeout)
-                *cmd,
+        except (asyncio.TimeoutError, PingUnavailable, OSError) as e:
-                stdout=asyncio.subprocess.PIPE,
-                stderr=asyncio.subprocess.PIPE,
-            )
-            stdout, _ = await asyncio.wait_for(
-                proc.communicate(),
-                timeout=self.timeout * self.count + 2,
-            )
-            output = stdout.decode(errors="replace")
-        except (asyncio.TimeoutError, FileNotFoundError, OSError) as e:
             self.logger.warning("ping_monitor: ping failed for %s: %s", host, e)
             return {"rtt_min": float("inf"), "rtt_avg": float("inf"),
                     "rtt_max": float("inf"), "loss": 100.0}
 
-        # Parse packet loss
-        loss = 100.0
-        loss_match = re.search(r"(\d+(?:\.\d+)?)\s*%\s*packet\s*loss", output)
-        if loss_match:
-            loss = float(loss_match.group(1))
-
-        # Parse rtt min/avg/max — Linux: "rtt min/avg/max/mdev = x/x/x/x ms"
-        #                          macOS: "round-trip min/avg/max/stddev = x/x/x/x ms"
-        rtt_match = re.search(
-            r"(?:rtt|round-trip)\s+min/avg/max/\S+\s*=\s*([\d.]+)/([\d.]+)/([\d.]+)",
-            output,
-        )
-        if rtt_match:
-            return {
-                "rtt_min": float(rtt_match.group(1)),
-                "rtt_avg": float(rtt_match.group(2)),
-                "rtt_max": float(rtt_match.group(3)),
-                "loss": loss,
-            }
-
-        # Host unreachable or all packets lost
-        return {"rtt_min": float("inf"), "rtt_avg": float("inf"),
-                "rtt_max": float("inf"), "loss": loss}
-
     async def _collect_metrics(self) -> Dict[str, Any]:
         data: Dict[str, Any] = {}
         tasks = {host: asyncio.create_task(self._ping(host)) for host in self.hosts}

hbd/server/hbdclass.py

@@ -29,6 +29,7 @@ class Connection:
     UP = "up"
     DOWN = "down"
     OVERDUE = "overdue"
+    ABSENT = "absent"
 
     def __init__(self, host, cid, addr, afam):
         self.host = host
@@ -94,7 +95,9 @@ class Connection:
         now = time.time()
         if not Null:
             d["addr"] = self.addr
-            if self.rtts[-1]:
+            if self.state == Connection.ABSENT:
+                d["rtt"] = ""
+            elif self.rtts[-1]:
                 d["rtt"] = "%d" % round(self.rtts[-1])
             elif self.state == Connection.UNKNOWN:
                 d["rtt"] = ""
@@ -299,6 +302,8 @@ class Host:
         self.alert_states = {}
         # Stale-data timers: {plugin_name: asyncio.TimerHandle}
         self.plugin_timers = {}
+        # Client-reported protocol presence: {afam: {"present": bool, "ts": float}}
+        self.presence: dict = {}
         # User access control
         self.owner: str | None = None       # username of owner
         self.managers: list = []            # usernames with manager role
@@ -322,6 +327,7 @@ class Host:
             d["name"] = "<b>%s</b>" % d["name"]
         d["dyn"] = str(self.dyn)
         d["num"] = self.num
+        d["status"] = self.status()
         
         # Add alert counts (split by acknowledged status)
         warning_unacked = 0
@@ -444,6 +450,7 @@ class Host:
             _, os_data = latest_os
             hbc_version = os_data.get("hbc_version")
         ddict["hbc_version"] = hbc_version
+        ddict["status"] = self.status()
 
         return ddict
 
@@ -459,6 +466,47 @@ class Host:
         else:
             return addr.find(".") > 0
 
+    def set_presence(self, afam, present, now):
+        """Record the client's report of whether it has internet connectivity on afam."""
+        self.presence[afam] = {"present": bool(present), "ts": now}
+
+    def flag_present(self, afam):
+        """Client-reported presence; True when the client never reported (legacy clients)."""
+        entry = getattr(self, "presence", {}).get(afam)
+        return True if entry is None else entry["present"]
+
+    def family_present(self, afam):
+        """True if afam should be monitored: the client says it is present, or heartbeats
+        are demonstrably arriving on it (real traffic overrides a false "absent")."""
+        conn = self.connections.get(afam)
+        return self.flag_present(afam) or (conn is not None and conn.state == Connection.UP)
+
+    def status(self):
+        """Host-level status: down, overdue, simplex, up or unknown.
+
+        overdue: every present family is overdue. simplex: both present, one up, one overdue
+        (same severity as up). Families the client reports absent are not considered,
+        unless none is present: then every connection not parked as ABSENT is considered.
+        """
+        conns = self.connections
+        if any(c.state == Connection.DOWN for c in conns.values()):
+            return "down"
+        present = [c for afam, c in conns.items() if self.family_present(afam)]
+        if not present:
+            # The client reports no protocol present (e.g. its internet probe fails on every
+            # family) yet may still be heard: monitor whatever is not parked as absent, so a
+            # host that then dies is "overdue", not silently "unknown".
+            present = [c for c in conns.values() if c.state != Connection.ABSENT]
+        up = sum(c.state == Connection.UP for c in present)
+        overdue = sum(c.state == Connection.OVERDUE for c in present)
+        if up and overdue:
+            return "simplex"
+        if up:
+            return "up"
+        if overdue:
+            return "overdue"
+        return "unknown"
+
     def conndata(self, cid, addr, rtt, now):
         if addr[0:7] == "::ffff:":
             addr = addr[7:]
@@ -499,6 +547,8 @@ class Host:
             self.monitors = []
         if not hasattr(self, "plugin_timers"):
             self.plugin_timers = {}
+        if not hasattr(self, "presence"):
+            self.presence = {}
 
         pass
 

hbd/server/templates/live.html

@@ -73,6 +73,7 @@
     .tipbox .up { color: var(--st-ok); font-weight: 700; }
     .tipbox .dn { color: var(--st-crit); font-weight: 700; }
     .tipbox .ov { color: var(--st-warn); font-weight: 700; }
+    .tipbox .ab { color: var(--st-faint); font-weight: 700; }
     .tipbox .meta { margin-top: 6px; padding-top: 6px; border-top: 1px solid var(--st-line-soft); font-size: 11px; color: var(--st-faint); }
 
     @media (max-width: 760px) {
@@ -134,18 +135,18 @@
       function worstConnection(data) {
         var worst = null;
         (data.connections || []).forEach(function (c) {
-          if (!c.state || c.state === 'unknown') return;
+          if (!c.state || c.state === 'unknown' || c.state === 'absent') return;
           if (!worst || (STATE_RANK[c.state] || 0) > (STATE_RANK[worst.state] || 0)) worst = c;
         });
         return worst;
       }
 
       function severity(data) {
-        var w = worstConnection(data);
+        var st = data.status;
         if ((data.alert_critical_unacked || 0) > 0 || (data.alert_critical_acked || 0) > 0) return 'crit';
-        if (w && w.state === 'down') return 'crit';
+        if (st === 'down') return 'crit';
         if ((data.alert_warning_unacked || 0) > 0 || (data.alert_warning_acked || 0) > 0) return 'warn';
-        if (w && w.state === 'overdue') return 'warn';
+        if (st === 'overdue') return 'warn';
         return 'ok';
       }
       var SEV_ORDER = { crit: 0, warn: 1, ok: 2 };
@@ -171,17 +172,17 @@
 
       // ---- host row rendering ---------------------------------------------
       function statusChip(data) {
-        var w = worstConnection(data);
+        var st = data.status;
-        if (!w) return '<span class="chip outline">—</span>';
+        if (!st || st === 'unknown') return '<span class="chip outline">—</span>';
-        var cls = w.state === 'up' ? 'ok' : (w.state === 'down' ? 'crit' : 'warn');
+        var cls = (st === 'up' || st === 'simplex') ? 'ok' : (st === 'down' ? 'crit' : 'warn');
-        return '<span class="chip ' + cls + '">' + escHtml(w.state) + '</span>';
+        return '<span class="chip ' + cls + '">' + escHtml(st) + '</span>';
       }
 
       function tipCard(data) {
         var rows = '';
         (data.connections || []).forEach(function (c) {
           if (!c.state || c.state === 'unknown') return;
-          var cls = c.state === 'up' ? 'up' : (c.state === 'down' ? 'dn' : 'ov');
+          var cls = c.state === 'up' ? 'up' : (c.state === 'down' ? 'dn' : (c.state === 'absent' ? 'ab' : 'ov'));
           var fam = (c.afam || '').replace('IPv', 'v');
           var rtt = (c.state === 'up' && c.rtts && c.rtts[0]) ? Math.round(c.rtts[0]) + ' ms' : '—';
           rows += '<tr><td>' + escHtml(fam) + '</td><td>' + escHtml(c.addr || '—') + '</td>'
@@ -247,7 +248,9 @@
         else if (data.hbc_version !== HBD_VERSION) {
           staleTag = '<span class="stale" title="agent v' + escHtml(String(data.hbc_version)) + ' ≠ server v' + HBD_VERSION + '">v' + escHtml(String(data.hbc_version)) + '</span>';
         }
-        var dotCls = w ? (w.state === 'up' ? sev : (w.state === 'down' ? 'down' : 'overdue')) : 'off';
+        var st = data.status;
+        var dotCls = (!st || st === 'unknown') ? 'off'
+          : ((st === 'up' || st === 'simplex') ? sev : (st === 'down' ? 'down' : 'overdue'));
         var flapTag = '';
         var flapping = data.flapping || [];
         if (flapping.length) {
@@ -286,11 +289,11 @@
         var total = 0, up = 0, overdue = 0, down = 0;
         Object.values(hostData).forEach(function (d) {
           total++;
-          var w = worstConnection(d);
+          var st = d.status;
-          if (!w) return;
+          if (!st || st === 'unknown') return;
-          if (w.state === 'up') up++;
+          if (st === 'up' || st === 'simplex') up++;
-          else if (w.state === 'overdue') overdue++;
+          else if (st === 'overdue') overdue++;
-          else if (w.state === 'down') down++;
+          else if (st === 'down') down++;
         });
         var parts = [total + ' hosts', up + ' up'];
         if (overdue) parts.push(overdue + ' overdue');

hbd/server/templates/plugins.html

@@ -456,7 +456,7 @@
             </tr></thead><tbody>`;
           for (const c of data.connections) {
             const st = escHtml(c.state || '—');
-            const stCls = c.state === 'up' ? 'status-up' : (c.state ? 'status-down' : '');
+            const stCls = c.state === 'up' ? 'status-up' : (c.state === 'absent' ? '' : (c.state ? 'status-down' : ''));
             const rtt = (c.state === 'up' && c.rtt) ? Math.round(c.rtt) + ' ms' : '—';
             const chg = c.statetime ? new Date(c.statetime * 1000).toLocaleString() : '—';
             const seen = c.lastbeat ? new Date(c.lastbeat * 1000).toLocaleString() : '—';
@@ -827,13 +827,10 @@
         if (!el || pts.length < 2) { if (el) el.style.display = 'none'; return; }
 
         const unitSuffix = opts.unitSuffix || '';
-        const W = 600, H = 80, PAD = { top: 6, right: 8, bottom: 18, left: 28 };
+        const W = 690, H = 92, PAD = { top: 6, right: 8, bottom: 18, left: 28 };
-        const cW = W - PAD.left - PAD.right;
-        const cH = H - PAD.top - PAD.bottom;
 
         const tMin = pts[0].t, tMax = pts[pts.length - 1].t;
         const tRange = tMax - tMin || 1;
-        const x = t => PAD.left + ((t - tMin) / tRange) * cW;
 
         // Auto-scale Y axis with 10% padding, optionally clamped to opts.yDomain
         const vMin = Math.min(...pts.map(p => p.v));
@@ -845,39 +842,80 @@
         const yLow  = Math.max(domainLow,  vMin - vPad);
         const yHigh = Math.min(domainHigh, vMax + vPad);
         const yRange = yHigh - yLow || 1;
-        const y = v => PAD.top + cH - ((v - yLow) / yRange) * cH;
 
-        // Build polyline points and filled area path
+        // Compute nice tick step for ~3-5 grid lines, then size the left
-        const linePoints = pts.map(p => `${x(p.t).toFixed(1)},${y(p.v).toFixed(1)}`).join(' ');
+        // margin to fit the widest label (values/units can run to 3+ digits,
-        const areaPath = `M${x(pts[0].t).toFixed(1)},${(PAD.top + cH).toFixed(1)} ` +
+        // e.g. RTT samples above 99ms) instead of a fixed guess that clips them.
-          pts.map(p => `L${x(p.t).toFixed(1)},${y(p.v).toFixed(1)}`).join(' ') +
+        const rawStep = yRange / 4;
-          ` L${x(pts[pts.length-1].t).toFixed(1)},${(PAD.top + cH).toFixed(1)} Z`;
+        const mag = Math.pow(10, Math.floor(Math.log10(rawStep || 1)));
+        const niceStep = [1, 2, 5, 10].map(f => f * mag).find(s => yRange / s <= 5) || mag * 10;
+        const tickStart = Math.ceil(yLow / niceStep) * niceStep;
+        const yTicks = [];
+        for (let v = tickStart; v <= yHigh + 0.001; v += niceStep) yTicks.push(v);
+        const yTickLabels = yTicks.map(v => (Number.isInteger(v) ? v : v.toFixed(1)) + unitSuffix);
+        const maxLabelLen = yTickLabels.reduce((m, s) => Math.max(m, s.length), 0);
+        PAD.left = Math.max(28, Math.ceil(maxLabelLen * 5.5) + 10);
+
+        const cW = W - PAD.left - PAD.right;
+        const cH = H - PAD.top - PAD.bottom;
+        const x = t => PAD.left + ((t - tMin) / tRange) * cW;
+        const y = v => PAD.top + cH - ((v - yLow) / yRange) * cH;
 
         // Color based on latest value
         const latest = pts[pts.length - 1].v;
         const { stroke: strokeColor, fill: fillColor } = opts.colorFor(latest);
 
-        // Compute nice tick step for ~3-5 grid lines
+        // Split into segments wherever the gap between consecutive samples is
-        const rawStep = yRange / 4;
+        // much larger than the typical spacing (e.g. the host was overdue/down
-        const mag = Math.pow(10, Math.floor(Math.log10(rawStep || 1)));
+        // for a while) — draw each segment separately so missing data reads as
-        const niceStep = [1, 2, 5, 10].map(f => f * mag).find(s => yRange / s <= 5) || mag * 10;
+        // a visual gap instead of an interpolated line across dead time.
-        const tickStart = Math.ceil(yLow / niceStep) * niceStep;
+        const deltas = [];
-        let gridLines = '';
+        for (let i = 1; i < pts.length; i++) deltas.push(pts[i].t - pts[i - 1].t);
-        for (let v = tickStart; v <= yHigh + 0.001; v += niceStep) {
+        deltas.sort((a, b) => a - b);
-          const yy = y(v).toFixed(1);
+        const medianDelta = deltas[Math.floor(deltas.length / 2)];
-          const label = (Number.isInteger(v) ? v : v.toFixed(1)) + unitSuffix;
+        const gapThreshold = medianDelta * 2.5;
-          gridLines += `<line x1="${PAD.left}" y1="${yy}" x2="${PAD.left + cW}" y2="${yy}" stroke="#e0e0e0" stroke-width="1"/>`;
+
-          gridLines += `<text x="${(PAD.left - 3).toFixed(1)}" y="${yy}" text-anchor="end" dominant-baseline="middle" font-size="8" fill="#999">${label}</text>`;
+        const segments = [[pts[0]]];
+        for (let i = 1; i < pts.length; i++) {
+          if (pts[i].t - pts[i - 1].t > gapThreshold) segments.push([]);
+          segments[segments.length - 1].push(pts[i]);
         }
 
-        // X-axis time labels
+        let linePolylines = '';
+        let areaPaths = '';
+        for (const seg of segments) {
+          if (seg.length < 2) continue;
+          const segPoints = seg.map(p => `${x(p.t).toFixed(1)},${y(p.v).toFixed(1)}`).join(' ');
+          linePolylines += `<polyline points="${segPoints}" fill="none" stroke="${strokeColor}" stroke-width="1.5" stroke-linejoin="round"/>`;
+          const segArea = `M${x(seg[0].t).toFixed(1)},${(PAD.top + cH).toFixed(1)} ` +
+            seg.map(p => `L${x(p.t).toFixed(1)},${y(p.v).toFixed(1)}`).join(' ') +
+            ` L${x(seg[seg.length-1].t).toFixed(1)},${(PAD.top + cH).toFixed(1)} Z`;
+          areaPaths += `<path d="${segArea}" fill="${fillColor}" opacity="0.6"/>`;
+        }
+
+        let gridLines = '';
+        yTicks.forEach((v, i) => {
+          const yy = y(v).toFixed(1);
+          gridLines += `<line x1="${PAD.left}" y1="${yy}" x2="${PAD.left + cW}" y2="${yy}" stroke="#e0e0e0" stroke-width="1"/>`;
+          gridLines += `<text x="${(PAD.left - 3).toFixed(1)}" y="${yy}" text-anchor="end" dominant-baseline="middle" font-size="8" fill="#999">${yTickLabels[i]}</text>`;
+        });
+
+        // X-axis: start/end plus evenly spaced intermediate tickmarks + labels
         const fmt = ts => {
           const d = new Date(ts * 1000);
           return d.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' });
         };
-        const xLabels = `
+        const xTickCount = 5;
-          <text x="${PAD.left}" y="${H - 2}" text-anchor="start" font-size="8" fill="#999">${fmt(pts[0].t)}</text>
+        const xAxisY = (PAD.top + cH).toFixed(1);
-          <text x="${PAD.left + cW}" y="${H - 2}" text-anchor="end" font-size="8" fill="#999">${fmt(pts[pts.length-1].t)}</text>`;
+        let xAxisMarks = '';
+        let xLabels = '';
+        for (let i = 0; i < xTickCount; i++) {
+          const t = tMin + (tRange * i) / (xTickCount - 1);
+          const xx = x(t).toFixed(1);
+          const anchor = i === 0 ? 'start' : (i === xTickCount - 1 ? 'end' : 'middle');
+          xAxisMarks += `<line x1="${xx}" y1="${xAxisY}" x2="${xx}" y2="${(PAD.top + cH + 3).toFixed(1)}" stroke="#ccc" stroke-width="1"/>`;
+          xLabels += `<text x="${xx}" y="${H - 2}" text-anchor="${anchor}" font-size="8" fill="#999">${fmt(t)}</text>`;
+        }
 
         el.innerHTML = `<svg viewBox="0 0 ${W} ${H}" preserveAspectRatio="none"
           style="width:100%;height:${H}px;display:block;">
@@ -890,9 +928,10 @@
           <line x1="${PAD.left}" y1="${PAD.top}" x2="${PAD.left}" y2="${PAD.top + cH}" stroke="#ccc" stroke-width="1"/>
           <line x1="${PAD.left}" y1="${PAD.top + cH}" x2="${PAD.left + cW}" y2="${PAD.top + cH}" stroke="#ccc" stroke-width="1"/>
           <g clip-path="url(#${opts.clipId})">
-            <path d="${areaPath}" fill="${fillColor}" opacity="0.6"/>
+            ${areaPaths}
-            <polyline points="${linePoints}" fill="none" stroke="${strokeColor}" stroke-width="1.5" stroke-linejoin="round"/>
+            ${linePolylines}
           </g>
+          ${xAxisMarks}
           ${xLabels}
         </svg>`;
       }
@@ -1104,6 +1143,24 @@
       function renderNetworkTables(d) {
         let html = '';
 
+        if (d.connectivity) {
+          let t = '<table class="data-table"><thead><tr><th>Protocol</th><th>Internet</th>'
+            + '<th>Address</th><th>Gateway</th><th class="num">RTT</th></tr></thead><tbody>';
+          for (const fam of ['ipv4', 'ipv6']) {
+            const c = d.connectivity[fam];
+            if (!c) continue;
+            const known = c.present === true || c.present === false;
+            const cls = c.present === true ? 'status-up' : '';
+            const txt = c.present === true ? '✓ present' : (c.present === false ? 'absent' : '?');
+            t += `<tr><td>${fam === 'ipv4' ? 'IPv4' : 'IPv6'}</td>
+              <td class="${cls}">${txt}</td>
+              <td>${escHtml(c.addr || '—')}</td><td>${escHtml(c.gateway || '—')}</td>
+              <td class="num">${known && c.rtt != null ? Math.round(c.rtt) + ' ms' : '—'}</td></tr>`;
+          }
+          t += '</tbody></table>';
+          html += `<div class="table-section-label">Connectivity</div>${t}`;
+        }
+
         if (d.interfaces && Object.keys(d.interfaces).length) {
           const sample = Object.values(d.interfaces)[0];
           const hasDelta = 'bytes_sent_delta' in sample;

hbd/server/udp.py

@@ -194,6 +194,40 @@ def _set_connectivity_alert(host, afam, level_name):
     state.update(level, level_name)
 
 
+_PRESENCE_KEYS = {"p4": "IPv4", "p6": "IPv6"}
+
+
+def _apply_presence(host, msg, now, uname, ctx, skip_conn):
+    """Record client-reported protocol presence and settle the affected connections.
+
+    A family reported absent whose connection is already overdue/unknown is parked as
+    ABSENT (timer cancelled, connectivity alert cleared). One still UP is left alone; its
+    timer expiry parks it. A family reported present again while ABSENT restarts a grace
+    timer. The connection that carried this very heartbeat (skip_conn) is never touched.
+    """
+    cfg = ctx.get("config", {})
+    for key, afam in _PRESENCE_KEYS.items():
+        if key not in msg:
+            continue
+        present = bool(msg[key])
+        host.set_presence(afam, present, now)
+        conn = host.connections.get(afam)
+        if conn is None or conn is skip_conn:
+            continue
+        cls = conn.__class__
+        if not present:
+            if conn.getstate() in (cls.OVERDUE, cls.UNKNOWN):
+                conn.cancel_overdue_timer()
+                conn.newstate(cls.ABSENT, now)
+                _set_connectivity_alert(host, afam, "OK")
+        elif conn.getstate() == cls.ABSENT:
+            conn.newstate(cls.UNKNOWN, now)
+            interval = int(msg.get("interval", 0) or 0) or host.interval
+            if interval > 0:
+                on_overdue, _ = _make_timer_callbacks(uname, host, ctx)
+                conn.reset_overdue_timer(interval + cfg.get("grace", 2), on_overdue)
+
+
 def _make_timer_callbacks(uname, host, ctx):
     """Return (on_overdue, on_unknown) async callbacks for connection timer logic.
 
@@ -210,26 +244,52 @@ def _make_timer_callbacks(uname, host, ctx):
             msg_to_websockets("host", host.stateinfo())
 
     async def on_overdue(connection):
-        if connection.getstate() != connection.__class__.UP:
+        cls = connection.__class__
+        if connection.getstate() not in (cls.UP, cls.UNKNOWN):
             return
         now = time.time()
-        connection.newstate(connection.__class__.OVERDUE, now, cfg.get("grace", 2))
+        afam = connection.afam
-        msg = f"{connection.afam} overdue"
+        # Park only while some other family is still present: the last monitored family
-        eventlog(uname, "CRITICAL", msg)
+        # must never be parked silently, or a host whose client reports no internet on any
-        if host.watched:
+        # protocol would die without an alert.
-            asyncio.create_task(notify_mod.send_notification(
+        if not host.flag_present(afam) and any(
-                uname,
+                af != afam and host.family_present(af) for af in host.connections):
-                notify_mod.Notification(title=f"[CRITICAL] {uname}", body=msg, level="CRITICAL"),
+            # The client says this protocol has no connectivity: not an outage.
-            ))
+            connection.newstate(cls.ABSENT, now)
-        # Track in alert_states so the Alerts Dashboard shows this
+            connection.cancel_overdue_timer()
-        _set_connectivity_alert(host, connection.afam, "CRITICAL")
+            _set_connectivity_alert(host, afam, "OK")
-        if threshold_checker:
+            if msg_to_websockets:
-            threshold_checker.check_value(
+                msg_to_websockets("host", host.stateinfo())
-                host_name=uname,
+            return
-                metric_path="rtt",
+
-                value=float("inf"),
+        # If any connection is DOWN (clean shutdown reported) Host.status() is "down", so a
-                alert_states=host.alert_states,
+        # later overdue of the other family raises no separate event/alert; the shutdown did.
-            )
+        prev = host.status()
+        connection.newstate(cls.OVERDUE, now, cfg.get("grace", 2))
+        new = host.status()
+        if new == "overdue" and prev != "overdue":
+            names = ", ".join(sorted(af for af, c in host.connections.items()
+                                     if c.getstate() == cls.OVERDUE))
+            msg = f"{names} overdue"
+            eventlog(uname, "CRITICAL", msg)
+            if host.watched:
+                asyncio.create_task(notify_mod.send_notification(
+                    uname,
+                    notify_mod.Notification(title=f"[CRITICAL] {uname}", body=msg, level="CRITICAL"),
+                ))
+            # Track in alert_states so the Alerts Dashboard shows this
+            for af, c in host.connections.items():
+                if c.getstate() == cls.OVERDUE:
+                    _set_connectivity_alert(host, af, "CRITICAL")
+            if threshold_checker:
+                threshold_checker.check_value(
+                    host_name=uname,
+                    metric_path="rtt",
+                    value=float("inf"),
+                    alert_states=host.alert_states,
+                )
+        elif new == "simplex":
+            eventlog(uname, "INFO", f"{afam} overdue (simplex)")
         if msg_to_websockets:
             msg_to_websockets("host", host.stateinfo())
         connection.reset_overdue_timer(DROPOVERDUE, on_unknown)
@@ -270,6 +330,8 @@ def restore_connection_timers(hbdclass, ctx):
         interval = host.interval
         for afam, conn in list(host.connections.items()):
             state = conn.getstate()
+            if state == hbdclass.Connection.ABSENT:
+                continue
             if state == hbdclass.Connection.DOWN:
                 _set_connectivity_alert(host, afam, "CRITICAL")
                 continue
@@ -295,6 +357,7 @@ def restore_connection_timers(hbdclass, ctx):
                 restored += 1
 
             elif state == hbdclass.Connection.OVERDUE:
+                host_overdue = host.status() == "overdue"
                 elapsed_overdue = now - conn.statetime
                 remaining = DROPOVERDUE - elapsed_overdue
                 if remaining <= 1:
@@ -304,6 +367,9 @@ def restore_connection_timers(hbdclass, ctx):
                         "Marking %s/%s UNKNOWN (overdue %.1f days)",
                         uname, afam, elapsed_overdue / 86400,
                     )
+                    # A connection dropped to UNKNOWN keeps its alert, like the
+                    # UNKNOWN branch above and on_unknown do.
+                    host_overdue = True
                 else:
                     conn.reset_overdue_timer(remaining, on_unknown)
                     logger.debug(
@@ -313,7 +379,8 @@ def restore_connection_timers(hbdclass, ctx):
                 # Ensure the connectivity alert is set — it may be missing if
                 # hbd was shut down before the on_overdue callback had a chance
                 # to record it.
-                _set_connectivity_alert(host, afam, "CRITICAL")
+                if host_overdue:
+                    _set_connectivity_alert(host, afam, "CRITICAL")
                 restored += 1
 
     logger.info("Restored timers for %d connection(s)", restored)
@@ -517,25 +584,36 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
     if message:
         eventlog(uname, "INFO", message, service=service)
 
+    _apply_presence(host, msg, now, uname, ctx, conn)
+    prev_status = host.status()
+
     if conn.getstate() != hbdcls.Connection.UP:
         # Transition to UP and log/notify if appropriate
         lasts = conn.state
         d = conn.newstate(hbdcls.Connection.UP, now)
-        # On reboot, pre-boot plugin data and derived alerts are stale.
+        if boot:
-        # Cancel all plugin timers and wipe plugin state so timers restart
+            # On reboot, pre-boot plugin data and derived alerts are stale.
-        # cleanly from the first two post-boot samples.
+            # Cancel all plugin timers and wipe plugin state so timers restart
-        for pname in list(host.plugin_timers):
+            # cleanly from the first two post-boot samples. An ordinary
-            host.cancel_plugin_timer(pname)
+            # reconnect (no boot flag) doesn't invalidate the client's
-        for pname in [k for k in host.plugin_data if not _is_rtt_key(k)]:
+            # already-collected data, so it's left alone — this keeps chart
-            del host.plugin_data[pname]
+            # history intact across a transient network blip.
+            for pname in list(host.plugin_timers):
+                host.cancel_plugin_timer(pname)
+            for pname in [k for k in host.plugin_data if not _is_rtt_key(k)]:
+                del host.plugin_data[pname]
         stale_plugin_keys = [
             k for k in host.alert_states
             if k not in ("rtt",) and not k.startswith("connectivity.")
         ]
         for k in stale_plugin_keys:
             del host.alert_states[k]
-        # Clear connectivity alert now that the host is back up
+        # Clear connectivity alert now that the host is back up; once the host is no longer
+        # overdue no family may keep a connectivity alert (e.g. the simplex case).
         _set_connectivity_alert(host, conn.afam, "OK")
+        if host.status() in ("up", "simplex"):
+            for af in host.connections:
+                _set_connectivity_alert(host, af, "OK")
         # Don't log/notify RECOVER for a brand-new host seen for the first time —
         # it was never down, it just hasn't been seen before.
         if not newh:
@@ -547,8 +625,11 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
             else:
                 m = "%s back after being %s for %s" % (conn.afam, lasts, dur(d))
             if m:
-                eventlog(uname, "RECOVER", m)
+                # RECOVER (and a notification) only if the host itself was overdue/down;
-                if host.watched:
+                # a single family coming back while another stayed up is informational.
+                level = "RECOVER" if prev_status in ("overdue", "down", "unknown") else "INFO"
+                eventlog(uname, level, m)
+                if host.watched and level == "RECOVER":
                     asyncio.create_task(notify_mod.send_notification(
                         uname,
                         notify_mod.Notification(title=f"[RECOVER] {uname}", body=m, level="RECOVER"),

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "hbd"
-version = "5.4.1"
+version = "5.4.5"
 description = "Heartbeat monitoring system — client (hbc) and server (hbd)"
 readme = "README.md"
 requires-python = ">=3.11"

scripts/bumpminor.sh

@@ -9,8 +9,10 @@ sed -i".bak"  "s/\*\*Package:\*\* \`hbd\` v[0-9.]*/\*\*Package:\*\* \`hbd\` v$VE
 
 # Update CHANGELOG.md with commits since last tag
 LASTTAG=$(git describe --tags --abbrev=0 2>/dev/null || true)
-ADDED=$(git log "${LASTTAG:+$LASTTAG..}HEAD" --pretty="%s" | grep "^feat:" | sed 's/^feat: /- /')
+# Matches both unscoped ("feat: ...") and scoped ("feat(server): ...") conventional-commit
-FIXED=$(git log "${LASTTAG:+$LASTTAG..}HEAD" --pretty="%s" | grep "^fix:" | sed 's/^fix: /- /')
+# subjects; the scope is dropped from the changelog line.
+ADDED=$(git log "${LASTTAG:+$LASTTAG..}HEAD" --pretty="%s" | grep -E "^feat(\([^)]*\))?:" | sed -E 's/^feat(\([^)]*\))?: /- /')
+FIXED=$(git log "${LASTTAG:+$LASTTAG..}HEAD" --pretty="%s" | grep -E "^fix(\([^)]*\))?:" | sed -E 's/^fix(\([^)]*\))?: /- /')
 {
     printf "## [%s]\n" "$VER"
     [ -n "$ADDED" ] && printf "\n### Added\n%s\n" "$ADDED"

scripts/c/.gitignore

@@ -1,2 +1,3 @@
 hbc_mini
 hbc_mini_dbg
+test_probe

scripts/c/Makefile

@@ -7,7 +7,7 @@ SRC     = hbc_mini.c
 # FreeBSD/NetBSD keep zlib in base; no extra flags needed.
 # On some NetBSD installs pthreads may need -lpthread from pkgsrc.
 
-.PHONY: all clean debug
+.PHONY: all clean debug test
 
 all: $(TARGET)
 
@@ -17,5 +17,11 @@ $(TARGET): $(SRC)
 debug: $(SRC)
 	$(CC) -g -fsanitize=address,undefined -o $(TARGET)_dbg $< $(LDFLAGS)
 
+test: test_probe
+	./test_probe
+
+test_probe: test_probe.c $(SRC)
+	$(CC) $(CFLAGS) -o $@ test_probe.c $(LDFLAGS)
+
 clean:
-	rm -f $(TARGET) $(TARGET)_dbg
+	rm -f $(TARGET) $(TARGET)_dbg test_probe

scripts/c/hbc_mini.c

@@ -6,6 +6,12 @@
  *
  * Drop on any host and run: ./hbc_mini <hbd-host>
  * Config: ~/.hbc.json (same keys as Python version)
+ *
+ * Presence probe (built in): reports per protocol (IPv4/IPv6) whether the host has internet
+ * connectivity, as p4/p6 flags on every heartbeat, so hbd stops treating a missing IPv6 (or
+ * IPv4) path as an outage. Config, in the network_monitor section of ~/.hbc.json:
+ *   probe_enabled (default true), probe_v4 (default 1.1.1.1), probe_v6 (default
+ *   2606:4700:4700::1111; IP literals only), probe_interval (default 60 s)
  */
 
 #ifdef __linux__
@@ -396,6 +402,10 @@ typedef struct {
 
     char net_skip[MAX_NET_SKIP][64];
     int  net_skip_count;
+
+    bool probe_enabled;
+    int  probe_interval;
+    char probe_v4[64], probe_v6[64];
 } config_t;
 
 static void config_defaults(config_t *c) {
@@ -410,6 +420,10 @@ static void config_defaults(config_t *c) {
     c->net_interval   = 300;
     snprintf(c->net_skip[0], 64, "lo");
     c->net_skip_count = 1;
+    c->probe_enabled  = true;
+    c->probe_interval = 60;
+    snprintf(c->probe_v4, sizeof(c->probe_v4), "1.1.1.1");
+    snprintf(c->probe_v6, sizeof(c->probe_v6), "2606:4700:4700::1111");
 }
 
 static void config_load(config_t *cfg, const char *path) {
@@ -498,6 +512,10 @@ static void config_load(config_t *cfg, const char *path) {
     jval_t *net = plugins ? jget(plugins,"network_monitor") : jget(root,"network_monitor");
     if (net && net->type == JT_OBJECT) {
         if ((v=jget(net,"interval"))) cfg->net_interval = jint(v, cfg->net_interval);
+        if ((v=jget(net,"probe_enabled")))  cfg->probe_enabled  = jint(v, 1) != 0;
+        if ((v=jget(net,"probe_interval"))) cfg->probe_interval = jint(v, cfg->probe_interval);
+        if ((v=jget(net,"probe_v4"))) snprintf(cfg->probe_v4, sizeof(cfg->probe_v4), "%s", jstr(v, cfg->probe_v4));
+        if ((v=jget(net,"probe_v6"))) snprintf(cfg->probe_v6, sizeof(cfg->probe_v6), "%s", jstr(v, cfg->probe_v6));
         jval_t *sk = jget(net,"skip_interfaces");
         if (sk && sk->type == JT_ARRAY) {
             cfg->net_skip_count = 0;
@@ -984,6 +1002,341 @@ static void plugin_disk_monitor(conn_t *c, const config_t *cfg) {
     LOGD("sent disk_monitor");
 }
 
+/* ============================================================
+ * Connectivity probe: per-protocol internet presence (IPv4 / IPv6)
+ *
+ * Tier 1 (every PROBE_TIER1 s, sends no packets): a UDP connect() to the probe address shows
+ * whether a route exists and which source address would be used. Tier 2 (every probe_interval
+ * s, sooner after a failure or a change of address or gateway): one ICMP echo, only when tier 1
+ * found a route. Absent = no route, or PROBE_FAIL_LIMIT consecutive failed echoes; present =
+ * one success; unknown (flag omitted) when the family cannot be probed. The result rides on
+ * every HTB as p4/p6 and inside the network_monitor metric. Mirrors hbd/client/connectivity.py.
+ * ============================================================ */
+
+#define PROBE_UNKNOWN     (-1)
+#define PROBE_FAIL_LIMIT  2
+#define PROBE_TIER1       5      /* seconds between route checks */
+
+typedef struct {
+    int    present;                          /* PROBE_UNKNOWN, 0 = absent, 1 = present */
+    char   addr[INET6_ADDRSTRLEN];
+    char   gateway[INET6_ADDRSTRLEN];
+    double rtt;                              /* ms, < 0 when unknown */
+    int    fails;                            /* consecutive failed echoes */
+    double next_ping;                        /* monotonic seconds */
+    char   last_key[2 * INET6_ADDRSTRLEN + 2];
+    bool   ping_disabled;                    /* no usable ping binary */
+} probe_fam_t;
+
+typedef struct {
+    /* 1 = route exists (source address in out), 0 = no route, -1 = family unsupported/unusable */
+    int    (*local_address)(int af, const char *target, char *out, size_t n);
+    /* RTT in ms; -1 = no reply; -2 = ping unavailable */
+    double (*ping)(int af, const char *target);
+    /* fills out with the default gateway, "" when unknown */
+    void   (*gateway)(int af, char *out, size_t n);
+} probe_ops_t;
+
+typedef struct {
+    probe_fam_t     fam[2];                  /* [0] = IPv4, [1] = IPv6 */
+    char            target[2][64];
+    double          probe_interval, retry_interval;
+    bool            cheap_gateway;           /* gateway read is cheap (/proc), so read every tier 1 */
+    pthread_mutex_t mu;                      /* guards fam[] between the probe thread and readers */
+} probe_t;
+
+/* Lowest-metric default gateway from /proc/net/route. The kernel prints the raw network-order
+ * u32 in host order, so it can be stored straight into s_addr on any endianness. */
+static void gw_parse_proc_v4(const char *text, char *out, size_t n) {
+    long best = -1; out[0] = '\0';
+    const char *p = strchr(text, '\n');                    /* skip the header line */
+    while (p && *++p) {
+        char iface[64], dest[16], gw[16]; unsigned flags; int refcnt, use, metric;
+        if (sscanf(p, "%63s %15s %15s %x %d %d %d", iface, dest, gw, &flags, &refcnt, &use, &metric) == 7
+                && !strcmp(dest, "00000000") && (flags & 0x2)) {
+            if (best < 0 || metric < best) {
+                struct in_addr a; a.s_addr = (uint32_t)strtoul(gw, NULL, 16);
+                if (inet_ntop(AF_INET, &a, out, n)) best = metric;
+            }
+        }
+        p = strchr(p, '\n');
+    }
+}
+
+/* Lowest-metric default gateway from /proc/net/ipv6_route (skips RTF_REJECT and "lo"). */
+static void gw_parse_proc_v6(const char *text, char *out, size_t n) {
+    long best = -1; out[0] = '\0';
+    const char *zero = "00000000000000000000000000000000";
+    for (const char *p = text; p && *p; ) {
+        char dest[40], src[40], nh[40], iface[64];
+        unsigned plen, splen, metric, refcnt, use, flags;
+        if (sscanf(p, "%32s %x %32s %x %32s %x %x %x %x %63s",
+                   dest, &plen, src, &splen, nh, &metric, &refcnt, &use, &flags, iface) == 10
+                && !strcmp(dest, zero) && plen == 0 && strcmp(nh, zero) && strcmp(iface, "lo")
+                && (flags & 0x2) && !(flags & 0x200)) {
+            if (best < 0 || (long)metric < best) {
+                struct in6_addr a; memset(&a, 0, sizeof(a));
+                bool ok = strlen(nh) == 32;
+                for (int i = 0; ok && i < 16; i++) {
+                    unsigned b;
+                    if (sscanf(nh + 2 * i, "%2x", &b) != 1) ok = false; else a.s6_addr[i] = (uint8_t)b;
+                }
+                if (ok && inet_ntop(AF_INET6, &a, out, n)) best = (long)metric;
+            }
+        }
+        p = strchr(p, '\n'); if (p) p++;
+    }
+}
+
+/* Gateway from `route -n get [-inet6] default` (the BSDs). */
+static void gw_parse_route_get(const char *text, char *out, size_t n) {
+    out[0] = '\0';
+    const char *p = strstr(text, "gateway:");
+    if (!p) return;
+    p += 8;
+    while (*p == ' ' || *p == '\t') p++;
+    size_t i = 0;
+    while (*p && !isspace((unsigned char)*p) && i + 1 < n) out[i++] = *p++;
+    out[i] = '\0';
+}
+
+/* Average RTT from ping output: -1.0 no reply, -2.0 ping could not be run. Works for Linux
+ * ("rtt min/avg/max/mdev") and BSD ("round-trip min/avg/max/stddev"). */
+static double ping_parse_rtt(const char *out) {
+    if (strstr(out, "not found") || strstr(out, "No such file")) return -2.0;
+    const char *p = strstr(out, "min/avg/max");
+    if (!p) return -1.0;
+    p = strchr(p, '=');
+    double mn, avg, mx;
+    if (p && sscanf(p + 1, " %lf/%lf/%lf", &mn, &avg, &mx) == 3) return avg;
+    return -1.0;
+}
+
+/* One probe step for one family (f: 0 = IPv4, 1 = IPv6). Works on *s only, so the probe thread
+ * can run it on a private copy without holding the mutex during the (blocking) echo. */
+static void probe_step(const probe_t *p, probe_fam_t *s, int f, const probe_ops_t *ops, double now) {
+    int af = f ? AF_INET6 : AF_INET;
+    char addr[INET6_ADDRSTRLEN] = "", gw[INET6_ADDRSTRLEN] = "";
+
+    int r = ops->local_address(af, p->target[f], addr, sizeof(addr));
+    if (r < 0) { s->present = PROBE_UNKNOWN; return; }
+    if (r == 0) {                                          /* tier 1: no route => absent immediately */
+        s->fails = 0; s->next_ping = 0; s->last_key[0] = '\0';
+        s->present = 0; s->addr[0] = '\0'; s->gateway[0] = '\0'; s->rtt = -1;
+        return;
+    }
+
+    if (p->cheap_gateway) ops->gateway(af, gw, sizeof(gw));
+    else snprintf(gw, sizeof(gw), "%s", s->gateway);
+    char key[sizeof(s->last_key)];
+    snprintf(key, sizeof(key), "%s|%s", addr, gw);
+    bool due = strcmp(key, s->last_key) != 0 || now >= s->next_ping;
+    if (due && !p->cheap_gateway) {
+        ops->gateway(af, gw, sizeof(gw));
+        snprintf(key, sizeof(key), "%s|%s", addr, gw);
+    }
+    snprintf(s->addr, sizeof(s->addr), "%s", addr);
+    snprintf(s->gateway, sizeof(s->gateway), "%s", gw);
+    snprintf(s->last_key, sizeof(s->last_key), "%s", key);
+
+    if (s->ping_disabled) { s->present = PROBE_UNKNOWN; return; } /* cannot probe: unknown, never absent, while a route exists */
+    if (!due) return;
+    double rtt = ops->ping(af, p->target[f]);
+    if (rtt < -1.5) { s->ping_disabled = true; s->present = PROBE_UNKNOWN; return; }
+    if (rtt < 0) {
+        s->fails++;
+        s->next_ping = now + p->retry_interval;
+        if (s->fails >= PROBE_FAIL_LIMIT) { s->present = 0; s->rtt = -1; }
+    } else {
+        s->fails = 0;
+        s->next_ping = now + p->probe_interval;
+        s->present = 1; s->rtt = rtt;
+    }
+}
+
+/* HTB flags for the families whose presence is known. */
+static void probe_flags(probe_t *p, kvdict_t *d) {
+    pthread_mutex_lock(&p->mu);
+    if (p->fam[0].present >= 0) kv_set_int(d, "p4", p->fam[0].present);
+    if (p->fam[1].present >= 0) kv_set_int(d, "p6", p->fam[1].present);
+    pthread_mutex_unlock(&p->mu);
+}
+
+/* The `connectivity` metric as a JSON object; returns its length or -1 on overflow. */
+static int probe_metric_json(probe_t *p, char *out, size_t n) {
+    static const char *names[2] = { "ipv4", "ipv6" };
+    size_t len = 0;
+    len += (size_t)snprintf(out + len, n - len, "{");
+    pthread_mutex_lock(&p->mu);
+    for (int f = 0; f < 2 && len < n; f++) {
+        const probe_fam_t *s = &p->fam[f];
+        char pres[8], addr[64], gw[64], rtt[32];
+        snprintf(pres, sizeof(pres), "%s", s->present < 0 ? "null" : s->present ? "true" : "false");
+        if (s->addr[0]) snprintf(addr, sizeof(addr), "\"%s\"", s->addr); else snprintf(addr, sizeof(addr), "null");
+        if (s->gateway[0]) snprintf(gw, sizeof(gw), "\"%s\"", s->gateway); else snprintf(gw, sizeof(gw), "null");
+        if (s->rtt >= 0) snprintf(rtt, sizeof(rtt), "%.3f", s->rtt); else snprintf(rtt, sizeof(rtt), "null");
+        len += (size_t)snprintf(out + len, n - len,
+            "%s\"%s\":{\"present\":%s,\"addr\":%s,\"gateway\":%s,\"rtt\":%s}",
+            f ? "," : "", names[f], pres, addr, gw, rtt);
+    }
+    pthread_mutex_unlock(&p->mu);
+    if (len + 2 > n) { out[0] = '\0'; return -1; }
+    out[len++] = '}'; out[len] = '\0';
+    return (int)len;
+}
+
+static double mono_ts(void) {
+    struct timespec ts;
+    clock_gettime(CLOCK_MONOTONIC, &ts);
+    return ts.tv_sec + ts.tv_nsec * 1e-9;
+}
+
+/* Source address the kernel would use to reach target (an IP literal). A UDP connect() sends
+ * nothing. Returns 1 = route (address in out), 0 = no route, -1 = family unsupported or the
+ * target is not an IP literal (unknown, never absent). */
+static int sys_local_address(int af, const char *target, char *out, size_t n) {
+    int fd = socket(af, SOCK_DGRAM, 0);
+    if (fd < 0) return -1;
+    struct sockaddr_storage sa; socklen_t sl;
+    memset(&sa, 0, sizeof(sa));
+    if (af == AF_INET) {
+        struct sockaddr_in *s4 = (struct sockaddr_in *)&sa;
+        s4->sin_family = AF_INET; s4->sin_port = htons(9);
+        if (inet_pton(AF_INET, target, &s4->sin_addr) != 1) { close(fd); return -1; }
+        sl = sizeof(*s4);
+    } else {
+        struct sockaddr_in6 *s6 = (struct sockaddr_in6 *)&sa;
+        s6->sin6_family = AF_INET6; s6->sin6_port = htons(9);
+        if (inet_pton(AF_INET6, target, &s6->sin6_addr) != 1) { close(fd); return -1; }
+        sl = sizeof(*s6);
+    }
+    int rc = 0;
+    if (connect(fd, (struct sockaddr *)&sa, sl) == 0) {
+        struct sockaddr_storage me; socklen_t ml = sizeof(me);
+        if (getsockname(fd, (struct sockaddr *)&me, &ml) == 0) {
+            const void *src = (af == AF_INET)
+                ? (const void *)&((struct sockaddr_in *)&me)->sin_addr
+                : (const void *)&((struct sockaddr_in6 *)&me)->sin6_addr;
+            if (inet_ntop(af, src, out, (socklen_t)n)) rc = 1;
+        }
+    }
+    close(fd);
+    return rc;
+}
+
+static char *slurp(FILE *f) {
+    size_t cap = 4096, len = 0;
+    char *buf = malloc(cap);
+    if (!buf) return NULL;
+    size_t r;
+    while ((r = fread(buf + len, 1, cap - len - 1, f)) > 0) {
+        len += r;
+        if (len + 1 >= cap) {
+            if (cap >= 65536) break;
+            cap *= 2;
+            char *nb = realloc(buf, cap);
+            if (!nb) { free(buf); return NULL; }
+            buf = nb;
+        }
+    }
+    buf[len] = '\0';
+    return buf;
+}
+
+static bool g_have_timeout = false;   /* timeout(1) exists: bounds ping6 on the BSDs */
+
+/* One echo request: RTT in ms, -1.0 no reply, -2.0 ping could not be run. The target is an IP
+ * literal from the config; anything else is refused before it can reach the shell. */
+static double sys_ping(int af, const char *target) {
+    for (const char *c = target; *c; c++)
+        if (!(isxdigit((unsigned char)*c) || *c == '.' || *c == ':')) return -2.0;
+    char cmd[256];
+#ifdef __linux__
+    snprintf(cmd, sizeof(cmd), "ping -c 1 -W 2 -w 3 %s %s 2>&1", af == AF_INET ? "-4" : "-6", target);
+#else
+    /* BSD ping: -W is in milliseconds, and IPv6 is reached through ping6 */
+    if (af == AF_INET)
+        snprintf(cmd, sizeof(cmd), "%sping -c 1 -W 2000 %s 2>&1", g_have_timeout ? "timeout 4 " : "", target);
+    else
+        snprintf(cmd, sizeof(cmd), "%sping6 -c 1 %s 2>&1", g_have_timeout ? "timeout 4 " : "", target);
+#endif
+    FILE *f = popen(cmd, "r");
+    if (!f) return -2.0;
+    char *out = slurp(f);
+    pclose(f);
+    if (!out) return -1.0;
+    double rtt = ping_parse_rtt(out);
+    free(out);
+    return rtt;
+}
+
+/* Default gateway for the family, "" when unknown. Best effort; never affects presence. */
+static void sys_gateway(int af, char *out, size_t n) {
+    out[0] = '\0';
+#ifdef __linux__
+    FILE *f = fopen(af == AF_INET ? "/proc/net/route" : "/proc/net/ipv6_route", "r");
+    if (!f) return;
+    char *text = slurp(f);
+    fclose(f);
+    if (!text) return;
+    if (af == AF_INET) gw_parse_proc_v4(text, out, n); else gw_parse_proc_v6(text, out, n);
+    free(text);
+#else
+    FILE *f = popen(af == AF_INET ? "route -n get default 2>/dev/null"
+                                  : "route -n get -inet6 default 2>/dev/null", "r");
+    if (!f) return;
+    char *text = slurp(f);
+    pclose(f);
+    if (!text) return;
+    gw_parse_route_get(text, out, n);
+    free(text);
+#endif
+}
+
+static probe_t g_probe;
+static bool    g_probe_enabled = false;
+
+static void probe_init(probe_t *p, const config_t *cfg) {
+    memset(p, 0, sizeof(*p));
+    snprintf(p->target[0], sizeof(p->target[0]), "%s", cfg->probe_v4);
+    snprintf(p->target[1], sizeof(p->target[1]), "%s", cfg->probe_v6);
+    p->probe_interval = cfg->probe_interval > 0 ? cfg->probe_interval : 60;
+    p->retry_interval = 10;
+#ifdef __linux__
+    p->cheap_gateway = true;                 /* /proc read; elsewhere a subprocess only when due */
+#endif
+    p->fam[0].present = p->fam[1].present = PROBE_UNKNOWN;
+    p->fam[0].rtt = p->fam[1].rtt = -1;
+    pthread_mutex_init(&p->mu, NULL);
+    g_have_timeout = system("command -v timeout >/dev/null 2>&1") == 0;
+    g_probe_enabled = cfg->probe_enabled;
+
+    /* Diagnostic only: sys_local_address already turns a non-literal target into PROBE_UNKNOWN
+     * (via inet_pton failing there), so this does not change behavior — it just stops that from
+     * being silent. */
+    struct in_addr  a4;
+    struct in6_addr a6;
+    if (inet_pton(AF_INET, p->target[0], &a4) != 1)
+        LOGW("probe_v4 is not an IPv4 literal: %s", p->target[0]);
+    if (inet_pton(AF_INET6, p->target[1], &a6) != 1)
+        LOGW("probe_v6 is not an IPv6 literal: %s", p->target[1]);
+}
+
+static void *probe_thread(void *arg) {
+    probe_t *p = arg;
+    static const probe_ops_t ops = { sys_local_address, sys_ping, sys_gateway };
+    while (g_running) {
+        double now = mono_ts();
+        for (int f = 0; f < 2 && g_running; f++) {
+            pthread_mutex_lock(&p->mu); probe_fam_t tmp = p->fam[f]; pthread_mutex_unlock(&p->mu);
+            probe_step(p, &tmp, f, &ops, now);          /* may block in ping: no lock held */
+            pthread_mutex_lock(&p->mu); p->fam[f] = tmp; pthread_mutex_unlock(&p->mu);
+        }
+        for (int i = 0; i < PROBE_TIER1 && g_running; i++) sleep(1);
+    }
+    return NULL;
+}
+
 /* ============================================================
  * Plugin: network_monitor
  * Linux: /proc/net/dev
@@ -1077,16 +1430,26 @@ static void plugin_network_monitor(conn_t *c, const config_t *cfg) {
     json[jlen++] = '}'; json[jlen] = '\0';
     memcpy(g_net_prev, curr, cn * sizeof(net_stat_t));
     g_net_prev_n = cn; g_net_prev_ts = now;
-    if (first) { free(json); return; }
-
     kvdict_t d; kv_clear(&d);
     kv_set(&d, "plugin", "network_monitor");
-    char *jval = malloc(MAX_VAL + 1);
+    if (!first) {
-    if (jval) { snprintf(jval, MAX_VAL, "@%s", json); kv_set(&d, "interfaces", jval); free(jval); }
+        char *jval = malloc(MAX_VAL + 1);
+        if (jval) { snprintf(jval, MAX_VAL, "@%s", json); kv_set(&d, "interfaces", jval); free(jval); }
+    }
+    bool have_conn = false;
+    if (g_probe_enabled) {
+        char cj[512];
+        cj[0] = '@';
+        if (probe_metric_json(&g_probe, cj + 1, sizeof(cj) - 1) > 0) {
+            kv_set(&d, "connectivity", cj);
+            have_conn = true;
+        }
+    }
+    free(json);
+    if (first && !have_conn) return;
     kv_set_dbl(&d, "_timestamp", now_ts());
     kv_set_int(&d, "_interval", cfg->net_interval);
     conn_send(c, "PLG", &d);
-    free(json);
     LOGD("sent network_monitor");
 }
 
@@ -1323,6 +1686,7 @@ int main(int argc, char **argv) {
 
     config_t cfg;
     config_load(&cfg, cfgpath);
+    probe_init(&g_probe, &cfg);
 
     if (do_daemon) {
         daemonize();
@@ -1405,6 +1769,13 @@ int main(int argc, char **argv) {
     pthread_t tid;
     pthread_create(&tid, NULL, monitor_thread, &tctx);
 
+    pthread_t ptid; bool probe_running = false;
+    if (g_probe_enabled) {
+        int prc = pthread_create(&ptid, NULL, probe_thread, &g_probe);
+        if (prc == 0) probe_running = true;
+        else LOGW("pthread_create(probe): %s", strerror(prc));
+    }
+
     /* Main heartbeat + receive loop */
     time_t last_hb = 0;
     while (g_running) {
@@ -1415,6 +1786,7 @@ int main(int argc, char **argv) {
                 kv_set_int(&hb, "acks",     g_conns[i].ack_count);
                 kv_set_dbl(&hb, "rtt",      g_conns[i].rtt);
                 kv_set_int(&hb, "interval", cfg.interval);
+                if (g_probe_enabled) probe_flags(&g_probe, &hb);
                 conn_send(&g_conns[i], "HTB", &hb);
             }
             last_hb = now;
@@ -1449,6 +1821,10 @@ int main(int argc, char **argv) {
         conn_send(primary, "HTB", &sh);
         usleep(300000);
     }
+    /* Joined after the shutdown HTB goes out: the probe thread is never cancelled (it can be
+     * inside popen), so joining it can block for a few seconds — long enough to lose the
+     * clean-shutdown notification to an impatient supervisor's SIGKILL. */
+    if (probe_running) pthread_join(ptid, NULL);
     for (int i = 0; i < g_nconns; i++) conn_close(&g_conns[i]);
     if (g_use_syslog) closelog();
     if (g_restart) { LOGI("restarting..."); execv(argv[0], argv); }

scripts/c/test_probe.c

@@ -0,0 +1,243 @@
+/* Unit tests for the connectivity-probe helpers in hbc_mini.c.
+ * Build and run:  make test        (live check of this host:  ./test_probe --live)
+ * The source file is included directly so its static functions can be exercised; its main()
+ * is renamed out of the way.
+ */
+#define main hbc_mini_main
+#include "hbc_mini.c"
+#undef main
+
+static int g_run = 0, g_fail = 0;
+#define CHECK(cond) do { g_run++; if (!(cond)) { g_fail++; \
+    fprintf(stderr, "FAIL %s:%d: %s\n", __FILE__, __LINE__, #cond); } } while (0)
+#define CHECK_STR(a, b) do { g_run++; if (strcmp((a), (b)) != 0) { g_fail++; \
+    fprintf(stderr, "FAIL %s:%d: \"%s\" != \"%s\"\n", __FILE__, __LINE__, (a), (b)); } } while (0)
+
+static const char PROC_V4[] =
+    "Iface\tDestination\tGateway \tFlags\tRefCnt\tUse\tMetric\tMask\t\tMTU\tWindow\tIRTT\n"
+    "wlan0\t00000000\t0100A8C0\t0003\t0\t0\t600\t00000000\t0\t0\t0\n"
+    "eth0\t00000000\t0101A8C0\t0003\t0\t0\t100\t00000000\t0\t0\t0\n"
+    "wlan0\t0000A8C0\t00000000\t0001\t0\t0\t600\t00FFFFFF\t0\t0\t0\n";
+
+static const char PROC_V6[] =
+    "00000000000000000000000000000000 00 00000000000000000000000000000000 00 "
+    "fe800000000000000000000000000001 00000400 00000001 00000000 00000003 wlan0\n"
+    "00000000000000000000000000000000 00 00000000000000000000000000000000 00 "
+    "00000000000000000000000000000000 ffffffff 00000001 00000000 00200200 lo\n"
+    "fe800000000000000000000000000000 40 00000000000000000000000000000000 00 "
+    "00000000000000000000000000000000 00000100 00000001 00000000 00000001 wlan0\n";
+
+static const char ROUTE_GET[] =
+    "   route to: 0.0.0.0\ndestination: 0.0.0.0\n       mask: 0.0.0.0\n"
+    "    gateway: 192.0.2.1\n        fib: 0\n  interface: vtnet0\n";
+
+static const char LINUX_PING[] =
+    "PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.\n"
+    "64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=11.2 ms\n\n"
+    "--- 1.1.1.1 ping statistics ---\n"
+    "1 packets transmitted, 1 received, 0% packet loss, time 0ms\n"
+    "rtt min/avg/max/mdev = 11.234/11.234/11.234/0.000 ms\n";
+
+static const char FREEBSD_PING[] =
+    "PING 1.1.1.1 (1.1.1.1): 56 data bytes\n"
+    "64 bytes from 1.1.1.1: icmp_seq=0 ttl=59 time=4.006 ms\n\n"
+    "--- 1.1.1.1 ping statistics ---\n"
+    "1 packets transmitted, 1 packets received, 0.0% packet loss, 1 packets out of wait time\n"
+    "round-trip min/avg/max/stddev = 4.006/4.006/4.006/0.000 ms\n";
+
+static void test_gateway_parsers(void) {
+    char gw[64];
+    gw_parse_proc_v4(PROC_V4, gw, sizeof(gw));
+    CHECK_STR(gw, "192.168.1.1");                    /* eth0, metric 100 beats wlan0, 600 */
+    gw_parse_proc_v4("Iface\tDestination\n", gw, sizeof(gw));
+    CHECK_STR(gw, "");
+    gw_parse_proc_v6(PROC_V6, gw, sizeof(gw));
+    CHECK_STR(gw, "fe80::1");                        /* reject and non-default lines skipped */
+    gw_parse_route_get(ROUTE_GET, gw, sizeof(gw));
+    CHECK_STR(gw, "192.0.2.1");
+    gw_parse_route_get("route: writing to routing socket: not in table\n", gw, sizeof(gw));
+    CHECK_STR(gw, "");
+}
+
+static void test_ping_parser(void) {
+    CHECK(ping_parse_rtt(LINUX_PING) > 11.23 && ping_parse_rtt(LINUX_PING) < 11.24);
+    CHECK(ping_parse_rtt(FREEBSD_PING) > 4.0 && ping_parse_rtt(FREEBSD_PING) < 4.01);
+    CHECK(ping_parse_rtt("1 packets transmitted, 0 received, 100% packet loss\n") == -1.0);
+    CHECK(ping_parse_rtt("sh: 1: ping: not found\n") == -2.0);
+    CHECK(ping_parse_rtt("ping: command not found\n") == -2.0);
+}
+
+/* ---- scriptable network for probe_step ------------------------------------------------ */
+static struct {
+    char   addr[2][INET6_ADDRSTRLEN];   /* "" = no route */
+    char   gw[2][INET6_ADDRSTRLEN];
+    double rtt[2];                      /* -1 no reply, -2 ping unavailable */
+    int    unsupported[2];
+    int    npings[2];
+} net;
+
+static int fake_local(int af, const char *t, char *out, size_t n) {
+    int f = af == AF_INET6; (void)t;
+    if (net.unsupported[f]) return -1;
+    if (!net.addr[f][0]) return 0;
+    snprintf(out, n, "%s", net.addr[f]);
+    return 1;
+}
+static double fake_ping(int af, const char *t) { int f = af == AF_INET6; (void)t; net.npings[f]++; return net.rtt[f]; }
+static void fake_gw(int af, char *out, size_t n) { snprintf(out, n, "%s", net.gw[af == AF_INET6]); }
+static const probe_ops_t FAKE = { fake_local, fake_ping, fake_gw };
+
+static void net_reset(void) {
+    memset(&net, 0, sizeof(net));
+    snprintf(net.addr[0], INET6_ADDRSTRLEN, "192.0.2.5");
+    snprintf(net.addr[1], INET6_ADDRSTRLEN, "2001:db8::5");
+    snprintf(net.gw[0], INET6_ADDRSTRLEN, "192.0.2.1");
+    snprintf(net.gw[1], INET6_ADDRSTRLEN, "fe80::1");
+    net.rtt[0] = 10.0; net.rtt[1] = 20.0;
+}
+
+static void probe_reset(probe_t *p) {
+    memset(p, 0, sizeof(*p));
+    snprintf(p->target[0], sizeof(p->target[0]), "1.1.1.1");
+    snprintf(p->target[1], sizeof(p->target[1]), "2606:4700:4700::1111");
+    p->probe_interval = 60; p->retry_interval = 10; p->cheap_gateway = true;
+    p->fam[0].present = p->fam[1].present = PROBE_UNKNOWN;
+    p->fam[0].rtt = p->fam[1].rtt = -1;
+    pthread_mutex_init(&p->mu, NULL);
+}
+
+static void step_both(probe_t *p, double now) {
+    probe_step(p, &p->fam[0], 0, &FAKE, now);
+    probe_step(p, &p->fam[1], 1, &FAKE, now);
+}
+
+static void test_present_with_details(void) {
+    net_reset(); probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    CHECK(p.fam[0].present == 1 && p.fam[1].present == 1);
+    CHECK_STR(p.fam[0].addr, "192.0.2.5");
+    CHECK_STR(p.fam[0].gateway, "192.0.2.1");
+    CHECK(p.fam[0].rtt == 10.0);
+    kvdict_t d; kv_clear(&d);
+    probe_flags(&p, &d);
+    CHECK_STR(kv_get(&d, "p4") ? kv_get(&d, "p4") : "(null)", "1");
+    CHECK_STR(kv_get(&d, "p6") ? kv_get(&d, "p6") : "(null)", "1");
+    char js[512];
+    CHECK(probe_metric_json(&p, js, sizeof(js)) > 0);
+    CHECK(strstr(js, "\"ipv6\":{\"present\":true,\"addr\":\"2001:db8::5\",\"gateway\":\"fe80::1\"") != NULL);
+}
+
+static void test_no_route_is_absent_and_wastes_no_ping(void) {
+    net_reset(); net.addr[1][0] = '\0'; probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    CHECK(p.fam[1].present == 0 && net.npings[1] == 0 && net.npings[0] == 1);
+    kvdict_t d; kv_clear(&d);
+    probe_flags(&p, &d);
+    CHECK_STR(kv_get(&d, "p6") ? kv_get(&d, "p6") : "(null)", "0");
+    char js[512];
+    probe_metric_json(&p, js, sizeof(js));
+    CHECK(strstr(js, "\"ipv6\":{\"present\":false,\"addr\":null,\"gateway\":null,\"rtt\":null}") != NULL);
+}
+
+static void test_debounce_and_rate_limit(void) {
+    net_reset(); probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    step_both(&p, 1000);                                  /* cached: no second ping */
+    CHECK(net.npings[0] == 1 && net.npings[1] == 1);
+    net.rtt[1] = -1;
+    step_both(&p, 1060);                                  /* first failure: still present */
+    CHECK(p.fam[1].present == 1);
+    step_both(&p, 1070);                                  /* retry interval elapsed: flips */
+    CHECK(p.fam[1].present == 0);
+    net.rtt[1] = 25;
+    step_both(&p, 1080);                                  /* first success recovers */
+    CHECK(p.fam[1].present == 1);
+}
+
+static void test_first_failure_is_unknown_and_omitted(void) {
+    net_reset(); net.rtt[0] = -1; probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    CHECK(p.fam[0].present == PROBE_UNKNOWN);
+    kvdict_t d; kv_clear(&d);
+    probe_flags(&p, &d);
+    CHECK(kv_get(&d, "p4") == NULL && kv_get(&d, "p6") != NULL);
+    char js[512];
+    probe_metric_json(&p, js, sizeof(js));
+    CHECK(strstr(js, "\"ipv4\":{\"present\":null") != NULL);
+}
+
+static void test_address_or_gateway_change_repings(void) {
+    net_reset(); probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    snprintf(net.addr[0], INET6_ADDRSTRLEN, "198.51.100.7");
+    step_both(&p, 1000);
+    CHECK(net.npings[0] == 2);
+    snprintf(net.gw[0], INET6_ADDRSTRLEN, "192.0.2.254");
+    step_both(&p, 1000);
+    CHECK(net.npings[0] == 3);
+}
+
+static void test_route_loss_then_return_pings_immediately(void) {
+    net_reset(); probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    net.addr[1][0] = '\0';
+    step_both(&p, 1000);
+    CHECK(p.fam[1].present == 0);
+    snprintf(net.addr[1], INET6_ADDRSTRLEN, "2001:db8::5");
+    step_both(&p, 1000);
+    CHECK(net.npings[1] == 2 && p.fam[1].present == 1);
+}
+
+static void test_missing_ping_stays_unknown_even_after_route_loss(void) {
+    net_reset(); net.rtt[0] = -2; probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    step_both(&p, 1600);
+    CHECK(p.fam[0].present == PROBE_UNKNOWN && net.npings[0] == 1);
+    net.addr[0][0] = '\0';
+    step_both(&p, 1700);
+    CHECK(p.fam[0].present == 0);                         /* no route is definitive */
+    snprintf(net.addr[0], INET6_ADDRSTRLEN, "192.0.2.5");
+    step_both(&p, 1800);
+    CHECK(p.fam[0].present == PROBE_UNKNOWN && net.npings[0] == 1);
+}
+
+static void test_unsupported_family_is_unknown(void) {
+    net_reset(); net.unsupported[1] = 1; probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    CHECK(p.fam[1].present == PROBE_UNKNOWN && p.fam[0].present == 1);
+}
+
+static int live_mode(void) {
+    static const probe_ops_t sys_ops = { sys_local_address, sys_ping, sys_gateway };
+    probe_t p; probe_reset(&p);
+    p.cheap_gateway = false;
+    snprintf(p.target[0], sizeof(p.target[0]), "1.1.1.1");
+    for (int round = 0; round < 3; round++) {
+        double now = mono_ts();
+        for (int f = 0; f < 2; f++) probe_step(&p, &p.fam[f], f, &sys_ops, now);
+        sleep(round < 2 ? 11 : 0);
+    }
+    char js[512];
+    probe_metric_json(&p, js, sizeof(js));
+    kvdict_t d; kv_clear(&d);
+    probe_flags(&p, &d);
+    printf("%s\np4=%s p6=%s\n", js, kv_get(&d, "p4") ? kv_get(&d, "p4") : "-",
+           kv_get(&d, "p6") ? kv_get(&d, "p6") : "-");
+    return 0;
+}
+
+int main(int argc, char **argv) {
+    if (argc > 1 && !strcmp(argv[1], "--live")) return live_mode();
+    test_gateway_parsers();
+    test_ping_parser();
+    test_present_with_details();
+    test_no_route_is_absent_and_wastes_no_ping();
+    test_debounce_and_rate_limit();
+    test_first_failure_is_unknown_and_omitted();
+    test_address_or_gateway_change_repings();
+    test_route_loss_then_return_pings_immediately();
+    test_missing_ping_stays_unknown_even_after_route_loss();
+    test_unsupported_family_is_unknown();
+    printf("%d checks, %d failed\n", g_run, g_fail);
+    return g_fail ? 1 : 0;
+}

scripts/hbc_mini.py

@@ -10,14 +10,20 @@ Built-in plugins (always available):
   os_info         — OS name, kernel, distro, Python version
   ping_monitor    — ICMP ping RTT to configured hosts
   nagios_runner   — run Nagios-compatible check scripts
+  network_monitor — per-interface TX/RX bytes/s (Linux); per-protocol connectivity (all platforms)
 
 Linux-only plugins (via /proc, no extra tools needed):
   cpu_monitor     — CPU %, load average, core count
   memory_monitor  — RAM and swap usage
-  network_monitor — per-interface TX/RX bytes/s
 
 Cross-platform (via df subprocess):
   disk_monitor    — per-mount usage and capacity
+
+Presence probe (built in): reports per protocol (IPv4/IPv6) whether the host has internet
+connectivity, as p4/p6 flags on every heartbeat, so hbd stops treating a missing IPv6 (or
+IPv4) path as an outage. Config, in the network_monitor section of ~/.hbc.json:
+  probe_enabled (default true), probe_v4 (default 1.1.1.1), probe_v6 (default
+  2606:4700:4700::1111; IP literals only), probe_interval (default 60 s)
 """
 
 import argparse
@@ -36,12 +42,13 @@ import time
 import zlib
 from abc import ABC, abstractmethod
 from collections import defaultdict
+from dataclasses import asdict, dataclass
 from logging.handlers import SysLogHandler
 from pathlib import Path
-from typing import Any, Dict, List, Optional, Tuple
+from typing import Any, Awaitable, Callable, Dict, List, Optional, Set, Tuple
 
 # updated by scripts/bumpminor.sh
-__version__ = "5.4.1"
+__version__ = "5.4.5"
 
 # ---------------------------------------------------------------------------
 # Protocol  (mirrors hbd/common/proto.py)
@@ -637,6 +644,315 @@ class DiskMonitorPlugin(MonitorPlugin):
         return {"partitions": partitions} if partitions else {}
 
 
+# ---------------------------------------------------------------------------
+# Connectivity probe — per-protocol internet presence (IPv4 / IPv6)
+#
+# Tier 1 (every tier1_interval s, sends no packets): a UDP connect() to the probe address
+# shows whether a route exists and which source address would be used. Tier 2 (every
+# probe_interval s, sooner after a failure or a change of address or gateway): one ICMP echo,
+# only when tier 1 found a route. A family is absent after no route at all or fail_limit
+# consecutive failed echoes, present after one success, and unknown (flag omitted) when it
+# cannot be probed. The result rides on every HTB as p4/p6 and, when network_monitor runs, as
+# its `connectivity` metric. Mirrors hbd/client/connectivity.py.
+# ---------------------------------------------------------------------------
+
+_FAMILIES = ("ipv4", "ipv6")
+_FLAG_KEYS = {"ipv4": "p4", "ipv6": "p6"}
+_DEFAULT_PROBES = {"ipv4": "1.1.1.1", "ipv6": "2606:4700:4700::1111"}
+_BSD_PLATFORMS = ("darwin", "freebsd", "netbsd", "openbsd", "dragonfly")
+
+
+class _ProbeUnavailable(Exception):
+    """The address family cannot be probed on this host (e.g. no IPv6 support)."""
+
+
+class _PingUnavailable(Exception):
+    """The system ping binary could not be executed."""
+
+
+def _parse_proc_route_v4(text: str) -> Optional[str]:
+    """Default gateway from /proc/net/route (lowest metric wins)."""
+    best: Optional[Tuple[int, str]] = None
+    for line in text.splitlines()[1:]:
+        f = line.split()
+        if len(f) < 8 or f[1] != "00000000" or not int(f[3], 16) & 0x2:
+            continue
+        metric = int(f[6])
+        if best is None or metric < best[0]:
+            # the kernel prints the raw network-order u32 in host order: use the host byte order
+            best = (metric, socket.inet_ntoa(int(f[2], 16).to_bytes(4, sys.byteorder)))
+    return best[1] if best else None
+
+
+def _parse_proc_route_v6(text: str) -> Optional[str]:
+    """Default gateway from /proc/net/ipv6_route (lowest metric wins)."""
+    best: Optional[Tuple[int, str]] = None
+    for line in text.splitlines():
+        f = line.split()
+        if len(f) < 10:
+            continue
+        dest, plen, nexthop, metric, flags, iface = f[0], f[1], f[4], f[5], f[8], f[9]
+        if dest != "0" * 32 or plen != "00" or nexthop == "0" * 32 or iface == "lo":
+            continue
+        fl = int(flags, 16)
+        if not fl & 0x2 or fl & 0x200:            # need RTF_GATEWAY, skip RTF_REJECT
+            continue
+        m = int(metric, 16)
+        if best is None or m < best[0]:
+            best = (m, socket.inet_ntop(socket.AF_INET6, bytes.fromhex(nexthop)))
+    return best[1] if best else None
+
+
+def _parse_route_get(text: str) -> Optional[str]:
+    """Gateway from `route -n get [-inet6] default` (macOS and the BSDs)."""
+    m = re.search(r"^\s*gateway:\s*(\S+)", text, re.MULTILINE)
+    return m.group(1) if m else None
+
+
+def _parse_windows_route(text: str, family: str) -> Optional[str]:
+    """Default gateway from `route print -4` / `route print -6` (lowest metric wins)."""
+    best: Optional[Tuple[int, str]] = None
+    for line in text.splitlines():
+        if family == "ipv4":
+            f = line.split()
+            if not (len(f) >= 5 and f[0] == "0.0.0.0" and f[1] == "0.0.0.0" and f[2] != "On-link"):
+                continue
+            cand = (int(f[4]), f[2])
+        else:
+            m = re.match(r"^\s*\d+\s+(\d+)\s+::/0\s+(\S+)\s*$", line)
+            if not m or m.group(2) == "On-link":
+                continue
+            cand = (int(m.group(1)), m.group(2))
+        if best is None or cand[0] < best[0]:
+            best = cand
+    return best[1] if best else None
+
+
+async def _run_cmd(cmd: List[str], timeout: float = 5.0) -> str:
+    proc = await asyncio.create_subprocess_exec(
+        *cmd, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.DEVNULL)
+    try:
+        out, _ = await asyncio.wait_for(proc.communicate(), timeout=timeout)
+    except asyncio.TimeoutError:
+        proc.kill()
+        await proc.wait()
+        raise
+    return out.decode(errors="replace")
+
+
+async def _default_gateway(family: str) -> Optional[str]:
+    """Default gateway for "ipv4"/"ipv6", or None on any failure (informational only)."""
+    v4 = family == "ipv4"
+    try:
+        if sys.platform.startswith("linux"):
+            with open("/proc/net/route" if v4 else "/proc/net/ipv6_route") as fh:
+                text = fh.read()
+            return _parse_proc_route_v4(text) if v4 else _parse_proc_route_v6(text)
+        if sys.platform.startswith(_BSD_PLATFORMS):
+            cmd = ["route", "-n", "get"] + ([] if v4 else ["-inet6"]) + ["default"]
+            return _parse_route_get(await _run_cmd(cmd))
+        if sys.platform == "win32":
+            text = await _run_cmd(["route", "print", "-4" if v4 else "-6"])
+            return _parse_windows_route(text, family)
+    except Exception:
+        return None
+    return None
+
+
+def _probe_ping_command(host: str, family: str) -> List[str]:
+    """Command for ONE echo request with a ~2 s reply wait."""
+    v4 = family == "ipv4"
+    if sys.platform == "win32":
+        return ["ping", "-n", "1", "-w", "2000", "-4" if v4 else "-6", host]
+    if sys.platform.startswith(_BSD_PLATFORMS):
+        # BSD/macOS -W is in milliseconds, and IPv6 is reached through ping6.
+        return ["ping", "-c", "1", "-W", "2000", host] if v4 else ["ping6", "-c", "1", host]
+    return ["ping", "-c", "1", "-W", "2", "-4" if v4 else "-6", host]
+
+
+def _parse_ping_rtt(output: str) -> Optional[float]:
+    """Average RTT in ms from ping output, or None when there was no reply."""
+    m = re.search(r"(?:rtt|round-trip)\s+min/avg/max/\S+\s*=\s*([\d.]+)/([\d.]+)/([\d.]+)", output)
+    if m:
+        return float(m.group(2))
+    if "TTL=" in output:                           # Windows reply line (also when localized)
+        m = re.search(r"[=<]\s*(\d+)\s*ms", output)
+        return float(m.group(1)) if m else 0.0
+    return None
+
+
+async def _ping_rtt(target: str, family: str) -> Optional[float]:
+    """One ICMP echo; RTT in ms, or None if no reply. Raises _PingUnavailable without ping."""
+    try:
+        proc = await asyncio.create_subprocess_exec(
+            *_probe_ping_command(target, family),
+            stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE)
+    except FileNotFoundError as e:
+        raise _PingUnavailable(str(e))
+    try:
+        out, _ = await asyncio.wait_for(proc.communicate(), timeout=6)
+    except asyncio.TimeoutError:
+        proc.kill()
+        await proc.wait()
+        return None
+    return _parse_ping_rtt(out.decode(errors="replace"))
+
+
+def _local_address(family: str, target: str) -> Optional[str]:
+    """Source address the kernel would use to reach target, or None if there is no route.
+
+    A UDP connect() sends nothing. Raises _ProbeUnavailable if the family is unsupported.
+    """
+    af = socket.AF_INET if family == "ipv4" else socket.AF_INET6
+    try:
+        sock = socket.socket(af, socket.SOCK_DGRAM)
+    except OSError as e:
+        raise _ProbeUnavailable(str(e))
+    try:
+        sock.connect((target, 9))
+        return str(sock.getsockname()[0])
+    except OSError:
+        return None
+    finally:
+        sock.close()
+
+
+@dataclass
+class _FamilyState:
+    present: Optional[bool] = None      # None = unknown (probe cannot run yet/at all)
+    addr: Optional[str] = None
+    gateway: Optional[str] = None
+    rtt: Optional[float] = None
+
+
+class _Connectivity:
+    def __init__(
+        self,
+        probes: Optional[Dict[str, str]] = None,
+        probe_interval: float = 60.0,
+        retry_interval: float = 10.0,
+        tier1_interval: float = 5.0,
+        fail_limit: int = 2,
+        local_address_fn: Callable[[str, str], Optional[str]] = _local_address,
+        ping_fn: Callable[[str, str], Awaitable[Optional[float]]] = _ping_rtt,
+        gateway_fn: Callable[[str], Awaitable[Optional[str]]] = _default_gateway,
+        clock: Callable[[], float] = time.monotonic,
+    ):
+        self.probes = dict(_DEFAULT_PROBES)
+        self.probes.update(probes or {})
+        self.probe_interval = probe_interval
+        self.retry_interval = retry_interval
+        self.tier1_interval = tier1_interval
+        self.fail_limit = fail_limit
+        self._local_address = local_address_fn
+        self._ping = ping_fn
+        self._gateway_fn = gateway_fn
+        self._clock = clock
+        # Reading the gateway is a cheap /proc read on Linux but a subprocess elsewhere, so
+        # elsewhere it is refreshed only when a tier-2 probe is due.
+        self._cheap_gateway = sys.platform.startswith("linux")
+        self.states: Dict[str, _FamilyState] = {f: _FamilyState() for f in _FAMILIES}
+        self._fails: Dict[str, int] = {f: 0 for f in _FAMILIES}
+        self._next_ping: Dict[str, float] = {f: 0.0 for f in _FAMILIES}
+        self._last_key: Dict[str, Optional[Tuple[Optional[str], Optional[str]]]] = {
+            f: None for f in _FAMILIES}
+        self._ping_disabled: Set[str] = set()
+
+    @classmethod
+    def from_config(cls, cfg: Optional[Dict[str, Any]]) -> "_Connectivity":
+        cfg = cfg or {}
+        probes = {}
+        if cfg.get("probe_v4"):
+            probes["ipv4"] = str(cfg["probe_v4"])
+        if cfg.get("probe_v6"):
+            probes["ipv6"] = str(cfg["probe_v6"])
+        return cls(probes=probes, probe_interval=float(cfg.get("probe_interval", 60)))
+
+    # ---- readers (cheap; safe to call on every heartbeat) ---------------------------
+    def snapshot(self) -> Dict[str, _FamilyState]:
+        return {f: _FamilyState(**asdict(s)) for f, s in self.states.items()}
+
+    def flags(self) -> Dict[str, bool]:
+        """HTB flags for families whose presence is known."""
+        return {_FLAG_KEYS[f]: s.present for f, s in self.states.items() if s.present is not None}
+
+    def as_metric(self) -> Dict[str, Dict[str, Any]]:
+        return {f: asdict(s) for f, s in self.states.items()}
+
+    # ---- probing ----------------------------------------------------------------------
+    async def refresh(self) -> None:
+        for fam in _FAMILIES:
+            try:
+                await self._refresh_family(fam)
+            except Exception:
+                logging.getLogger("hbc.probe").exception("connectivity probe failed for %s", fam)
+
+    async def run(self) -> None:
+        while _running:
+            await self.refresh()
+            await _sleep(self.tier1_interval)
+
+    async def _gateway(self, fam: str) -> Optional[str]:
+        try:
+            return await self._gateway_fn(fam)
+        except Exception:
+            return None
+
+    async def _refresh_family(self, fam: str) -> None:
+        st = self.states[fam]
+        try:
+            addr = self._local_address(fam, self.probes[fam])
+        except _ProbeUnavailable:
+            st.present = None
+            return
+
+        if addr is None:                       # tier 1: no route => absent immediately
+            self._fails[fam] = 0
+            self._next_ping[fam] = 0.0
+            self._last_key[fam] = None
+            st.present, st.addr, st.gateway, st.rtt = False, None, None, None
+            return
+
+        now = self._clock()
+        gw = await self._gateway(fam) if self._cheap_gateway else st.gateway
+        key = (addr, gw)
+        due = key != self._last_key[fam] or now >= self._next_ping[fam]
+        if due and not self._cheap_gateway:
+            gw = await self._gateway(fam)
+            key = (addr, gw)
+        st.addr, st.gateway = addr, gw
+        self._last_key[fam] = key
+        if fam in self._ping_disabled:
+            st.present = None                  # cannot probe: unknown, never absent, while a route exists
+        elif due:
+            await self._tier2(fam, st, now)
+
+    async def _tier2(self, fam: str, st: _FamilyState, now: float) -> None:
+        try:
+            rtt = await self._ping(self.probes[fam], fam)
+        except _PingUnavailable:
+            logging.getLogger("hbc.probe").warning("ping unavailable; %s presence stays unknown", fam)
+            self._ping_disabled.add(fam)
+            st.present = None
+            return
+        except (asyncio.TimeoutError, OSError):
+            rtt = None
+        if rtt is None:
+            self._fails[fam] += 1
+            self._next_ping[fam] = now + self.retry_interval
+            if self._fails[fam] >= self.fail_limit:
+                st.present, st.rtt = False, None
+        else:
+            self._fails[fam] = 0
+            self._next_ping[fam] = now + self.probe_interval
+            st.present, st.rtt = True, rtt
+
+
+# The running probe (None when probe_enabled is false); shared by the heartbeat sender and
+# the network_monitor plugin. Set in _async_main.
+_probe: Optional[_Connectivity] = None
+
+
 # ---------------------------------------------------------------------------
 # Plugin: network_monitor  (Linux /proc/net/dev)
 # ---------------------------------------------------------------------------
@@ -658,7 +974,7 @@ def _read_net_dev() -> Dict[str, Tuple[int, int]]:
 
 class NetworkMonitorPlugin(MonitorPlugin):
     name = "network_monitor"
-    description = "Network I/O rates via /proc/net/dev (Linux)"
+    description = "Network I/O rates via /proc/net/dev (Linux) and per-protocol internet connectivity"
     interval = 300
 
     def __init__(self, config: Optional[Dict[str, Any]] = None):
@@ -667,19 +983,25 @@ class NetworkMonitorPlugin(MonitorPlugin):
         self.interval = cfg.get("interval", 300)
         self.skip_ifaces: set = set(cfg.get("skip_interfaces", ["lo"]))
         self._prev: Optional[Tuple[float, Dict[str, Tuple[int, int]]]] = None
+        self._have_dev = False
 
     async def initialize(self) -> bool:
-        if platform.system() != "Linux":
+        dev = _read_net_dev() if platform.system() == "Linux" else {}
-            self.skip_reason = "Linux only (/proc/net/dev not available)"
+        self._have_dev = bool(dev)
+        if not self._have_dev and _probe is None:
+            self.skip_reason = "no /proc/net/dev (Linux only) and the connectivity probe is disabled"
             return False
-        dev = _read_net_dev()
+        if self._have_dev:
-        if not dev:
+            self._prev = (time.time(), dev)  # prime first delta
-            self.skip_reason = "/proc/net/dev not readable"
-            return False
-        self._prev = (time.time(), dev)  # prime first delta
         return True
 
     async def _collect_metrics(self) -> Dict[str, Any]:
+        data: Dict[str, Any] = await self._collect_interfaces() if self._have_dev else {}
+        if _probe is not None:
+            data["connectivity"] = _probe.as_metric()
+        return data
+
+    async def _collect_interfaces(self) -> Dict[str, Any]:
         now = time.time()
         curr = _read_net_dev()
         if not curr or self._prev is None:
@@ -940,15 +1262,21 @@ async def _handle_update(conn: AsyncConnection):
 # Heartbeat sender
 # ---------------------------------------------------------------------------
 
-async def _heartbeat_sender(conn: AsyncConnection, interval: int):
+def _build_heartbeat_msg(conn: AsyncConnection, interval: int,
+                         probe: Optional[_Connectivity] = None) -> Dict[str, Any]:
+    """HTB payload; carries p4/p6 presence flags when the probe knows them."""
+    msg: Dict[str, Any] = {"acks": conn.ackcount, "rtt": conn.rtts[-1], "interval": interval}
+    if probe is not None:
+        msg.update(probe.flags())
+    return msg
+
+
+async def _heartbeat_sender(conn: AsyncConnection, interval: int,
+                            probe: Optional[_Connectivity] = None):
     log = logging.getLogger("hbc.hb")
     while _running:
         try:
-            await conn.sendto({
+            await conn.sendto(_build_heartbeat_msg(conn, interval, probe))
-                "acks": conn.ackcount,
-                "rtt": conn.rtts[-1],
-                "interval": interval,
-            })
         except Exception as e:
             log.error("send: %s", e)
         await _sleep(interval)
@@ -1056,7 +1384,7 @@ def _reconfigure_syslog(level: int):
 # ---------------------------------------------------------------------------
 
 async def _async_main(args, cfg: Dict[str, Any]) -> int:
-    global _running, _shutdown_event, _active_tasks, send_shutdown
+    global _running, _shutdown_event, _active_tasks, send_shutdown, _probe
     _running = True
     _shutdown_event = asyncio.Event()
     _active_tasks = []
@@ -1113,6 +1441,10 @@ async def _async_main(args, cfg: Dict[str, Any]) -> int:
                 c.close()
             return 0
 
+    # Per-protocol internet presence probe (shared with the network_monitor plugin)
+    net_cfg = dict((cfg.get("plugins") or {}).get("network_monitor") or cfg.get("network_monitor") or {})
+    _probe = _Connectivity.from_config(net_cfg) if net_cfg.get("probe_enabled", True) else None
+
     plugins = await _load_plugins(cfg)
 
     loop = asyncio.get_event_loop()
@@ -1126,8 +1458,10 @@ async def _async_main(args, cfg: Dict[str, Any]) -> int:
 
     loop.add_signal_handler(signal.SIGHUP, _sighup)
 
+    if _probe is not None:
+        _active_tasks.append(asyncio.create_task(_probe.run()))
     for conn in connections:
-        _active_tasks.append(asyncio.create_task(_heartbeat_sender(conn, interval)))
+        _active_tasks.append(asyncio.create_task(_heartbeat_sender(conn, interval, _probe)))
 
     if plugins and connections:
         _active_tasks.append(asyncio.create_task(_plugin_collector(connections[0], plugins)))

tests/test_client_heartbeat.py

@@ -0,0 +1,28 @@
+from hbd.client.main import build_heartbeat_msg
+
+
+class _Conn:
+    ackcount = 3
+    rtts = [0, 12.5]
+
+
+class _Conn2:
+    def __init__(self, flags):
+        self._flags = flags
+
+    def flags(self):
+        return self._flags
+
+
+def test_heartbeat_without_connectivity_is_unchanged():
+    assert build_heartbeat_msg(_Conn(), 60) == {"acks": 3, "rtt": 12.5, "interval": 60}
+
+
+def test_heartbeat_carries_presence_flags():
+    msg = build_heartbeat_msg(_Conn(), 60, _Conn2({"p4": True, "p6": False}))
+    assert msg == {"acks": 3, "rtt": 12.5, "interval": 60, "p4": True, "p6": False}
+
+
+def test_heartbeat_omits_unknown_flags():
+    msg = build_heartbeat_msg(_Conn(), 60, _Conn2({}))
+    assert "p4" not in msg and "p6" not in msg

tests/test_connectivity.py

@@ -0,0 +1,173 @@
+import asyncio
+
+from hbd.client.connectivity import Connectivity, ProbeUnavailable
+from hbd.client.pingutil import PingUnavailable
+
+
+class Net:
+    """Scriptable network: addr None = no route, 'unavailable' = family unsupported."""
+
+    def __init__(self):
+        self.addr = {"ipv4": "192.0.2.5", "ipv6": "2001:db8::5"}
+        self.gw = {"ipv4": "192.0.2.1", "ipv6": "fe80::1"}
+        self.rtt = {"ipv4": 10.0, "ipv6": 20.0}   # None = ping fails, 'unavailable' = no ping binary
+        self.pings = []
+        self.now = 1000.0
+
+    def local(self, fam, target):
+        a = self.addr[fam]
+        if a == "unavailable":
+            raise ProbeUnavailable("no such family")
+        return a
+
+    async def ping(self, target, fam):
+        self.pings.append(fam)
+        r = self.rtt[fam]
+        if r == "unavailable":
+            raise PingUnavailable("no ping")
+        return r
+
+    async def gateway(self, fam):
+        return self.gw[fam]
+
+    def clock(self):
+        return self.now
+
+
+def make(net, **kw):
+    c = Connectivity(local_address_fn=net.local, ping_fn=net.ping, gateway_fn=net.gateway,
+                     clock=net.clock, **kw)
+    c._cheap_gateway = True
+    return c
+
+
+def refresh(c):
+    asyncio.run(c.refresh())
+
+
+def test_present_with_addr_gateway_rtt():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    s = c.snapshot()["ipv4"]
+    assert (s.present, s.addr, s.gateway, s.rtt) == (True, "192.0.2.5", "192.0.2.1", 10.0)
+    assert c.flags() == {"p4": True, "p6": True}
+    assert c.as_metric()["ipv6"]["gateway"] == "fe80::1"
+
+
+def test_no_route_is_immediately_absent():
+    net = Net()
+    net.addr["ipv6"] = None
+    c = make(net)
+    refresh(c)
+    assert c.flags() == {"p4": True, "p6": False}
+    assert net.pings == ["ipv4"]          # no ping wasted on a family without a route
+    assert c.snapshot()["ipv6"].addr is None
+
+
+def test_ping_is_debounced_and_rate_limited():
+    net = Net()
+    c = make(net, probe_interval=60.0, retry_interval=10.0)
+    refresh(c)
+    refresh(c)                            # same instant: cached, no second ping
+    assert net.pings == ["ipv4", "ipv6"]
+    net.now += 60
+    net.rtt["ipv6"] = None                # first failure: still present (debounce)
+    refresh(c)
+    assert c.flags()["p6"] is True
+    net.now += 10                         # retry_interval elapsed: second failure flips it
+    refresh(c)
+    assert c.flags()["p6"] is False
+
+
+def test_first_probe_failing_once_is_unknown():
+    net = Net()
+    net.rtt["ipv4"] = None
+    c = make(net)
+    refresh(c)
+    assert "p4" not in c.flags() and c.flags()["p6"] is True
+
+
+def test_recovers_on_first_success():
+    net = Net()
+    net.rtt["ipv6"] = None
+    c = make(net, retry_interval=10.0)
+    refresh(c)
+    net.now += 10
+    refresh(c)
+    assert c.flags()["p6"] is False
+    net.rtt["ipv6"] = 25.0
+    net.now += 10
+    refresh(c)
+    assert c.flags()["p6"] is True
+
+
+def test_address_change_invalidates_cache():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    net.addr["ipv4"] = "198.51.100.7"     # moved networks, no time has passed
+    refresh(c)
+    assert net.pings.count("ipv4") == 2
+    assert c.snapshot()["ipv4"].addr == "198.51.100.7"
+
+
+def test_gateway_change_invalidates_cache():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    net.gw["ipv4"] = "192.0.2.254"
+    refresh(c)
+    assert net.pings.count("ipv4") == 2
+
+
+def test_route_loss_then_return_pings_immediately():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    net.addr["ipv6"] = None
+    refresh(c)
+    net.addr["ipv6"] = "2001:db8::5"
+    refresh(c)
+    assert net.pings.count("ipv6") == 2 and c.flags()["p6"] is True
+
+
+def test_missing_ping_binary_leaves_presence_unknown_and_stops_retrying():
+    net = Net()
+    net.rtt["ipv4"] = "unavailable"
+    c = make(net)
+    refresh(c)
+    net.now += 600
+    refresh(c)
+    assert "p4" not in c.flags()
+    assert net.pings.count("ipv4") == 1
+
+
+def test_unsupported_family_is_unknown():
+    net = Net()
+    net.addr["ipv6"] = "unavailable"
+    c = make(net)
+    refresh(c)
+    assert "p6" not in c.flags() and c.flags()["p4"] is True
+
+
+def test_from_config():
+    c = Connectivity.from_config({"probe_v4": "9.9.9.9", "probe_interval": 30})
+    assert c.probes["ipv4"] == "9.9.9.9"
+    assert c.probes["ipv6"] == "2606:4700:4700::1111"
+    assert c.probe_interval == 30.0
+
+
+def test_ping_less_family_stays_unknown_after_route_returns():
+    net = Net()
+    net.rtt["ipv4"] = "unavailable"
+    c = make(net)
+    refresh(c)
+    assert "p4" not in c.flags()
+    net.addr["ipv4"] = None
+    refresh(c)
+    assert c.flags()["p4"] is False       # no route is definitive
+    net.addr["ipv4"] = "192.0.2.5"
+    refresh(c)
+    assert "p4" not in c.flags()
+    assert net.pings.count("ipv4") == 1   # missing binary is not retried

tests/test_gateway.py

@@ -0,0 +1,75 @@
+import asyncio
+import struct
+
+from hbd.client import gateway
+
+PROC_V4 = (
+    "Iface\tDestination\tGateway \tFlags\tRefCnt\tUse\tMetric\tMask\t\tMTU\tWindow\tIRTT\n"
+    "wlan0\t00000000\t0100A8C0\t0003\t0\t0\t600\t00000000\t0\t0\t0\n"
+    "eth0\t00000000\t0101A8C0\t0003\t0\t0\t100\t00000000\t0\t0\t0\n"
+    "wlan0\t0000A8C0\t00000000\t0001\t0\t0\t600\t00FFFFFF\t0\t0\t0\n"
+)
+
+PROC_V6 = (
+    "00000000000000000000000000000000 00 00000000000000000000000000000000 00 "
+    "fe800000000000000000000000000001 00000400 00000001 00000000 00000003 wlan0\n"
+    "00000000000000000000000000000000 00 00000000000000000000000000000000 00 "
+    "00000000000000000000000000000000 ffffffff 00000001 00000000 00200200 lo\n"
+    "fe800000000000000000000000000000 40 00000000000000000000000000000000 00 "
+    "00000000000000000000000000000000 00000100 00000001 00000000 00000001 wlan0\n"
+)
+
+MACOS = """   route to: default
+destination: default
+       mask: default
+    gateway: 192.168.1.1
+  interface: en0
+"""
+
+WIN_V4 = """IPv4 Route Table
+Active Routes:
+Network Destination        Netmask          Gateway       Interface  Metric
+          0.0.0.0          0.0.0.0      192.168.1.1    192.168.1.50     25
+          0.0.0.0          0.0.0.0        10.0.0.1        10.0.0.9     50
+        127.0.0.0        255.0.0.0         On-link         127.0.0.1    331
+"""
+
+WIN_V6 = """IPv6 Route Table
+Active Routes:
+ If Metric Network Destination      Gateway
+ 12    281 ::/0                     fe80::1
+  1    331 ::1/128                  On-link
+"""
+
+
+def test_proc_v4_picks_lowest_metric_gateway():
+    assert gateway.parse_proc_route_v4(PROC_V4) == "192.168.1.1"  # eth0, metric 100
+
+
+def test_proc_v4_none_without_default_route():
+    assert gateway.parse_proc_route_v4(PROC_V4.splitlines()[0] + "\n") is None
+
+
+def test_proc_v6_skips_reject_and_non_default():
+    assert gateway.parse_proc_route_v6(PROC_V6) == "fe80::1"
+
+
+def test_macos():
+    assert gateway.parse_macos_route(MACOS) == "192.168.1.1"
+    assert gateway.parse_macos_route("route: writing to routing socket: not in table") is None
+
+
+def test_windows_v4_lowest_metric():
+    assert gateway.parse_windows_route(WIN_V4, "ipv4") == "192.168.1.1"
+
+
+def test_windows_v6():
+    assert gateway.parse_windows_route(WIN_V6, "ipv6") == "fe80::1"
+
+
+def test_default_gateway_never_raises_on_parser_error(monkeypatch):
+    """default_gateway catches all exceptions; never raises even if parser fails."""
+    monkeypatch.setattr("sys.platform", "linux")
+    monkeypatch.setattr(gateway, "parse_proc_route_v4", lambda _: (_ for _ in ()).throw(struct.error))
+    result = asyncio.run(gateway.default_gateway("ipv4"))
+    assert result is None

tests/test_hbc_mini_probe.py

@@ -0,0 +1,381 @@
+"""Connectivity probe in the standalone scripts/hbc_mini.py (stdlib-only single-file client)."""
+import asyncio
+import importlib.util
+import pathlib
+import sys
+
+import pytest
+
+_PATH = pathlib.Path(__file__).resolve().parents[1] / "scripts" / "hbc_mini.py"
+_SPEC = importlib.util.spec_from_file_location("hbc_mini", _PATH)
+mini = importlib.util.module_from_spec(_SPEC)
+sys.modules["hbc_mini"] = mini          # dataclasses resolve their module by name
+_SPEC.loader.exec_module(mini)
+
+PROC_V4 = (
+    "Iface\tDestination\tGateway \tFlags\tRefCnt\tUse\tMetric\tMask\t\tMTU\tWindow\tIRTT\n"
+    "wlan0\t00000000\t0100A8C0\t0003\t0\t0\t600\t00000000\t0\t0\t0\n"
+    "eth0\t00000000\t0101A8C0\t0003\t0\t0\t100\t00000000\t0\t0\t0\n"
+    "wlan0\t0000A8C0\t00000000\t0001\t0\t0\t600\t00FFFFFF\t0\t0\t0\n"
+)
+PROC_V6 = (
+    "00000000000000000000000000000000 00 00000000000000000000000000000000 00 "
+    "fe800000000000000000000000000001 00000400 00000001 00000000 00000003 wlan0\n"
+    "00000000000000000000000000000000 00 00000000000000000000000000000000 00 "
+    "00000000000000000000000000000000 ffffffff 00000001 00000000 00200200 lo\n"
+    "fe800000000000000000000000000000 40 00000000000000000000000000000000 00 "
+    "00000000000000000000000000000000 00000100 00000001 00000000 00000001 wlan0\n"
+)
+ROUTE_GET = """   route to: 0.0.0.0
+destination: 0.0.0.0
+       mask: 0.0.0.0
+    gateway: 192.0.2.1
+        fib: 0
+  interface: vtnet0
+"""
+WIN_V4 = """IPv4 Route Table
+Active Routes:
+Network Destination        Netmask          Gateway       Interface  Metric
+          0.0.0.0          0.0.0.0      192.168.1.1    192.168.1.50     25
+          0.0.0.0          0.0.0.0        10.0.0.1        10.0.0.9     50
+        127.0.0.0        255.0.0.0         On-link         127.0.0.1    331
+"""
+WIN_V6 = """IPv6 Route Table
+Active Routes:
+ If Metric Network Destination      Gateway
+ 12    281 ::/0                     fe80::1
+  1    331 ::1/128                  On-link
+"""
+LINUX_PING = """PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
+64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=11.2 ms
+
+--- 1.1.1.1 ping statistics ---
+1 packets transmitted, 1 received, 0% packet loss, time 0ms
+rtt min/avg/max/mdev = 11.234/11.234/11.234/0.000 ms
+"""
+FREEBSD_PING = """PING 1.1.1.1 (1.1.1.1): 56 data bytes
+64 bytes from 1.1.1.1: icmp_seq=0 ttl=59 time=4.006 ms
+
+--- 1.1.1.1 ping statistics ---
+1 packets transmitted, 1 packets received, 0.0% packet loss, 1 packets out of wait time
+round-trip min/avg/max/stddev = 4.006/4.006/4.006/0.000 ms
+"""
+WIN_PING = """Pinging 1.1.1.1 with 32 bytes of data:
+Reply from 1.1.1.1: bytes=32 time=12ms TTL=57
+
+Ping statistics for 1.1.1.1:
+    Packets: Sent = 1, Received = 1, Lost = 0 (0% loss),
+"""
+LOST_PING = """--- 1.1.1.1 ping statistics ---
+1 packets transmitted, 0 received, 100% packet loss, time 0ms
+"""
+
+
+# ---- gateway parsers ---------------------------------------------------------------
+
+def test_proc_v4_lowest_metric_wins():
+    assert mini._parse_proc_route_v4(PROC_V4) == "192.168.1.1"      # eth0, metric 100
+
+
+def test_proc_v4_none_without_default_route():
+    assert mini._parse_proc_route_v4(PROC_V4.splitlines()[0] + "\n") is None
+
+
+def test_proc_v6_skips_reject_and_non_default():
+    assert mini._parse_proc_route_v6(PROC_V6) == "fe80::1"
+
+
+def test_route_get_bsd_and_macos():
+    assert mini._parse_route_get(ROUTE_GET) == "192.0.2.1"
+    assert mini._parse_route_get("route: writing to routing socket: not in table") is None
+
+
+def test_windows_routes():
+    assert mini._parse_windows_route(WIN_V4, "ipv4") == "192.168.1.1"
+    assert mini._parse_windows_route(WIN_V6, "ipv6") == "fe80::1"
+
+
+def test_default_gateway_never_raises(monkeypatch):
+    monkeypatch.setattr(sys, "platform", "linux")
+    monkeypatch.setattr(mini, "_parse_proc_route_v4", lambda t: 1 / 0)
+    assert asyncio.run(mini._default_gateway("ipv4")) is None
+
+
+# ---- ping ---------------------------------------------------------------------------
+
+@pytest.mark.parametrize("platform,family,expected", [
+    ("linux", "ipv4", ["ping", "-c", "1", "-W", "2", "-4", "h"]),
+    ("linux", "ipv6", ["ping", "-c", "1", "-W", "2", "-6", "h"]),
+    ("freebsd15", "ipv4", ["ping", "-c", "1", "-W", "2000", "h"]),
+    ("freebsd15", "ipv6", ["ping6", "-c", "1", "h"]),
+    ("darwin", "ipv4", ["ping", "-c", "1", "-W", "2000", "h"]),
+    ("darwin", "ipv6", ["ping6", "-c", "1", "h"]),
+    ("win32", "ipv4", ["ping", "-n", "1", "-w", "2000", "-4", "h"]),
+    ("win32", "ipv6", ["ping", "-n", "1", "-w", "2000", "-6", "h"]),
+])
+def test_probe_ping_command(monkeypatch, platform, family, expected):
+    monkeypatch.setattr(sys, "platform", platform)
+    assert mini._probe_ping_command("h", family) == expected
+
+
+def test_parse_ping_rtt():
+    assert mini._parse_ping_rtt(LINUX_PING) == 11.234
+    assert mini._parse_ping_rtt(FREEBSD_PING) == 4.006
+    assert mini._parse_ping_rtt(WIN_PING) == 12.0
+    assert mini._parse_ping_rtt(LOST_PING) is None
+    assert mini._parse_ping_rtt("ping: connect: Network is unreachable") is None
+
+
+# ---- probe state machine ------------------------------------------------------------
+
+class Net:
+    """Scriptable network: addr None = no route, 'unavailable' = family unsupported."""
+
+    def __init__(self):
+        self.addr = {"ipv4": "192.0.2.5", "ipv6": "2001:db8::5"}
+        self.gw = {"ipv4": "192.0.2.1", "ipv6": "fe80::1"}
+        self.rtt = {"ipv4": 10.0, "ipv6": 20.0}   # None = ping fails, 'unavailable' = no ping binary
+        self.pings = []
+        self.now = 1000.0
+
+    def local(self, fam, target):
+        a = self.addr[fam]
+        if a == "unavailable":
+            raise mini._ProbeUnavailable("no such family")
+        return a
+
+    async def ping(self, target, fam):
+        self.pings.append(fam)
+        r = self.rtt[fam]
+        if r == "unavailable":
+            raise mini._PingUnavailable("no ping")
+        return r
+
+    async def gateway(self, fam):
+        return self.gw[fam]
+
+    def clock(self):
+        return self.now
+
+
+def make(net, **kw):
+    c = mini._Connectivity(local_address_fn=net.local, ping_fn=net.ping, gateway_fn=net.gateway,
+                           clock=net.clock, **kw)
+    c._cheap_gateway = True
+    return c
+
+
+def refresh(c):
+    asyncio.run(c.refresh())
+
+
+def test_present_with_addr_gateway_rtt():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    s = c.snapshot()["ipv4"]
+    assert (s.present, s.addr, s.gateway, s.rtt) == (True, "192.0.2.5", "192.0.2.1", 10.0)
+    assert c.flags() == {"p4": True, "p6": True}
+    assert c.as_metric()["ipv6"]["gateway"] == "fe80::1"
+
+
+def test_no_route_is_immediately_absent():
+    net = Net()
+    net.addr["ipv6"] = None
+    c = make(net)
+    refresh(c)
+    assert c.flags() == {"p4": True, "p6": False}
+    assert net.pings == ["ipv4"]
+    assert c.snapshot()["ipv6"].addr is None
+
+
+def test_ping_is_debounced_and_rate_limited():
+    net = Net()
+    c = make(net, probe_interval=60.0, retry_interval=10.0)
+    refresh(c)
+    refresh(c)
+    assert net.pings == ["ipv4", "ipv6"]
+    net.now += 60
+    net.rtt["ipv6"] = None
+    refresh(c)
+    assert c.flags()["p6"] is True
+    net.now += 10
+    refresh(c)
+    assert c.flags()["p6"] is False
+
+
+def test_first_probe_failing_once_is_unknown():
+    net = Net()
+    net.rtt["ipv4"] = None
+    c = make(net)
+    refresh(c)
+    assert "p4" not in c.flags() and c.flags()["p6"] is True
+
+
+def test_recovers_on_first_success():
+    net = Net()
+    net.rtt["ipv6"] = None
+    c = make(net, retry_interval=10.0)
+    refresh(c)
+    net.now += 10
+    refresh(c)
+    assert c.flags()["p6"] is False
+    net.rtt["ipv6"] = 25.0
+    net.now += 10
+    refresh(c)
+    assert c.flags()["p6"] is True
+
+
+def test_address_change_invalidates_cache():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    net.addr["ipv4"] = "198.51.100.7"
+    refresh(c)
+    assert net.pings.count("ipv4") == 2
+    assert c.snapshot()["ipv4"].addr == "198.51.100.7"
+
+
+def test_gateway_change_invalidates_cache():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    net.gw["ipv4"] = "192.0.2.254"
+    refresh(c)
+    assert net.pings.count("ipv4") == 2
+
+
+def test_route_loss_then_return_pings_immediately():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    net.addr["ipv6"] = None
+    refresh(c)
+    net.addr["ipv6"] = "2001:db8::5"
+    refresh(c)
+    assert net.pings.count("ipv6") == 2 and c.flags()["p6"] is True
+
+
+def test_missing_ping_binary_leaves_presence_unknown_and_stops_retrying():
+    net = Net()
+    net.rtt["ipv4"] = "unavailable"
+    c = make(net)
+    refresh(c)
+    net.now += 600
+    refresh(c)
+    assert "p4" not in c.flags()
+    assert net.pings.count("ipv4") == 1
+
+
+def test_ping_less_family_stays_unknown_after_route_returns():
+    net = Net()
+    net.rtt["ipv4"] = "unavailable"
+    c = make(net)
+    refresh(c)
+    assert "p4" not in c.flags()
+    net.addr["ipv4"] = None
+    refresh(c)
+    assert c.flags()["p4"] is False              # no route is definitive
+    net.addr["ipv4"] = "192.0.2.5"
+    refresh(c)
+    assert "p4" not in c.flags() and net.pings.count("ipv4") == 1
+
+
+def test_unsupported_family_is_unknown():
+    net = Net()
+    net.addr["ipv6"] = "unavailable"
+    c = make(net)
+    refresh(c)
+    assert "p6" not in c.flags() and c.flags()["p4"] is True
+
+
+def test_from_config():
+    c = mini._Connectivity.from_config({"probe_v4": "9.9.9.9", "probe_interval": 30})
+    assert c.probes["ipv4"] == "9.9.9.9"
+    assert c.probes["ipv6"] == "2606:4700:4700::1111"
+    assert c.probe_interval == 30.0
+
+
+def test_local_address_on_this_host_never_raises():
+    # Real UDP connect(): either an address string, None (no route) or _ProbeUnavailable.
+    for fam, target in (("ipv4", "1.1.1.1"), ("ipv6", "2606:4700:4700::1111")):
+        try:
+            r = mini._local_address(fam, target)
+        except mini._ProbeUnavailable:
+            continue
+        assert r is None or isinstance(r, str)
+
+
+# ---- heartbeat message and plugin wiring ---------------------------------------------
+
+class _Conn:
+    ackcount = 3
+    rtts = [0.0, 12.5]
+
+
+class _FlagsProbe:
+    def __init__(self, flags):
+        self._flags = flags
+
+    def flags(self):
+        return self._flags
+
+
+def test_heartbeat_without_probe_is_unchanged():
+    assert mini._build_heartbeat_msg(_Conn(), 60) == {"acks": 3, "rtt": 12.5, "interval": 60}
+
+
+def test_heartbeat_carries_presence_flags():
+    msg = mini._build_heartbeat_msg(_Conn(), 60, _FlagsProbe({"p4": True, "p6": False}))
+    assert msg == {"acks": 3, "rtt": 12.5, "interval": 60, "p4": True, "p6": False}
+
+
+def test_heartbeat_omits_unknown_flags():
+    msg = mini._build_heartbeat_msg(_Conn(), 60, _FlagsProbe({}))
+    assert "p4" not in msg and "p6" not in msg
+
+
+def test_flags_survive_the_wire_encoding():
+    msg = mini._build_heartbeat_msg(_Conn(), 60, _FlagsProbe({"p4": True, "p6": False}))
+    out = mini._stodict(mini._dicttos("HTB", dict(msg, name="h", id=1)))
+    assert out["p4"] == 1 and out["p6"] == 0
+
+
+class _FakeProbe:
+    def as_metric(self):
+        return {"ipv4": {"present": True, "addr": "192.0.2.5", "gateway": "192.0.2.1", "rtt": 9.0},
+                "ipv6": {"present": False, "addr": None, "gateway": None, "rtt": None}}
+
+
+def test_plugin_reports_connectivity_alongside_interfaces(monkeypatch):
+    monkeypatch.setattr(mini, "_probe", _FakeProbe())
+    monkeypatch.setattr(mini.platform, "system", lambda: "Linux")
+    plugin = mini.NetworkMonitorPlugin({})
+    assert asyncio.run(plugin.initialize()) is True
+    data = asyncio.run(plugin._collect_metrics())
+    assert data["connectivity"]["ipv4"]["present"] is True
+    assert data["connectivity"]["ipv6"]["present"] is False
+
+
+def test_plugin_probe_only_mode_on_non_linux(monkeypatch):
+    monkeypatch.setattr(mini, "_probe", _FakeProbe())
+    monkeypatch.setattr(mini.platform, "system", lambda: "FreeBSD")
+    plugin = mini.NetworkMonitorPlugin({})
+    assert asyncio.run(plugin.initialize()) is True
+    assert list(asyncio.run(plugin._collect_metrics())) == ["connectivity"]
+
+
+def test_plugin_still_skips_on_non_linux_without_probe(monkeypatch):
+    monkeypatch.setattr(mini, "_probe", None)
+    monkeypatch.setattr(mini.platform, "system", lambda: "FreeBSD")
+    plugin = mini.NetworkMonitorPlugin({})
+    assert asyncio.run(plugin.initialize()) is False
+    assert plugin.skip_reason
+
+
+def test_plugin_omits_connectivity_without_probe(monkeypatch):
+    monkeypatch.setattr(mini, "_probe", None)
+    monkeypatch.setattr(mini.platform, "system", lambda: "Linux")
+    plugin = mini.NetworkMonitorPlugin({})
+    assert asyncio.run(plugin.initialize()) is True
+    assert "connectivity" not in asyncio.run(plugin._collect_metrics())

tests/test_host_status.py

@@ -0,0 +1,120 @@
+import time
+
+import pytest
+
+from hbd.server import hbdclass
+from hbd.server.hbdclass import Connection as C
+
+
+def _host(name):
+    hbdclass.Host.hosts.pop(name, None)
+    return hbdclass.Host(name)
+
+
+def _conn(host, afam, state):
+    addr = "192.0.2.1" if afam == "IPv4" else "2001:db8::1"
+    conn, _ = host.conndata(0, addr, None, 1000.0)
+    conn.state = state
+    return conn
+
+
+CASES = [
+    # v4 state, v6 state, presence flags, expected status
+    (C.UP, C.UP, {}, "up"),
+    (C.UP, C.OVERDUE, {}, "simplex"),                       # both present (legacy) => simplex
+    (C.UP, C.OVERDUE, {"IPv6": True}, "simplex"),
+    (C.UP, C.OVERDUE, {"IPv6": False}, "up"),               # v6 not present => not monitored
+    (C.OVERDUE, C.OVERDUE, {}, "overdue"),
+    (C.OVERDUE, C.ABSENT, {"IPv6": False}, "overdue"),      # only present family is overdue
+    (C.UP, C.ABSENT, {"IPv6": False}, "up"),
+    (C.DOWN, C.UP, {}, "down"),
+    (C.UNKNOWN, C.UNKNOWN, {}, "unknown"),
+    (C.OVERDUE, C.UNKNOWN, {}, "overdue"),
+    (C.UP, C.UNKNOWN, {}, "up"),
+    (C.OVERDUE, C.UP, {"IPv6": False}, "simplex"),          # HTB arriving => present despite flag
+]
+
+
+@pytest.mark.parametrize("s4,s6,flags,expected", CASES)
+def test_status_matrix(s4, s6, flags, expected):
+    h = _host("status-matrix")
+    _conn(h, "IPv4", s4)
+    _conn(h, "IPv6", s6)
+    for afam, present in flags.items():
+        h.set_presence(afam, present, time.time())
+    assert h.status() == expected
+
+
+def test_status_single_family_host():
+    h = _host("status-v4only")
+    _conn(h, "IPv4", C.UP)
+    assert h.status() == "up"
+    h.connections["IPv4"].state = C.OVERDUE
+    assert h.status() == "overdue"
+
+
+def test_status_no_connections_is_unknown():
+    assert _host("status-empty").status() == "unknown"
+
+
+def test_family_present_rules():
+    h = _host("fp")
+    v6 = _conn(h, "IPv6", C.OVERDUE)
+    assert h.family_present("IPv6") is True            # never reported => present
+    h.set_presence("IPv6", False, 1.0)
+    assert h.family_present("IPv6") is False
+    v6.state = C.UP                                    # real traffic overrides the flag
+    assert h.family_present("IPv6") is True
+    assert h.flag_present("IPv6") is False
+
+
+def test_stateinfo_and_statedict_expose_status():
+    h = _host("status-exposed")
+    _conn(h, "IPv4", C.UP)
+    assert h.stateinfo()["status"] == "up"
+    assert h.statedict()["status"] == "up"
+
+
+def test_absent_connection_statedict_is_neutral():
+    h = _host("absent-dict")
+    conn = _conn(h, "IPv6", C.ABSENT)
+    conn.rtts.append(42.0)
+    d = conn.statedict()
+    assert d["state"] == "absent" and d["rtt"] == "" and d["rttstate"] == "absent"
+
+
+def test_presence_survives_pickle_and_old_pickles_get_default():
+    import pickle
+    h = _host("pickled")
+    h.set_presence("IPv6", False, 5.0)
+    h2 = pickle.loads(pickle.dumps(h))
+    assert h2.presence == {"IPv6": {"present": False, "ts": 5.0}}
+    del h2.__dict__["presence"]                        # simulate a pre-feature pickle
+    h2.fixup()
+    assert h2.presence == {}
+
+
+def _all_absent(name, s4, s6=None):
+    h = _host(name)
+    _conn(h, "IPv4", s4)
+    h.set_presence("IPv4", False, time.time())
+    if s6 is not None:
+        _conn(h, "IPv6", s6)
+        h.set_presence("IPv6", False, time.time())
+    return h
+
+
+def test_all_absent_single_family_overdue_is_overdue():
+    assert _all_absent("abs-single", C.OVERDUE).status() == "overdue"
+
+
+def test_all_absent_dual_stack_one_absent_one_overdue_is_overdue():
+    assert _all_absent("abs-dual-od", C.ABSENT, C.OVERDUE).status() == "overdue"
+
+
+def test_all_absent_dual_stack_v6_up_is_up():
+    assert _all_absent("abs-dual-up", C.ABSENT, C.UP).status() == "up"
+
+
+def test_all_absent_all_unknown_is_unknown():
+    assert _all_absent("abs-unknown", C.UNKNOWN, C.UNKNOWN).status() == "unknown"

tests/test_network_monitor_connectivity.py

@@ -0,0 +1,26 @@
+import asyncio
+
+from hbd.client import connectivity
+from hbd.client.plugins.network_monitor import NetworkMonitorPlugin
+
+
+class _FakeShared:
+    def as_metric(self):
+        return {"ipv4": {"present": True, "addr": "192.0.2.5", "gateway": "192.0.2.1", "rtt": 9.0},
+                "ipv6": {"present": False, "addr": None, "gateway": None, "rtt": None}}
+
+
+def test_collect_includes_connectivity_when_probe_running():
+    connectivity.set_shared(_FakeShared())
+    try:
+        data = asyncio.run(NetworkMonitorPlugin()._collect_metrics())
+    finally:
+        connectivity.set_shared(None)
+    assert data["connectivity"]["ipv4"]["present"] is True
+    assert data["connectivity"]["ipv6"]["present"] is False
+
+
+def test_collect_omits_connectivity_without_probe():
+    connectivity.set_shared(None)
+    data = asyncio.run(NetworkMonitorPlugin()._collect_metrics())
+    assert "connectivity" not in data

tests/test_pingutil.py

@@ -0,0 +1,60 @@
+import sys
+
+import pytest
+
+from hbd.client import pingutil
+
+LINUX_OK = """PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
+64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=11.2 ms
+
+--- 1.1.1.1 ping statistics ---
+1 packets transmitted, 1 received, 0% packet loss, time 0ms
+rtt min/avg/max/mdev = 11.234/11.234/11.234/0.000 ms
+"""
+
+MACOS_OK = """PING 1.1.1.1 (1.1.1.1): 56 data bytes
+64 bytes from 1.1.1.1: icmp_seq=0 ttl=57 time=12.3 ms
+
+--- 1.1.1.1 ping statistics ---
+1 packets transmitted, 1 packets received, 0.0% packet loss
+round-trip min/avg/max/stddev = 12.3/12.3/12.3/0.000 ms
+"""
+
+LOST = """--- 1.1.1.1 ping statistics ---
+1 packets transmitted, 0 received, 100% packet loss, time 0ms
+"""
+
+
+def test_parse_linux():
+    r = pingutil.parse_ping_output(LINUX_OK)
+    assert r == {"rtt_min": 11.234, "rtt_avg": 11.234, "rtt_max": 11.234, "loss": 0.0}
+
+
+def test_parse_macos():
+    r = pingutil.parse_ping_output(MACOS_OK)
+    assert r["rtt_avg"] == 12.3 and r["loss"] == 0.0
+
+
+def test_parse_lost():
+    r = pingutil.parse_ping_output(LOST)
+    assert r["rtt_avg"] == float("inf") and r["loss"] == 100.0
+
+
+def test_parse_garbage_is_unreachable():
+    r = pingutil.parse_ping_output("ping: connect: Network is unreachable")
+    assert r["rtt_avg"] == float("inf") and r["loss"] == 100.0
+
+
+@pytest.mark.parametrize("platform,family,expected", [
+    ("linux", None, ["ping", "-c", "3", "-W", "5", "h"]),
+    ("linux", 6, ["ping", "-c", "3", "-W", "5", "-6", "h"]),
+    ("linux", 4, ["ping", "-c", "3", "-W", "5", "-4", "h"]),
+    ("win32", None, ["ping", "-n", "3", "-w", "5000", "h"]),
+    ("win32", 6, ["ping", "-n", "3", "-w", "5000", "-6", "h"]),
+    ("darwin", None, ["ping", "-c", "3", "-W", "5", "h"]),
+    ("darwin", 4, ["ping", "-c", "3", "-W", "5000", "h"]),
+    ("darwin", 6, ["ping6", "-c", "3", "h"]),
+])
+def test_ping_command(monkeypatch, platform, family, expected):
+    monkeypatch.setattr(sys, "platform", platform)
+    assert pingutil.ping_command("h", 3, 5, family) == expected

tests/test_udp_presence.py

@@ -0,0 +1,250 @@
+"""Host-level overdue alerting driven by client-reported protocol presence."""
+import asyncio
+import time
+
+from hbd.common.proto import dicttos
+from hbd.server import hbdclass, udp
+from hbd.server.hbdclass import Connection as C
+from hbd.server.udp import handle_datagram, parse_message
+
+
+class _FakeTransport:
+    def sendto(self, data, addr):
+        pass
+
+
+def _ctx():
+    return {"config": {}, "hbdclass": hbdclass, "msg_to_websockets": None,
+            "DEBUG": 0, "verbose": False}
+
+
+def _fresh(name):
+    hbdclass.Host.hosts.pop(name, None)
+
+
+def _send(name, addr, ctx, **fields):
+    msg = parse_message(dicttos("HTB", {"name": name, "id": 0, **fields}))
+    handle_datagram(msg, (addr, 50000), _FakeTransport(), ctx)
+
+
+def _events(monkeypatch):
+    events = []
+    monkeypatch.setattr(udp, "eventlog", lambda u, level, msg, **kw: events.append((level, msg)))
+    return events
+
+
+def _dual_stack(name, ctx):
+    _fresh(name)
+    _send(name, "127.0.0.1", ctx)
+    _send(name, "::1", ctx)
+    return hbdclass.Host.hosts[name]
+
+
+def test_flags_are_stored_on_host():
+    ctx = _ctx()
+    _fresh("pres-store")
+    _send("pres-store", "127.0.0.1", ctx, p4=1, p6=0)
+    h = hbdclass.Host.hosts["pres-store"]
+    assert h.presence["IPv4"]["present"] is True
+    assert h.presence["IPv6"]["present"] is False
+
+
+def test_no_flags_leaves_presence_unknown():
+    ctx = _ctx()
+    _fresh("pres-legacy")
+    _send("pres-legacy", "127.0.0.1", ctx)
+    assert hbdclass.Host.hosts["pres-legacy"].presence == {}
+
+
+def test_absent_flag_moves_overdue_family_to_absent_and_clears_alert(monkeypatch):
+    _events(monkeypatch)
+    ctx = _ctx()
+    h = _dual_stack("pres-absent", ctx)
+    v6 = h.connections["IPv6"]
+    v6.state = C.OVERDUE
+    udp._set_connectivity_alert(h, "IPv6", "CRITICAL")
+    _send("pres-absent", "127.0.0.1", ctx, p4=1, p6=0)
+    assert v6.state == C.ABSENT
+    assert "connectivity.IPv6" not in h.alert_states
+    assert h.status() == "up"
+
+
+def test_absent_flag_does_not_touch_family_that_is_up():
+    ctx = _ctx()
+    h = _dual_stack("pres-up-stays", ctx)
+    _send("pres-up-stays", "127.0.0.1", ctx, p4=1, p6=0)
+    assert h.connections["IPv6"].state == C.UP          # traffic overrides the flag
+
+
+def test_present_again_arms_grace_timer():
+    ctx = _ctx()
+
+    async def scenario():
+        _fresh("pres-back")
+        _send("pres-back", "127.0.0.1", ctx, interval=10)
+        _send("pres-back", "::1", ctx, interval=10)
+        h = hbdclass.Host.hosts["pres-back"]
+        v6 = h.connections["IPv6"]
+        v6.cancel_overdue_timer()
+        v6.state = C.ABSENT
+        _send("pres-back", "127.0.0.1", ctx, interval=10, p6=1)
+        assert v6.state == C.UNKNOWN and v6.overdue_timer is not None
+        for c in h.connections.values():
+            c.cancel_overdue_timer()
+
+    asyncio.run(scenario())
+
+
+def test_one_family_overdue_is_simplex_all_overdue_is_critical(monkeypatch):
+    events = _events(monkeypatch)
+    ctx = _ctx()
+    h = _dual_stack("pres-simplex", ctx)
+    on_overdue, _ = udp._make_timer_callbacks("pres-simplex", h, ctx)
+
+    asyncio.run(on_overdue(h.connections["IPv6"]))
+    assert h.status() == "simplex"
+    assert ("INFO", "IPv6 overdue (simplex)") in events
+    assert not any(level == "CRITICAL" for level, _ in events)
+    assert "connectivity.IPv6" not in h.alert_states
+
+    asyncio.run(on_overdue(h.connections["IPv4"]))
+    assert h.status() == "overdue"
+    assert [e for e in events if e[0] == "CRITICAL"] == [("CRITICAL", "IPv4, IPv6 overdue")]
+    assert {"connectivity.IPv4", "connectivity.IPv6"} <= set(h.alert_states)
+
+
+def test_timer_expiry_on_absent_family_is_silent(monkeypatch):
+    events = _events(monkeypatch)
+    ctx = _ctx()
+    h = _dual_stack("pres-quiet", ctx)
+    _send("pres-quiet", "127.0.0.1", ctx, p4=1, p6=0)   # v6 still UP => left alone
+    events.clear()
+    on_overdue, _ = udp._make_timer_callbacks("pres-quiet", h, ctx)
+    asyncio.run(on_overdue(h.connections["IPv6"]))
+    assert h.connections["IPv6"].state == C.ABSENT
+    assert events == []
+    assert h.status() == "up"
+
+
+def test_recovery_from_simplex_is_info_only(monkeypatch):
+    events = _events(monkeypatch)
+    ctx = _ctx()
+    h = _dual_stack("pres-rec-simplex", ctx)
+    v6 = h.connections["IPv6"]
+    v6.state, v6.statetime = C.OVERDUE, time.time() - 100
+    events.clear()
+    _send("pres-rec-simplex", "::1", ctx)
+    assert v6.state == C.UP
+    assert [e[0] for e in events] == ["INFO"] and "back after" in events[0][1]
+
+
+def test_recovery_from_overdue_is_recover_and_clears_all_alerts(monkeypatch):
+    events = _events(monkeypatch)
+    ctx = _ctx()
+    h = _dual_stack("pres-rec-overdue", ctx)
+    for afam in ("IPv4", "IPv6"):
+        c = h.connections[afam]
+        c.state, c.statetime = C.OVERDUE, time.time() - 100
+        udp._set_connectivity_alert(h, afam, "CRITICAL")
+    events.clear()
+    _send("pres-rec-overdue", "127.0.0.1", ctx)
+    assert [e[0] for e in events] == ["RECOVER"]
+    assert h.status() == "simplex"
+    assert "connectivity.IPv6" not in h.alert_states     # left overdue => all cleared
+    assert "connectivity.IPv4" not in h.alert_states
+
+
+def test_restore_skips_absent_and_sets_no_alert_for_simplex():
+    ctx = _ctx()
+
+    async def scenario():
+        h = _dual_stack("pres-restore", ctx)
+        h.connections["IPv6"].state = C.OVERDUE          # v4 UP + v6 OVERDUE => simplex
+        udp.restore_connection_timers(hbdclass, ctx)
+        assert "connectivity.IPv6" not in h.alert_states
+        h.connections["IPv6"].cancel_overdue_timer()
+        h.connections["IPv6"].state = C.ABSENT
+        h.set_presence("IPv6", False, time.time())
+        udp.restore_connection_timers(hbdclass, ctx)
+        assert h.connections["IPv6"].state == C.ABSENT
+        for c in h.connections.values():
+            c.cancel_overdue_timer()
+
+    asyncio.run(scenario())
+
+
+def _past_drop(conn):
+    conn.state, conn.statetime = C.OVERDUE, time.time() - (udp.DROPOVERDUE + 10)
+
+
+def test_restore_past_drop_window_keeps_alert_single_family():
+    ctx = _ctx()
+
+    async def scenario():
+        _fresh("pres-drop-single")
+        _send("pres-drop-single", "127.0.0.1", ctx)
+        h = hbdclass.Host.hosts["pres-drop-single"]
+        _past_drop(h.connections["IPv4"])
+        udp.restore_connection_timers(hbdclass, ctx)
+        assert h.connections["IPv4"].state == C.UNKNOWN
+        assert "connectivity.IPv4" in h.alert_states
+
+    asyncio.run(scenario())
+
+
+def test_restore_past_drop_window_keeps_alerts_dual_family():
+    ctx = _ctx()
+
+    async def scenario():
+        h = _dual_stack("pres-drop-dual", ctx)
+        for c in h.connections.values():
+            _past_drop(c)
+        udp.restore_connection_timers(hbdclass, ctx)
+        assert {"connectivity.IPv4", "connectivity.IPv6"} <= set(h.alert_states)
+
+    asyncio.run(scenario())
+
+
+def test_v4_only_host_overdue_is_single_critical(monkeypatch):
+    events = _events(monkeypatch)
+    ctx = _ctx()
+    _fresh("pres-v4only")
+    _send("pres-v4only", "127.0.0.1", ctx)
+    h = hbdclass.Host.hosts["pres-v4only"]
+    on_overdue, _ = udp._make_timer_callbacks("pres-v4only", h, ctx)
+    asyncio.run(on_overdue(h.connections["IPv4"]))
+    assert [e for e in events if e[0] == "CRITICAL"] == [("CRITICAL", "IPv4 overdue")]
+    assert "connectivity.IPv4" in h.alert_states
+    asyncio.run(on_overdue(h.connections["IPv4"]))
+    assert len([e for e in events if e[0] == "CRITICAL"]) == 1
+
+
+def test_all_families_absent_v4_only_host_still_alerts_when_it_dies(monkeypatch):
+    events = _events(monkeypatch)
+    ctx = _ctx()
+    _fresh("pres-allabsent")
+    _send("pres-allabsent", "127.0.0.1", ctx, p4=0)
+    h = hbdclass.Host.hosts["pres-allabsent"]
+    on_overdue, _ = udp._make_timer_callbacks("pres-allabsent", h, ctx)
+    asyncio.run(on_overdue(h.connections["IPv4"]))
+    assert [e for e in events if e[0] == "CRITICAL"] == [("CRITICAL", "IPv4 overdue")]
+    assert h.connections["IPv4"].state == C.OVERDUE
+    assert h.status() == "overdue"
+    assert "connectivity.IPv4" in h.alert_states
+
+
+def test_last_present_family_is_not_parked_when_other_is_absent(monkeypatch):
+    events = _events(monkeypatch)
+    ctx = _ctx()
+    h = _dual_stack("pres-lastfam", ctx)
+    _send("pres-lastfam", "127.0.0.1", ctx, p4=0, p6=0)
+    events.clear()
+    on_overdue, _ = udp._make_timer_callbacks("pres-lastfam", h, ctx)
+    asyncio.run(on_overdue(h.connections["IPv4"]))
+    assert h.connections["IPv4"].state == C.ABSENT
+    assert not any(level == "CRITICAL" for level, _ in events)
+    assert h.status() == "up"
+    asyncio.run(on_overdue(h.connections["IPv6"]))
+    assert [e for e in events if e[0] == "CRITICAL"] == [("CRITICAL", "IPv6 overdue")]
+    assert h.connections["IPv6"].state == C.OVERDUE
+    assert h.status() == "overdue"

tests/test_udp_rtt_history.py

@@ -14,10 +14,12 @@ class _FakeTransport:
         self.sent.append((data, addr))
 
 
-def _htb(name, rtt=None, interval=0):
+def _htb(name, rtt=None, interval=0, boot=0):
     d = {"name": name, "interval": interval, "id": 0}
     if rtt is not None:
         d["rtt"] = rtt
+    if boot:
+        d["boot"] = boot
     return parse_message(dicttos("HTB", d))
 
 
@@ -94,10 +96,11 @@ def test_request_update_fires_on_recovery_even_with_rtt_history():
     assert ack.get("request_update")
 
 
-def test_recovery_clears_real_plugin_data_but_preserves_rtt_history():
+def test_ordinary_recovery_preserves_real_plugin_data_and_rtt_history():
-    """Regression for Finding 2: host.plugin_data.clear() on recovery must
+    """An ordinary reconnect (no boot flag) — e.g. OVERDUE->UP after a
-    wipe real client-collected plugin data while leaving rtt_* history
+    transient network blip — must NOT wipe already-collected real plugin
-    samples intact.
+    data (e.g. cpu_monitor, os_info) or rtt_* history. Only an actual
+    client reboot invalidates that data (see the boot-flag test below).
     """
     hbdclass.Host.hosts.pop("rtt-hist-host5", None)
     transport = _FakeTransport()
@@ -116,8 +119,35 @@ def test_recovery_clears_real_plugin_data_but_preserves_rtt_history():
     host.add_plugin_data("os_info", {"os": "linux"}, timestamp=time.time())
     assert "os_info" in host.plugin_data
 
-    # Recovery heartbeat.
+    # Ordinary recovery heartbeat — no boot flag.
     handle_datagram(_htb("rtt-hist-host5", rtt=13.0), ("127.0.0.1", 50000), transport, ctx)
 
+    assert "os_info" in host.plugin_data
+    assert len(host.plugin_data["rtt_ipv4"]) == 4
+
+
+def test_boot_recovery_clears_real_plugin_data_but_preserves_rtt_history():
+    """A recovery heartbeat carrying the boot flag (client process actually
+    restarted) must still wipe stale real plugin data, while rtt_* history
+    (still a valid measurement, unaffected by a client reboot) survives.
+    """
+    hbdclass.Host.hosts.pop("rtt-hist-host6", None)
+    transport = _FakeTransport()
+    ctx = _base_ctx()
+
+    for rtt in (10.0, 11.0, 12.0):
+        handle_datagram(_htb("rtt-hist-host6", rtt=rtt), ("127.0.0.1", 50000), transport, ctx)
+
+    host = hbdclass.Host.hosts["rtt-hist-host6"]
+    assert len(host.plugin_data["rtt_ipv4"]) == 3
+
+    conn = host.connections["IPv4"]
+    conn.state = hbdclass.Connection.DOWN
+    host.add_plugin_data("os_info", {"os": "linux"}, timestamp=time.time())
+    assert "os_info" in host.plugin_data
+
+    # Recovery heartbeat with boot=1 — client process actually restarted.
+    handle_datagram(_htb("rtt-hist-host6", rtt=13.0, boot=1), ("127.0.0.1", 50000), transport, ctx)
+
     assert "os_info" not in host.plugin_data
     assert len(host.plugin_data["rtt_ipv4"]) == 4