version 5.4.7

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -2,6 +2,54 @@
 
 All notable changes to this project are documented here, organized by release.
 
+## [5.4.7]
+
+### Fixed
+- clear all mypy errors
+- honour the legacy thresholds: config; repair the stale test suite
+- ignore datagrams that are not named HTB/PLG messages
+
+---
+
+## [5.4.6]
+
+### Added
+- re-resolve the server hostnames at the DNS TTL
+
+---
+
+## [5.4.5]
+
+### Added
+- `hbc_mini.py` and `hbc_mini.c` (the dependency-free clients) get the same per-protocol connectivity probe as `hbc`: `p4`/`p6` flags on every heartbeat and a `connectivity` metric (presence, own address, gateway, RTT) via `network_monitor`, with gateway lookup on Linux, FreeBSD/NetBSD/DragonFly (both clients) and macOS/Windows (`hbc_mini.py` only)
+- `scripts/c/test_probe.c` and a `make test` target — the C client's first automated test suite
+
+### Fixed
+- `hbc_mini.c`: the clean-shutdown heartbeat could be delayed or lost behind an in-flight connectivity probe on SIGTERM; it is now sent before the probe thread is joined
+- `hbc_mini.c`: a connectivity probe that fails to start, or is configured with a non-IP-literal target, now logs a warning instead of failing silently
+- `hbc_mini.py`: removed an unused import
+
+---
+
+## [5.4.4]
+
+### Added
+- client reports per-protocol (IPv4/IPv6) internet presence: a connectivity probe (route check every ~5s, ICMP echo every ~60s, debounced) sends `p4`/`p6` flags on every heartbeat and a `connectivity` metric (presence, own address, gateway, RTT) via `network_monitor`
+- server monitors only the protocols a client reports as present; a protocol reported absent is shown as `absent` and raises no alert
+- host status (`up`, `simplex`, `overdue`, `down`, `unknown`) computed on the server: a host is overdue only when every present protocol is overdue; `simplex` (both present, one silent) has the same severity as `up` and is logged without notification
+- live and plugins pages show the host status, `simplex`/`absent` protocols and a Connectivity table
+- `hbc` config keys under `network_monitor`: `probe_enabled`, `probe_v4`, `probe_v6`, `probe_interval`
+
+### Changed
+- overdue and recovery alerts are raised per host, not per protocol: a single CRITICAL when the last present protocol goes overdue; a protocol coming back while another stayed up is logged as INFO instead of a RECOVER notification (also applies to clients that do not report presence)
+
+### Fixed
+- never park the last present family as absent, so a host that reports no protocol as present still alerts when it dies
+- keep the connectivity alert for a host overdue past the drop window across an `hbd` restart
+- `default_gateway` never raises; a protocol whose probe cannot run stays unknown rather than absent
+
+---
+
 ## [5.4.3]
 
 ### Fixed

CLAUDE.md

@@ -33,7 +33,7 @@ hbd passwd <username>
 
 Tests live in `tests/` (pytest, imported as a package). A second directory `test/` contains only TLS fixtures (`.pem` files), not test code.
 
-Line length: 111 characters (`black`, `flake8`, both configured).
+Line length: `black` and `flake8` both use 165 characters (set in `pyproject.toml`, `tox.ini` and `.flake8`).
 
 ## Architecture
 

README.md

@@ -20,7 +20,7 @@ A lightweight UDP-based host monitoring system. Monitored hosts run a client (`h
   └────────────────────┘              └────────────────────────────┘
 ```
 
-**Package:** `hbd` v5.4.3
+**Package:** `hbd` v5.4.7
 **Python:** 3.11+
 
 ### Subpackages
@@ -242,6 +242,7 @@ plugins:
 ### Connection behaviour
 
 - The client sends heartbeats over UDP to each server address resolved from the hostname (IPv4 and IPv6).
+- The server hostname is re-resolved every TTL seconds (the smallest A/AAAA TTL, at least 30s; 300s when the TTL cannot be determined, e.g. an IP literal), so an address change is followed without restarting the client. A failed lookup keeps the current addresses.
 - If a connection fails to open at startup, IPv6 connections are dropped after 3 consecutive failures. IPv4 connections retry indefinitely.
 - In daemon mode (`-d`), all log output goes to syslog (`LOG_DAEMON` facility).
 
@@ -659,7 +660,28 @@ Config: `~/.hbc.json` (JSON format, same keys as `~/.hbc.yaml`).
 | `cpu_monitor` | Linux (`/proc/stat`; no per-core, no frequency) |
 | `memory_monitor` | Linux (`/proc/meminfo`) |
 | `disk_monitor` | Linux, macOS, BSD (`df -P`) |
-| `network_monitor` | Linux (`/proc/net/dev`) |
+| `network_monitor` | Linux (`/proc/net/dev`); per-protocol connectivity probe (all platforms) |
+
+`network_monitor` also reports per-protocol (IPv4/IPv6) internet presence — whether the host
+has a working route and reply to a well-known address, plus its own address, the local
+gateway, and RTT — as `p4`/`p6` flags on every heartbeat and a `connectivity` object in the
+plugin data. Configurable, defaults shown:
+
+```json
+{
+  "plugins": {
+    "network_monitor": {
+      "probe_enabled": true,
+      "probe_v4": "1.1.1.1",
+      "probe_v6": "2606:4700:4700::1111",
+      "probe_interval": 60
+    }
+  }
+}
+```
+
+`probe_v4`/`probe_v6` must be IP literals (no hostnames). Gateway lookup works on Linux,
+FreeBSD/NetBSD/DragonFly/macOS (`route -n get`), and Windows (`route print`).
 
 Not available vs full `hbc`: no YAML config, no `filesystem_info`, no `zfs_monitor`, no IPv6 early-fail protection.
 
@@ -712,6 +734,7 @@ SIGHUP causes the process to restart itself via `execv()`. SIGTERM/SIGINT trigge
 | `disk_monitor` | All | `df -P` subprocess |
 | `network_monitor` | Linux | `/proc/net/dev` |
 | `network_monitor` | FreeBSD, NetBSD, DragonFly | `getifaddrs()` + `AF_LINK` |
+| `network_monitor` (connectivity probe) | Linux, FreeBSD, NetBSD, DragonFly | UDP `connect()` route check + one ICMP echo per protocol |
 | `ping_monitor` | All | `ping` subprocess |
 | `nagios_runner` | All | `popen()` subprocess |
 
@@ -743,12 +766,33 @@ SIGHUP causes the process to restart itself via `execv()`. SIGTERM/SIGINT trigge
 }
 ```
 
+`network_monitor` also runs the per-protocol connectivity probe described above, in a
+dedicated thread (its ICMP echo can block for a few seconds). Same config keys and defaults
+as `hbc_mini.py`:
+
+```json
+{
+  "plugins": {
+    "network_monitor": {
+      "probe_enabled": true,
+      "probe_v4": "1.1.1.1",
+      "probe_v6": "2606:4700:4700::1111",
+      "probe_interval": 60
+    }
+  }
+}
+```
+
+Gateway lookup on the C client covers Linux and FreeBSD/NetBSD/DragonFly only (no macOS or
+Windows build of `hbc_mini.c` exists).
+
 ### Differences from `hbc_mini.py`
 
 - No `filesystem_info` or `zfs_monitor` plugins
 - `UPD` (self-update) messages are logged but not acted on
 - No IPv6 early-fail protection
 - Config is JSON only (`~/.hbc.json`), no YAML
+- No macOS or Windows build (Linux and the BSDs only)
 
 ---
 

hbd/__init__.py

@@ -4,7 +4,7 @@ This package contains both the heartbeat client (hbc) and server (hbd) component
 organized into separate subpackages:
 
 - hbd.client: Client component with system monitoring plugins
-- hbd.server: Server/daemon component with web UI and notifications  
+- hbd.server: Server/daemon component with web UI and notifications
 - hbd.common: Shared utilities and protocol definitions
 
 Install options:
@@ -14,4 +14,4 @@ Install options:
 """
 
 __all__ = ["__version__"]
-__version__ = "5.4.3"
+__version__ = "5.4.7"

hbd/client/__init__.py

@@ -1,3 +1,3 @@
 """HeartBeat Client (hbc) - System monitoring client."""
 
-from hbd import __version__
+from hbd import __version__  # noqa: F401

hbd/client/config.py

@@ -2,16 +2,16 @@
 
 import logging
 import os
-import logging
+from typing import Any, Dict
 
 logger = logging.getLogger(__name__)
 
 try:
     import yaml
 except Exception:
-    yaml = None
+    yaml = None  # type: ignore[assignment]
 
-CLIENT_DEFAULTS = {
+CLIENT_DEFAULTS: Dict[str, Any] = {
     # Network settings
     "hb_port": 50003,          # Port where hbd servers listen
     "interval": 10,             # Heartbeat interval in seconds
@@ -37,10 +37,10 @@ def load_config(path=None):
     """Load configuration from a YAML file and merge with client defaults.
 
     If YAML is not available or the file does not exist, defaults are returned.
-    
+
     Args:
         path: Path to YAML config file (default: ~/.hbc.yaml)
-        
+
     Returns:
         Dictionary with configuration
     """

hbd/client/connectivity.py

@@ -0,0 +1,204 @@
+"""Per-protocol internet presence probe.
+
+Tier 1 (every tier1_interval seconds, sends no packets): a UDP connect() to the probe
+address reveals whether a route exists and which source address would be used.
+Tier 2 (every probe_interval seconds, sooner after a failure or a change of address or
+gateway): one ICMP echo to the probe address. Absence needs fail_limit consecutive
+failures (or no route at all); presence needs one success.
+
+The result is read by the heartbeat sender (p4/p6 flags on every HTB) and by the
+network_monitor plugin (connectivity metric) via the shared instance.
+"""
+
+import asyncio
+import logging
+import socket
+import sys
+import time
+from dataclasses import asdict, dataclass, replace
+from typing import Any, Awaitable, Callable, Dict, Optional, Set, Tuple
+
+from hbd.client import gateway as gateway_mod
+from hbd.client.pingutil import PingUnavailable, run_ping
+
+logger = logging.getLogger("hbc.connectivity")
+
+FAMILIES = ("ipv4", "ipv6")
+FLAG_KEYS = {"ipv4": "p4", "ipv6": "p6"}
+DEFAULT_PROBES = {"ipv4": "1.1.1.1", "ipv6": "2606:4700:4700::1111"}
+
+
+class ProbeUnavailable(Exception):
+    """The address family cannot be probed on this host (e.g. no IPv6 support)."""
+
+
+@dataclass
+class FamilyState:
+    present: Optional[bool] = None      # None = unknown (probe cannot run yet/at all)
+    addr: Optional[str] = None
+    gateway: Optional[str] = None
+    rtt: Optional[float] = None
+
+
+def local_address(family: str, target: str) -> Optional[str]:
+    """Source address the kernel would use to reach target, or None if there is no route.
+
+    A UDP connect() sends nothing. Raises ProbeUnavailable if the family is unsupported.
+    """
+    af = socket.AF_INET if family == "ipv4" else socket.AF_INET6
+    try:
+        sock = socket.socket(af, socket.SOCK_DGRAM)
+    except OSError as e:
+        raise ProbeUnavailable(str(e)) from e
+    try:
+        sock.connect((target, 9))
+        return str(sock.getsockname()[0])
+    except OSError:
+        return None
+    finally:
+        sock.close()
+
+
+async def ping_rtt(target: str, family: str) -> Optional[float]:
+    """One ICMP echo; returns RTT in ms or None if no reply."""
+    result = await run_ping(target, 1, 2, 4 if family == "ipv4" else 6)
+    rtt = result["rtt_avg"]
+    return None if rtt == float("inf") else rtt
+
+
+class Connectivity:
+    def __init__(
+        self,
+        probes: Optional[Dict[str, str]] = None,
+        probe_interval: float = 60.0,
+        retry_interval: float = 10.0,
+        tier1_interval: float = 5.0,
+        fail_limit: int = 2,
+        local_address_fn: Callable[[str, str], Optional[str]] = local_address,
+        ping_fn: Callable[[str, str], Awaitable[Optional[float]]] = ping_rtt,
+        gateway_fn: Callable[[str], Awaitable[Optional[str]]] = gateway_mod.default_gateway,
+        clock: Callable[[], float] = time.monotonic,
+    ):
+        self.probes = {**DEFAULT_PROBES, **(probes or {})}
+        self.probe_interval = probe_interval
+        self.retry_interval = retry_interval
+        self.tier1_interval = tier1_interval
+        self.fail_limit = fail_limit
+        self._local_address = local_address_fn
+        self._ping = ping_fn
+        self._gateway_fn = gateway_fn
+        self._clock = clock
+        # Reading the gateway is a cheap /proc read on Linux but a subprocess elsewhere,
+        # so elsewhere it is refreshed only when a tier-2 probe is due.
+        self._cheap_gateway = sys.platform.startswith("linux")
+        self.states: Dict[str, FamilyState] = {f: FamilyState() for f in FAMILIES}
+        self._fails: Dict[str, int] = {f: 0 for f in FAMILIES}
+        self._next_ping: Dict[str, float] = {f: 0.0 for f in FAMILIES}
+        self._last_key: Dict[str, Optional[Tuple[Optional[str], Optional[str]]]] = {
+            f: None for f in FAMILIES}
+        self._ping_disabled: Set[str] = set()
+
+    @classmethod
+    def from_config(cls, cfg: Optional[Dict[str, Any]]) -> "Connectivity":
+        cfg = cfg or {}
+        probes = {}
+        if cfg.get("probe_v4"):
+            probes["ipv4"] = str(cfg["probe_v4"])
+        if cfg.get("probe_v6"):
+            probes["ipv6"] = str(cfg["probe_v6"])
+        return cls(probes=probes, probe_interval=float(cfg.get("probe_interval", 60)))
+
+    # ---- readers (cheap; safe to call on every heartbeat) --------------------------
+    def snapshot(self) -> Dict[str, FamilyState]:
+        return {f: replace(s) for f, s in self.states.items()}
+
+    def flags(self) -> Dict[str, bool]:
+        """HTB flags for families whose presence is known."""
+        return {FLAG_KEYS[f]: s.present for f, s in self.states.items() if s.present is not None}
+
+    def as_metric(self) -> Dict[str, Dict[str, Any]]:
+        return {f: asdict(s) for f, s in self.states.items()}
+
+    # ---- probing -------------------------------------------------------------------
+    async def refresh(self) -> None:
+        for fam in FAMILIES:
+            try:
+                await self._refresh_family(fam)
+            except Exception:
+                logger.exception("connectivity probe failed for %s", fam)
+
+    async def run(self, stop: asyncio.Event) -> None:
+        while not stop.is_set():
+            await self.refresh()
+            try:
+                await asyncio.wait_for(stop.wait(), timeout=self.tier1_interval)
+            except asyncio.TimeoutError:
+                pass
+
+    async def _gateway(self, fam: str) -> Optional[str]:
+        try:
+            return await self._gateway_fn(fam)
+        except Exception:
+            return None
+
+    async def _refresh_family(self, fam: str) -> None:
+        st = self.states[fam]
+        try:
+            addr = self._local_address(fam, self.probes[fam])
+        except ProbeUnavailable:
+            st.present = None
+            return
+
+        if addr is None:                       # tier 1: no route => absent immediately
+            self._fails[fam] = 0
+            self._next_ping[fam] = 0.0
+            self._last_key[fam] = None
+            st.present, st.addr, st.gateway, st.rtt = False, None, None, None
+            return
+
+        now = self._clock()
+        gw = await self._gateway(fam) if self._cheap_gateway else st.gateway
+        key = (addr, gw)
+        due = key != self._last_key[fam] or now >= self._next_ping[fam]
+        if due and not self._cheap_gateway:
+            gw = await self._gateway(fam)
+            key = (addr, gw)
+        st.addr, st.gateway = addr, gw
+        self._last_key[fam] = key
+        if fam in self._ping_disabled:
+            st.present = None          # cannot probe: unknown, never absent, while a route exists
+        elif due:
+            await self._tier2(fam, st, now)
+
+    async def _tier2(self, fam: str, st: FamilyState, now: float) -> None:
+        try:
+            rtt = await self._ping(self.probes[fam], fam)
+        except PingUnavailable:
+            logger.warning("ping unavailable; %s presence stays unknown", fam)
+            self._ping_disabled.add(fam)
+            st.present = None
+            return
+        except (asyncio.TimeoutError, OSError):
+            rtt = None
+        if rtt is None:
+            self._fails[fam] += 1
+            self._next_ping[fam] = now + self.retry_interval
+            if self._fails[fam] >= self.fail_limit:
+                st.present, st.rtt = False, None
+        else:
+            self._fails[fam] = 0
+            self._next_ping[fam] = now + self.probe_interval
+            st.present, st.rtt = True, rtt
+
+
+# ---- shared instance (heartbeat sender and network_monitor plugin) -----------------
+_shared: Optional[Connectivity] = None
+
+
+def set_shared(conn: Optional[Connectivity]) -> None:
+    global _shared
+    _shared = conn
+
+
+def get_shared() -> Optional[Connectivity]:
+    return _shared

hbd/client/dnsttl.py

@@ -0,0 +1,170 @@
+"""Re-resolution of the server hostnames at the TTL of their A/AAAA records.
+
+getaddrinfo() hides the TTL, so a minimal stub-resolver query (resolv.conf nameservers, search
+list) is made only to learn it. The addresses themselves still come from getaddrinfo(). When
+the TTL cannot be learned (IP literal, /etc/hosts name, no resolv.conf, resolver timeout)
+DEFAULT_TTL is used.
+"""
+
+import ipaddress
+import logging
+import os
+import socket
+import struct
+from typing import List, Optional, Tuple
+
+logger = logging.getLogger("hbc.dns")
+
+DEFAULT_TTL = 300       # used when the real TTL is unknown
+MIN_TTL = 30            # a TTL of 0 must not turn into a resolution loop
+FAILED_RETRY = 60       # re-try delay after a failed resolution
+QUERY_TIMEOUT = 2.0
+
+TYPE_A, TYPE_CNAME, TYPE_AAAA = 1, 5, 28
+
+
+def parse_resolv_conf(path: str = "/etc/resolv.conf") -> Tuple[List[str], List[str]]:
+    """Return (nameservers, search domains)."""
+    servers: List[str] = []
+    search: List[str] = []
+    try:
+        with open(path) as f:
+            for line in f:
+                parts = line.split()
+                if len(parts) < 2:
+                    continue
+                if parts[0] == "nameserver":
+                    servers.append(parts[1].split("%")[0])
+                elif parts[0] in ("search", "domain"):
+                    search = parts[1:]
+    except OSError:
+        pass
+    return servers, search
+
+
+def build_query(name: str, qtype: int, qid: int) -> bytes:
+    out = struct.pack("!HHHHHH", qid, 0x0100, 1, 0, 0, 0)
+    for label in name.rstrip(".").split("."):
+        out += bytes([len(label)]) + label.encode("idna")
+    return out + b"\x00" + struct.pack("!HH", qtype, 1)
+
+
+def _skip_name(buf: bytes, off: int) -> int:
+    while True:
+        n = buf[off]
+        if n == 0:
+            return off + 1
+        if n & 0xC0:
+            return off + 2
+        off += n + 1
+
+
+def min_answer_ttl(resp: bytes, qid: int, qtype: int) -> Optional[int]:
+    """Smallest TTL among the answer RRs (CNAMEs included), or None if there is no qtype answer."""
+    try:
+        rid, flags, qd, an = struct.unpack("!HHHH", resp[:8])
+        if rid != qid or flags & 0x000F:
+            return None
+        off = 12
+        for _ in range(qd):
+            off = _skip_name(resp, off) + 4
+        ttls: List[int] = []
+        found = False
+        for _ in range(an):
+            off = _skip_name(resp, off)
+            rtype, _cls, ttl, rdlen = struct.unpack("!HHIH", resp[off:off + 10])
+            off += 10 + rdlen
+            if rtype in (qtype, TYPE_CNAME):
+                ttls.append(ttl)
+            found = found or rtype == qtype
+        return min(ttls) if found else None
+    except (struct.error, IndexError):
+        return None
+
+
+def _query_ttl(server: str, name: str, qtype: int) -> Optional[int]:
+    qid = struct.unpack("!H", os.urandom(2))[0]
+    try:
+        af = socket.AF_INET6 if ":" in server else socket.AF_INET
+        with socket.socket(af, socket.SOCK_DGRAM) as s:
+            s.settimeout(QUERY_TIMEOUT)
+            s.sendto(build_query(name, qtype, qid), (server, 53))
+            return min_answer_ttl(s.recv(4096), qid, qtype)
+    except (OSError, UnicodeError):
+        return None
+
+
+def lookup_ttl(host: str, af_filter: int = 0) -> Optional[int]:
+    """Blocking. Smallest A/AAAA TTL for host, or None when it cannot be determined."""
+    try:
+        ipaddress.ip_address(host)
+        return None
+    except ValueError:
+        pass
+    servers, search = parse_resolv_conf()
+    if host.endswith("."):
+        names = [host]
+    elif "." in host:
+        names = [host] + [f"{host}.{d}" for d in search]
+    else:
+        names = [f"{host}.{d}" for d in search] + [host]
+    qtypes = [t for t, af in ((TYPE_A, socket.AF_INET), (TYPE_AAAA, socket.AF_INET6))
+              if af_filter in (0, af)]
+    for name in names:
+        ttls = []
+        for qtype in qtypes:
+            for server in servers:
+                ttl = _query_ttl(server, name, qtype)
+                if ttl is not None:
+                    ttls.append(ttl)
+                    break
+        if ttls:
+            return min(ttls)
+    return None
+
+
+def refresh_delay(ttls: List[Optional[int]]) -> int:
+    """Seconds until the next resolution: the smallest known TTL, floored at MIN_TTL."""
+    known = [t for t in ttls if t is not None]
+    return max(min(known), MIN_TTL) if known else DEFAULT_TTL
+
+
+def resolve(hosts: List[str], port: int, af_filter: int) -> Optional[List[Tuple[int, str]]]:
+    """Blocking. Every (family, address) the hosts resolve to, deduplicated, or None if any lookup fails."""
+    found: List[Tuple[int, str]] = []
+    for host in hosts:
+        try:
+            infos = socket.getaddrinfo(host, port, af_filter, 0, socket.SOL_UDP)
+        except socket.gaierror as e:
+            logger.warning(f"Cannot resolve {host}: {e}")
+            return None
+        for info in infos:
+            entry = (info[0], str(info[4][0]))
+            if entry not in found:
+                found.append(entry)
+    return found or None
+
+
+def retarget(connections: list, resolved: List[Tuple[int, str]]) -> Tuple[List[Tuple[int, str]], list]:
+    """Point the connections at the resolved addresses, in place.
+
+    A connection whose address is gone is re-pointed at an address that is new, in list order, so
+    the first connection (the one plugin data uses) survives as long as anything resolves.
+    Returns (new addresses still needing a connection, connections to drop).
+    """
+    current = {c.addr for c in connections}
+    wanted = {addr for _, addr in resolved}
+    new = [(af, addr) for af, addr in resolved if addr not in current]
+    gone = [c for c in connections if c.addr not in wanted]
+    paired = min(len(gone), len(new))
+    for conn, (af, addr) in zip(gone, new):
+        logger.info(f"Server address {conn.addr} -> {addr}")
+        conn.set_target(addr, af)
+    drop = gone[paired:]
+    if connections and connections[0] in drop:
+        # plugin data goes out on the first connection: hand it a surviving address and drop that one instead
+        keep = next(c for c in connections if c not in gone)
+        connections[0].set_target(keep.addr, keep.af)
+        drop.remove(connections[0])
+        drop.append(keep)
+    return new[paired:], drop

hbd/client/gateway.py

@@ -0,0 +1,103 @@
+"""Best-effort default-gateway lookup (informational only; never affects presence)."""
+
+import asyncio
+import re
+import socket
+import struct
+import sys
+from typing import List, Optional, Tuple
+
+_RTF_GATEWAY = 0x2
+_RTF_REJECT = 0x200
+
+
+def parse_proc_route_v4(text: str) -> Optional[str]:
+    """Default gateway from /proc/net/route (lowest metric wins)."""
+    best: Optional[Tuple[int, str]] = None
+    for line in text.splitlines()[1:]:
+        f = line.split()
+        if len(f) < 8:
+            continue
+        _iface, dest, gw, flags = f[0], f[1], f[2], f[3]  # noqa: F841
+        if dest != "00000000" or not int(flags, 16) & _RTF_GATEWAY:
+            continue
+        metric = int(f[6])
+        if best is None or metric < best[0]:
+            best = (metric, socket.inet_ntoa(struct.pack("<L", int(gw, 16))))
+    return best[1] if best else None
+
+
+def parse_proc_route_v6(text: str) -> Optional[str]:
+    """Default gateway from /proc/net/ipv6_route (lowest metric wins)."""
+    best: Optional[Tuple[int, str]] = None
+    for line in text.splitlines():
+        f = line.split()
+        if len(f) < 10:
+            continue
+        dest, plen, nexthop, metric, flags, iface = f[0], f[1], f[4], f[5], f[8], f[9]
+        if dest != "0" * 32 or plen != "00" or nexthop == "0" * 32 or iface == "lo":
+            continue
+        fl = int(flags, 16)
+        if not fl & _RTF_GATEWAY or fl & _RTF_REJECT:
+            continue
+        m = int(metric, 16)
+        if best is None or m < best[0]:
+            best = (m, socket.inet_ntop(socket.AF_INET6, bytes.fromhex(nexthop)))
+    return best[1] if best else None
+
+
+def parse_macos_route(text: str) -> Optional[str]:
+    """Gateway from `route -n get [-inet6] default`."""
+    m = re.search(r"^\s*gateway:\s*(\S+)", text, re.MULTILINE)
+    return m.group(1) if m else None
+
+
+def parse_windows_route(text: str, family: str) -> Optional[str]:
+    """Default gateway from `route print -4` / `route print -6` (lowest metric wins)."""
+    best: Optional[Tuple[int, str]] = None
+    for line in text.splitlines():
+        if family == "ipv4":
+            f = line.split()
+            if len(f) >= 5 and f[0] == "0.0.0.0" and f[1] == "0.0.0.0" and f[2] != "On-link":
+                cand = (int(f[4]), f[2])
+            else:
+                continue
+        else:
+            m = re.match(r"^\s*\d+\s+(\d+)\s+::/0\s+(\S+)\s*$", line)
+            if not m or m.group(2) == "On-link":
+                continue
+            cand = (int(m.group(1)), m.group(2))
+        if best is None or cand[0] < best[0]:
+            best = cand
+    return best[1] if best else None
+
+
+async def _run(cmd: List[str]) -> str:
+    proc = await asyncio.create_subprocess_exec(
+        *cmd, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.DEVNULL,
+    )
+    try:
+        out, _ = await asyncio.wait_for(proc.communicate(), timeout=5)
+    except asyncio.TimeoutError:
+        proc.kill()
+        await proc.wait()
+        raise
+    return out.decode(errors="replace")
+
+
+async def default_gateway(family: str) -> Optional[str]:
+    """Return the default gateway address for "ipv4"/"ipv6", or None on any failure."""
+    v4 = family == "ipv4"
+    try:
+        if sys.platform.startswith("linux"):
+            with open("/proc/net/route" if v4 else "/proc/net/ipv6_route") as fh:
+                text = fh.read()
+            return parse_proc_route_v4(text) if v4 else parse_proc_route_v6(text)
+        if sys.platform == "darwin":
+            cmd = ["route", "-n", "get"] + ([] if v4 else ["-inet6"]) + ["default"]
+            return parse_macos_route(await _run(cmd))
+        if sys.platform == "win32":
+            return parse_windows_route(await _run(["route", "print", "-4" if v4 else "-6"]), family)
+    except Exception:
+        return None
+    return None

hbd/client/main.py

@@ -16,7 +16,7 @@ import sys
 import time
 from logging.handlers import SysLogHandler
 from pathlib import Path
-from typing import Dict, List, Optional
+from typing import List, Optional
 
 # Import protocol and config
 from .config import load_config
@@ -25,6 +25,8 @@ from .. import __version__
 
 # Import plugin system
 from .plugin import PluginRegistry, PluginLoader, InfoPlugin, MonitorPlugin
+from . import dnsttl
+from .connectivity import Connectivity, set_shared
 
 # Constants
 PORT = 50003
@@ -34,6 +36,7 @@ MAXRECV = 32767
 # Global state
 running = True
 dorestart = False
+send_shutdown = False
 shutdown_event: Optional[asyncio.Event] = None
 active_tasks: List[asyncio.Task] = []
 
@@ -44,20 +47,20 @@ allow_remote_command = False
 
 class AsyncConnection:
     """Async UDP connection to a heartbeat server."""
-    
+
     def __init__(self, conn_id: int, addr: str, port: int, af: int, name: str):
         self.conn_id = conn_id
         self.addr = addr
         self.port = port
         self.af = af
         self.name = name
-        
+
         self.ackcount = 0
         self.lastack = 0.0
         self.send_count = 0
         self.lastsend = 0.0
         self.rtts = [0.0]
-        
+
         self.transport: Optional[asyncio.DatagramTransport] = None
         self.protocol: Optional[asyncio.DatagramProtocol] = None
         self._dead = False
@@ -87,17 +90,25 @@ class AsyncConnection:
         except Exception as e:
             self.logger.error(f"Failed to open connection: {e}")
             return False
-    
+
     def close(self):
         """Close the connection."""
         if self.transport:
             self.transport.close()
             self.transport = None
             self.protocol = None
-    
+
+    def set_target(self, addr: str, af: int):
+        """Re-point at a new server address; the socket is reopened on the next send if the family changed."""
+        if af != self.af:
+            self.close()
+            self.af = af
+        self.addr = addr
+        self.logger = logging.getLogger(f"hbc.conn.{addr}")
+
     async def sendto(self, msg: dict, msg_id: str = "HTB"):
         """Send a message to the server.
-        
+
         Args:
             msg: Message dictionary
             msg_id: Message ID (HTB, PLG, etc.)
@@ -111,36 +122,36 @@ class AsyncConnection:
         if not self.transport:
             self.logger.error("Cannot send - no transport")
             return
-        
+
         # Add standard fields
         msg["name"] = shortname(self.name)
         msg["id"] = self.conn_id
         msg["time"] = time.time()
-        
+
         # Encode message
         data = dicttos(msg_id, msg)
-        
+
         # Send
         self.transport.sendto(data, (self.addr, self.port))
         self.send_count += 1
         self.lastsend = time.time()
-        
+
         self.logger.debug(f"Sent {msg_id} message ({len(data)} bytes)")
-    
+
     def handle_ack(self, msg: dict, now: float):
         """Handle ACK message from server.
-        
+
         RTT is calculated as: (time ACK received) - (time HTB sent)
         """
         self.lastack = now
-        
+
         # Calculate RTT: time ACK received minus time HTB sent
         rtt = (now - self.lastsend) * 1000.0  # Convert to ms
-        
+
         self.rtts.append(rtt)
         if len(self.rtts) > 10:
             self.rtts.pop(0)
-        
+
         self.ackcount += 1
         self.logger.debug(f"ACK received, RTT: {rtt:.1f}ms")
         if msg.get("request_update"):
@@ -150,11 +161,11 @@ class AsyncConnection:
 
 class HeartbeatProtocol(asyncio.DatagramProtocol):
     """Protocol handler for incoming UDP messages."""
-    
+
     def __init__(self, connection: AsyncConnection):
         self.connection = connection
         self.logger = logging.getLogger("hbc.protocol")
-    
+
     def datagram_received(self, data: bytes, addr):
         """Handle incoming datagram."""
         try:
@@ -162,10 +173,10 @@ class HeartbeatProtocol(asyncio.DatagramProtocol):
             if not msg:
                 self.logger.warning(f"Failed to parse message from {addr}")
                 return
-            
+
             now = time.time()
             msg_id = msg.get("ID")
-            
+
             if msg_id == "ACK":
                 self.connection.handle_ack(msg, now)
             elif msg_id == "CMD":
@@ -176,10 +187,10 @@ class HeartbeatProtocol(asyncio.DatagramProtocol):
                 asyncio.create_task(handle_update(self.connection, msg))
             else:
                 self.logger.warning(f"Unknown message type: {msg_id}")
-                
+
         except Exception as e:
             self.logger.error(f"Error processing datagram: {e}", exc_info=True)
-    
+
     def error_received(self, exc):
         """Handle protocol errors — close transport so the heartbeat sender retries."""
         self.logger.warning(f"Protocol error on {self.connection.addr}: {exc} — will retry")
@@ -220,7 +231,7 @@ async def handle_command(conn: AsyncConnection, msg: dict):
     except Exception as e:
         result = str(e)
         status = "Error"
-    
+
     # Send response
     response = {
         "service": "command",
@@ -281,7 +292,16 @@ async def handle_update(conn: AsyncConnection, _msg: dict):  # pyright: ignore[r
     stop()
 
 
-async def heartbeat_sender(conn: AsyncConnection, interval: int):
+def build_heartbeat_msg(conn, interval: int, connectivity: Optional[Connectivity] = None) -> dict:
+    """HTB payload; carries p4/p6 presence flags when the probe knows them."""
+    msg = {"acks": conn.ackcount, "rtt": conn.rtts[-1], "interval": interval}
+    if connectivity is not None:
+        msg.update(connectivity.flags())
+    return msg
+
+
+async def heartbeat_sender(conn: AsyncConnection, interval: int,
+                           connectivity: Optional[Connectivity] = None):
     """Send periodic heartbeats, retrying the connection if it is not open.
 
     IPv6 connections that fail to open before their first successful send are
@@ -291,6 +311,7 @@ async def heartbeat_sender(conn: AsyncConnection, interval: int):
     Args:
         conn: Connection to send on
         interval: Heartbeat interval in seconds
+        connectivity: Optional probe whose presence flags are attached to every heartbeat
     """
     logger = logging.getLogger("hbc.heartbeat")
     IPV6_EARLY_FAIL_LIMIT = 3
@@ -328,11 +349,7 @@ async def heartbeat_sender(conn: AsyncConnection, interval: int):
                 continue
 
         try:
-            msg = {
-                "acks": conn.ackcount,
-                "rtt": conn.rtts[-1],
-                "interval": interval
-            }
+            msg = build_heartbeat_msg(conn, interval, connectivity)
             await conn.sendto(msg, "HTB")
 
         except asyncio.CancelledError:
@@ -430,14 +447,14 @@ async def plugin_collector_interval(
     interval: int
 ):
     """Collect plugins on a specific interval.
-    
+
     Args:
         conn: Connection to send on
         plugins: List of plugins to collect
         interval: Collection interval in seconds
     """
     logger = logging.getLogger(f"hbc.plugins.{interval}s")
-    
+
     while running:
         for plugin in plugins:
             try:
@@ -455,7 +472,7 @@ async def plugin_collector_interval(
                     f"Error collecting {plugin.name}: {e}",
                     exc_info=True
                 )
-        
+
         # Wait for next interval or shutdown event
         try:
             if shutdown_event:
@@ -482,22 +499,64 @@ def stop():
     """Stop the event loop."""
     global running
     running = False
-    
+
     # Set shutdown event to wake up sleeping tasks
     if shutdown_event:
         shutdown_event.set()
-    
+
     # Cancel all active tasks
     for task in active_tasks:
         if not task.done():
             task.cancel()
 
 
+async def dns_refresher(connections: List[AsyncConnection], hosts: List[str], port: int, af_filter: int,
+                        name: str, interval: int, connectivity: Optional[Connectivity], next_id: int):
+    """Re-resolve the server hosts every TTL seconds and follow address changes.
+
+    A failed lookup leaves the connections untouched. Addresses that disappear are re-pointed at
+    new ones where possible; the rest are retired, and surplus new ones get a connection.
+    """
+    logger = logging.getLogger("hbc.dns")
+    loop = asyncio.get_running_loop()
+    delay = await loop.run_in_executor(None, _ttl_delay, hosts, af_filter)
+    while running:
+        try:
+            if shutdown_event:
+                await asyncio.wait_for(shutdown_event.wait(), timeout=delay)
+                break
+            await asyncio.sleep(delay)
+        except asyncio.TimeoutError:
+            pass
+        resolved = await loop.run_in_executor(None, dnsttl.resolve, hosts, port, af_filter)
+        if resolved is None:
+            delay = dnsttl.FAILED_RETRY
+            continue
+        new, gone = dnsttl.retarget(connections, resolved)
+        for conn in gone:
+            if len(connections) > 1:
+                logger.info(f"Dropping server address {conn.addr}")
+                conn._dead = True
+                conn.close()
+                connections.remove(conn)
+        for af, addr in new:
+            conn = AsyncConnection(next_id, addr, port, af, name)
+            next_id += 1
+            logger.info(f"Adding server address {addr}")
+            connections.append(conn)
+            active_tasks.append(asyncio.create_task(heartbeat_sender(conn, interval, connectivity)))
+        delay = await loop.run_in_executor(None, _ttl_delay, hosts, af_filter)
+
+
+def _ttl_delay(hosts: List[str], af_filter: int) -> int:
+    return dnsttl.refresh_delay([dnsttl.lookup_ttl(h, af_filter) for h in hosts])
+
+
 async def cleanup(connections: List[AsyncConnection]):
     """Cleanup connections on shutdown."""
     logger = logging.getLogger("hbc.cleanup")
     logger.info("Cleaning up connections")
-    
+
     target = next((c for c in connections if c.transport), connections[0] if connections else None)
     # A SIGHUP restart is not a host shutdown, so don't announce one.
     if target and send_shutdown and not dorestart:
@@ -507,35 +566,35 @@ async def cleanup(connections: List[AsyncConnection]):
             logger.error(f"Error sending shutdown: {e}")
     for conn in connections:
         conn.close()
-    
+
     # Give messages time to send
     await asyncio.sleep(0.5)
 
 
 async def async_main(args, config):
     """Async main function."""
-    global running, shutdown_event, active_tasks, send_shutdown, allow_remote_command
+    global shutdown_event, active_tasks, send_shutdown, allow_remote_command
 
     # Create shutdown event
     shutdown_event = asyncio.Event()
     active_tasks = []
     allow_remote_command = bool(config.get("allow_remote_command", False))
-    
+
     logger = logging.getLogger("hbc.main")
-    
+
     # Setup
     iam = socket.gethostname()
     if args.name:
         iam = args.name
-    
+
     hb_hosts = args.hosts
     hb_port = config.get("hb_port", PORT)
     interval = config.get("interval", INTERVAL)
-    
+
     logger.info(f"hbc {__version__} on {iam} -> {hb_hosts} port={hb_port}, interval={interval}s")
     if allow_remote_command:
         logger.warning("allow_remote_command is true — CMD packets from the server will be executed")
-    
+
     af_filter = (socket.AF_INET if getattr(args, "ipv4_only", False)
                  else socket.AF_INET6 if getattr(args, "ipv6_only", False)
                  else 0)
@@ -574,7 +633,7 @@ async def async_main(args, config):
         return 1
 
     logger.info(f"Created {len(connections)} connections")
-    
+
     # Send boot/message if requested
     send_shutdown = False
     if args.boot or args.message:
@@ -585,27 +644,27 @@ async def async_main(args, config):
         if args.message:
             boot_msg["service"] = "service"
             boot_msg["msg"] = args.message
-        
+
         boot_msg["acks"] = 0
         target = next((c for c in connections if c.transport), connections[0])
         await target.sendto(boot_msg)
-        
+
         if args.message and not args.daemon:
             # Message-only mode
             await cleanup(connections)
             return 0
-    
+
     # Load plugins
     registry = PluginRegistry()
     loader = PluginLoader(registry)
-    
+
     plugin_dir = Path(__file__).parent / "plugins"
     if plugin_dir.exists():
         count = await loader.load_from_directory(plugin_dir, config)
         logger.info(f"Loaded {count} plugins")
     else:
         logger.warning(f"Plugin directory not found: {plugin_dir}")
-    
+
     # Setup signal handlers
     loop = asyncio.get_event_loop()
     for sig in (signal.SIGTERM, signal.SIGINT):
@@ -617,29 +676,41 @@ async def async_main(args, config):
         stop()
 
     loop.add_signal_handler(signal.SIGHUP, _sighup)
-    
+
     # Start async tasks
+    # Per-protocol internet presence probe (shared with the network_monitor plugin)
+    net_cfg = dict((config.get("plugins") or {}).get("network_monitor")
+                   or config.get("network_monitor") or {})
+    connectivity = None
+    if net_cfg.get("probe_enabled", True):
+        connectivity = Connectivity.from_config(net_cfg)
+        set_shared(connectivity)
+        active_tasks.append(asyncio.create_task(connectivity.run(shutdown_event)))
+
     # Heartbeat senders (one per connection)
     for conn in connections:
-        task = asyncio.create_task(heartbeat_sender(conn, interval))
+        task = asyncio.create_task(heartbeat_sender(conn, interval, connectivity))
         active_tasks.append(task)
-    
+
+    active_tasks.append(asyncio.create_task(
+        dns_refresher(connections, hb_hosts, hb_port, af_filter, iam, interval, connectivity, conn_id)))
+
     # Plugin collector (uses all connections, but we'll use first one)
     if connections and registry.get_enabled():
         task = asyncio.create_task(plugin_collector(connections[0], registry))
         active_tasks.append(task)
-    
+
     # Wait for stop or tasks to complete
     try:
         await asyncio.gather(*active_tasks, return_exceptions=True)
     except asyncio.CancelledError:
         logger.info("Tasks cancelled")
-    
+
     # Cleanup
     logger.info("Shutting down...")
     await cleanup(connections)
     await loader.unload_all()
-    
+
     return 0
 
 
@@ -657,11 +728,11 @@ def daemonize(
     except OSError as e:
         sys.stderr.write(f"fork #1 failed: {e}\n")
         os._exit(1)
-    
+
     os.chdir(working_dir)
     os.setsid()
     os.umask(0)
-    
+
     try:
         pid = os.fork()
         if pid > 0:
@@ -669,14 +740,14 @@ def daemonize(
     except OSError as e:
         sys.stderr.write(f"fork #2 failed: {e}\n")
         sys.exit(1)
-    
+
     sys.stdout.flush()
     sys.stderr.flush()
-    
+
     si = open(stdin, "r")
     so = open(stdout, "a+")
     se = open(stderr, "a+")
-    
+
     os.dup2(si.fileno(), sys.stdin.fileno())
     os.dup2(so.fileno(), sys.stdout.fileno())
     os.dup2(se.fileno(), sys.stderr.fileno())
@@ -768,18 +839,17 @@ def build_parser():
 
 def main(argv=None):
     """Main entry point."""
-    global running, dorestart
-    
+
     parser = build_parser()
     args = parser.parse_args(argv)
-    
+
     # Setup logging
     log_level = logging.WARNING
     if args.verbose:
         log_level = logging.INFO
     if args.debug:
         log_level = logging.DEBUG
-    
+
     logging.basicConfig(
         level=log_level,
         format="%(asctime)s %(name)s %(levelname)s: %(message)s",
@@ -788,14 +858,14 @@ def main(argv=None):
 
     # Load config
     config = load_config(args.configfile)
-    
+
     # Daemonize if requested
     if args.daemon:
         logging.info("Daemonizing...")
         daemonize()
         _reconfigure_logging_for_daemon(log_level)
         logging.info(f"hbc starting, sending heartbeat to {', '.join(args.hosts)}")
-    
+
     # Run async main
     try:
         exit_code = asyncio.run(async_main(args, config))
@@ -805,7 +875,7 @@ def main(argv=None):
     except Exception as e:
         logging.error(f"Fatal error: {e}", exc_info=True)
         exit_code = 1
-    
+
     # Handle restart
     if dorestart:
         logging.info("Restarting...")
@@ -813,7 +883,7 @@ def main(argv=None):
         # a SIGHUP restart is not a host reboot.
         restart_argv = [sys.argv[0]] + [a for a in sys.argv[1:] if a not in ("-b", "--boot")]
         os.execv(restart_argv[0], restart_argv)
-    
+
     sys.exit(exit_code)
 
 

hbd/client/pingutil.py

@@ -0,0 +1,73 @@
+"""System ping helpers shared by ping_monitor and the connectivity probe."""
+
+import asyncio
+import re
+import sys
+from typing import Dict, List, Optional
+
+_INF = float("inf")
+
+
+class PingUnavailable(Exception):
+    """The system ping binary could not be executed."""
+
+
+def ping_command(host: str, count: int, timeout: int, family: Optional[int] = None) -> List[str]:
+    """Build the platform ping command. family is 4, 6 or None (let ping decide)."""
+    if sys.platform == "win32":
+        cmd = ["ping", "-n", str(count), "-w", str(timeout * 1000)]
+        if family:
+            cmd.append(f"-{family}")
+        return cmd + [host]
+    if sys.platform == "darwin" and family:
+        if family == 6:
+            return ["ping6", "-c", str(count), host]
+        return ["ping", "-c", str(count), "-W", str(timeout * 1000), host]
+    cmd = ["ping", "-c", str(count), "-W", str(timeout)]
+    if family:
+        cmd.append(f"-{family}")
+    return cmd + [host]
+
+
+def parse_ping_output(output: str) -> Dict[str, float]:
+    """Parse rtt_min/avg/max/loss from ping output (Linux, macOS, BSD)."""
+    loss = 100.0
+    loss_match = re.search(r"(\d+(?:\.\d+)?)\s*%\s*packet\s*loss", output)
+    if loss_match:
+        loss = float(loss_match.group(1))
+
+    # Linux: "rtt min/avg/max/mdev = x/x/x/x ms"; macOS: "round-trip min/avg/max/stddev = ..."
+    rtt_match = re.search(
+        r"(?:rtt|round-trip)\s+min/avg/max/\S+\s*=\s*([\d.]+)/([\d.]+)/([\d.]+)",
+        output,
+    )
+    if rtt_match:
+        return {
+            "rtt_min": float(rtt_match.group(1)),
+            "rtt_avg": float(rtt_match.group(2)),
+            "rtt_max": float(rtt_match.group(3)),
+            "loss": loss,
+        }
+    return {"rtt_min": _INF, "rtt_avg": _INF, "rtt_max": _INF, "loss": loss}
+
+
+async def run_ping(host: str, count: int, timeout: int, family: Optional[int] = None) -> Dict[str, float]:
+    """Run ping and return parsed metrics.
+
+    Raises PingUnavailable if the ping binary is missing and asyncio.TimeoutError
+    (after killing the process) if it hangs.
+    """
+    cmd = ping_command(host, count, timeout, family)
+    try:
+        proc = await asyncio.create_subprocess_exec(
+            *cmd, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE,
+        )
+    except FileNotFoundError as e:
+        raise PingUnavailable(str(e)) from e
+    try:
+        stdout, _ = await asyncio.wait_for(proc.communicate(), timeout=timeout * count + 2)
+    except asyncio.TimeoutError:
+        proc.kill()
+        await proc.wait()
+        raise
+    return parse_ping_output(stdout.decode(errors="replace"))

hbd/client/plugin.py

@@ -17,7 +17,7 @@ import logging
 import sys
 from abc import ABC, abstractmethod
 from pathlib import Path
-from typing import Any, Dict, List, Optional, Type
+from typing import Any, Dict, List, Optional
 
 
 class Plugin(ABC):
@@ -31,13 +31,13 @@ class Plugin(ABC):
         enabled: Whether plugin is active (can be disabled via config)
         skip_reason: Set by plugin before returning False from initialize(); causes loader to log INFO instead of WARNING.
     """
-    
+
     name: str = ""
     version: str = "1.0.0"
     description: str = ""
     interval: int = 0
     enabled: bool = True
-    
+
     def __init__(self, config: Optional[Dict[str, Any]] = None):
         """Initialize plugin with optional configuration.
 
@@ -48,51 +48,51 @@ class Plugin(ABC):
         self.logger = logging.getLogger(f"plugin.{self.name}")
         self._initialized = False
         self.skip_reason: Optional[str] = None
-        
+
     @abstractmethod
     async def initialize(self) -> bool:
         """Initialize plugin (load resources, check dependencies).
-        
+
         Called once when plugin is loaded. Plugins should validate dependencies
         (e.g., check if psutil is available) and prepare any resources.
-        
+
         Returns:
             True if initialization succeeded, False otherwise
         """
         pass
-    
+
     @abstractmethod
     async def collect(self) -> Dict[str, Any]:
         """Collect data from the system.
-        
+
         This is the main method called on each collection interval. Should return
         a dictionary of key-value pairs representing the collected data.
-        
+
         Keys should be strings (metric names). Values can be:
         - Scalars: int, float, str, bool
         - Lists/dicts (will be serialized appropriately)
-        
+
         Returns:
             Dictionary of collected metrics, or empty dict on error
         """
         pass
-    
+
     async def cleanup(self) -> None:
         """Cleanup plugin resources before shutdown.
-        
+
         Called when plugin is being unloaded or on system shutdown.
         Override to release resources, close connections, etc.
         """
         pass
-    
+
     def validate_data(self, data: Dict[str, Any]) -> bool:
         """Validate collected data before sending to server.
-        
+
         Override to implement custom validation logic.
-        
+
         Args:
             data: Data returned from collect()
-            
+
         Returns:
             True if data is valid, False otherwise
         """
@@ -101,46 +101,46 @@ class Plugin(ABC):
 
 class InfoPlugin(Plugin):
     """Plugin for collecting static or rarely-changing information.
-    
+
     InfoPlugins collect data that doesn't change frequently:
     - OS name and version
     - Hardware specifications (CPU model, RAM size)
     - Network interface MAC addresses
-    
+
     Characteristics:
     - interval = 0 (collected once at startup by default)
     - Can specify interval > 0 for periodic refresh (e.g., check for hardware changes)
     - Data is cached and reused until next collection
     """
-    
+
     interval: int = 0  # Collect once at startup
-    
+
     def __init__(self, config: Optional[Dict[str, Any]] = None):
         super().__init__(config)
         self._cached_data: Optional[Dict[str, Any]] = None
-    
+
     async def get_cached_data(self) -> Optional[Dict[str, Any]]:
         """Get cached data if available (avoids re-collection).
-        
+
         Returns:
             Cached data dict, or None if not yet collected
         """
         return self._cached_data
-    
+
     async def collect(self) -> Dict[str, Any]:
         """Collect and cache static information."""
         if self._cached_data is None:
             self._cached_data = await self._collect_info()
         return self._cached_data
-    
+
     @abstractmethod
     async def _collect_info(self) -> Dict[str, Any]:
         """Internal method to perform actual data collection.
-        
+
         Override this method instead of collect() for InfoPlugins.
         """
         pass
-    
+
     def invalidate_cache(self) -> None:
         """Force re-collection on next collect() call."""
         self._cached_data = None
@@ -148,33 +148,33 @@ class InfoPlugin(Plugin):
 
 class MonitorPlugin(Plugin):
     """Plugin for collecting periodic monitoring data.
-    
+
     MonitorPlugins collect time-series metrics that change frequently:
     - CPU usage percentage
     - Memory consumption
     - Disk I/O statistics
     - Network traffic
-    
+
     Characteristics:
     - interval > 0 (e.g., 30 seconds for CPU, 60 for disk)
     - Collected continuously on schedule
     - Data includes timestamps for time-series tracking
     """
-    
+
     interval: int = 30  # Default: collect every 30 seconds
-    
+
     def __init__(self, config: Optional[Dict[str, Any]] = None):
         super().__init__(config)
         self._last_reading: Optional[Dict[str, Any]] = None
-    
+
     def get_last_reading(self) -> Optional[Dict[str, Any]]:
         """Get the last collected reading.
-        
+
         Returns:
             Last reading dict with timestamp, or None if not yet collected
         """
         return self._last_reading
-    
+
     async def collect(self) -> Dict[str, Any]:
         """Collect monitoring data and store as last reading."""
         data = await self._collect_metrics()
@@ -184,11 +184,11 @@ class MonitorPlugin(Plugin):
             data['_timestamp'] = time.time()
             self._last_reading = data
         return data
-    
+
     @abstractmethod
     async def _collect_metrics(self) -> Dict[str, Any]:
         """Internal method to perform actual metric collection.
-        
+
         Override this method instead of collect() for MonitorPlugins.
         """
         pass
@@ -196,38 +196,38 @@ class MonitorPlugin(Plugin):
 
 class PluginRegistry:
     """Registry for managing loaded plugins.
-    
+
     Maintains a collection of loaded plugins and provides methods to
     query plugins by name, type, or interval.
     """
-    
+
     def __init__(self):
         self._plugins: Dict[str, Plugin] = {}
         self.logger = logging.getLogger("plugin.registry")
-    
+
     def register(self, plugin: Plugin) -> bool:
         """Register a plugin instance.
-        
+
         Args:
             plugin: Plugin instance to register
-            
+
         Returns:
             True if registered successfully, False if name conflict
         """
         if plugin.name in self._plugins:
             self.logger.error(f"Plugin '{plugin.name}' already registered")
             return False
-        
+
         self._plugins[plugin.name] = plugin
         self.logger.info(f"Registered plugin: {plugin.name} v{plugin.version}")
         return True
-    
+
     def unregister(self, name: str) -> bool:
         """Unregister a plugin by name.
-        
+
         Args:
             name: Plugin name to unregister
-            
+
         Returns:
             True if unregistered, False if not found
         """
@@ -236,43 +236,43 @@ class PluginRegistry:
             self.logger.info(f"Unregistered plugin: {name}")
             return True
         return False
-    
+
     def get(self, name: str) -> Optional[Plugin]:
         """Get plugin by name.
-        
+
         Args:
             name: Plugin name
-            
+
         Returns:
             Plugin instance or None if not found
         """
         return self._plugins.get(name)
-    
+
     def get_all(self) -> List[Plugin]:
         """Get all registered plugins."""
         return list(self._plugins.values())
-    
+
     def get_enabled(self) -> List[Plugin]:
         """Get all enabled plugins."""
         return [p for p in self._plugins.values() if p.enabled]
-    
-    def get_by_type(self, plugin_type: Type[Plugin]) -> List[Plugin]:
+
+    def get_by_type(self, plugin_type: type) -> List[Plugin]:
         """Get all plugins of a specific type.
-        
+
         Args:
             plugin_type: Plugin class (InfoPlugin or MonitorPlugin)
-            
+
         Returns:
             List of plugins matching the type
         """
         return [p for p in self._plugins.values() if isinstance(p, plugin_type)]
-    
+
     def get_by_interval(self, interval: int) -> List[Plugin]:
         """Get all plugins with a specific collection interval.
-        
+
         Args:
             interval: Interval in seconds (0 for one-time collection)
-            
+
         Returns:
             List of plugins with matching interval
         """
@@ -281,51 +281,51 @@ class PluginRegistry:
 
 class PluginLoader:
     """Load plugins from filesystem and instantiate them.
-    
+
     Scans plugin directories for Python modules containing Plugin subclasses,
     loads them dynamically, and registers them with the PluginRegistry.
     """
-    
+
     def __init__(self, registry: PluginRegistry):
         self.registry = registry
         self.logger = logging.getLogger("plugin.loader")
         self._loaded_modules: Dict[str, Any] = {}
-    
+
     async def load_from_directory(
         self,
         directory: Path,
         config: Optional[Dict[str, Any]] = None
     ) -> int:
         """Load all plugins from a directory.
-        
+
         Scans for .py files, imports them, finds Plugin subclasses,
         instantiates them with config, initializes, and registers.
-        
+
         Args:
             directory: Path to plugin directory
             config: Configuration dict (may contain per-plugin config)
-            
+
         Returns:
             Number of plugins successfully loaded
         """
         if not directory.exists() or not directory.is_dir():
             self.logger.warning(f"Plugin directory not found: {directory}")
             return 0
-        
+
         loaded_count = 0
         raw_config = config or {}
         # Per-plugin config lives under the 'plugins' key or at top-level.
         # CLIENT_DEFAULTS seeds "plugins": {} so the key always exists; check
         # both the subdict and top-level so that either layout in .hbc.yaml works.
         plugins_subconfig = raw_config.get("plugins", {})
-        
+
         # Scan for Python files
         for plugin_file in directory.glob("*.py"):
             if plugin_file.name.startswith("_"):
                 continue  # Skip __init__.py and private modules
-            
+
             self.logger.debug(f"Processing plugin file: {plugin_file.name}")
-            
+
             try:
                 # Load module dynamically
                 module_name = f"plugins.{plugin_file.stem}"
@@ -333,17 +333,17 @@ class PluginLoader:
                 if not spec or not spec.loader:
                     self.logger.warning(f"Could not create spec for {plugin_file}")
                     continue
-                
+
                 module = importlib.util.module_from_spec(spec)
                 sys.modules[module_name] = module
                 spec.loader.exec_module(module)
                 self._loaded_modules[module_name] = module
-                
+
                 self.logger.debug(f"Loaded module: {module_name}")
-                
+
                 # Track which plugin classes we've already processed to avoid duplicates
                 processed_classes = set()
-                
+
                 # Find Plugin subclasses in module
                 for name, obj in inspect.getmembers(module, inspect.isclass):
                     # Skip base classes and non-Plugin classes
@@ -353,15 +353,15 @@ class PluginLoader:
                     if not issubclass(obj, Plugin):
                         self.logger.debug(f"Skipping non-Plugin class: {name}")
                         continue
-                    
+
                     # Skip if we've already processed this class (handles module-level aliases)
                     if id(obj) in processed_classes:
                         self.logger.debug(f"Skipping duplicate reference to: {obj.__name__}")
                         continue
                     processed_classes.add(id(obj))
-                    
+
                     self.logger.debug(f"Found plugin class: {name}")
-                    
+
                     # Instantiate plugin with config — check plugins subdict first,
                     # then top-level keys (e.g. nagios_runner: ... at root of config).
                     plugin_instance_config = dict(plugins_subconfig.get(obj.name) or raw_config.get(obj.name) or {})
@@ -369,7 +369,7 @@ class PluginLoader:
                     if "owner" in raw_config and "owner" not in plugin_instance_config:
                         plugin_instance_config["owner"] = raw_config["owner"]
                     plugin = obj(config=plugin_instance_config)
-                    
+
                     # Initialize plugin
                     try:
                         initialized = await plugin.initialize()
@@ -389,7 +389,7 @@ class PluginLoader:
                             exc_info=True
                         )
                         continue
-                    
+
                     # Register with registry
                     if self.registry.register(plugin):
                         loaded_count += 1
@@ -397,15 +397,15 @@ class PluginLoader:
                             f"Loaded plugin: {plugin.name} v{plugin.version} "
                             f"(interval: {plugin.interval}s)"
                         )
-                
+
             except Exception as e:
                 self.logger.error(
                     f"Error loading plugin from {plugin_file}: {e}",
                     exc_info=True
                 )
-        
+
         return loaded_count
-    
+
     async def unload_all(self) -> None:
         """Unload all plugins and cleanup resources."""
         for plugin in self.registry.get_all():
@@ -417,7 +417,7 @@ class PluginLoader:
                     exc_info=True
                 )
             self.registry.unregister(plugin.name)
-        
+
         # Remove loaded modules
         for module_name in self._loaded_modules:
             if module_name in sys.modules:

hbd/client/plugins/cpu_monitor.py

@@ -5,8 +5,6 @@ load average, and process counts.
 """
 
 from typing import Any, Dict, Optional
-import sys
-from pathlib import Path
 
 # Import from parent package
 from hbd.client.plugin import MonitorPlugin
@@ -14,7 +12,7 @@ from hbd.client.plugin import MonitorPlugin
 
 class CPUMonitorPlugin(MonitorPlugin):
     """Monitor CPU usage and load.
-    
+
     Collects:
     - Overall CPU usage percentage
     - Per-core CPU usage (if enabled in config)
@@ -22,28 +20,28 @@ class CPUMonitorPlugin(MonitorPlugin):
     - Process count
     - CPU frequency (if available)
     """
-    
+
     name = "cpu_monitor"
     version = "1.0.0"
     description = "CPU usage and load monitoring"
     interval = 300  # MonitorPlugin: collect every 5 minutes by default
-    
+
     def __init__(self, config: Optional[Dict[str, Any]] = None):
         super().__init__(config)
-        self.psutil = None
+        self.psutil: Any = None
         self.per_core = config.get("per_core", False) if config else False
         self.interval = config.get("interval", 300) if config else 300
-    
+
     async def initialize(self) -> bool:
         """Initialize the CPU monitor plugin.
-        
+
         Checks if psutil is available.
-        
+
         Returns:
             True if psutil is available, False otherwise
         """
         self.logger.info(f"Initializing {self.name} plugin")
-        
+
         try:
             import psutil
             self.psutil = psutil
@@ -54,23 +52,23 @@ class CPUMonitorPlugin(MonitorPlugin):
                 "psutil module not available. Install with: pip install psutil"
             )
             return False
-    
+
     async def _collect_metrics(self) -> Dict[str, Any]:
         """Collect CPU metrics.
-        
+
         Returns:
             Dictionary with CPU metrics
         """
         if not self.psutil:
             return {}
-        
+
         try:
             data = {}
-            
+
             # Overall CPU usage percentage (non-blocking, interval=1 for accuracy)
             # Note: first call to cpu_percent() returns 0.0, subsequent calls work correctly
             data["cpu_percent"] = self.psutil.cpu_percent(interval=1)
-            
+
             # Per-core CPU usage (if enabled)
             if self.per_core:
                 per_core_percents = self.psutil.cpu_percent(interval=0, percpu=True)
@@ -79,7 +77,7 @@ class CPUMonitorPlugin(MonitorPlugin):
             else:
                 # Just report core count
                 data["cpu_core_count"] = self.psutil.cpu_count()
-            
+
             # Load average (Unix-like systems only)
             try:
                 load_avg = self.psutil.getloadavg()
@@ -89,13 +87,13 @@ class CPUMonitorPlugin(MonitorPlugin):
             except (AttributeError, OSError):
                 # Not available on Windows
                 pass
-            
+
             # Process count
             try:
                 data["process_count"] = len(self.psutil.pids())
             except Exception as e:
                 self.logger.warning(f"Could not get process count: {e}")
-            
+
             # CPU frequency (if available)
             try:
                 freq = self.psutil.cpu_freq()
@@ -107,7 +105,7 @@ class CPUMonitorPlugin(MonitorPlugin):
                 # Not available on all systems, or may fail on FreeBSD with sysctl issues
                 self.logger.debug(f"CPU frequency not available: {e}")
                 pass
-            
+
             # CPU times (user, system, idle, etc.)
             try:
                 cpu_times = self.psutil.cpu_times_percent(interval=0)
@@ -125,12 +123,12 @@ class CPUMonitorPlugin(MonitorPlugin):
                 data["uptime_seconds"] = int(time.time() - self.psutil.boot_time())
             except Exception as e:
                 self.logger.debug(f"Could not get uptime: {e}")
-            
+
             self.logger.debug(
                 f"Collected CPU metrics: {data.get('cpu_percent', 'N/A')}% usage"
             )
             return data
-            
+
         except Exception as e:
             self.logger.error(f"Error collecting CPU metrics: {e}", exc_info=True)
             return {}

hbd/client/plugins/disk_monitor.py

@@ -5,12 +5,12 @@ Collects disk usage and I/O statistics using psutil.
 """
 
 import logging
-from typing import Dict, Any, Optional, List
+from typing import Dict, Any, Optional
 
 try:
     import psutil
 except ImportError:
-    psutil = None
+    psutil = None  # type: ignore[assignment]
 
 from hbd.client.plugin import MonitorPlugin
 
@@ -20,27 +20,27 @@ logger = logging.getLogger(__name__)
 class DiskMonitorPlugin(MonitorPlugin):
     """
     Monitor disk usage and I/O statistics.
-    
+
     Collects:
     - Disk partition information
     - Disk usage per partition (total, used, free, percent)
     - Disk I/O counters (read/write bytes, read/write count)
     - Disk I/O time statistics
-    
+
     Configuration:
         interval: Collection interval in seconds (default: 300)
         partitions: List of mount points to monitor (default: all)
         include_io: Include disk I/O statistics (default: True)
         exclude_types: List of filesystem types to exclude (default: tmpfs, devtmpfs, squashfs)
     """
-    
+
     name = "disk_monitor"
     interval = 300  # Collect every 5 minutes by default
-    
+
     def __init__(self, config: Optional[Dict[str, Any]] = None):
         """
         Initialize the disk monitor plugin.
-        
+
         Args:
             config: Optional configuration dict with keys:
                    - interval: Collection interval in seconds (default: 300)
@@ -53,34 +53,34 @@ class DiskMonitorPlugin(MonitorPlugin):
         self.include_io = self.config.get('include_io', True)
         self.exclude_types = set(self.config.get('exclude_types', ['tmpfs', 'devtmpfs', 'squashfs']))
         self.interval = self.config.get('interval', 300)
-        
+
         if psutil is None:
             raise ImportError("psutil library is required for disk_monitor plugin")
-        
+
         # Store previous I/O counters for delta calculation
-        self._prev_io = {}
-    
+        self._prev_io: Dict[str, Any] = {}
+
     async def initialize(self):
         """Initialize the plugin (check psutil availability)."""
         if psutil is None:
             logger.error("psutil not available - disk_monitor cannot run")
             return False
-        
+
         logger.info(f"Disk monitor initialized (interval: {self.interval}s, io: {self.include_io})")
-        
+
         # Initialize I/O counters if available
         if self.include_io:
             try:
                 self._prev_io = psutil.disk_io_counters(perdisk=True)
             except Exception as e:
                 logger.warning(f"Could not initialize disk I/O counters: {e}")
-        
+
         return True
-    
+
     async def collect(self) -> Dict[str, Any]:
         """
         Collect current disk statistics.
-        
+
         Returns:
             Dictionary with disk metrics organized by partition:
             - partitions: Dict of partition data, keyed by mount point
@@ -103,7 +103,7 @@ class DiskMonitorPlugin(MonitorPlugin):
         if psutil is None:
             logger.error("psutil not available")
             return {}
-        
+
         try:
             data = await self._collect_metrics()
             logger.debug(f"Collected disk metrics: {len(data.get('partitions', {}))} partitions")
@@ -111,24 +111,24 @@ class DiskMonitorPlugin(MonitorPlugin):
         except Exception as e:
             logger.error(f"Error collecting disk metrics: {e}")
             return {"error": str(e)}
-    
+
     async def _collect_metrics(self) -> Dict[str, Any]:
         """Collect disk metrics from psutil."""
-        metrics = {}
-        
+        metrics: Dict[str, Any] = {}
+
         # Collect partition usage
         partitions_data = {}
         partitions = psutil.disk_partitions(all=False)
-        
+
         for partition in partitions:
             # Skip unwanted filesystem types
             if partition.fstype in self.exclude_types:
                 continue
-            
+
             # Skip if we're only monitoring specific partitions
             if self.partitions and partition.mountpoint not in self.partitions:
                 continue
-            
+
             try:
                 usage = psutil.disk_usage(partition.mountpoint)
                 partitions_data[partition.mountpoint] = {
@@ -145,15 +145,15 @@ class DiskMonitorPlugin(MonitorPlugin):
             except Exception as e:
                 logger.warning(f"Error reading {partition.mountpoint}: {e}")
                 continue
-        
+
         metrics['partitions'] = partitions_data
-        
+
         # Collect I/O statistics
         if self.include_io:
             try:
                 io_counters = psutil.disk_io_counters(perdisk=True)
                 io_data = {}
-                
+
                 for disk_name, counters in io_counters.items():
                     disk_stats = {
                         'read_count': counters.read_count,
@@ -161,7 +161,7 @@ class DiskMonitorPlugin(MonitorPlugin):
                         'read_bytes': counters.read_bytes,
                         'write_bytes': counters.write_bytes,
                     }
-                    
+
                     # Add time statistics if available
                     if hasattr(counters, 'read_time'):
                         disk_stats['read_time'] = counters.read_time
@@ -169,7 +169,7 @@ class DiskMonitorPlugin(MonitorPlugin):
                         disk_stats['write_time'] = counters.write_time
                     if hasattr(counters, 'busy_time'):
                         disk_stats['busy_time'] = counters.busy_time
-                    
+
                     # Calculate deltas from previous collection
                     if disk_name in self._prev_io:
                         prev = self._prev_io[disk_name]
@@ -177,19 +177,19 @@ class DiskMonitorPlugin(MonitorPlugin):
                         disk_stats['write_bytes_delta'] = counters.write_bytes - prev.write_bytes
                         disk_stats['read_count_delta'] = counters.read_count - prev.read_count
                         disk_stats['write_count_delta'] = counters.write_count - prev.write_count
-                    
+
                     io_data[disk_name] = disk_stats
-                
+
                 metrics['io_counters'] = io_data
-                
+
                 # Store current counters for next delta calculation
                 self._prev_io = io_counters
-                
+
             except Exception as e:
                 logger.warning(f"Could not collect disk I/O statistics: {e}")
-        
+
         return metrics
-    
+
     async def cleanup(self):
         """Cleanup (nothing to do for this plugin)."""
         logger.info("Disk monitor cleanup")

hbd/client/plugins/filesystem_info.py

@@ -10,7 +10,7 @@ from typing import Dict, Any, Optional
 try:
     import psutil
 except ImportError:
-    psutil = None
+    psutil = None  # type: ignore[assignment]
 
 from hbd.client.plugin import InfoPlugin
 
@@ -20,29 +20,29 @@ logger = logging.getLogger(__name__)
 class FilesystemInfoPlugin(InfoPlugin):
     """
     Collect filesystem and partition information.
-    
+
     This is an InfoPlugin that collects static information once during startup.
-    
+
     By default, only reports physical mounted filesystems (e.g., ext4, xfs, btrfs).
     Set include_pseudo=True to also include pseudo filesystems (proc, sysfs, tmpfs, etc.).
-    
+
     Collects:
     - List of mounted filesystems
     - Partition details (device, mount point, filesystem type, options)
     - Filesystem capabilities and features
-    
+
     Configuration:
         include_pseudo: Include pseudo/virtual filesystems (default: False)
         exclude_types: List of additional filesystem types to exclude (default: [])
     """
-    
+
     name = "filesystem_info"
     interval = 0  # InfoPlugin - collect once
-    
+
     def __init__(self, config: Optional[Dict[str, Any]] = None):
         """
         Initialize the filesystem info plugin.
-        
+
         Args:
             config: Optional configuration dict with keys:
                    - include_pseudo: Include pseudo/virtual filesystems (default: False)
@@ -53,25 +53,25 @@ class FilesystemInfoPlugin(InfoPlugin):
         # By default, no exclusions since all=False filters most pseudo filesystems
         # Users can add specific types to exclude if needed
         self.exclude_types = set(self.config.get('exclude_types', []))
-        
+
         if psutil is None:
             raise ImportError("psutil library is required for filesystem_info plugin")
-    
+
     async def initialize(self):
         """Initialize the plugin (check psutil availability)."""
         if psutil is None:
             logger.error("psutil not available - filesystem_info cannot run")
             return False
-        
+
         logger.info(f"Filesystem info initialized (pseudo: {self.include_pseudo})")
         return True
-    
+
     async def collect(self) -> Dict[str, Any]:
         """
         Collect filesystem information.
-        
+
         Returns only physical mounted filesystems by default.
-        
+
         Returns:
             Dictionary with filesystem data:
             - filesystems: List of filesystem dictionaries:
@@ -87,7 +87,7 @@ class FilesystemInfoPlugin(InfoPlugin):
         if psutil is None:
             logger.error("psutil not available")
             return {}
-        
+
         try:
             data = await self._collect_info()
             logger.info(f"Collected filesystem info: {len(data.get('filesystems', []))} filesystems")
@@ -95,30 +95,30 @@ class FilesystemInfoPlugin(InfoPlugin):
         except Exception as e:
             logger.error(f"Error collecting filesystem info: {e}")
             return {"error": str(e)}
-    
+
     async def _collect_info(self) -> Dict[str, Any]:
         """Collect filesystem information from psutil."""
-        info = {}
-        filesystems = []
+        info: Dict[str, Any] = {}
+        filesystems: list = []
         filesystem_types = set()
-        
+
         # Get mounted disk partitions
         # all=False returns only physical devices (real mounted filesystems)
         # all=True would include pseudo filesystems (proc, sysfs, etc.)
         partitions = psutil.disk_partitions(all=self.include_pseudo)
-        
+
         for partition in partitions:
             # Additional filtering if exclude_types is specified
             if partition.fstype in self.exclude_types:
                 continue
-            
-            fs_info = {
+
+            fs_info: Dict[str, Any] = {
                 'device': partition.device,
                 'mountpoint': partition.mountpoint,
                 'fstype': partition.fstype,
                 'opts': partition.opts,
             }
-            
+
             # Try to get filesystem capabilities
             try:
                 # Get path configuration for this mount point
@@ -141,14 +141,14 @@ class FilesystemInfoPlugin(InfoPlugin):
                         pass
             except Exception as e:
                 logger.debug(f"Could not get pathconf for {partition.mountpoint}: {e}")
-            
+
             filesystems.append(fs_info)
             filesystem_types.add(partition.fstype)
-        
+
         info['filesystems'] = filesystems
         info['filesystem_types'] = sorted(list(filesystem_types))
         info['mount_count'] = len(filesystems)
-        
+
         # Add some additional filesystem statistics
         try:
             # Get boot time (useful for determining filesystem mount times)
@@ -156,9 +156,9 @@ class FilesystemInfoPlugin(InfoPlugin):
             info['boot_time'] = boot_time
         except Exception as e:
             logger.debug(f"Could not get boot time: {e}")
-        
+
         return info
-    
+
     async def cleanup(self):
         """Cleanup (nothing to do for this plugin)."""
         logger.info("Filesystem info cleanup")

hbd/client/plugins/memory_monitor.py

@@ -10,7 +10,7 @@ from typing import Dict, Any, Optional
 try:
     import psutil
 except ImportError:
-    psutil = None
+    psutil = None  # type: ignore[assignment]
 
 from hbd.client.plugin import MonitorPlugin
 
@@ -32,31 +32,32 @@ def _zfs_arc_bytes() -> int:
         pass
     return 0
 
+
 logger = logging.getLogger(__name__)
 
 
 class MemoryMonitorPlugin(MonitorPlugin):
     """
     Monitor memory and swap usage.
-    
+
     Collects:
     - Physical memory (RAM) usage and statistics
     - Virtual memory details
     - Swap memory usage and statistics
     - Memory available for applications
-    
+
     Configuration:
         interval: Collection interval in seconds (default: 300)
         include_swap: Include swap statistics (default: True)
     """
-    
+
     name = "memory_monitor"
     interval = 300  # Collect every 5 minutes by default
-    
+
     def __init__(self, config: Optional[Dict[str, Any]] = None):
         """
         Initialize the memory monitor plugin.
-        
+
         Args:
             config: Optional configuration dict with keys:
                    - interval: Collection interval in seconds (default: 300)
@@ -65,23 +66,23 @@ class MemoryMonitorPlugin(MonitorPlugin):
         super().__init__(config)
         self.include_swap = self.config.get('include_swap', True)
         self.interval = self.config.get('interval', 300)
-        
+
         if psutil is None:
             raise ImportError("psutil library is required for memory_monitor plugin")
-    
+
     async def initialize(self):
         """Initialize the plugin (check psutil availability)."""
         if psutil is None:
             logger.error("psutil not available - memory_monitor cannot run")
             return False
-        
+
         logger.info(f"Memory monitor initialized (interval: {self.interval}s, swap: {self.include_swap})")
         return True
-    
+
     async def collect(self) -> Dict[str, Any]:
         """
         Collect current memory statistics.
-        
+
         Returns:
             Dictionary with memory metrics:
             - memory_total: Total physical RAM in bytes
@@ -104,7 +105,7 @@ class MemoryMonitorPlugin(MonitorPlugin):
         if psutil is None:
             logger.error("psutil not available")
             return {}
-        
+
         try:
             data = await self._collect_metrics()
             logger.debug(f"Collected memory metrics: {len(data)} fields")
@@ -112,11 +113,11 @@ class MemoryMonitorPlugin(MonitorPlugin):
         except Exception as e:
             logger.error(f"Error collecting memory metrics: {e}")
             return {"error": str(e)}
-    
+
     async def _collect_metrics(self) -> Dict[str, Any]:
         """Collect memory metrics from psutil."""
-        metrics = {}
-        
+        metrics: Dict[str, Any] = {}
+
         # Virtual (physical) memory statistics
         vmem = psutil.virtual_memory()
 
@@ -134,7 +135,7 @@ class MemoryMonitorPlugin(MonitorPlugin):
         metrics['memory_used'] = used
         metrics['memory_free'] = vmem.free
         metrics['memory_percent'] = percent
-        
+
         # Platform-specific memory details
         if hasattr(vmem, 'active'):
             metrics['memory_active'] = vmem.active
@@ -146,7 +147,7 @@ class MemoryMonitorPlugin(MonitorPlugin):
             metrics['memory_cached'] = vmem.cached
         if hasattr(vmem, 'shared'):
             metrics['memory_shared'] = vmem.shared
-        
+
         # Swap memory statistics
         if self.include_swap:
             try:
@@ -155,7 +156,7 @@ class MemoryMonitorPlugin(MonitorPlugin):
                 metrics['swap_used'] = swap.used
                 metrics['swap_free'] = swap.free
                 metrics['swap_percent'] = swap.percent
-                
+
                 # Swap in/out counters (may not be available on all platforms)
                 if hasattr(swap, 'sin'):
                     metrics['swap_sin'] = swap.sin
@@ -163,9 +164,9 @@ class MemoryMonitorPlugin(MonitorPlugin):
                     metrics['swap_sout'] = swap.sout
             except Exception as e:
                 logger.warning(f"Could not collect swap statistics: {e}")
-        
+
         return metrics
-    
+
     async def cleanup(self):
         """Cleanup (nothing to do for this plugin)."""
         logger.info("Memory monitor cleanup")

hbd/client/plugins/nagios_runner.py

@@ -43,10 +43,10 @@ STATUS_NAMES = {
 
 class NagiosRunnerPlugin(MonitorPlugin):
     """Run Nagios-compatible monitoring plugins.
-    
+
     This plugin executes external Nagios plugins and collects their output,
     including status codes, messages, and performance data.
-    
+
     Configuration:
         interval: Collection interval in seconds (default: 300)
         commands: List of command definitions with 'name' and 'command' keys
@@ -62,20 +62,20 @@ class NagiosRunnerPlugin(MonitorPlugin):
             - name: check_load
               command: /usr/lib/nagios/plugins/check_load -w 5,4,3 -c 10,8,6
     """
-    
+
     name = "nagios_runner"
     version = "1.0.0"
     description = "Execute Nagios-compatible monitoring plugins"
     interval = 300  # MonitorPlugin: collect every 5 minutes by default
-    
+
     def __init__(self, config: Optional[Dict[str, Any]] = None):
         super().__init__(config)
-        
+
         # Extract configuration
         self.commands: List[Dict[str, str]] = config.get("commands", []) if config else []
         self.timeout: int = config.get("timeout", 30) if config else 30
         self.interval = config.get("interval", 300) if config else 300
-    
+
     async def initialize(self) -> bool:
         """Initialize the Nagios runner plugin.
 
@@ -117,14 +117,14 @@ class NagiosRunnerPlugin(MonitorPlugin):
                     )
 
         return True
-    
+
     async def _collect_metrics(self) -> Dict[str, Any]:
         """Collect metrics from all configured Nagios plugins.
-        
+
         Returns:
             Dictionary with results from all plugins
         """
-        results = {}
+        results: Dict[str, Any] = {}
 
         for cmd_config in self.commands:
             name = cmd_config.get("name")
@@ -159,7 +159,7 @@ class NagiosRunnerPlugin(MonitorPlugin):
                 results[f"{name}_output"] = str(e)
 
         return results
-    
+
     async def _run_nagios_plugin(
         self,
         command: str
@@ -181,7 +181,8 @@ class NagiosRunnerPlugin(MonitorPlugin):
                 self.logger.error(f"Command timed out: {command}")
                 return NAGIOS_UNKNOWN, f"Command timed out after {self.timeout}s", {}
 
-            status_code = proc.returncode
+            # communicate() has finished, so the process has exited and returncode is set
+            status_code = proc.returncode if proc.returncode is not None else NAGIOS_UNKNOWN
 
             if status_code < 0:
                 return NAGIOS_UNKNOWN, f"Process killed by signal {-status_code}", {}
@@ -210,33 +211,33 @@ class NagiosRunnerPlugin(MonitorPlugin):
         except Exception as e:
             self.logger.error(f"Error executing command: {e}")
             return NAGIOS_UNKNOWN, f"Execution error: {str(e)}", {}
-    
+
     def _parse_perfdata(self, output: str) -> Dict[str, Any]:
         """Parse Nagios performance data from plugin output.
-        
+
         Nagios performance data format:
         'label'=value[UOM];[warn];[crit];[min];[max]
-        
+
         Multiple metrics separated by spaces.
-        
+
         Args:
             output: Plugin output string
-            
+
         Returns:
             Dictionary of metric_name: value
         """
-        perfdata = {}
-        
+        perfdata: Dict[str, Any] = {}
+
         # Performance data comes after the pipe character
         if '|' not in output:
             return perfdata
-        
+
         perf_section = output.split('|', 1)[1].strip()
-        
+
         # Regex to match performance data format
         # Matches: 'label'=value or label=value
         perf_regex = r"'?([^'=]+)'?=([\d.]+)([a-zA-Z%]*);?([\d.]*);?([\d.]*);?([\d.]*);?([\d.]*)"
-        
+
         for match in re.finditer(perf_regex, perf_section):
             label = match.group(1).strip()
             value_str = match.group(2)
@@ -245,43 +246,43 @@ class NagiosRunnerPlugin(MonitorPlugin):
             crit = match.group(5)
             min_val = match.group(6)
             max_val = match.group(7)
-            
+
             # Convert value to float
             try:
                 value = float(value_str)
             except ValueError:
                 continue
-            
+
             # Store the value
             perfdata[label] = value
-            
+
             # Optionally store UOM as separate field
             if uom:
                 perfdata[f"{label}_uom"] = uom
-            
+
             # Store thresholds if present
             if warn:
                 try:
                     perfdata[f"{label}_warn"] = float(warn)
                 except ValueError:
                     pass
-            
+
             if crit:
                 try:
                     perfdata[f"{label}_crit"] = float(crit)
                 except ValueError:
                     pass
-            
+
             if min_val:
                 try:
                     perfdata[f"{label}_min"] = float(min_val)
                 except ValueError:
                     pass
-            
+
             if max_val:
                 try:
                     perfdata[f"{label}_max"] = float(max_val)
                 except ValueError:
                     pass
-        
+
         return perfdata

hbd/client/plugins/network_monitor.py

@@ -5,14 +5,15 @@ Collects network interface statistics and connection information using psutil.
 """
 
 import logging
-from typing import Dict, Any, Optional, List
+from typing import Dict, Any, Optional
 
 try:
     import psutil
 except ImportError:
-    psutil = None
+    psutil = None  # type: ignore[assignment]
 
 from hbd.client.plugin import MonitorPlugin
+from hbd.client.connectivity import get_shared
 
 logger = logging.getLogger(__name__)
 
@@ -20,27 +21,31 @@ logger = logging.getLogger(__name__)
 class NetworkMonitorPlugin(MonitorPlugin):
     """
     Monitor network interface statistics and connections.
-    
+
     Collects:
     - Network interface I/O counters (bytes sent/received, packets, errors, drops)
     - Per-interface statistics
     - Network connection counts by state
     - Interface addresses and configuration
-    
+
     Configuration:
         interval: Collection interval in seconds (default: 300)
         interfaces: List of interfaces to monitor (default: all)
         include_connections: Include connection statistics (default: True)
         include_addresses: Include interface addresses (default: False)
+        probe_enabled: Run the per-protocol internet presence probe (default: True)
+        probe_v4: IPv4 address to ping (default: 1.1.1.1)
+        probe_v6: IPv6 address to ping (default: 2606:4700:4700::1111)
+        probe_interval: Seconds between ICMP probes while healthy (default: 60)
     """
-    
+
     name = "network_monitor"
     interval = 300  # Collect every 5 minutes by default
-    
+
     def __init__(self, config: Optional[Dict[str, Any]] = None):
         """
         Initialize the network monitor plugin.
-        
+
         Args:
             config: Optional configuration dict with keys:
                    - interval: Collection interval in seconds (default: 300)
@@ -53,34 +58,34 @@ class NetworkMonitorPlugin(MonitorPlugin):
         self.include_connections = self.config.get('include_connections', True)
         self.include_addresses = self.config.get('include_addresses', False)
         self.interval = self.config.get('interval', 300)
-        
+
         if psutil is None:
             raise ImportError("psutil library is required for network_monitor plugin")
-        
+
         # Store previous I/O counters for delta calculation
-        self._prev_io = {}
-    
+        self._prev_io: Dict[str, Any] = {}
+
     async def initialize(self):
         """Initialize the plugin (check psutil availability)."""
         if psutil is None:
             logger.error("psutil not available - network_monitor cannot run")
             return False
-        
+
         logger.info(f"Network monitor initialized (interval: {self.interval}s, "
-                   f"connections: {self.include_connections})")
-        
+                    f"connections: {self.include_connections})")
+
         # Initialize I/O counters
         try:
             self._prev_io = psutil.net_io_counters(pernic=True)
         except Exception as e:
             logger.warning(f"Could not initialize network I/O counters: {e}")
-        
+
         return True
-    
+
     async def collect(self) -> Dict[str, Any]:
         """
         Collect current network statistics.
-        
+
         Returns:
             Dictionary with network metrics:
             - interfaces: Dict of interface statistics, keyed by interface name
@@ -103,11 +108,12 @@ class NetworkMonitorPlugin(MonitorPlugin):
               - etc.
             - addresses: Interface address information (if include_addresses)
               - Dict keyed by interface name with address details
+            - connectivity: per-protocol presence {ipv4, ipv6} -> {present, addr, gateway, rtt}
         """
         if psutil is None:
             logger.error("psutil not available")
             return {}
-        
+
         try:
             data = await self._collect_metrics()
             logger.debug(f"Collected network metrics: {len(data.get('interfaces', {}))} interfaces")
@@ -115,21 +121,21 @@ class NetworkMonitorPlugin(MonitorPlugin):
         except Exception as e:
             logger.error(f"Error collecting network metrics: {e}")
             return {"error": str(e)}
-    
+
     async def _collect_metrics(self) -> Dict[str, Any]:
         """Collect network metrics from psutil."""
-        metrics = {}
-        
+        metrics: Dict[str, Any] = {}
+
         # Collect per-interface I/O counters
         try:
             io_counters = psutil.net_io_counters(pernic=True)
             interfaces_data = {}
-            
+
             for iface_name, counters in io_counters.items():
                 # Skip if we're only monitoring specific interfaces
                 if self.interfaces and iface_name not in self.interfaces:
                     continue
-                
+
                 iface_stats = {
                     'bytes_sent': counters.bytes_sent,
                     'bytes_recv': counters.bytes_recv,
@@ -140,7 +146,7 @@ class NetworkMonitorPlugin(MonitorPlugin):
                     'dropin': counters.dropin,
                     'dropout': counters.dropout,
                 }
-                
+
                 # Calculate deltas from previous collection
                 if iface_name in self._prev_io:
                     prev = self._prev_io[iface_name]
@@ -148,46 +154,46 @@ class NetworkMonitorPlugin(MonitorPlugin):
                     iface_stats['bytes_recv_delta'] = counters.bytes_recv - prev.bytes_recv
                     iface_stats['packets_sent_delta'] = counters.packets_sent - prev.packets_sent
                     iface_stats['packets_recv_delta'] = counters.packets_recv - prev.packets_recv
-                
+
                 interfaces_data[iface_name] = iface_stats
-            
+
             metrics['interfaces'] = interfaces_data
-            
+
             # Store current counters for next delta calculation
             self._prev_io = io_counters
-            
+
         except Exception as e:
             logger.warning(f"Could not collect network I/O counters: {e}")
-        
+
         # Collect connection statistics
         if self.include_connections:
             try:
                 connections = psutil.net_connections(kind='inet')
-                conn_stats = {}
-                
+                conn_stats: Dict[str, int] = {}
+
                 # Count connections by state
                 for conn in connections:
                     state = conn.status
                     conn_stats[state] = conn_stats.get(state, 0) + 1
-                
+
                 metrics['connections'] = conn_stats
-                
+
             except (PermissionError, psutil.AccessDenied):
                 logger.debug("Permission denied for net_connections (requires root/admin)")
             except Exception as e:
                 logger.warning(f"Could not collect connection statistics: {e}")
-        
+
         # Collect interface addresses
         if self.include_addresses:
             try:
                 addresses = psutil.net_if_addrs()
                 addr_data = {}
-                
+
                 for iface_name, addrs in addresses.items():
                     # Skip if we're only monitoring specific interfaces
                     if self.interfaces and iface_name not in self.interfaces:
                         continue
-                    
+
                     iface_addrs = []
                     for addr in addrs:
                         addr_info = {
@@ -199,38 +205,43 @@ class NetworkMonitorPlugin(MonitorPlugin):
                         if addr.broadcast:
                             addr_info['broadcast'] = addr.broadcast
                         iface_addrs.append(addr_info)
-                    
+
                     addr_data[iface_name] = iface_addrs
-                
+
                 metrics['addresses'] = addr_data
-                
+
             except Exception as e:
                 logger.warning(f"Could not collect interface addresses: {e}")
-        
+
         # Add interface stats (up/down status, speed, mtu)
         try:
             if_stats = psutil.net_if_stats()
             stats_data = {}
-            
+
             for iface_name, stats in if_stats.items():
                 # Skip if we're only monitoring specific interfaces
                 if self.interfaces and iface_name not in self.interfaces:
                     continue
-                
+
                 stats_data[iface_name] = {
                     'isup': stats.isup,
                     'duplex': str(stats.duplex) if hasattr(stats, 'duplex') else None,
                     'speed': stats.speed,
                     'mtu': stats.mtu,
                 }
-            
+
             metrics['interface_stats'] = stats_data
-            
+
         except Exception as e:
             logger.warning(f"Could not collect interface stats: {e}")
-        
+
+        # Per-protocol internet presence, address and gateway (from the shared probe)
+        shared = get_shared()
+        if shared is not None:
+            metrics['connectivity'] = shared.as_metric()
+
         return metrics
-    
+
     async def cleanup(self):
         """Cleanup (nothing to do for this plugin)."""
         logger.info("Network monitor cleanup")

hbd/client/plugins/os_info.py

@@ -5,7 +5,6 @@ kernel, architecture, and distribution details.
 """
 
 import platform
-import sys
 from pathlib import Path
 from typing import Any, Dict, Optional
 
@@ -15,7 +14,7 @@ from hbd.client.plugin import InfoPlugin
 
 class OSInfoPlugin(InfoPlugin):
     """Collect operating system information.
-    
+
     This plugin gathers static OS information that rarely changes:
     - OS name and version
     - Kernel version
@@ -23,27 +22,27 @@ class OSInfoPlugin(InfoPlugin):
     - Distribution details (for Linux)
     - Python version (used by hbc)
     """
-    
+
     name = "os_info"
     version = "1.0.0"
     description = "Operating system and platform information"
     interval = 0  # InfoPlugin: collect once at startup
-    
+
     def __init__(self, config: Optional[Dict[str, Any]] = None):
         super().__init__(config)
-    
+
     async def initialize(self) -> bool:
         """Initialize the OS info plugin.
-        
+
         Returns:
             True (always succeeds - platform module is stdlib)
         """
         self.logger.info(f"Initializing {self.name} plugin")
         return True
-    
+
     async def _collect_info(self) -> Dict[str, Any]:
         """Collect OS information.
-        
+
         Returns:
             Dictionary with OS details
         """
@@ -65,15 +64,15 @@ class OSInfoPlugin(InfoPlugin):
             if self.config.get("owner"):
                 self.logger.debug(f"Adding owner from config: {self.config['owner']}")
                 data["owner"] = self.config["owner"]
-            
+
             # Add Linux-specific distribution info
             if platform.system() == "Linux":
                 data.update(self._get_linux_distro())
-            
+
             # Add macOS-specific info
             elif platform.system() == "Darwin":
                 data["macos_version"] = platform.mac_ver()[0]
-            
+
             # Add Windows-specific info
             elif platform.system() == "Windows":
                 win_ver = platform.win32_ver()
@@ -81,22 +80,22 @@ class OSInfoPlugin(InfoPlugin):
                 data["windows_version"] = win_ver[1]
                 data["windows_sp"] = win_ver[2]
                 data["windows_type"] = win_ver[3]
-            
+
             self.logger.debug(f"Collected OS info: {data['system']} {data['release']}")
             return data
-            
+
         except Exception as e:
             self.logger.error(f"Error collecting OS info: {e}", exc_info=True)
             return {}
-    
+
     def _get_linux_distro(self) -> Dict[str, str]:
         """Get Linux distribution information.
-        
+
         Returns:
             Dictionary with distribution details
         """
         distro_info = {}
-        
+
         # Try reading /etc/os-release (standard on modern Linux)
         os_release = Path("/etc/os-release")
         if os_release.exists():
@@ -121,7 +120,7 @@ class OSInfoPlugin(InfoPlugin):
                                 distro_info["distro_pretty_name"] = value
             except Exception as e:
                 self.logger.warning(f"Could not read /etc/os-release: {e}")
-        
+
         # Fallback: try lsb_release (older systems)
         elif Path("/etc/lsb-release").exists():
             try:
@@ -138,5 +137,5 @@ class OSInfoPlugin(InfoPlugin):
                                 distro_info["distro_name"] = value
             except Exception as e:
                 self.logger.warning(f"Could not read /etc/lsb-release: {e}")
-        
+
         return distro_info

hbd/client/plugins/ping_monitor.py

@@ -40,9 +40,9 @@ threshold_configs:
 
 import asyncio
 import re
-import sys
 from typing import Any, Dict, Optional
 
+from hbd.client.pingutil import PingUnavailable, run_ping
 from hbd.client.plugin import MonitorPlugin
 
 
@@ -67,6 +67,7 @@ class PingMonitorPlugin(MonitorPlugin):
         self.timeout = int(cfg.get("timeout", 5))
         # hosts: dict of {hostname: {warning: x, critical: y}} or list of hostnames
         raw_hosts = cfg.get("hosts", {})
+        self.hosts: Dict[str, Any]
         if isinstance(raw_hosts, list):
             self.hosts = {h: {} for h in raw_hosts}
         else:
@@ -84,51 +85,13 @@ class PingMonitorPlugin(MonitorPlugin):
 
     async def _ping(self, host: str) -> Dict[str, float]:
         """Run a system ping command and return rtt_min/avg/max/loss."""
-        if sys.platform == "win32":
-            cmd = ["ping", "-n", str(self.count), "-w", str(self.timeout * 1000), host]
-        else:
-            cmd = ["ping", "-c", str(self.count), "-W", str(self.timeout), host]
-
         try:
-            proc = await asyncio.create_subprocess_exec(
-                *cmd,
-                stdout=asyncio.subprocess.PIPE,
-                stderr=asyncio.subprocess.PIPE,
-            )
-            stdout, _ = await asyncio.wait_for(
-                proc.communicate(),
-                timeout=self.timeout * self.count + 2,
-            )
-            output = stdout.decode(errors="replace")
-        except (asyncio.TimeoutError, FileNotFoundError, OSError) as e:
+            return await run_ping(host, self.count, self.timeout)
+        except (asyncio.TimeoutError, PingUnavailable, OSError) as e:
             self.logger.warning("ping_monitor: ping failed for %s: %s", host, e)
             return {"rtt_min": float("inf"), "rtt_avg": float("inf"),
                     "rtt_max": float("inf"), "loss": 100.0}
 
-        # Parse packet loss
-        loss = 100.0
-        loss_match = re.search(r"(\d+(?:\.\d+)?)\s*%\s*packet\s*loss", output)
-        if loss_match:
-            loss = float(loss_match.group(1))
-
-        # Parse rtt min/avg/max — Linux: "rtt min/avg/max/mdev = x/x/x/x ms"
-        #                          macOS: "round-trip min/avg/max/stddev = x/x/x/x ms"
-        rtt_match = re.search(
-            r"(?:rtt|round-trip)\s+min/avg/max/\S+\s*=\s*([\d.]+)/([\d.]+)/([\d.]+)",
-            output,
-        )
-        if rtt_match:
-            return {
-                "rtt_min": float(rtt_match.group(1)),
-                "rtt_avg": float(rtt_match.group(2)),
-                "rtt_max": float(rtt_match.group(3)),
-                "loss": loss,
-            }
-
-        # Host unreachable or all packets lost
-        return {"rtt_min": float("inf"), "rtt_avg": float("inf"),
-                "rtt_max": float("inf"), "loss": loss}
-
     async def _collect_metrics(self) -> Dict[str, Any]:
         data: Dict[str, Any] = {}
         tasks = {host: asyncio.create_task(self._ping(host)) for host in self.hosts}

hbd/common/__init__.py

@@ -1,3 +1,3 @@
 """Common utilities shared between hbc and hbd."""
 
-from hbd import __version__
+from hbd import __version__  # noqa: F401

hbd/common/proto.py

@@ -8,17 +8,17 @@ Message Types:
     PLG: Plugin data message (client -> server)
 """
 
-from typing import Dict, Any, Union
+from typing import Dict, Any
 import json
 import zlib
 
 
 def encode_value(v: Any) -> str:
     """Encode a value for protocol transmission.
-    
+
     Args:
         v: Value to encode (int, float, str, bool, list, dict, etc.)
-        
+
     Returns:
         String representation suitable for protocol
     """
@@ -35,23 +35,23 @@ def encode_value(v: Any) -> str:
 
 def decode_value(val: str) -> Any:
     """Decode a value from protocol format.
-    
+
     Args:
         val: String value from protocol
-        
+
     Returns:
         Decoded Python object
     """
     if not val:
         return val
-    
+
     # Check for JSON-encoded complex types
     if val.startswith("@"):
         try:
             return json.loads(val[1:])
         except Exception:
             return val[1:]  # Return as string without @
-    
+
     # Try numeric conversion (avoid eval to prevent SyntaxWarnings on version strings)
     if val[0].isdigit() or (val[0] == '-' and len(val) > 1 and val[1].isdigit()):
         try:
@@ -63,7 +63,7 @@ def decode_value(val: str) -> Any:
         except ValueError:
             pass
         return val
-    
+
     return val
 
 
@@ -92,7 +92,7 @@ def stodict(msg: bytes):
     '!' and decodes accordingly. Returns a dict with key 'ID' set to the
     message ID and the parsed key/value pairs.
     """
-    d = {}
+    d: Dict[str, Any] = {}
     if len(msg) > 0 and chr(msg[0]) == "!":
         # message is: b'!ID:' + compressed_payload
         # original code used msg[1:4].decode() for ID (3 bytes including colon)
@@ -135,12 +135,12 @@ def oldmtodict(msg: bytes):
 
 def encode_plugin_data(plugin_name: str, data: Dict[str, Any]) -> bytes:
     """Encode plugin data into a PLG message.
-    
+
     Args:
         plugin_name: Name of the plugin (e.g., "os_info", "cpu_monitor")
         data: Plugin data dictionary
         compress: Whether to compress the payload
-        
+
     Returns:
         Encoded message bytes
     """
@@ -151,12 +151,11 @@ def encode_plugin_data(plugin_name: str, data: Dict[str, Any]) -> bytes:
 
 def decode_plugin_data(msg: bytes) -> Dict[str, Any]:
     """Decode a PLG message into plugin data.
-    
+
     Args:
         msg: Raw message bytes
-        
+
     Returns:
         Dictionary with 'ID', 'plugin', and plugin data fields
     """
     return stodict(msg)
-

hbd/server/__init__.py

@@ -1,3 +1,3 @@
 """HeartBeat Daemon (hbd) - Server/daemon component."""
 
-from hbd import __version__
+from hbd import __version__  # noqa: F401

hbd/server/cli.py

@@ -120,7 +120,7 @@ def cmd_passwd(args):
 def cmd_notify(args):
     """Send a test message via a single notification channel."""
     from .config import load_config
-    from .notify import Notification, _dispatch_to_channel, setup
+    from .notify import Notification, setup
 
     config = load_config(args.configfile)
     setup(config)
@@ -193,7 +193,9 @@ def _read_pid(configfile) -> int | None:
 
 
 def cmd_stop(args):
-    import os, signal as _signal, time
+    import os
+    import signal as _signal
+    import time
     pid = _read_pid(args.configfile)
     if pid is None:
         sys.exit(1)
@@ -212,7 +214,8 @@ def cmd_stop(args):
 
 
 def cmd_reload(args):
-    import os, signal as _signal
+    import os
+    import signal as _signal
     pid = _read_pid(args.configfile)
     if pid is None:
         sys.exit(1)
@@ -222,7 +225,10 @@ def cmd_reload(args):
 
 
 def cmd_restart(args):
-    import os, signal as _signal, time, subprocess
+    import os
+    import signal as _signal
+    import time
+    import subprocess
     pid = _read_pid(args.configfile)
     if pid is not None:
         print(f"Stopping hbd (pid {pid})...")

hbd/server/config.py

@@ -7,7 +7,7 @@ import os
 try:
     import yaml
 except Exception:
-    yaml = None
+    yaml = None  # type: ignore[assignment]
 
 SERVER_DEFAULTS = {
     # Network settings
@@ -43,30 +43,30 @@ SERVER_DEFAULTS = {
 
     # Host management
     "hosts": {},                # Unified host definitions
-    "dyndomains": ["example.org"], # Domains to update via nsupdate when a host with dyndns: true is updated
+    "dyndomains": ["example.org"],  # Domains to update via nsupdate when a host with dyndns: true is updated
 
     # DNS updates
-    "nsupdate_bin": "/usr/bin/nsupdate", # Path to nsupdate binary
-    
+    "nsupdate_bin": "/usr/bin/nsupdate",  # Path to nsupdate binary
+
     # WebSocket settings
-    "ws_port": 50005, 
+    "ws_port": 50005,
     "wss_port": None,
     "cert_path": "/usr/local/etc/ssl/",
     "wss_pem": "fullchain.pem",
     "wss_key": "privkey.pem",
-    
+
     # Message journal configuration
     "journal_enabled": True,
     "journal_dir": "/var/log/heartbeat",
     "journal_file": "messages.journal",
     "journal_max_size": 100 * 1024 * 1024,  # 100MB
     "journal_max_backups": 10,
-    
+
     # Runtime flags
     "foreground": False,
     "verbose": False,
     "debug": 0,
-    
+
     # Plugin/threshold configs (for clients reporting to this server)
     "plugins": {},
     "thresholds": {},
@@ -76,11 +76,11 @@ THRESHOLD_DEFAULTS = {
     'thresholds': {
         'cpu_monitor': {
             'cpu_percent': {
-                'warning': 80.0, 
+                'warning': 80.0,
                 'critical': 90.0
                 }
             },
-            'memory_monitor': {
+        'memory_monitor': {
                 'memory_percent': {
                     'warning': 85.0,
                     'critical': 95.0
@@ -90,7 +90,7 @@ THRESHOLD_DEFAULTS = {
                     'critical': 75.0
                 }
             },
-            'disk_monitor': {
+        'disk_monitor': {
                 'partitions': {
                     '/': {
                         'percent': {
@@ -100,18 +100,18 @@ THRESHOLD_DEFAULTS = {
                     }
                 }
             },
-            'rtt': {
+        'rtt': {
                 'warning': 200,
                 'critical': 250.0,
                 'count': 3  # Optional: number of consecutive breaches before alerting
             },
-            'nagios_runner': {
+        'nagios_runner': {
                 'status_code': {
                     'display': '{check_name} {output}',
                     'operator': "nagios"
                 }
             },
-            'zfs_monitor': {
+        'zfs_monitor': {
                 'pools': {
                     '*': {
                         'status': {
@@ -137,10 +137,10 @@ def load_config(path=None):
     """Load configuration from a YAML file and merge with server defaults.
 
     If YAML is not available or the file does not exist, defaults are returned.
-    
+
     Args:
         path: Path to YAML config file (default: ~/.hb.yaml)
-        
+
     Returns:
         Dictionary with configuration
     """
@@ -165,16 +165,16 @@ def load_config(path=None):
 
 class ReloadableConfig:
     """Thread-safe/async-safe configuration wrapper that supports runtime reloading.
-    
+
     This class wraps the configuration dictionary and provides:
     - Thread-safe config reloading via SIGHUP
     - Backward-compatible dict-like access
     - Async lock to prevent concurrent reloads
     """
-    
+
     def __init__(self, initial_config, config_path=None):
         """Initialize with initial configuration.
-        
+
         Args:
             initial_config: Initial configuration dictionary
             config_path: Path to config file for reloading (optional)
@@ -183,65 +183,62 @@ class ReloadableConfig:
         self._config_path = config_path
         self._lock = asyncio.Lock()
         self._logger = logging.getLogger(__name__)
-    
+
     async def reload(self, config_path=None):
         """Reload configuration from file.
-        
+
         Args:
             config_path: Path to config file (uses stored path if not provided)
-            
+
         Returns:
             New configuration dictionary
-            
+
         Raises:
             Exception if reload fails (keeps existing config)
         """
         path = config_path or self._config_path
         if not path:
             raise ValueError("No config path specified for reload")
-        
+
         async with self._lock:
             try:
                 # Load new config
                 new_config = load_config(path)
-                
-                # Store old config for rollback if needed
-                old_config = self._config
-                
+
                 # Update config
                 self._config = new_config
                 self._logger.info(f"Configuration reloaded from {path}")
-                
+
                 return new_config
             except Exception as e:
                 self._logger.error(f"Failed to reload config from {path}: {e}", exc_info=True)
                 # Keep existing config on error
                 raise
-    
+
     def get(self, key, default=None):
         """Get a config value (dict-compatible)."""
         return self._config.get(key, default)
-    
+
     def __getitem__(self, key):
         """Get a config value via subscript (dict-compatible)."""
         return self._config[key]
-    
+
     def __contains__(self, key):
         """Check if key exists (dict-compatible)."""
         return key in self._config
-    
+
     def keys(self):
         """Return config keys (dict-compatible)."""
         return self._config.keys()
-    
+
     def items(self):
         """Return config items (dict-compatible)."""
         return self._config.items()
-    
+
     def values(self):
         """Return config values (dict-compatible)."""
         return self._config.values()
-    
+
     @property
     def config(self):
         """Get the underlying config dict (for components that need full dict)."""
@@ -319,7 +316,7 @@ def get_host_access(config, hostname) -> dict:
     """
     host_cfg = get_host_config(config, hostname)
 
-    owner = host_cfg.get("owner") # or get_default_owner(config)
+    owner = host_cfg.get("owner")  # or get_default_owner(config)
 
     managers = host_cfg.get("managers", [])
     if isinstance(managers, str):

hbd/server/configio.py

@@ -15,6 +15,7 @@ def _make_yaml() -> YAML:
     y.preserve_quotes = True
     return y
 
+
 # Top-level keys managed by the 'server' logical section
 _SERVER_KEYS = [
     "hbd_port", "hbd_host", "ws_port", "wss_port", "hb_port",

hbd/server/data.py

@@ -1,12 +1,14 @@
-msgs = []  # in-memory list of recent messages for new websocket clients; also logged to file via notify.eventlog
+msgs: list = []  # in-memory list of recent messages for new websocket clients; also logged to file via notify.eventlog
+
+
 class Data:
 
     def __init__(self, config):
         self.config = config
         self.data = {}
-    
+
     def update(self, new_data):
         self.data.update(new_data)
-    
+
     def get(self, key, default=None):
-        return self.data.get(key, default)
+        return self.data.get(key, default)

hbd/server/dns.py

@@ -2,7 +2,7 @@
 
 from __future__ import annotations
 from subprocess import Popen, PIPE, STDOUT
-from typing import Optional
+from typing import Any, Callable, Optional
 import asyncio
 import logging
 
@@ -79,8 +79,8 @@ async def dns_update_worker(
     hbdclass,
     cfg: dict,
     async_queue=None,
-    log: Optional[callable] = None,
-    pushmsg: Optional[callable] = None,
+    log: Optional[Callable[..., Any]] = None,
+    pushmsg: Optional[Callable[..., Any]] = None,
     loop: Optional[asyncio.AbstractEventLoop] = None,
 ):
     """Pure async DNS worker that processes updates from asyncio.Queue.
@@ -164,7 +164,7 @@ async def dns_update_worker(
 def start_dns_worker(
     hbdclass,
     cfg: dict,
-    log: Optional[callable] = None,
+    log: Optional[Callable[..., Any]] = None,
     loop: Optional[asyncio.AbstractEventLoop] = None,
 ):
     """Start the async DNS worker and return the Task.
@@ -176,7 +176,7 @@ def start_dns_worker(
         loop = asyncio.get_event_loop()
 
     # Create asyncio.Queue and wrap in a bridge for thread-safe puts
-    async_q = asyncio.Queue()
+    async_q: asyncio.Queue[Any] = asyncio.Queue()
 
     class _QueueBridge:
         """Thread-safe wrapper around asyncio.Queue for synchronous callers."""

hbd/server/hbdclass.py

@@ -8,6 +8,7 @@ import time
 import json
 import copy
 import queue
+from typing import Any, Dict
 
 num = 0
 
@@ -24,11 +25,12 @@ def log(host, m):
 class Connection:
     # map of addrs to names
 
-    htab = {}
+    htab: Dict[str, Any] = {}
     UNKNOWN = "unknown"
     UP = "up"
     DOWN = "down"
     OVERDUE = "overdue"
+    ABSENT = "absent"
 
     def __init__(self, host, cid, addr, afam):
         self.host = host
@@ -42,7 +44,7 @@ class Connection:
         self.statetime = self.lastbeat
         self.deltastatetime = "computed"
         self.state = Connection.UNKNOWN
-        
+
         # Timer-based reachability monitoring
         self.overdue_timer = None
         self.overdue_callback = None
@@ -94,7 +96,9 @@ class Connection:
         now = time.time()
         if not Null:
             d["addr"] = self.addr
-            if self.rtts[-1]:
+            if self.state == Connection.ABSENT:
+                d["rtt"] = ""
+            elif self.rtts[-1]:
                 d["rtt"] = "%d" % round(self.rtts[-1])
             elif self.state == Connection.UNKNOWN:
                 d["rtt"] = ""
@@ -193,39 +197,39 @@ class Connection:
             if self.host.isDynDns():
                 Host.dnsQ.put((self.host.name, self.addr))
         return r
-    
+
     def reset_overdue_timer(self, timeout_seconds, callback):
         """Reset the overdue timer for this connection.
-        
+
         Cancels any existing timer and sets a new one that will mark
         the connection as overdue if no heartbeat arrives before timeout.
-        
+
         Args:
             timeout_seconds: Seconds before marking as overdue
             callback: Async function to call when timer expires
         """
         import asyncio
-        
+
         # Cancel existing timer if any
         if self.overdue_timer and not self.overdue_timer.cancelled():
             self.overdue_timer.cancel()
-        
+
         # Store parameters for later reference
         self.timeout_duration = timeout_seconds
         self.overdue_callback = callback
-        
+
         # Create new timer
         async def timer_expired():
             await callback(self)
-        
+
         try:
             loop = asyncio.get_event_loop()
-            self.overdue_timer = loop.call_later(timeout_seconds, 
-                                                  lambda: asyncio.create_task(timer_expired()))
+            self.overdue_timer = loop.call_later(timeout_seconds,
+                                                 lambda: asyncio.create_task(timer_expired()))
         except RuntimeError:
             # No event loop running yet
             pass
-    
+
     def cancel_overdue_timer(self):
         """Cancel the overdue timer if it exists and clear all timer references."""
         if self.overdue_timer:
@@ -238,45 +242,45 @@ class Connection:
         self.overdue_timer = None
         self.overdue_callback = None
         self.timeout_duration = None
-    
+
     def get_avg_rtt(self):
         """Get average RTT from recent samples."""
         valid_rtts = [r for r in self.rtts if r > 0]
         if valid_rtts:
             return sum(valid_rtts) / len(valid_rtts)
         return 0
-    
+
     def get_current_rtt(self):
         """Get most recent RTT value."""
         return self.rtts[-1] if self.rtts else 0
-    
+
     def check_rtt_threshold(self, warning_threshold=None, critical_threshold=None):
         """Check if RTT exceeds thresholds.
-        
+
         Args:
             warning_threshold: RTT in ms for warning level
             critical_threshold: RTT in ms for critical level
-            
+
         Returns:
             Tuple of (level, rtt_value) where level is None, 'WARNING', or 'CRITICAL'
         """
         rtt = self.get_current_rtt()
         if rtt <= 0:
             return (None, rtt)
-        
+
         if critical_threshold and rtt > critical_threshold:
             return ('CRITICAL', rtt)
         elif warning_threshold and rtt > warning_threshold:
             return ('WARNING', rtt)
-        
+
         return (None, rtt)
 
 
 #
 class Host:
     # Table of Hosts
-    hosts = {}
-    dnsQ = queue.Queue()
+    hosts: Dict[str, "Host"] = {}
+    dnsQ: "queue.Queue[Any]" = queue.Queue()
 
     def __init__(self, name):
         global num
@@ -299,6 +303,8 @@ class Host:
         self.alert_states = {}
         # Stale-data timers: {plugin_name: asyncio.TimerHandle}
         self.plugin_timers = {}
+        # Client-reported protocol presence: {afam: {"present": bool, "ts": float}}
+        self.presence: dict = {}
         # User access control
         self.owner: str | None = None       # username of owner
         self.managers: list = []            # usernames with manager role
@@ -322,7 +328,8 @@ class Host:
             d["name"] = "<b>%s</b>" % d["name"]
         d["dyn"] = str(self.dyn)
         d["num"] = self.num
-        
+        d["status"] = self.status()
+
         # Add alert counts (split by acknowledged status)
         warning_unacked = 0
         warning_acked = 0
@@ -341,12 +348,12 @@ class Host:
                     critical_acked += 1
                 else:
                     critical_unacked += 1
-        
+
         d["alert_warning_unacked"] = warning_unacked
         d["alert_warning_acked"] = warning_acked
         d["alert_critical_unacked"] = critical_unacked
         d["alert_critical_acked"] = critical_acked
-        
+
         for c in ["IPv4", "IPv6"]:
             if c in self.connections:
                 cs = self.connections[c].statedict()
@@ -403,7 +410,7 @@ class Host:
                 ddict[d] = cl
             else:
                 ddict[d] = self.__dict__[d]
-        
+
         # Add alert counts (computed from alert_states)
         warning_unacked = 0
         warning_acked = 0
@@ -422,7 +429,7 @@ class Host:
                         critical_acked += 1
                     else:
                         critical_unacked += 1
-        
+
         ddict["alert_warning_unacked"] = warning_unacked
         ddict["alert_warning_acked"] = warning_acked
         ddict["alert_critical_unacked"] = critical_unacked
@@ -444,6 +451,7 @@ class Host:
             _, os_data = latest_os
             hbc_version = os_data.get("hbc_version")
         ddict["hbc_version"] = hbc_version
+        ddict["status"] = self.status()
 
         return ddict
 
@@ -459,6 +467,47 @@ class Host:
         else:
             return addr.find(".") > 0
 
+    def set_presence(self, afam, present, now):
+        """Record the client's report of whether it has internet connectivity on afam."""
+        self.presence[afam] = {"present": bool(present), "ts": now}
+
+    def flag_present(self, afam):
+        """Client-reported presence; True when the client never reported (legacy clients)."""
+        entry = getattr(self, "presence", {}).get(afam)
+        return True if entry is None else entry["present"]
+
+    def family_present(self, afam):
+        """True if afam should be monitored: the client says it is present, or heartbeats
+        are demonstrably arriving on it (real traffic overrides a false "absent")."""
+        conn = self.connections.get(afam)
+        return self.flag_present(afam) or (conn is not None and conn.state == Connection.UP)
+
+    def status(self):
+        """Host-level status: down, overdue, simplex, up or unknown.
+
+        overdue: every present family is overdue. simplex: both present, one up, one overdue
+        (same severity as up). Families the client reports absent are not considered,
+        unless none is present: then every connection not parked as ABSENT is considered.
+        """
+        conns = self.connections
+        if any(c.state == Connection.DOWN for c in conns.values()):
+            return "down"
+        present = [c for afam, c in conns.items() if self.family_present(afam)]
+        if not present:
+            # The client reports no protocol present (e.g. its internet probe fails on every
+            # family) yet may still be heard: monitor whatever is not parked as absent, so a
+            # host that then dies is "overdue", not silently "unknown".
+            present = [c for c in conns.values() if c.state != Connection.ABSENT]
+        up = sum(c.state == Connection.UP for c in present)
+        overdue = sum(c.state == Connection.OVERDUE for c in present)
+        if up and overdue:
+            return "simplex"
+        if up:
+            return "up"
+        if overdue:
+            return "overdue"
+        return "unknown"
+
     def conndata(self, cid, addr, rtt, now):
         if addr[0:7] == "::ffff:":
             addr = addr[7:]
@@ -499,12 +548,14 @@ class Host:
             self.monitors = []
         if not hasattr(self, "plugin_timers"):
             self.plugin_timers = {}
+        if not hasattr(self, "presence"):
+            self.presence = {}
 
         pass
 
     def add_plugin_data(self, plugin_name, data, timestamp=None):
         """Store plugin data with timestamp.
-        
+
         Args:
             plugin_name: Name of the plugin (e.g., "cpu_monitor")
             data: Dict of plugin data
@@ -512,24 +563,24 @@ class Host:
         """
         if timestamp is None:
             timestamp = time.time()
-        
+
         if plugin_name not in self.plugin_data:
             self.plugin_data[plugin_name] = []
-        
+
         # Add new data
         self.plugin_data[plugin_name].append((timestamp, data))
-        
+
         # Enforce retention limit (keep last N samples)
         if len(self.plugin_data[plugin_name]) > self.plugin_retention:
             self.plugin_data[plugin_name] = self.plugin_data[plugin_name][-self.plugin_retention:]
-    
+
     def get_plugin_data(self, plugin_name, limit=None):
         """Retrieve plugin data for a specific plugin.
-        
+
         Args:
             plugin_name: Name of the plugin
             limit: Optional limit on number of recent samples to return
-            
+
         Returns:
             List of (timestamp, data) tuples, most recent last
         """
@@ -537,19 +588,19 @@ class Host:
         if limit and len(data) > limit:
             return data[-limit:]
         return data
-    
+
     def get_latest_plugin_data(self, plugin_name):
         """Get the most recent plugin data for a plugin.
-        
+
         Args:
             plugin_name: Name of the plugin
-            
+
         Returns:
             (timestamp, data) tuple or None if no data
         """
         data = self.plugin_data.get(plugin_name, [])
         return data[-1] if data else None
-    
+
     def get_all_plugin_data(self):
         """Get all plugin data for this host.
 

hbd/server/http.py

@@ -109,6 +109,7 @@ def _insert_threshold_metric(thresholds: dict, metric_path: str, values: dict) -
         else:
             thresholds[plugin].setdefault(intermediate, {})[leaf] = cfg
 
+
 def _render_template(html_str: str, **context) -> str:
     tmpl = jinja2.Template(html_str)
     return tmpl.render(**context)
@@ -563,7 +564,7 @@ async def start(
         if err:
             return err
         hostname = request.match_info.get("hostname")
-        
+
         if hostname not in hbdclass.Host.hosts:
             return web.json_response({"error": f"Host '{hostname}' not found"}, status=404)
 
@@ -582,7 +583,7 @@ async def start(
                     "data": data,
                     "sample_count": len(samples),
                 }
-        
+
         return web.json_response({
             "hostname": hostname,
             "plugins": plugins_summary,
@@ -602,23 +603,23 @@ async def start(
         host = hbdclass.Host.hosts[hostname]
         if not _can_view_host(user, host):
             return web.json_response({"error": "Forbidden"}, status=403)
-        
+
         # Get limit from query parameter
         limit = request.rel_url.query.get("limit", "10")
         try:
             limit = int(limit)
         except ValueError:
             limit = 10
-        
+
         # Get plugin data
         samples = host.get_plugin_data(plugin_name, limit=limit)
-        
+
         if not samples:
             return web.json_response(
                 {"error": f"No data for plugin '{plugin_name}' on host '{hostname}'"},
                 status=404
             )
-        
+
         # Format samples
         formatted_samples = [
             {
@@ -627,7 +628,7 @@ async def start(
             }
             for ts, data in samples
         ]
-        
+
         return web.json_response({
             "hostname": hostname,
             "plugin": plugin_name,
@@ -648,17 +649,17 @@ async def start(
         host = hbdclass.Host.hosts[hostname]
         if not _can_view_host(user, host):
             return web.json_response({"error": "Forbidden"}, status=403)
-        
+
         # Get alert states
         alerts = []
         for metric_path, alert_state in host.alert_states.items():
             alerts.append(alert_state.to_dict())
-        
+
         # Get summary if threshold_checker available
         summary = {"ok": 0, "warning": 0, "critical": 0, "unknown": 0}
         if threshold_checker:
             summary = threshold_checker.get_alert_summary(host.alert_states)
-        
+
         return web.json_response({
             "hostname": hostname,
             "alerts": alerts,
@@ -684,31 +685,31 @@ async def start(
                     state for state in host.alert_states.values()
                     if state.level != AlertLevel.OK
                 ]
-            
+
             for alert in active_alerts:
                 alert_dict = alert.to_dict()
                 alert_dict["hostname"] = hostname
                 all_alerts.append(alert_dict)
-        
+
         # Sort by level (critical first) then by hostname
         level_order = {"CRITICAL": 0, "WARNING": 1, "UNKNOWN": 2, "OK": 3}
         all_alerts.sort(
             key=lambda a: (level_order.get(a["level"], 99), a["hostname"], a["metric_path"])
         )
-        
+
         # Get summary counts
         summary = {"critical": 0, "warning": 0, "unknown": 0, "total": len(all_alerts)}
         for alert in all_alerts:
             level = alert["level"].lower()
             if level in summary:
                 summary[level] += 1
-        
+
         return web.json_response({
             "alerts": all_alerts,
             "summary": summary,
             "host_count": len(hbdclass.Host.hosts),
         })
-    
+
     async def api_acknowledge_alert(request):
         """Acknowledge an alert to stop reminder notifications."""
         user, err = _require_auth(request)
@@ -721,22 +722,22 @@ async def start(
                 {"error": "Invalid JSON in request body"},
                 status=400
             )
-        
+
         hostname = data.get("hostname")
         metric_path = data.get("metric_path")
-        
+
         if not hostname or not metric_path:
             return web.json_response(
                 {"error": "Missing required fields: hostname and metric_path"},
                 status=400
             )
-        
+
         if hostname not in hbdclass.Host.hosts:
             return web.json_response(
                 {"error": f"Host '{hostname}' not found"},
                 status=404
             )
-        
+
         host = hbdclass.Host.hosts[hostname]
         if not _can_view_host(user, host):
             return web.json_response({"error": "Forbidden"}, status=403)
@@ -746,10 +747,10 @@ async def start(
                 {"error": f"Alert '{metric_path}' not found for host '{hostname}'"},
                 status=404
             )
-        
+
         alert_state = host.alert_states[metric_path]
         alert_state.acknowledge()
-        
+
         return web.json_response({
             "success": True,
             "hostname": hostname,
@@ -1132,7 +1133,6 @@ async def start(
                 elif is_mon:
                     monitored.append(hostname)
 
-
         # Resolve notification channel configs for display
         notif_channels = []
         if current_user:

hbd/server/journal.py

@@ -11,7 +11,7 @@ import os
 import asyncio
 from datetime import datetime
 from pathlib import Path
-from typing import Dict, Any, Optional, Callable, Iterable, List, Tuple, Union
+from typing import Dict, Any, Optional, Callable, Iterable, List, TextIO, Tuple, Union
 
 logger = logging.getLogger(__name__)
 
@@ -19,14 +19,14 @@ logger = logging.getLogger(__name__)
 class MessageJournal:
     """
     Journal logger for heartbeat messages with size-based rotation.
-    
+
     Features:
     - Logs all received messages in JSON format
     - Automatic rotation when file size exceeds threshold
     - Keeps configurable number of rotated logs
     - Thread-safe and async-safe operation
     - Configurable log directory and file naming
-    
+
     Configuration:
         journal_dir: Directory for journal files (default: /var/log/heartbeat/)
         journal_file: Base filename (default: messages.journal)
@@ -34,69 +34,69 @@ class MessageJournal:
         max_backups: Number of backup files to keep (default: 10)
         enabled: Enable/disable journaling (default: True)
     """
-    
+
     def __init__(self, config: Optional[Dict[str, Any]] = None):
         """
         Initialize the message journal.
-        
+
         Args:
             config: Configuration dictionary with journal settings
         """
         self.config = config or {}
-        
+
         # Configuration options
         self.journal_dir = Path(self.config.get('journal_dir', '/var/log/heartbeat'))
         self.journal_file = self.config.get('journal_file', 'messages.journal')
         self.max_size = self.config.get('journal_max_size', 100 * 1024 * 1024)  # 100MB default
         self.max_backups = self.config.get('journal_max_backups', 10)
         self.enabled = self.config.get('journal_enabled', True)
-        
+
         # Runtime state
-        self._file_handle = None
+        self._file_handle: Optional[TextIO] = None
         self._current_size = 0
         self._lock = asyncio.Lock()
         self._initialized = False
-        
+
         # Full path to current journal file
         self.journal_path = self.journal_dir / self.journal_file
-        
+
     async def initialize(self) -> bool:
         """
         Initialize the journal.
-        
+
         Creates journal directory if needed and opens the journal file.
-        
+
         Returns:
             True if initialization successful, False otherwise
         """
         if not self.enabled:
             logger.info("Message journal disabled in configuration")
             return True
-            
+
         try:
             # Create journal directory if it doesn't exist
             self.journal_dir.mkdir(parents=True, exist_ok=True)
-            
+
             # Open journal file in append mode
             self._file_handle = open(self.journal_path, 'a', encoding='utf-8')
-            
+
             # Get current file size
             try:
                 self._current_size = os.path.getsize(self.journal_path)
             except OSError:
                 self._current_size = 0
-            
+
             self._initialized = True
             logger.info(f"Message journal initialized: {self.journal_path} "
-                       f"(current size: {self._current_size:,} bytes, "
-                       f"max: {self.max_size:,} bytes)")
+                        f"(current size: {self._current_size:,} bytes, "
+                        f"max: {self.max_size:,} bytes)")
             return True
-            
+
         except Exception as e:
             logger.error(f"Failed to initialize message journal: {e}")
             self.enabled = False
             return False
-    
+
     async def log_message(
         self,
         msg: Dict[str, Any],
@@ -105,7 +105,7 @@ class MessageJournal:
     ):
         """
         Log a received message to the journal.
-        
+
         Args:
             msg: Parsed message dictionary
             addr: Source address (ip, port) tuple
@@ -113,19 +113,19 @@ class MessageJournal:
         """
         if not self.enabled or not self._initialized:
             return
-        
+
         # Skip HTB (heartbeat) messages - too verbose
         msg_id = msg.get('ID', '')
         if msg_id == 'HTB':
             return
-        
+
         async with self._lock:
             try:
                 # Prepare journal entry
                 if timestamp is None:
                     import time
                     timestamp = time.time()
-                
+
                 entry = {
                     'timestamp': timestamp,
                     'datetime': datetime.fromtimestamp(timestamp).isoformat(),
@@ -133,21 +133,21 @@ class MessageJournal:
                     'source_port': addr[1] if isinstance(addr, (tuple, list)) and len(addr) > 1 else None,
                     'message': msg
                 }
-                
+
                 # Serialize to JSON (one line per entry)
                 json_line = json.dumps(entry, separators=(',', ':')) + '\n'
                 json_bytes = json_line.encode('utf-8')
-                
+
                 # Check if rotation is needed
                 if self._current_size + len(json_bytes) > self.max_size:
                     await self._rotate()
-                
+
                 # Write to journal
                 if self._file_handle:
                     self._file_handle.write(json_line)
                     self._file_handle.flush()  # Ensure data is written
                     self._current_size += len(json_bytes)
-                    
+
                     logger.debug(f"Logged message from {addr[0]}: {msg.get('ID', 'UNKNOWN')}")
 
             except Exception as e:
@@ -181,7 +181,7 @@ class MessageJournal:
     async def _rotate(self):
         """
         Rotate the journal file.
-        
+
         Renames current file with timestamp, opens new file, and removes
         old backups exceeding max_backups limit.
         """
@@ -190,25 +190,25 @@ class MessageJournal:
             if self._file_handle:
                 self._file_handle.close()
                 self._file_handle = None
-            
+
             # Generate backup filename with timestamp
             timestamp_str = datetime.now().strftime('%Y%m%d-%H%M%S')
             backup_name = f"{self.journal_file}.{timestamp_str}"
             backup_path = self.journal_dir / backup_name
-            
+
             # Rename current file to backup
             if self.journal_path.exists():
                 self.journal_path.rename(backup_path)
                 logger.info(f"Rotated journal: {backup_path} "
-                           f"(size: {self._current_size:,} bytes)")
-            
+                            f"(size: {self._current_size:,} bytes)")
+
             # Open new journal file
             self._file_handle = open(self.journal_path, 'a', encoding='utf-8')
             self._current_size = 0
-            
+
             # Clean up old backups
             await self._cleanup_old_backups()
-            
+
         except Exception as e:
             logger.error(f"Error rotating journal: {e}")
             # Try to reopen the file even if rotation failed
@@ -217,18 +217,18 @@ class MessageJournal:
             except Exception as e2:
                 logger.error(f"Failed to reopen journal after rotation error: {e2}")
                 self.enabled = False
-    
+
     async def _cleanup_old_backups(self):
         """
         Remove old backup files exceeding max_backups limit.
-        
+
         Keeps only the most recent backups based on filename (which includes timestamp).
         """
         try:
             # Find all backup files
             backup_pattern = f"{self.journal_file}.*"
             backup_files = sorted(self.journal_dir.glob(backup_pattern))
-            
+
             # Remove oldest backups if we have too many
             if len(backup_files) > self.max_backups:
                 files_to_remove = backup_files[:len(backup_files) - self.max_backups]
@@ -238,10 +238,10 @@ class MessageJournal:
                         logger.info(f"Removed old backup: {backup_file.name}")
                     except Exception as e:
                         logger.warning(f"Failed to remove old backup {backup_file}: {e}")
-                        
+
         except Exception as e:
             logger.error(f"Error cleaning up old backups: {e}")
-    
+
     async def log_threshold_event(
         self,
         host_name: str,
@@ -253,7 +253,7 @@ class MessageJournal:
     ):
         """
         Log a threshold state change event.
-        
+
         Args:
             host_name: Name of the host
             metric_path: Full metric path (e.g., "cpu_monitor.cpu_percent")
@@ -264,11 +264,11 @@ class MessageJournal:
         """
         if not self.enabled or not self._initialized:
             return
-        
+
         try:
             if timestamp is None:
                 timestamp = __import__('time').time()
-            
+
             event = {
                 'timestamp': timestamp,
                 'iso_time': datetime.fromtimestamp(timestamp).isoformat(),
@@ -279,30 +279,30 @@ class MessageJournal:
                 'new_level': new_level,
                 'value': value,
             }
-            
+
             async with self._lock:
                 if not self._file_handle:
                     return
-                
+
                 # Check if rotation is needed
                 if self._current_size >= self.max_size:
                     await self._rotate()
-                
+
                 # Write event
                 line = json.dumps(event) + '\n'
                 self._file_handle.write(line)
                 self._file_handle.flush()
-                
+
                 # Update size
                 self._current_size += len(line.encode('utf-8'))
-                
+
         except Exception as e:
             logger.error(f"Error logging threshold event: {e}")
-    
+
     async def close(self):
         """
         Close the journal and release resources.
-        
+
         Should be called during shutdown.
         """
         async with self._lock:
@@ -315,11 +315,11 @@ class MessageJournal:
                 finally:
                     self._file_handle = None
                     self._initialized = False
-    
+
     def get_stats(self) -> Dict[str, Any]:
         """
         Get journal statistics.
-        
+
         Returns:
             Dictionary with journal stats
         """
@@ -423,10 +423,10 @@ _journal_instance: Optional[MessageJournal] = None
 def get_journal(config: Optional[Dict[str, Any]] = None) -> MessageJournal:
     """
     Get or create the global journal instance.
-    
+
     Args:
         config: Configuration dictionary (only used on first call)
-        
+
     Returns:
         MessageJournal instance
     """

hbd/server/main.py

@@ -5,8 +5,6 @@ import logging
 import socket
 import time
 import signal
-import sys
-import ssl
 from . import __version__
 
 from . import udp
@@ -23,6 +21,7 @@ eventlog = notify_mod.eventlog
 
 # shared runtime collections and helpers
 
+
 def save_state(config, hbdclass):
     """Save current state to pickle file. Safe to call at any time."""
     import pickle
@@ -68,21 +67,21 @@ def cleanup_function(config, hbdclass):
 
 async def reload_configuration(config_obj, config_path, components):
     """Reload configuration and update all components.
-    
+
     Args:
         config_obj: ReloadableConfig instance
         config_path: Path to config file
         components: Dict with threshold_checker and other components
-        
+
     Returns:
         True if reload succeeded, False otherwise
     """
     try:
         logger.info("Starting configuration reload...")
-        
+
         # Reload config file
         new_config = await config_obj.reload(config_path)
-        
+
         # Update notify module
         notify_mod.reload_config(new_config)
 
@@ -103,23 +102,23 @@ async def reload_configuration(config_obj, config_path, components):
         if 'threshold_checker' in components:
             components['threshold_checker'].reload(new_config)
             components['threshold_checker'].purge_stale_alerts(hbdclass)
-        
+
         # Note: Changes to the following require restart:
         # - hb_port, hbd_port, ws_port (already bound)
         # - SSL certificates (already loaded)
         # - pickfile (already opened)
         # - journal settings (journal already initialized)
-        
+
         # These are reloadable and effective immediately:
         # - notification_channels
         # - threshold_configs
         # - hosts (watchhosts, dyndns, notification_channels)
         # - grace period (used on next heartbeat)
         # - debug/verbose flags (used on next message)
-        
+
         logger.info("Configuration reload completed successfully")
         return True
-        
+
     except Exception as e:
         logger.error("=" * 60)
         logger.error(f"Failed to reload configuration: {e}", exc_info=True)
@@ -142,7 +141,7 @@ async def _run_async(config, config_path=None):
         sig_name = signal.Signals(signum).name if hasattr(signal, "Signals") else signum
         logger.info(f"Received {sig_name}, initiating shutdown...")
         loop.call_soon_threadsafe(shutdown_event.set)
-    
+
     def reload_handler(signum, frame):
         sig_name = signal.Signals(signum).name if hasattr(signal, "Signals") else signum
         logger.info(f"Received {sig_name}, initiating config reload...")
@@ -181,7 +180,7 @@ async def _run_async(config, config_path=None):
         journal=msg_journal,
     )
     logger.info("Threshold checker initialized")
-    
+
     # Components dict for reload orchestration
     components = {
         'threshold_checker': threshold_checker,
@@ -324,7 +323,7 @@ async def _run_async(config, config_path=None):
                 ],
                 return_when=asyncio.FIRST_COMPLETED
             )
-            
+
             # Check which event was triggered
             if shutdown_event.is_set():
                 logger.info("Shutdown signal received, stopping services...")
@@ -332,24 +331,24 @@ async def _run_async(config, config_path=None):
                 for task in pending:
                     task.cancel()
                 break
-            
+
             if reload_event.is_set():
                 # Clear the event for next reload
                 reload_event.clear()
-                
+
                 # Cancel pending wait tasks
                 for task in pending:
                     task.cancel()
-                
+
                 # Perform reload if config_path is available
                 if config_path:
                     await reload_configuration(config, config_path, components)
                 else:
                     logger.warning("Cannot reload: no config path available")
-                
+
                 # Continue main loop
                 continue
-                
+
     except Exception as e:
         logger.exception("Error in main loop: %s", e)
     finally:
@@ -381,7 +380,7 @@ async def _run_async(config, config_path=None):
                 logger.warning("Timeout waiting for tasks to cancel")
             except Exception as e:
                 logger.debug("Exception during task cancellation: %s", e)
-        
+
         # Close message journal
         try:
             await msg_journal.close()
@@ -487,7 +486,7 @@ def run(config, config_path=None):
     """Start the hbd service (blocking).
 
     Manually manages the event loop to ensure clean shutdown.
-    
+
     Args:
         config: Configuration dictionary
         config_path: Path to config file (for reload support)

hbd/server/monitor.py

@@ -1,15 +1,13 @@
 """Monitor helper for heartbeat daemon.
 
 This module provides monitoring tasks for the heartbeat daemon.
-The primary reachability monitoring is now event-driven (timers set/reset 
+The primary reachability monitoring is now event-driven (timers set/reset
 on HTB arrival in udp.py) rather than periodic polling.
 
 This module can be extended for additional monitoring tasks.
 """
 
 from __future__ import annotations
-import asyncio
-import time
 from . import notify as notify_mod
 
 DROPOVERDUE = 7 * 24 * 3600
@@ -18,11 +16,10 @@ eventlog = notify_mod.eventlog
 
 async def cleanup_connections(hbdclass):
     """Clean up connection timers on shutdown.
-    
+
     Cancels all active overdue timers to prevent callbacks after shutdown.
     """
     for hostname, host in list(hbdclass.Host.hosts.items()):
         for conn_type, conn in host.connections.items():
             if hasattr(conn, 'cancel_overdue_timer'):
                 conn.cancel_overdue_timer()
-

hbd/server/notify.py

@@ -21,7 +21,7 @@ import smtplib
 import subprocess
 import time
 import sys
-from dataclasses import dataclass, field
+from dataclasses import dataclass
 from typing import Optional
 
 from . import data
@@ -49,6 +49,7 @@ logf = None
 
 _LEVEL_ORDER = {"RECOVER": 0, "INFO": 0, "WARNING": 1, "CRITICAL": 2}
 
+
 def _level_value(level: str) -> int:
     return _LEVEL_ORDER.get(level.upper(), 0)
 
@@ -103,7 +104,6 @@ def initlog(logfile):
 
 
 def closelog():
-    global logf
     if logf and logf != sys.stderr:
         try:
             logf.close()
@@ -331,8 +331,6 @@ async def _send_sms_voipms_async(channel_cfg: dict, notif: Notification) -> bool
         return False
 
 
-
-
 async def _send_matrix_async(channel_cfg: dict, notif: Notification) -> bool:
     """Send a Matrix message using matrix-nio."""
     try:
@@ -341,7 +339,6 @@ async def _send_matrix_async(channel_cfg: dict, notif: Notification) -> bool:
         logger.error("matrix-nio not installed; pip install matrix-nio")
         return False
 
-    from nio import AsyncClient
     homeserver = channel_cfg.get("homeserver", "")
     access_token = channel_cfg.get("access_token", "")
     room_id = channel_cfg.get("room_id", "")
@@ -359,7 +356,6 @@ async def _send_matrix_async(channel_cfg: dict, notif: Notification) -> bool:
     client = AsyncClient(homeserver)
     client.access_token = access_token
     try:
-        from nio import RoomSendResponse
         content = {
             "msgtype": "m.text",
             "body": text,

hbd/server/oauth.py

@@ -128,7 +128,7 @@ def get_providers(config: dict) -> list[ResolvedProvider]:
     Entries with missing required fields or unknown types are skipped with
     a warning log.  Order follows config declaration order.
     """
-    result = []
+    result: list = []
     oauth_cfg = config.get("oauth", {})
     if not isinstance(oauth_cfg, dict):
         return result

hbd/server/settings.py

@@ -420,7 +420,7 @@ def get_settings_sections(config: dict, threshold_checker=None, user=None) -> li
             "fields": [
                 field("journal_enabled",     "Enabled",          "boolean",
                       "Turn journalling on or off.", editable=True),
-                field("journal_dir",         "Journal directory","path",
+                field("journal_dir",         "Journal directory", "path",
                       "Directory where journal files are written.", editable=True),
                 field("journal_file",        "Journal filename", "text",
                       "Base filename for the journal (rotated copies get a numeric suffix)."),

hbd/server/templates/live.html

@@ -73,6 +73,7 @@
     .tipbox .up { color: var(--st-ok); font-weight: 700; }
     .tipbox .dn { color: var(--st-crit); font-weight: 700; }
     .tipbox .ov { color: var(--st-warn); font-weight: 700; }
+    .tipbox .ab { color: var(--st-faint); font-weight: 700; }
     .tipbox .meta { margin-top: 6px; padding-top: 6px; border-top: 1px solid var(--st-line-soft); font-size: 11px; color: var(--st-faint); }
 
     @media (max-width: 760px) {
@@ -134,18 +135,18 @@
       function worstConnection(data) {
         var worst = null;
         (data.connections || []).forEach(function (c) {
-          if (!c.state || c.state === 'unknown') return;
+          if (!c.state || c.state === 'unknown' || c.state === 'absent') return;
           if (!worst || (STATE_RANK[c.state] || 0) > (STATE_RANK[worst.state] || 0)) worst = c;
         });
         return worst;
       }
 
       function severity(data) {
-        var w = worstConnection(data);
+        var st = data.status;
         if ((data.alert_critical_unacked || 0) > 0 || (data.alert_critical_acked || 0) > 0) return 'crit';
-        if (w && w.state === 'down') return 'crit';
+        if (st === 'down') return 'crit';
         if ((data.alert_warning_unacked || 0) > 0 || (data.alert_warning_acked || 0) > 0) return 'warn';
-        if (w && w.state === 'overdue') return 'warn';
+        if (st === 'overdue') return 'warn';
         return 'ok';
       }
       var SEV_ORDER = { crit: 0, warn: 1, ok: 2 };
@@ -171,17 +172,17 @@
 
       // ---- host row rendering ---------------------------------------------
       function statusChip(data) {
-        var w = worstConnection(data);
-        if (!w) return '<span class="chip outline">—</span>';
-        var cls = w.state === 'up' ? 'ok' : (w.state === 'down' ? 'crit' : 'warn');
-        return '<span class="chip ' + cls + '">' + escHtml(w.state) + '</span>';
+        var st = data.status;
+        if (!st || st === 'unknown') return '<span class="chip outline">—</span>';
+        var cls = (st === 'up' || st === 'simplex') ? 'ok' : (st === 'down' ? 'crit' : 'warn');
+        return '<span class="chip ' + cls + '">' + escHtml(st) + '</span>';
       }
 
       function tipCard(data) {
         var rows = '';
         (data.connections || []).forEach(function (c) {
           if (!c.state || c.state === 'unknown') return;
-          var cls = c.state === 'up' ? 'up' : (c.state === 'down' ? 'dn' : 'ov');
+          var cls = c.state === 'up' ? 'up' : (c.state === 'down' ? 'dn' : (c.state === 'absent' ? 'ab' : 'ov'));
           var fam = (c.afam || '').replace('IPv', 'v');
           var rtt = (c.state === 'up' && c.rtts && c.rtts[0]) ? Math.round(c.rtts[0]) + ' ms' : '—';
           rows += '<tr><td>' + escHtml(fam) + '</td><td>' + escHtml(c.addr || '—') + '</td>'
@@ -247,7 +248,9 @@
         else if (data.hbc_version !== HBD_VERSION) {
           staleTag = '<span class="stale" title="agent v' + escHtml(String(data.hbc_version)) + ' ≠ server v' + HBD_VERSION + '">v' + escHtml(String(data.hbc_version)) + '</span>';
         }
-        var dotCls = w ? (w.state === 'up' ? sev : (w.state === 'down' ? 'down' : 'overdue')) : 'off';
+        var st = data.status;
+        var dotCls = (!st || st === 'unknown') ? 'off'
+          : ((st === 'up' || st === 'simplex') ? sev : (st === 'down' ? 'down' : 'overdue'));
         var flapTag = '';
         var flapping = data.flapping || [];
         if (flapping.length) {
@@ -286,11 +289,11 @@
         var total = 0, up = 0, overdue = 0, down = 0;
         Object.values(hostData).forEach(function (d) {
           total++;
-          var w = worstConnection(d);
-          if (!w) return;
-          if (w.state === 'up') up++;
-          else if (w.state === 'overdue') overdue++;
-          else if (w.state === 'down') down++;
+          var st = d.status;
+          if (!st || st === 'unknown') return;
+          if (st === 'up' || st === 'simplex') up++;
+          else if (st === 'overdue') overdue++;
+          else if (st === 'down') down++;
         });
         var parts = [total + ' hosts', up + ' up'];
         if (overdue) parts.push(overdue + ' overdue');

hbd/server/templates/plugins.html

@@ -456,7 +456,7 @@
             </tr></thead><tbody>`;
           for (const c of data.connections) {
             const st = escHtml(c.state || '—');
-            const stCls = c.state === 'up' ? 'status-up' : (c.state ? 'status-down' : '');
+            const stCls = c.state === 'up' ? 'status-up' : (c.state === 'absent' ? '' : (c.state ? 'status-down' : ''));
             const rtt = (c.state === 'up' && c.rtt) ? Math.round(c.rtt) + ' ms' : '—';
             const chg = c.statetime ? new Date(c.statetime * 1000).toLocaleString() : '—';
             const seen = c.lastbeat ? new Date(c.lastbeat * 1000).toLocaleString() : '—';
@@ -1143,6 +1143,24 @@
       function renderNetworkTables(d) {
         let html = '';
 
+        if (d.connectivity) {
+          let t = '<table class="data-table"><thead><tr><th>Protocol</th><th>Internet</th>'
+            + '<th>Address</th><th>Gateway</th><th class="num">RTT</th></tr></thead><tbody>';
+          for (const fam of ['ipv4', 'ipv6']) {
+            const c = d.connectivity[fam];
+            if (!c) continue;
+            const known = c.present === true || c.present === false;
+            const cls = c.present === true ? 'status-up' : '';
+            const txt = c.present === true ? '✓ present' : (c.present === false ? 'absent' : '?');
+            t += `<tr><td>${fam === 'ipv4' ? 'IPv4' : 'IPv6'}</td>
+              <td class="${cls}">${txt}</td>
+              <td>${escHtml(c.addr || '—')}</td><td>${escHtml(c.gateway || '—')}</td>
+              <td class="num">${known && c.rtt != null ? Math.round(c.rtt) + ' ms' : '—'}</td></tr>`;
+          }
+          t += '</tbody></table>';
+          html += `<div class="table-section-label">Connectivity</div>${t}`;
+        }
+
         if (d.interfaces && Object.keys(d.interfaces).length) {
           const sample = Object.values(d.interfaces)[0];
           const hasDelta = 'bytes_sent_delta' in sample;

hbd/server/threshold.py

@@ -13,13 +13,14 @@ import asyncio
 import logging
 import time
 from enum import Enum
-from typing import Dict, List, Any, Optional, Tuple, Callable
+from typing import Dict, List, Any, Optional, Tuple
 from . import notify as notify_mod
 from .config import THRESHOLD_DEFAULTS
 
 logger = logging.getLogger(__name__)
 eventlog = notify_mod.eventlog
 
+
 class AlertLevel(Enum):
     """Alert severity levels."""
     OK = 0
@@ -52,42 +53,42 @@ class AlertState:
         self.metric_path = metric_path
         self.level = AlertLevel.OK
         self.since = time.time()
-        self.last_value = None
+        self.last_value: Any = None
         self.last_check = time.time()
         self.notification_count = 0
-        self.last_notification = None
-        self.threshold_value = None  # The threshold value that triggered alert
-        self.operator = None  # The comparison operator (>, <, >=, etc.)
+        self.last_notification: Optional[float] = None
+        self.threshold_value: Optional[float] = None  # The threshold value that triggered alert
+        self.operator: Optional[str] = None  # The comparison operator (>, <, >=, etc.)
         self.hysteresis: Optional[float] = None  # Hysteresis fraction used for recovery
-        self.formatted_message = None  # Formatted display message for UI
+        self.formatted_message: Optional[str] = None  # Formatted display message for UI
         self.acknowledged = False  # Whether alert has been acknowledged
-        self.acknowledged_at = None  # Timestamp when acknowledged
+        self.acknowledged_at: Optional[float] = None  # Timestamp when acknowledged
         self.consecutive_count = 0  # Consecutive exceedances while still OK (for count gating)
         self.pending_since: Optional[float] = None  # non-None while waiting out grace period before notifying
-    
+
     def update(
-        self, 
-        level: AlertLevel, 
+        self,
+        level: AlertLevel,
         value: Any,
         threshold_value: Optional[float] = None,
         operator: Optional[str] = None
     ) -> bool:
         """
         Update alert state.
-        
+
         Args:
             level: New alert level
             value: Current metric value
             threshold_value: The threshold value that was exceeded (if applicable)
             operator: The comparison operator (>, <, >=, etc.)
-            
+
         Returns:
             True if state changed (notification needed), False otherwise
         """
         now = time.time()
         self.last_check = now
         self.last_value = value
-        
+
         # Update threshold info when alert is active
         if level != AlertLevel.OK:
             self.threshold_value = threshold_value
@@ -96,7 +97,7 @@ class AlertState:
             # Clear threshold info when returning to OK
             self.threshold_value = None
             self.operator = None
-        
+
         # Check if state changed
         if level != self.level:
             logger.info(
@@ -116,13 +117,13 @@ class AlertState:
                 self.acknowledged = False
                 self.acknowledged_at = None
             return True
-        
+
         return False
-    
+
     def to_dict(self) -> dict:
         """Convert alert state to dictionary for serialization."""
         import math
-        
+
         # Helper to sanitize numeric values for JSON (handle inf/nan)
         def sanitize_value(val):
             if isinstance(val, float):
@@ -131,7 +132,7 @@ class AlertState:
                 if math.isnan(val):
                     return None
             return val
-        
+
         result = {
             "metric_path": self.metric_path,
             "level": self.level.name,
@@ -141,11 +142,11 @@ class AlertState:
             "notification_count": self.notification_count,
             "acknowledged": self.acknowledged,
         }
-        
+
         # Include acknowledgment timestamp if acknowledged
         if self.acknowledged_at is not None:
             result["acknowledged_at"] = self.acknowledged_at
-        
+
         # Include threshold info if available
         if self.threshold_value is not None:
             result["threshold_value"] = sanitize_value(self.threshold_value)
@@ -164,7 +165,7 @@ class AlertState:
                 result["recovery_threshold"] = round(self.threshold_value + ha, 4)
 
         return result
-    
+
     def __setstate__(self, state):
         """Restore from pickle, backfilling fields added after the pickle was written."""
         self.__dict__.update(state)
@@ -182,6 +183,7 @@ class AlertState:
     def __str__(self):
         return self.to_dict().__str__()
 
+
 class ThresholdConfig:
     """Configuration for a single threshold check."""
 
@@ -218,7 +220,7 @@ class ThresholdConfig:
         self.display = display
         self.count = max(1, int(count))
         self.grace = float(grace) if grace is not None else None
-        
+
         # Parse operator
         try:
             self.operator = ComparisonOperator(operator)
@@ -229,7 +231,7 @@ class ThresholdConfig:
                 metric_path
             )
             self.operator = ComparisonOperator.GT
-    
+
     def evaluate(self, value: float) -> AlertLevel:
         """
         Evaluate a value against this threshold.
@@ -271,7 +273,7 @@ class ThresholdConfig:
                 return AlertLevel.WARNING
 
         return AlertLevel.OK
-    
+
     def evaluate_with_hysteresis(
         self,
         value: float,
@@ -279,11 +281,11 @@ class ThresholdConfig:
     ) -> AlertLevel:
         """
         Evaluate with hysteresis to prevent flapping.
-        
+
         Args:
             value: Current metric value
             current_level: Current alert level
-            
+
         Returns:
             New alert level considering hysteresis
         """
@@ -296,7 +298,7 @@ class ThresholdConfig:
         # If no hysteresis, return new level
         if self.hysteresis == 0.0:
             return new_level
-        
+
         # If improving (going to a lower severity), apply hysteresis
         if new_level.value < current_level.value:
             # For recovery, value must be better by hysteresis amount
@@ -306,10 +308,10 @@ class ThresholdConfig:
                 threshold = self.warning
             else:
                 return new_level
-            
+
             # Calculate hysteresis threshold
             hysteresis_amount = abs(threshold * self.hysteresis)
-            
+
             if self.operator in [ComparisonOperator.GT, ComparisonOperator.GTE]:
                 # For "greater than" thresholds, value must go below by hysteresis
                 recovery_threshold = threshold - hysteresis_amount
@@ -322,9 +324,9 @@ class ThresholdConfig:
                 if value <= recovery_threshold:
                     # Not enough improvement, keep current level
                     return current_level
-        
+
         return new_level
-    
+
     def _compare(self, value: float, threshold: float) -> bool:
         """Perform comparison based on operator."""
         if self.operator == ComparisonOperator.GT:
@@ -344,7 +346,7 @@ class ThresholdConfig:
 
 class ThresholdChecker:
     """Main threshold checking and alerting system."""
-    
+
     def __init__(
         self,
         config: Dict[str, Any],
@@ -353,34 +355,34 @@ class ThresholdChecker:
     ):
         """
         Initialize threshold checker.
-        
+
         Args:
             config: Threshold configuration dictionary from YAML
             renotify_interval: Seconds between repeat notifications (default: 1 hour)
             journal: Optional MessageJournal instance for logging threshold events
         """
         # Named threshold configurations (pre-merged: defaults + overrides): {config_name: {metric_path: ThresholdConfig}}
-        self.threshold_configs = {}
+        self.threshold_configs: Dict[str, Dict[str, ThresholdConfig]] = {}
 
         # Raw overrides only for each named config (no defaults baked in): {config_name: {metric_path: ThresholdConfig}}
         self.threshold_raw_configs: Dict[str, Dict[str, ThresholdConfig]] = {}
 
         # Single threshold set for backward compatibility: {metric_path: ThresholdConfig}
-        self.thresholds = {}
+        self.thresholds: Dict[str, ThresholdConfig] = {}
 
         # Host to ordered list of config names: {host_name: [config_name, ...]}
         self.host_config_mapping: Dict[str, List[str]] = {}
 
         # Default config name to use when no mapping exists
         self.default_config = "default"
-        
+
         self.renotify_interval = renotify_interval
         self.grace_seconds: float = float(config.get("grace", 2))
         self.journal = journal
 
         # Parse configuration
         self._parse_config(config)
-        
+
         total_thresholds = sum(len(cfg) for cfg in self.threshold_configs.values())
         if total_thresholds == 0 and len(self.thresholds) > 0:
             # Backward compatibility: using single threshold set
@@ -392,18 +394,18 @@ class ThresholdChecker:
                 len(self.threshold_configs),
                 total_thresholds
             )
-    
+
     def reload(self, config: Dict[str, Any]):
         """Reload threshold configuration from new config dict.
-        
+
         This clears all existing thresholds and re-parses from the new configuration.
         Alert states are preserved to maintain hysteresis across reloads.
-        
+
         Args:
             config: New configuration dictionary
         """
         logger.info("Reloading threshold configuration...")
-        
+
         # Clear old configuration
         self.threshold_configs.clear()
         self.threshold_raw_configs.clear()
@@ -413,13 +415,13 @@ class ThresholdChecker:
 
         # Parse new configuration
         self._parse_config(config)
-        
+
         total_thresholds = sum(len(cfg) for cfg in self.threshold_configs.values())
         if total_thresholds == 0 and len(self.thresholds) > 0:
             total_thresholds = len(self.thresholds)
-        
+
         logger.info("Threshold configuration reloaded: %d total thresholds", total_thresholds)
-    
+
     def _parse_config(self, config: Dict[str, Any]):
         """Parse threshold configuration from YAML structure.
 
@@ -449,7 +451,7 @@ class ThresholdChecker:
             self._parse_legacy_config(config)
         else:
             logger.info("No thresholds configured")
-    
+
     def _parse_multi_config(self, config: Dict[str, Any]):
         """Parse multiple named threshold configurations."""
         threshold_configs = config.get("threshold_configs", {})
@@ -545,34 +547,34 @@ class ThresholdChecker:
             if isinstance(legacy_mapping, dict):
                 for host_name, value in legacy_mapping.items():
                     self.host_config_mapping[host_name] = _normalise(value)
-        
+
         # Set default config (first one alphabetically or explicitly set)
         self.default_config = config.get("default_threshold_config", "default")
         if self.default_config not in self.threshold_configs and self.threshold_configs:
             # Use first available config as default
             self.default_config = sorted(self.threshold_configs.keys())[0]
             logger.info("Using '%s' as default threshold config", self.default_config)
-        
+
         logger.info(
             "Loaded %d threshold configurations with %d host mappings",
             len(self.threshold_configs),
             len(self.host_config_mapping)
         )
-    
+
     def _parse_legacy_config(self, config: Dict[str, Any]):
         """Parse legacy single threshold configuration for backward compatibility."""
         if not config or "thresholds" not in config:
             logger.info("No thresholds configured")
             return
-        
+
         thresholds_config = config["thresholds"]
-        
+
         for plugin_name, plugin_thresholds in thresholds_config.items():
             if not isinstance(plugin_thresholds, dict):
                 continue
-            
+
             self._parse_plugin_thresholds(plugin_name, plugin_thresholds, target_dict=self.thresholds)
-    
+
     def _parse_plugin_thresholds(
         self,
         plugin_name: str,
@@ -580,7 +582,7 @@ class ThresholdChecker:
         target_dict: Optional[Dict[str, ThresholdConfig]] = None
     ):
         """Parse thresholds for a specific plugin.
-        
+
         Args:
             plugin_name: Name of the plugin
             thresholds: Threshold configuration dictionary
@@ -588,7 +590,7 @@ class ThresholdChecker:
         """
         if target_dict is None:
             target_dict = self.thresholds
-        
+
         # Special handling for RTT thresholds (per-host)
         if plugin_name == "rtt":
             self._parse_rtt_thresholds(thresholds, target_dict)
@@ -606,7 +608,7 @@ class ThresholdChecker:
         for metric_name, threshold_config in thresholds.items():
             if not isinstance(threshold_config, dict):
                 continue
-            
+
             # Handle nested metrics (e.g., partitions./.percent or pools.*.status)
             if metric_name == "partitions":
                 self._parse_partition_thresholds(plugin_name, threshold_config, target_dict)
@@ -614,9 +616,9 @@ class ThresholdChecker:
             if metric_name == "pools":
                 self._parse_pool_thresholds(plugin_name, threshold_config, target_dict)
                 continue
-            
+
             metric_path = f"{plugin_name}.{metric_name}"
-            
+
             # Extract threshold values
             warning = threshold_config.get("warning")
             critical = threshold_config.get("critical")
@@ -643,7 +645,7 @@ class ThresholdChecker:
                 display=display,
                 grace=grace,
             )
-            
+
             target_dict[metric_path] = threshold
             logger.debug(
                 "Registered threshold for %s: warn=%s, crit=%s, op=%s",
@@ -652,7 +654,7 @@ class ThresholdChecker:
                 critical,
                 operator
             )
-    
+
     def _parse_partition_thresholds(
         self,
         plugin_name: str,
@@ -660,7 +662,7 @@ class ThresholdChecker:
         target_dict: Optional[Dict[str, ThresholdConfig]] = None
     ):
         """Parse partition-specific thresholds for disk monitoring.
-        
+
         Args:
             plugin_name: Name of the plugin
             partitions: Partition threshold configuration
@@ -668,18 +670,18 @@ class ThresholdChecker:
         """
         if target_dict is None:
             target_dict = self.thresholds
-        
+
         for partition, metrics in partitions.items():
             if not isinstance(metrics, dict):
                 continue
-            
+
             for metric_name, threshold_config in metrics.items():
                 if not isinstance(threshold_config, dict):
                     continue
-                
+
                 # Create metric path like "disk_monitor./dev/sda1.percent"
                 metric_path = f"{plugin_name}.{partition}.{metric_name}"
-                
+
                 warning = threshold_config.get("warning")
                 critical = threshold_config.get("critical")
                 operator = threshold_config.get("operator", ">")
@@ -700,7 +702,7 @@ class ThresholdChecker:
                     display=display,
                     grace=grace,
                 )
-                
+
                 target_dict[metric_path] = threshold
 
     def _parse_pool_thresholds(
@@ -761,26 +763,26 @@ class ThresholdChecker:
         target_dict: Optional[Dict[str, ThresholdConfig]] = None
     ):
         """Parse RTT thresholds (network latency thresholds).
-        
+
         RTT thresholds are configured as:
         thresholds:
           rtt:
             warning: 100.0   # ms
             critical: 500.0  # ms
-        
+
         Args:
             rtt_thresholds: RTT threshold configuration
             target_dict: Dictionary to store parsed thresholds
         """
         if target_dict is None:
             target_dict = self.thresholds
-        
+
         if not isinstance(rtt_thresholds, dict):
             return
-        
+
         # Metric path is simply "rtt" (not per-host)
         metric_path = "rtt"
-        
+
         warning = rtt_thresholds.get("warning")
         critical = rtt_thresholds.get("critical")
         operator = rtt_thresholds.get("operator", ">")
@@ -813,7 +815,7 @@ class ThresholdChecker:
             critical,
             count,
         )
-    
+
     def get_thresholds_for_host(self, host_name: str) -> Dict[str, ThresholdConfig]:
         """Get the effective threshold configuration for a host.
 
@@ -827,7 +829,7 @@ class ThresholdChecker:
             Dictionary of thresholds for this host
         """
         # Legacy mode: single threshold set for all hosts
-        if self.thresholds and not self.threshold_configs:
+        if self.thresholds:
             return self.thresholds
 
         if not self.threshold_configs:
@@ -864,7 +866,7 @@ class ThresholdChecker:
             else:
                 result.update(raw)
         return result
-    
+
     def check_value(
         self,
         host_name: str,
@@ -874,30 +876,30 @@ class ThresholdChecker:
     ) -> Optional[Tuple[AlertLevel, AlertLevel]]:
         """
         Check a single value against configured threshold.
-        
+
         Args:
             host_name: Name of the host
             metric_path: Full metric path (e.g., "rtt.hostname")
             value: The metric value to check
             alert_states: Host's alert_states dictionary
-            
+
         Returns:
             Tuple of (old_level, new_level) if state changed, None otherwise
         """
         # Get host-specific thresholds
         thresholds = self.get_thresholds_for_host(host_name)
-        
+
         if metric_path not in thresholds:
             return None
-        
+
         threshold = thresholds[metric_path]
-        
+
         # Get or create alert state
         if metric_path not in alert_states:
             alert_states[metric_path] = AlertState(metric_path)
-        
+
         alert_state = alert_states[metric_path]
-        
+
         # Evaluate threshold with hysteresis
         new_level = threshold.evaluate_with_hysteresis(
             value,
@@ -946,6 +948,7 @@ class ThresholdChecker:
             self._check_pending_or_renotify(host_name, alert_state, metric_path, value, threshold, None)
 
         return None
+
     def _find_threshold(
         self, thresholds: Dict[str, "ThresholdConfig"], metric_path: str
     ) -> Tuple[Optional["ThresholdConfig"], Optional[str]]:
@@ -985,21 +988,21 @@ class ThresholdChecker:
     ) -> list:
         """
         Check plugin data against configured thresholds.
-        
+
         Args:
             host_name: Name of the host
             plugin_name: Name of the plugin
             data: Plugin data dictionary
             alert_states: Host's alert_states dictionary
-            
+
         Returns:
             List of (metric_path, old_level, new_level, value) tuples for state changes
         """
         state_changes = []
-        
+
         # Get host-specific thresholds
         thresholds = self.get_thresholds_for_host(host_name)
-        
+
         # Check flat metrics
         for metric_name, value in data.items():
             metric_path = f"{plugin_name}.{metric_name}"
@@ -1045,9 +1048,9 @@ class ThresholdChecker:
             alert_states,
             state_changes
         )
-        
+
         return state_changes
-    
+
     def _check_nested_metrics(
         self,
         host_name: str,
@@ -1059,7 +1062,7 @@ class ThresholdChecker:
         """Check nested metrics like partition-specific thresholds."""
         # Get host-specific thresholds
         thresholds = self.get_thresholds_for_host(host_name)
-        
+
         # ZFS pool health checks
         if plugin_name == "zfs_monitor" and "pools" in data:
             pools = data["pools"]
@@ -1103,29 +1106,29 @@ class ThresholdChecker:
             partitions = data["partitions"]
             if not isinstance(partitions, dict):
                 return
-            
+
             for partition, metrics in partitions.items():
                 if not isinstance(metrics, dict):
                     continue
-                
+
                 for metric_name, value in metrics.items():
                     metric_path = f"{plugin_name}.{partition}.{metric_name}"
-                    
+
                     if metric_path not in thresholds:
                         continue
-                    
+
                     threshold = thresholds[metric_path]
-                    
+
                     if metric_path not in alert_states:
                         alert_states[metric_path] = AlertState(metric_path)
-                    
+
                     alert_state = alert_states[metric_path]
-                    
+
                     new_level = threshold.evaluate_with_hysteresis(
                         value,
                         alert_state.level
                     )
-                    
+
                     # Determine which threshold was exceeded
                     threshold_value = None
                     if new_level == AlertLevel.CRITICAL and threshold.critical is not None:
@@ -1141,7 +1144,7 @@ class ThresholdChecker:
                         self._apply_grace(host_name, alert_state, metric_path, old_level, new_level, value, threshold, data)
                     elif new_level != AlertLevel.OK:
                         self._check_pending_or_renotify(host_name, alert_state, metric_path, value, threshold, data)
-    
+
     def _trigger_notification(
         self,
         host_name: str,
@@ -1155,7 +1158,7 @@ class ThresholdChecker:
         metric_name: Optional[str] = None,
     ):
         """Trigger a notification for an alert state change.
-        
+
         Args:
             host_name: Name of the host
             metric_path: Full metric path
@@ -1171,7 +1174,7 @@ class ThresholdChecker:
             threshold_value = threshold.critical
         elif new_level == AlertLevel.WARNING and threshold.warning is not None:
             threshold_value = threshold.warning
-        
+
         # Format operator symbol
         op_symbol = threshold.operator.value
 
@@ -1223,7 +1226,7 @@ class ThresholdChecker:
         formatted_threshold_msg = _fmt() if has_display and new_level != AlertLevel.OK else None
 
         return lvl, message, formatted_threshold_msg
-    
+
     def _send_notification(
         self,
         host_name: str,
@@ -1254,7 +1257,7 @@ class ThresholdChecker:
                 service=short_path,
             ),
         ))
-        
+
         # Log to journal
         if self.journal is not None:
             try:
@@ -1270,10 +1273,10 @@ class ThresholdChecker:
                 logger.debug(f"Failed to log threshold event to journal: {e}")
         # Log to eventlog as well
         eventlog(host_name, lvl,  message, service="threshold")
-    
+
     def _format_display(
         self,
-        display_format: str,
+        display_format: Optional[str],
         value: Any,
         threshold_value: Optional[float],
         op_symbol: str,
@@ -1330,7 +1333,7 @@ class ThresholdChecker:
                 status = plugin_data.get(f"{check_name}_status")
                 if status is not None:
                     format_context['status'] = status
-        
+
         try:
             # Format the display string
             return display_format.format(**format_context)
@@ -1349,7 +1352,7 @@ class ThresholdChecker:
                 e
             )
             return f"(threshold: {op_symbol} {threshold_value})"
-    
+
     def _apply_grace(
         self,
         host_name: str,
@@ -1472,7 +1475,7 @@ class ThresholdChecker:
         metric_name: Optional[str] = None,
     ):
         """Check if we should send a repeat notification.
-        
+
         Args:
             host_name: Name of the host
             alert_state: Current alert state
@@ -1487,16 +1490,16 @@ class ThresholdChecker:
         # Skip reminders if alert has been acknowledged
         if alert_state.acknowledged:
             return
-        
+
         now = time.time()
-        
+
         # Check if we should re-notify
         if alert_state.last_notification is None:
             # First notification already sent during state change
             alert_state.last_notification = now
             alert_state.notification_count = 1
             return
-        
+
         if (now - alert_state.last_notification) >= self.renotify_interval:
             # Determine which threshold is active
             threshold_value = None
@@ -1504,7 +1507,7 @@ class ThresholdChecker:
                 threshold_value = threshold.critical
             elif alert_state.level == AlertLevel.WARNING and threshold.warning is not None:
                 threshold_value = threshold.warning
-            
+
             # Format operator symbol
             op_symbol = threshold.operator.value
             short_path = (metric_path.partition(".")[2] or metric_path).removesuffix("_status_code")
@@ -1541,7 +1544,7 @@ class ThresholdChecker:
                 logger.info("Re-notification sent: %s", message)
             alert_state.last_notification = now
             alert_state.notification_count += 1
-    
+
     def purge_stale_alerts(self, hbdclass) -> None:
         """Remove alert states that have no matching threshold configuration.
 
@@ -1578,10 +1581,10 @@ class ThresholdChecker:
     def get_active_alerts(self, alert_states: Dict[str, AlertState]) -> list:
         """
         Get all currently active (non-OK) alerts.
-        
+
         Args:
             alert_states: Host's alert_states dictionary
-            
+
         Returns:
             List of AlertState objects that are not OK
         """
@@ -1589,19 +1592,19 @@ class ThresholdChecker:
             state for state in alert_states.values()
             if state.level != AlertLevel.OK
         ]
-    
+
     def get_alert_summary(self, alert_states: Dict[str, AlertState]) -> Dict[str, int]:
         """
         Get summary counts of alert levels.
-        
+
         Args:
             alert_states: Host's alert_states dictionary
-            
+
         Returns:
             Dictionary with counts: {"ok": N, "warning": N, "critical": N}
         """
         summary = {"ok": 0, "warning": 0, "critical": 0, "unknown": 0}
-        
+
         for state in alert_states.values():
             if state.level == AlertLevel.OK:
                 summary["ok"] += 1
@@ -1611,5 +1614,5 @@ class ThresholdChecker:
                 summary["critical"] += 1
             elif state.level == AlertLevel.UNKNOWN:
                 summary["unknown"] += 1
-        
+
         return summary

hbd/server/udp.py

@@ -5,6 +5,7 @@ import socket
 import struct
 import time
 import zlib
+from typing import Optional
 import logging
 
 from platform import system as platform_system
@@ -23,14 +24,15 @@ _STALE_INTERVAL_MULTIPLIER = 3
 
 # SO_TIMESTAMP: kernel attaches a struct timeval to each received datagram.
 # Supported on Linux, FreeBSD, and macOS.  The constant is not exposed by
-# Python's socket module on all platforms 
+# Python's socket module on all platforms
 platform = platform_system()
+_SO_TIMESTAMP: Optional[int]
 if platform == "Darwin":
     _SO_TIMESTAMP = 1024  # SO_TIMESTAMP on macOS (not in Python's socket module)
 elif platform == "Linux":
     _SO_TIMESTAMP = 29  # Linux value (not in older Python versions)
 elif platform == "FreeBSD":
-     _SO_TIMESTAMP = 32  # FreeBSD value (not in older Python versions)
+    _SO_TIMESTAMP = 32  # FreeBSD value (not in older Python versions)
 else:
     logger.warning("SO_TIMESTAMP may not be supported on this platform (%s)", platform)
     _SO_TIMESTAMP = None
@@ -45,6 +47,8 @@ def enable_kernel_timestamps(sock) -> bool:
     Returns True if the kernel will supply receive timestamps, False otherwise
     (unsupported platform, older kernel, or insufficient permissions).
     """
+    if _SO_TIMESTAMP is None:
+        return False
     try:
         sock.setsockopt(socket.SOL_SOCKET, _SO_TIMESTAMP, 1)
         return True
@@ -71,6 +75,7 @@ class RecvmsgTransport:
     Exposes the same sendto() / close() interface as asyncio's DatagramTransport
     so the rest of the code does not need to know which path is in use.
     """
+
     def __init__(self, loop, sock):
         self._loop = loop
         self._sock = sock
@@ -194,6 +199,40 @@ def _set_connectivity_alert(host, afam, level_name):
     state.update(level, level_name)
 
 
+_PRESENCE_KEYS = {"p4": "IPv4", "p6": "IPv6"}
+
+
+def _apply_presence(host, msg, now, uname, ctx, skip_conn):
+    """Record client-reported protocol presence and settle the affected connections.
+
+    A family reported absent whose connection is already overdue/unknown is parked as
+    ABSENT (timer cancelled, connectivity alert cleared). One still UP is left alone; its
+    timer expiry parks it. A family reported present again while ABSENT restarts a grace
+    timer. The connection that carried this very heartbeat (skip_conn) is never touched.
+    """
+    cfg = ctx.get("config", {})
+    for key, afam in _PRESENCE_KEYS.items():
+        if key not in msg:
+            continue
+        present = bool(msg[key])
+        host.set_presence(afam, present, now)
+        conn = host.connections.get(afam)
+        if conn is None or conn is skip_conn:
+            continue
+        cls = conn.__class__
+        if not present:
+            if conn.getstate() in (cls.OVERDUE, cls.UNKNOWN):
+                conn.cancel_overdue_timer()
+                conn.newstate(cls.ABSENT, now)
+                _set_connectivity_alert(host, afam, "OK")
+        elif conn.getstate() == cls.ABSENT:
+            conn.newstate(cls.UNKNOWN, now)
+            interval = int(msg.get("interval", 0) or 0) or host.interval
+            if interval > 0:
+                on_overdue, _ = _make_timer_callbacks(uname, host, ctx)
+                conn.reset_overdue_timer(interval + cfg.get("grace", 2), on_overdue)
+
+
 def _make_timer_callbacks(uname, host, ctx):
     """Return (on_overdue, on_unknown) async callbacks for connection timer logic.
 
@@ -210,26 +249,52 @@ def _make_timer_callbacks(uname, host, ctx):
             msg_to_websockets("host", host.stateinfo())
 
     async def on_overdue(connection):
-        if connection.getstate() != connection.__class__.UP:
+        cls = connection.__class__
+        if connection.getstate() not in (cls.UP, cls.UNKNOWN):
             return
         now = time.time()
-        connection.newstate(connection.__class__.OVERDUE, now, cfg.get("grace", 2))
-        msg = f"{connection.afam} overdue"
-        eventlog(uname, "CRITICAL", msg)
-        if host.watched:
-            asyncio.create_task(notify_mod.send_notification(
-                uname,
-                notify_mod.Notification(title=f"[CRITICAL] {uname}", body=msg, level="CRITICAL"),
-            ))
-        # Track in alert_states so the Alerts Dashboard shows this
-        _set_connectivity_alert(host, connection.afam, "CRITICAL")
-        if threshold_checker:
-            threshold_checker.check_value(
-                host_name=uname,
-                metric_path="rtt",
-                value=float("inf"),
-                alert_states=host.alert_states,
-            )
+        afam = connection.afam
+        # Park only while some other family is still present: the last monitored family
+        # must never be parked silently, or a host whose client reports no internet on any
+        # protocol would die without an alert.
+        if not host.flag_present(afam) and any(
+                af != afam and host.family_present(af) for af in host.connections):
+            # The client says this protocol has no connectivity: not an outage.
+            connection.newstate(cls.ABSENT, now)
+            connection.cancel_overdue_timer()
+            _set_connectivity_alert(host, afam, "OK")
+            if msg_to_websockets:
+                msg_to_websockets("host", host.stateinfo())
+            return
+
+        # If any connection is DOWN (clean shutdown reported) Host.status() is "down", so a
+        # later overdue of the other family raises no separate event/alert; the shutdown did.
+        prev = host.status()
+        connection.newstate(cls.OVERDUE, now, cfg.get("grace", 2))
+        new = host.status()
+        if new == "overdue" and prev != "overdue":
+            names = ", ".join(sorted(af for af, c in host.connections.items()
+                                     if c.getstate() == cls.OVERDUE))
+            msg = f"{names} overdue"
+            eventlog(uname, "CRITICAL", msg)
+            if host.watched:
+                asyncio.create_task(notify_mod.send_notification(
+                    uname,
+                    notify_mod.Notification(title=f"[CRITICAL] {uname}", body=msg, level="CRITICAL"),
+                ))
+            # Track in alert_states so the Alerts Dashboard shows this
+            for af, c in host.connections.items():
+                if c.getstate() == cls.OVERDUE:
+                    _set_connectivity_alert(host, af, "CRITICAL")
+            if threshold_checker:
+                threshold_checker.check_value(
+                    host_name=uname,
+                    metric_path="rtt",
+                    value=float("inf"),
+                    alert_states=host.alert_states,
+                )
+        elif new == "simplex":
+            eventlog(uname, "INFO", f"{afam} overdue (simplex)")
         if msg_to_websockets:
             msg_to_websockets("host", host.stateinfo())
         connection.reset_overdue_timer(DROPOVERDUE, on_unknown)
@@ -270,6 +335,8 @@ def restore_connection_timers(hbdclass, ctx):
         interval = host.interval
         for afam, conn in list(host.connections.items()):
             state = conn.getstate()
+            if state == hbdclass.Connection.ABSENT:
+                continue
             if state == hbdclass.Connection.DOWN:
                 _set_connectivity_alert(host, afam, "CRITICAL")
                 continue
@@ -295,6 +362,7 @@ def restore_connection_timers(hbdclass, ctx):
                 restored += 1
 
             elif state == hbdclass.Connection.OVERDUE:
+                host_overdue = host.status() == "overdue"
                 elapsed_overdue = now - conn.statetime
                 remaining = DROPOVERDUE - elapsed_overdue
                 if remaining <= 1:
@@ -304,6 +372,9 @@ def restore_connection_timers(hbdclass, ctx):
                         "Marking %s/%s UNKNOWN (overdue %.1f days)",
                         uname, afam, elapsed_overdue / 86400,
                     )
+                    # A connection dropped to UNKNOWN keeps its alert, like the
+                    # UNKNOWN branch above and on_unknown do.
+                    host_overdue = True
                 else:
                     conn.reset_overdue_timer(remaining, on_unknown)
                     logger.debug(
@@ -313,7 +384,8 @@ def restore_connection_timers(hbdclass, ctx):
                 # Ensure the connectivity alert is set — it may be missing if
                 # hbd was shut down before the on_overdue callback had a chance
                 # to record it.
-                _set_connectivity_alert(host, afam, "CRITICAL")
+                if host_overdue:
+                    _set_connectivity_alert(host, afam, "CRITICAL")
                 restored += 1
 
     logger.info("Restored timers for %d connection(s)", restored)
@@ -343,7 +415,7 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
     if not msg:
         return
     now = ctx.get("recv_ts") or time.time()
-    
+
     # Log message to journal
     msg_journal = ctx.get("msg_journal")
     if msg_journal:
@@ -354,16 +426,21 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
             loop.create_task(msg_journal.log_message(msg, addr, now))
         except Exception as e:
             logger.debug(f"Failed to log message to journal: {e}")
-    
+
+    # Only named HTB/PLG datagrams come from clients; anything else (e.g. an internet scanner's
+    # SIP probe) is journaled above but must not sign on a host.
+    if msg.get("ID") not in ("HTB", "PLG") or not msg.get("name"):
+        logger.debug("Ignoring non-client datagram from %s", addr)
+        return
+
     cfg = ctx.get("config", {})
-    hbdcls = ctx.get("hbdclass")
+    hbdcls = ctx["hbdclass"]
     msg_to_websockets = ctx.get("msg_to_websockets")
     DEBUG = ctx.get("DEBUG", 0)
-    verbose = ctx.get("verbose", False)
 
     # normalize addr (ip, port)
     ip = addr[0] if isinstance(addr, (list, tuple)) else addr
-    name = msg.get("name", "unknown")
+    name = msg["name"]
     from ..common.utils import shortname
     from . import config as config_mod
 
@@ -384,12 +461,10 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
     else:
         host = hbdcls.Host.hosts[uname]
         newh = False
-    
+
     cid = msg.get("id", 0)
-    try:
-        rtt = float(msg.get("rtt"))
-    except TypeError:
-        rtt = None
+    rtt_raw = msg.get("rtt")
+    rtt = float(rtt_raw) if rtt_raw is not None else None
 
     if msg.get("ID") == "HTB":
         host.doesack = msg.get("acks", -1)
@@ -450,14 +525,14 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
 
             # If os_info reports an owner and none is configured server-side, apply it
             if plugin_name == "os_info":
-                config_owner =  config_mod.get_host_access(cfg, uname).get("owner")
+                config_owner = config_mod.get_host_access(cfg, uname).get("owner")
                 default_owner = config_mod.get_default_owner(cfg)
                 inferred_owner = config_owner or plugin_data.get("owner") or default_owner
                 host.owner = inferred_owner
                 logger.info(f"owner for {uname} is {host.owner}")
             if DEBUG > 1:
                 print(f"Stored plugin data for {uname}: {plugin_name}")
-            
+
             # Check thresholds if checker is available
             threshold_checker = ctx.get("threshold_checker")
             if threshold_checker:
@@ -472,7 +547,7 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
                         print(f"Threshold state changes for {uname}: {state_changes}")
                 except Exception as e:
                     logger.error(f"Error checking thresholds for {uname}.{plugin_name}: {e}")
-            
+
             # Notify websockets of plugin update
             if msg_to_websockets:
                 try:
@@ -517,6 +592,9 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
     if message:
         eventlog(uname, "INFO", message, service=service)
 
+    _apply_presence(host, msg, now, uname, ctx, conn)
+    prev_status = host.status()
+
     if conn.getstate() != hbdcls.Connection.UP:
         # Transition to UP and log/notify if appropriate
         lasts = conn.state
@@ -538,8 +616,12 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
         ]
         for k in stale_plugin_keys:
             del host.alert_states[k]
-        # Clear connectivity alert now that the host is back up
+        # Clear connectivity alert now that the host is back up; once the host is no longer
+        # overdue no family may keep a connectivity alert (e.g. the simplex case).
         _set_connectivity_alert(host, conn.afam, "OK")
+        if host.status() in ("up", "simplex"):
+            for af in host.connections:
+                _set_connectivity_alert(host, af, "OK")
         # Don't log/notify RECOVER for a brand-new host seen for the first time —
         # it was never down, it just hasn't been seen before.
         if not newh:
@@ -551,8 +633,11 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
             else:
                 m = "%s back after being %s for %s" % (conn.afam, lasts, dur(d))
             if m:
-                eventlog(uname, "RECOVER", m)
-                if host.watched:
+                # RECOVER (and a notification) only if the host itself was overdue/down;
+                # a single family coming back while another stayed up is informational.
+                level = "RECOVER" if prev_status in ("overdue", "down", "unknown") else "INFO"
+                eventlog(uname, level, m)
+                if host.watched and level == "RECOVER":
                     asyncio.create_task(notify_mod.send_notification(
                         uname,
                         notify_mod.Notification(title=f"[RECOVER] {uname}", body=m, level="RECOVER"),
@@ -588,13 +673,13 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
         timeout_seconds = interval + grace
         on_overdue, _ = _make_timer_callbacks(uname, host, ctx)
         conn.reset_overdue_timer(timeout_seconds, on_overdue)
-    
+
     # Check RTT thresholds using the threshold checker
     threshold_checker = ctx.get("threshold_checker")
     if threshold_checker and rtt and rtt > 0:
         # Metric path for RTT is simply "rtt"
         metric_path = "rtt"
-        
+
         # Check against configured thresholds (handles alerts, notifications, etc.)
         threshold_checker.check_value(
             host_name=uname,

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "hbd"
-version = "5.4.3"
+version = "5.4.7"
 description = "Heartbeat monitoring system — client (hbc) and server (hbd)"
 readme = "README.md"
 requires-python = ">=3.11"
@@ -58,9 +58,12 @@ all = [
 # Development dependencies
 dev = [
   "pytest>=7.0",
+  "pytest-asyncio>=0.23",
   "pytest-cov>=4.0",
   "flake8>=5.0",
   "mypy>=1.10",
+  "types-PyYAML",
+  "types-psutil",
   "black>=23.0",
   "isort>=5.0",
   "tox>=4.0",
@@ -83,10 +86,18 @@ include = ["hbd*"]
 
 
 [tool.black]
-line-length = 111
+line-length = 165
 
 [tool.flake8]
-max-line-length = 111
+max-line-length = 165
 
 [tool.pylint.format]
 max-line-length = 111
+
+[[tool.mypy.overrides]]
+# optional notification libraries that ship neither stubs nor py.typed
+module = ["nio", "nio.*", "mattermostdriver", "mattermostdriver.*"]
+ignore_missing_imports = true
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

scripts/bumpminor.sh

@@ -9,8 +9,10 @@ sed -i".bak"  "s/\*\*Package:\*\* \`hbd\` v[0-9.]*/\*\*Package:\*\* \`hbd\` v$VE
 
 # Update CHANGELOG.md with commits since last tag
 LASTTAG=$(git describe --tags --abbrev=0 2>/dev/null || true)
-ADDED=$(git log "${LASTTAG:+$LASTTAG..}HEAD" --pretty="%s" | grep "^feat:" | sed 's/^feat: /- /')
-FIXED=$(git log "${LASTTAG:+$LASTTAG..}HEAD" --pretty="%s" | grep "^fix:" | sed 's/^fix: /- /')
+# Matches both unscoped ("feat: ...") and scoped ("feat(server): ...") conventional-commit
+# subjects; the scope is dropped from the changelog line.
+ADDED=$(git log "${LASTTAG:+$LASTTAG..}HEAD" --pretty="%s" | grep -E "^feat(\([^)]*\))?:" | sed -E 's/^feat(\([^)]*\))?: /- /')
+FIXED=$(git log "${LASTTAG:+$LASTTAG..}HEAD" --pretty="%s" | grep -E "^fix(\([^)]*\))?:" | sed -E 's/^fix(\([^)]*\))?: /- /')
 {
     printf "## [%s]\n" "$VER"
     [ -n "$ADDED" ] && printf "\n### Added\n%s\n" "$ADDED"

scripts/c/.gitignore

@@ -1,2 +1,3 @@
 hbc_mini
 hbc_mini_dbg
+test_probe

scripts/c/Makefile

@@ -1,13 +1,15 @@
 CC      ?= cc
 CFLAGS  = -O2 -Wall -Wextra -std=c11
 LDFLAGS = -lz -lpthread -lm
+# res_search() lives in libresolv on older glibc; in libc on newer glibc, musl and the BSDs
+LDFLAGS += $(shell echo 'int main(void){return 0;}' | $(CC) -x c - -lresolv -o /dev/null 2>/dev/null && echo -lresolv)
 TARGET  = hbc_mini
 SRC     = hbc_mini.c
 
 # FreeBSD/NetBSD keep zlib in base; no extra flags needed.
 # On some NetBSD installs pthreads may need -lpthread from pkgsrc.
 
-.PHONY: all clean debug
+.PHONY: all clean debug test
 
 all: $(TARGET)
 
@@ -17,5 +19,11 @@ $(TARGET): $(SRC)
 debug: $(SRC)
 	$(CC) -g -fsanitize=address,undefined -o $(TARGET)_dbg $< $(LDFLAGS)
 
+test: test_probe
+	./test_probe
+
+test_probe: test_probe.c $(SRC)
+	$(CC) $(CFLAGS) -o $@ test_probe.c $(LDFLAGS)
+
 clean:
-	rm -f $(TARGET) $(TARGET)_dbg
+	rm -f $(TARGET) $(TARGET)_dbg test_probe

scripts/c/hbc_mini.c

@@ -6,6 +6,12 @@
  *
  * Drop on any host and run: ./hbc_mini <hbd-host>
  * Config: ~/.hbc.json (same keys as Python version)
+ *
+ * Presence probe (built in): reports per protocol (IPv4/IPv6) whether the host has internet
+ * connectivity, as p4/p6 flags on every heartbeat, so hbd stops treating a missing IPv6 (or
+ * IPv4) path as an outage. Config, in the network_monitor section of ~/.hbc.json:
+ *   probe_enabled (default true), probe_v4 (default 1.1.1.1), probe_v6 (default
+ *   2606:4700:4700::1111; IP literals only), probe_interval (default 60 s)
  */
 
 #ifdef __linux__
@@ -14,6 +20,7 @@
 #endif
 
 #include <arpa/inet.h>
+#include <arpa/nameser.h>
 #include <netinet/in.h>
 #include <ctype.h>
 #include <errno.h>
@@ -21,6 +28,7 @@
 #include <math.h>
 #include <netdb.h>
 #include <pthread.h>
+#include <resolv.h>
 #include <signal.h>
 #include <stdarg.h>
 #include <stdbool.h>
@@ -396,6 +404,10 @@ typedef struct {
 
     char net_skip[MAX_NET_SKIP][64];
     int  net_skip_count;
+
+    bool probe_enabled;
+    int  probe_interval;
+    char probe_v4[64], probe_v6[64];
 } config_t;
 
 static void config_defaults(config_t *c) {
@@ -410,6 +422,10 @@ static void config_defaults(config_t *c) {
     c->net_interval   = 300;
     snprintf(c->net_skip[0], 64, "lo");
     c->net_skip_count = 1;
+    c->probe_enabled  = true;
+    c->probe_interval = 60;
+    snprintf(c->probe_v4, sizeof(c->probe_v4), "1.1.1.1");
+    snprintf(c->probe_v6, sizeof(c->probe_v6), "2606:4700:4700::1111");
 }
 
 static void config_load(config_t *cfg, const char *path) {
@@ -498,6 +514,10 @@ static void config_load(config_t *cfg, const char *path) {
     jval_t *net = plugins ? jget(plugins,"network_monitor") : jget(root,"network_monitor");
     if (net && net->type == JT_OBJECT) {
         if ((v=jget(net,"interval"))) cfg->net_interval = jint(v, cfg->net_interval);
+        if ((v=jget(net,"probe_enabled")))  cfg->probe_enabled  = jint(v, 1) != 0;
+        if ((v=jget(net,"probe_interval"))) cfg->probe_interval = jint(v, cfg->probe_interval);
+        if ((v=jget(net,"probe_v4"))) snprintf(cfg->probe_v4, sizeof(cfg->probe_v4), "%s", jstr(v, cfg->probe_v4));
+        if ((v=jget(net,"probe_v6"))) snprintf(cfg->probe_v6, sizeof(cfg->probe_v6), "%s", jstr(v, cfg->probe_v6));
         jval_t *sk = jget(net,"skip_interfaces");
         if (sk && sk->type == JT_ARRAY) {
             cfg->net_skip_count = 0;
@@ -558,6 +578,7 @@ static int conn_send(conn_t *c, const char *msg_id, kvdict_t *d) {
 
     struct sockaddr_storage sa; socklen_t salen = 0;
     memset(&sa, 0, sizeof(sa));
+    pthread_mutex_lock(&c->mu);     /* conn_retarget() may change addr/af/sockfd */
     if (c->af == AF_INET) {
         struct sockaddr_in *s4 = (struct sockaddr_in *)&sa;
         s4->sin_family = AF_INET;
@@ -572,7 +593,6 @@ static int conn_send(conn_t *c, const char *msg_id, kvdict_t *d) {
         salen = sizeof(*s6);
     }
 
-    pthread_mutex_lock(&c->mu);
     c->last_send = now_ts();
     ssize_t sent = sendto(c->sockfd, wire, wlen, 0, (struct sockaddr *)&sa, salen);
     pthread_mutex_unlock(&c->mu);
@@ -984,6 +1004,341 @@ static void plugin_disk_monitor(conn_t *c, const config_t *cfg) {
     LOGD("sent disk_monitor");
 }
 
+/* ============================================================
+ * Connectivity probe: per-protocol internet presence (IPv4 / IPv6)
+ *
+ * Tier 1 (every PROBE_TIER1 s, sends no packets): a UDP connect() to the probe address shows
+ * whether a route exists and which source address would be used. Tier 2 (every probe_interval
+ * s, sooner after a failure or a change of address or gateway): one ICMP echo, only when tier 1
+ * found a route. Absent = no route, or PROBE_FAIL_LIMIT consecutive failed echoes; present =
+ * one success; unknown (flag omitted) when the family cannot be probed. The result rides on
+ * every HTB as p4/p6 and inside the network_monitor metric. Mirrors hbd/client/connectivity.py.
+ * ============================================================ */
+
+#define PROBE_UNKNOWN     (-1)
+#define PROBE_FAIL_LIMIT  2
+#define PROBE_TIER1       5      /* seconds between route checks */
+
+typedef struct {
+    int    present;                          /* PROBE_UNKNOWN, 0 = absent, 1 = present */
+    char   addr[INET6_ADDRSTRLEN];
+    char   gateway[INET6_ADDRSTRLEN];
+    double rtt;                              /* ms, < 0 when unknown */
+    int    fails;                            /* consecutive failed echoes */
+    double next_ping;                        /* monotonic seconds */
+    char   last_key[2 * INET6_ADDRSTRLEN + 2];
+    bool   ping_disabled;                    /* no usable ping binary */
+} probe_fam_t;
+
+typedef struct {
+    /* 1 = route exists (source address in out), 0 = no route, -1 = family unsupported/unusable */
+    int    (*local_address)(int af, const char *target, char *out, size_t n);
+    /* RTT in ms; -1 = no reply; -2 = ping unavailable */
+    double (*ping)(int af, const char *target);
+    /* fills out with the default gateway, "" when unknown */
+    void   (*gateway)(int af, char *out, size_t n);
+} probe_ops_t;
+
+typedef struct {
+    probe_fam_t     fam[2];                  /* [0] = IPv4, [1] = IPv6 */
+    char            target[2][64];
+    double          probe_interval, retry_interval;
+    bool            cheap_gateway;           /* gateway read is cheap (/proc), so read every tier 1 */
+    pthread_mutex_t mu;                      /* guards fam[] between the probe thread and readers */
+} probe_t;
+
+/* Lowest-metric default gateway from /proc/net/route. The kernel prints the raw network-order
+ * u32 in host order, so it can be stored straight into s_addr on any endianness. */
+static void gw_parse_proc_v4(const char *text, char *out, size_t n) {
+    long best = -1; out[0] = '\0';
+    const char *p = strchr(text, '\n');                    /* skip the header line */
+    while (p && *++p) {
+        char iface[64], dest[16], gw[16]; unsigned flags; int refcnt, use, metric;
+        if (sscanf(p, "%63s %15s %15s %x %d %d %d", iface, dest, gw, &flags, &refcnt, &use, &metric) == 7
+                && !strcmp(dest, "00000000") && (flags & 0x2)) {
+            if (best < 0 || metric < best) {
+                struct in_addr a; a.s_addr = (uint32_t)strtoul(gw, NULL, 16);
+                if (inet_ntop(AF_INET, &a, out, n)) best = metric;
+            }
+        }
+        p = strchr(p, '\n');
+    }
+}
+
+/* Lowest-metric default gateway from /proc/net/ipv6_route (skips RTF_REJECT and "lo"). */
+static void gw_parse_proc_v6(const char *text, char *out, size_t n) {
+    long best = -1; out[0] = '\0';
+    const char *zero = "00000000000000000000000000000000";
+    for (const char *p = text; p && *p; ) {
+        char dest[40], src[40], nh[40], iface[64];
+        unsigned plen, splen, metric, refcnt, use, flags;
+        if (sscanf(p, "%32s %x %32s %x %32s %x %x %x %x %63s",
+                   dest, &plen, src, &splen, nh, &metric, &refcnt, &use, &flags, iface) == 10
+                && !strcmp(dest, zero) && plen == 0 && strcmp(nh, zero) && strcmp(iface, "lo")
+                && (flags & 0x2) && !(flags & 0x200)) {
+            if (best < 0 || (long)metric < best) {
+                struct in6_addr a; memset(&a, 0, sizeof(a));
+                bool ok = strlen(nh) == 32;
+                for (int i = 0; ok && i < 16; i++) {
+                    unsigned b;
+                    if (sscanf(nh + 2 * i, "%2x", &b) != 1) ok = false; else a.s6_addr[i] = (uint8_t)b;
+                }
+                if (ok && inet_ntop(AF_INET6, &a, out, n)) best = (long)metric;
+            }
+        }
+        p = strchr(p, '\n'); if (p) p++;
+    }
+}
+
+/* Gateway from `route -n get [-inet6] default` (the BSDs). */
+static void gw_parse_route_get(const char *text, char *out, size_t n) {
+    out[0] = '\0';
+    const char *p = strstr(text, "gateway:");
+    if (!p) return;
+    p += 8;
+    while (*p == ' ' || *p == '\t') p++;
+    size_t i = 0;
+    while (*p && !isspace((unsigned char)*p) && i + 1 < n) out[i++] = *p++;
+    out[i] = '\0';
+}
+
+/* Average RTT from ping output: -1.0 no reply, -2.0 ping could not be run. Works for Linux
+ * ("rtt min/avg/max/mdev") and BSD ("round-trip min/avg/max/stddev"). */
+static double ping_parse_rtt(const char *out) {
+    if (strstr(out, "not found") || strstr(out, "No such file")) return -2.0;
+    const char *p = strstr(out, "min/avg/max");
+    if (!p) return -1.0;
+    p = strchr(p, '=');
+    double mn, avg, mx;
+    if (p && sscanf(p + 1, " %lf/%lf/%lf", &mn, &avg, &mx) == 3) return avg;
+    return -1.0;
+}
+
+/* One probe step for one family (f: 0 = IPv4, 1 = IPv6). Works on *s only, so the probe thread
+ * can run it on a private copy without holding the mutex during the (blocking) echo. */
+static void probe_step(const probe_t *p, probe_fam_t *s, int f, const probe_ops_t *ops, double now) {
+    int af = f ? AF_INET6 : AF_INET;
+    char addr[INET6_ADDRSTRLEN] = "", gw[INET6_ADDRSTRLEN] = "";
+
+    int r = ops->local_address(af, p->target[f], addr, sizeof(addr));
+    if (r < 0) { s->present = PROBE_UNKNOWN; return; }
+    if (r == 0) {                                          /* tier 1: no route => absent immediately */
+        s->fails = 0; s->next_ping = 0; s->last_key[0] = '\0';
+        s->present = 0; s->addr[0] = '\0'; s->gateway[0] = '\0'; s->rtt = -1;
+        return;
+    }
+
+    if (p->cheap_gateway) ops->gateway(af, gw, sizeof(gw));
+    else snprintf(gw, sizeof(gw), "%s", s->gateway);
+    char key[sizeof(s->last_key)];
+    snprintf(key, sizeof(key), "%s|%s", addr, gw);
+    bool due = strcmp(key, s->last_key) != 0 || now >= s->next_ping;
+    if (due && !p->cheap_gateway) {
+        ops->gateway(af, gw, sizeof(gw));
+        snprintf(key, sizeof(key), "%s|%s", addr, gw);
+    }
+    snprintf(s->addr, sizeof(s->addr), "%s", addr);
+    snprintf(s->gateway, sizeof(s->gateway), "%s", gw);
+    snprintf(s->last_key, sizeof(s->last_key), "%s", key);
+
+    if (s->ping_disabled) { s->present = PROBE_UNKNOWN; return; } /* cannot probe: unknown, never absent, while a route exists */
+    if (!due) return;
+    double rtt = ops->ping(af, p->target[f]);
+    if (rtt < -1.5) { s->ping_disabled = true; s->present = PROBE_UNKNOWN; return; }
+    if (rtt < 0) {
+        s->fails++;
+        s->next_ping = now + p->retry_interval;
+        if (s->fails >= PROBE_FAIL_LIMIT) { s->present = 0; s->rtt = -1; }
+    } else {
+        s->fails = 0;
+        s->next_ping = now + p->probe_interval;
+        s->present = 1; s->rtt = rtt;
+    }
+}
+
+/* HTB flags for the families whose presence is known. */
+static void probe_flags(probe_t *p, kvdict_t *d) {
+    pthread_mutex_lock(&p->mu);
+    if (p->fam[0].present >= 0) kv_set_int(d, "p4", p->fam[0].present);
+    if (p->fam[1].present >= 0) kv_set_int(d, "p6", p->fam[1].present);
+    pthread_mutex_unlock(&p->mu);
+}
+
+/* The `connectivity` metric as a JSON object; returns its length or -1 on overflow. */
+static int probe_metric_json(probe_t *p, char *out, size_t n) {
+    static const char *names[2] = { "ipv4", "ipv6" };
+    size_t len = 0;
+    len += (size_t)snprintf(out + len, n - len, "{");
+    pthread_mutex_lock(&p->mu);
+    for (int f = 0; f < 2 && len < n; f++) {
+        const probe_fam_t *s = &p->fam[f];
+        char pres[8], addr[64], gw[64], rtt[32];
+        snprintf(pres, sizeof(pres), "%s", s->present < 0 ? "null" : s->present ? "true" : "false");
+        if (s->addr[0]) snprintf(addr, sizeof(addr), "\"%s\"", s->addr); else snprintf(addr, sizeof(addr), "null");
+        if (s->gateway[0]) snprintf(gw, sizeof(gw), "\"%s\"", s->gateway); else snprintf(gw, sizeof(gw), "null");
+        if (s->rtt >= 0) snprintf(rtt, sizeof(rtt), "%.3f", s->rtt); else snprintf(rtt, sizeof(rtt), "null");
+        len += (size_t)snprintf(out + len, n - len,
+            "%s\"%s\":{\"present\":%s,\"addr\":%s,\"gateway\":%s,\"rtt\":%s}",
+            f ? "," : "", names[f], pres, addr, gw, rtt);
+    }
+    pthread_mutex_unlock(&p->mu);
+    if (len + 2 > n) { out[0] = '\0'; return -1; }
+    out[len++] = '}'; out[len] = '\0';
+    return (int)len;
+}
+
+static double mono_ts(void) {
+    struct timespec ts;
+    clock_gettime(CLOCK_MONOTONIC, &ts);
+    return ts.tv_sec + ts.tv_nsec * 1e-9;
+}
+
+/* Source address the kernel would use to reach target (an IP literal). A UDP connect() sends
+ * nothing. Returns 1 = route (address in out), 0 = no route, -1 = family unsupported or the
+ * target is not an IP literal (unknown, never absent). */
+static int sys_local_address(int af, const char *target, char *out, size_t n) {
+    int fd = socket(af, SOCK_DGRAM, 0);
+    if (fd < 0) return -1;
+    struct sockaddr_storage sa; socklen_t sl;
+    memset(&sa, 0, sizeof(sa));
+    if (af == AF_INET) {
+        struct sockaddr_in *s4 = (struct sockaddr_in *)&sa;
+        s4->sin_family = AF_INET; s4->sin_port = htons(9);
+        if (inet_pton(AF_INET, target, &s4->sin_addr) != 1) { close(fd); return -1; }
+        sl = sizeof(*s4);
+    } else {
+        struct sockaddr_in6 *s6 = (struct sockaddr_in6 *)&sa;
+        s6->sin6_family = AF_INET6; s6->sin6_port = htons(9);
+        if (inet_pton(AF_INET6, target, &s6->sin6_addr) != 1) { close(fd); return -1; }
+        sl = sizeof(*s6);
+    }
+    int rc = 0;
+    if (connect(fd, (struct sockaddr *)&sa, sl) == 0) {
+        struct sockaddr_storage me; socklen_t ml = sizeof(me);
+        if (getsockname(fd, (struct sockaddr *)&me, &ml) == 0) {
+            const void *src = (af == AF_INET)
+                ? (const void *)&((struct sockaddr_in *)&me)->sin_addr
+                : (const void *)&((struct sockaddr_in6 *)&me)->sin6_addr;
+            if (inet_ntop(af, src, out, (socklen_t)n)) rc = 1;
+        }
+    }
+    close(fd);
+    return rc;
+}
+
+static char *slurp(FILE *f) {
+    size_t cap = 4096, len = 0;
+    char *buf = malloc(cap);
+    if (!buf) return NULL;
+    size_t r;
+    while ((r = fread(buf + len, 1, cap - len - 1, f)) > 0) {
+        len += r;
+        if (len + 1 >= cap) {
+            if (cap >= 65536) break;
+            cap *= 2;
+            char *nb = realloc(buf, cap);
+            if (!nb) { free(buf); return NULL; }
+            buf = nb;
+        }
+    }
+    buf[len] = '\0';
+    return buf;
+}
+
+static bool g_have_timeout = false;   /* timeout(1) exists: bounds ping6 on the BSDs */
+
+/* One echo request: RTT in ms, -1.0 no reply, -2.0 ping could not be run. The target is an IP
+ * literal from the config; anything else is refused before it can reach the shell. */
+static double sys_ping(int af, const char *target) {
+    for (const char *c = target; *c; c++)
+        if (!(isxdigit((unsigned char)*c) || *c == '.' || *c == ':')) return -2.0;
+    char cmd[256];
+#ifdef __linux__
+    snprintf(cmd, sizeof(cmd), "ping -c 1 -W 2 -w 3 %s %s 2>&1", af == AF_INET ? "-4" : "-6", target);
+#else
+    /* BSD ping: -W is in milliseconds, and IPv6 is reached through ping6 */
+    if (af == AF_INET)
+        snprintf(cmd, sizeof(cmd), "%sping -c 1 -W 2000 %s 2>&1", g_have_timeout ? "timeout 4 " : "", target);
+    else
+        snprintf(cmd, sizeof(cmd), "%sping6 -c 1 %s 2>&1", g_have_timeout ? "timeout 4 " : "", target);
+#endif
+    FILE *f = popen(cmd, "r");
+    if (!f) return -2.0;
+    char *out = slurp(f);
+    pclose(f);
+    if (!out) return -1.0;
+    double rtt = ping_parse_rtt(out);
+    free(out);
+    return rtt;
+}
+
+/* Default gateway for the family, "" when unknown. Best effort; never affects presence. */
+static void sys_gateway(int af, char *out, size_t n) {
+    out[0] = '\0';
+#ifdef __linux__
+    FILE *f = fopen(af == AF_INET ? "/proc/net/route" : "/proc/net/ipv6_route", "r");
+    if (!f) return;
+    char *text = slurp(f);
+    fclose(f);
+    if (!text) return;
+    if (af == AF_INET) gw_parse_proc_v4(text, out, n); else gw_parse_proc_v6(text, out, n);
+    free(text);
+#else
+    FILE *f = popen(af == AF_INET ? "route -n get default 2>/dev/null"
+                                  : "route -n get -inet6 default 2>/dev/null", "r");
+    if (!f) return;
+    char *text = slurp(f);
+    pclose(f);
+    if (!text) return;
+    gw_parse_route_get(text, out, n);
+    free(text);
+#endif
+}
+
+static probe_t g_probe;
+static bool    g_probe_enabled = false;
+
+static void probe_init(probe_t *p, const config_t *cfg) {
+    memset(p, 0, sizeof(*p));
+    snprintf(p->target[0], sizeof(p->target[0]), "%s", cfg->probe_v4);
+    snprintf(p->target[1], sizeof(p->target[1]), "%s", cfg->probe_v6);
+    p->probe_interval = cfg->probe_interval > 0 ? cfg->probe_interval : 60;
+    p->retry_interval = 10;
+#ifdef __linux__
+    p->cheap_gateway = true;                 /* /proc read; elsewhere a subprocess only when due */
+#endif
+    p->fam[0].present = p->fam[1].present = PROBE_UNKNOWN;
+    p->fam[0].rtt = p->fam[1].rtt = -1;
+    pthread_mutex_init(&p->mu, NULL);
+    g_have_timeout = system("command -v timeout >/dev/null 2>&1") == 0;
+    g_probe_enabled = cfg->probe_enabled;
+
+    /* Diagnostic only: sys_local_address already turns a non-literal target into PROBE_UNKNOWN
+     * (via inet_pton failing there), so this does not change behavior — it just stops that from
+     * being silent. */
+    struct in_addr  a4;
+    struct in6_addr a6;
+    if (inet_pton(AF_INET, p->target[0], &a4) != 1)
+        LOGW("probe_v4 is not an IPv4 literal: %s", p->target[0]);
+    if (inet_pton(AF_INET6, p->target[1], &a6) != 1)
+        LOGW("probe_v6 is not an IPv6 literal: %s", p->target[1]);
+}
+
+static void *probe_thread(void *arg) {
+    probe_t *p = arg;
+    static const probe_ops_t ops = { sys_local_address, sys_ping, sys_gateway };
+    while (g_running) {
+        double now = mono_ts();
+        for (int f = 0; f < 2 && g_running; f++) {
+            pthread_mutex_lock(&p->mu); probe_fam_t tmp = p->fam[f]; pthread_mutex_unlock(&p->mu);
+            probe_step(p, &tmp, f, &ops, now);          /* may block in ping: no lock held */
+            pthread_mutex_lock(&p->mu); p->fam[f] = tmp; pthread_mutex_unlock(&p->mu);
+        }
+        for (int i = 0; i < PROBE_TIER1 && g_running; i++) sleep(1);
+    }
+    return NULL;
+}
+
 /* ============================================================
  * Plugin: network_monitor
  * Linux: /proc/net/dev
@@ -1077,16 +1432,26 @@ static void plugin_network_monitor(conn_t *c, const config_t *cfg) {
     json[jlen++] = '}'; json[jlen] = '\0';
     memcpy(g_net_prev, curr, cn * sizeof(net_stat_t));
     g_net_prev_n = cn; g_net_prev_ts = now;
-    if (first) { free(json); return; }
-
     kvdict_t d; kv_clear(&d);
     kv_set(&d, "plugin", "network_monitor");
-    char *jval = malloc(MAX_VAL + 1);
-    if (jval) { snprintf(jval, MAX_VAL, "@%s", json); kv_set(&d, "interfaces", jval); free(jval); }
+    if (!first) {
+        char *jval = malloc(MAX_VAL + 1);
+        if (jval) { snprintf(jval, MAX_VAL, "@%s", json); kv_set(&d, "interfaces", jval); free(jval); }
+    }
+    bool have_conn = false;
+    if (g_probe_enabled) {
+        char cj[512];
+        cj[0] = '@';
+        if (probe_metric_json(&g_probe, cj + 1, sizeof(cj) - 1) > 0) {
+            kv_set(&d, "connectivity", cj);
+            have_conn = true;
+        }
+    }
+    free(json);
+    if (first && !have_conn) return;
     kv_set_dbl(&d, "_timestamp", now_ts());
     kv_set_int(&d, "_interval", cfg->net_interval);
     conn_send(c, "PLG", &d);
-    free(json);
     LOGD("sent network_monitor");
 }
 
@@ -1292,6 +1657,199 @@ static void usage(const char *prog) {
         "  -h          Help\n", prog);
 }
 
+/* ============================================================
+ * Re-resolution of the server hosts at the TTL of their A/AAAA records
+ * getaddrinfo() hides the TTL, so a res_search() query is made only to learn it; the
+ * addresses still come from getaddrinfo(). TTL unknown (IP literal, /etc/hosts name, no
+ * resolv.conf, timeout) -> DNS_DEFAULT_TTL. Runs in a detached thread so a slow resolver
+ * cannot delay heartbeats; the main loop applies the result.
+ * ============================================================ */
+
+#define DNS_DEFAULT_TTL 300     /* used when the real TTL is unknown */
+#define DNS_MIN_TTL     30      /* a TTL of 0 must not turn into a resolution loop */
+#define DNS_FAILED_RETRY 60
+
+typedef struct { int af; char addr[INET6_ADDRSTRLEN]; } dns_addr_t;
+
+typedef struct {
+    const char **hosts;
+    int          nhost, port, af_filter;
+} dns_ctx_t;
+
+static struct {
+    pthread_mutex_t mu;
+    bool            ready;      /* a result is waiting for the main loop */
+    int             n;
+    dns_addr_t      addrs[MAX_HOSTS];
+} g_dns = { PTHREAD_MUTEX_INITIALIZER, false, 0, {{0, ""}} };
+
+/* Smallest A/AAAA TTL (CNAMEs included) of host, or -1 if it cannot be determined. */
+static long dns_host_ttl(const char *host, int af_filter) {
+    unsigned char probe[sizeof(struct in6_addr)];
+    if (inet_pton(AF_INET, host, probe) == 1 || inet_pton(AF_INET6, host, probe) == 1) return -1;
+
+    long best = -1;
+    int types[2] = { ns_t_a, ns_t_aaaa };
+    for (int t = 0; t < 2; t++) {
+        if (af_filter == AF_INET && t == 1) continue;
+        if (af_filter == AF_INET6 && t == 0) continue;
+        unsigned char ans[NS_PACKETSZ * 2];
+        int len = res_search(host, ns_c_in, types[t], ans, sizeof(ans));
+        ns_msg msg;
+        if (len < 0 || ns_initparse(ans, len, &msg) < 0) continue;
+        long min = -1; bool found = false;
+        for (int i = 0; i < ns_msg_count(msg, ns_s_an); i++) {
+            ns_rr rr;
+            if (ns_parserr(&msg, ns_s_an, i, &rr) < 0) continue;
+            int ty = ns_rr_type(rr);
+            if (ty != types[t] && ty != ns_t_cname) continue;
+            if (ty == types[t]) found = true;
+            if (min < 0 || (long)ns_rr_ttl(rr) < min) min = (long)ns_rr_ttl(rr);
+        }
+        if (found && (best < 0 || min < best)) best = min;
+    }
+    return best;
+}
+
+/* Seconds until the next resolution: the smallest known TTL, floored at DNS_MIN_TTL. */
+static int dns_refresh_delay(const dns_ctx_t *x) {
+    long best = -1;
+    for (int i = 0; i < x->nhost; i++) {
+        long t = dns_host_ttl(x->hosts[i], x->af_filter);
+        if (t >= 0 && (best < 0 || t < best)) best = t;
+    }
+    if (best < 0) return DNS_DEFAULT_TTL;
+    return best < DNS_MIN_TTL ? DNS_MIN_TTL : (best > 86400 ? 86400 : (int)best);
+}
+
+/* Every (family, address) the hosts resolve to, deduplicated; -1 if any lookup fails. */
+static int dns_resolve_all(const dns_ctx_t *x, dns_addr_t *out, int max) {
+    int n = 0;
+    for (int i = 0; i < x->nhost; i++) {
+        struct addrinfo hints = {0}, *res = NULL;
+        hints.ai_socktype = SOCK_DGRAM;
+        hints.ai_protocol = IPPROTO_UDP;
+        hints.ai_family   = x->af_filter;
+        char ps[16]; snprintf(ps, sizeof(ps), "%d", x->port);
+        if (getaddrinfo(x->hosts[i], ps, &hints, &res) != 0) {
+            LOGW("cannot resolve %s", x->hosts[i]);
+            return -1;
+        }
+        for (struct addrinfo *ai = res; ai && n < max; ai = ai->ai_next) {
+            dns_addr_t a; a.af = ai->ai_family;
+            void *ip = (ai->ai_family == AF_INET)
+                ? (void *)&((struct sockaddr_in  *)ai->ai_addr)->sin_addr
+                : (void *)&((struct sockaddr_in6 *)ai->ai_addr)->sin6_addr;
+            inet_ntop(ai->ai_family, ip, a.addr, sizeof(a.addr));
+            bool dup = false;
+            for (int k = 0; k < n; k++) if (strcmp(out[k].addr, a.addr) == 0) dup = true;
+            if (!dup) out[n++] = a;
+        }
+        freeaddrinfo(res);
+    }
+    return n;
+}
+
+static void *dns_thread(void *arg) {
+    const dns_ctx_t *x = arg;
+    int delay = dns_refresh_delay(x);
+    while (g_running) {
+        for (int s = 0; s < delay && g_running; s++) sleep(1);
+        if (!g_running) break;
+        dns_addr_t tmp[MAX_HOSTS];
+        int n = dns_resolve_all(x, tmp, MAX_HOSTS);
+        if (n <= 0) { delay = DNS_FAILED_RETRY; continue; }
+        pthread_mutex_lock(&g_dns.mu);
+        memcpy(g_dns.addrs, tmp, sizeof(tmp));
+        g_dns.n = n; g_dns.ready = true;
+        pthread_mutex_unlock(&g_dns.mu);
+        delay = dns_refresh_delay(x);
+    }
+    return NULL;
+}
+
+/* Point c at a new server address; the socket is replaced if the family changed. */
+static void conn_retarget(conn_t *c, int af, const char *addr) {
+    LOGI("server address %s -> %s", c->addr, addr);
+    pthread_mutex_lock(&c->mu);
+    if (af != c->af) {
+        int fd = socket(af, SOCK_DGRAM, 0);
+        if (fd < 0) { LOGE("socket: %s", strerror(errno)); pthread_mutex_unlock(&c->mu); return; }
+        fcntl(fd, F_SETFL, fcntl(fd, F_GETFL, 0) | O_NONBLOCK);
+        if (c->sockfd >= 0) close(c->sockfd);
+        c->sockfd = fd; c->af = af;
+    }
+    snprintf(c->addr, sizeof(c->addr), "%s", addr);
+    pthread_mutex_unlock(&c->mu);
+}
+
+/* Main-loop side: follow the latest resolution. Mirrors hbd.client.dnsttl.retarget(). Gone
+ * addresses are re-pointed at new ones in list order, so g_conns[0] (the one plugin data
+ * uses) survives as long as anything resolves; the rest are retired, surplus new ones added. */
+static void dns_apply(int *conn_id, const char *iam, int port) {
+    dns_addr_t r[MAX_HOSTS]; int nr;
+    pthread_mutex_lock(&g_dns.mu);
+    if (!g_dns.ready) { pthread_mutex_unlock(&g_dns.mu); return; }
+    memcpy(r, g_dns.addrs, sizeof(r)); nr = g_dns.n; g_dns.ready = false;
+    pthread_mutex_unlock(&g_dns.mu);
+
+    int gone[MAX_HOSTS], ng = 0, fresh[MAX_HOSTS], nf = 0;
+    for (int i = 0; i < g_nconns; i++) {
+        bool wanted = false;
+        for (int j = 0; j < nr; j++) if (strcmp(g_conns[i].addr, r[j].addr) == 0) wanted = true;
+        if (!wanted) gone[ng++] = i;
+    }
+    for (int j = 0; j < nr; j++) {
+        bool have = false;
+        for (int i = 0; i < g_nconns; i++) if (strcmp(g_conns[i].addr, r[j].addr) == 0) have = true;
+        if (!have) fresh[nf++] = j;
+    }
+
+    int paired = ng < nf ? ng : nf;
+    for (int k = 0; k < paired; k++) conn_retarget(&g_conns[gone[k]], r[fresh[k]].af, r[fresh[k]].addr);
+
+    /* drop[] holds connection indexes to retire, in ascending order */
+    int drop[MAX_HOSTS], nd = 0;
+    for (int k = paired; k < ng; k++) drop[nd++] = gone[k];
+    if (nd > 0 && drop[0] == 0) {
+        int keep = -1;
+        for (int i = 1; i < g_nconns && keep < 0; i++) {
+            bool isgone = false;
+            for (int k = 0; k < ng; k++) if (gone[k] == i) isgone = true;
+            if (!isgone) keep = i;
+        }
+        if (keep > 0) {
+            conn_retarget(&g_conns[0], g_conns[keep].af, g_conns[keep].addr);
+            drop[0] = keep;
+            for (int a = 0; a < nd - 1 && drop[a] > drop[a + 1]; a++) {
+                int tmp = drop[a]; drop[a] = drop[a + 1]; drop[a + 1] = tmp;
+            }
+        }
+    }
+    for (int k = nd - 1; k >= 0; k--) {
+        int i = drop[k];
+        if (i == 0) continue;
+        LOGI("dropping server address %s", g_conns[i].addr);
+        conn_close(&g_conns[i]);
+        pthread_mutex_destroy(&g_conns[i].mu);
+        int last = --g_nconns;
+        if (i != last) {
+            memcpy(&g_conns[i], &g_conns[last], sizeof(conn_t));
+            pthread_mutex_init(&g_conns[i].mu, NULL);
+        }
+    }
+
+    for (int k = paired; k < nf && g_nconns < MAX_HOSTS; k++) {
+        conn_t *c = &g_conns[g_nconns];
+        memset(c, 0, sizeof(*c));
+        c->conn_id = (*conn_id)++; c->port = port;
+        c->af = r[fresh[k]].af; c->sockfd = -1;
+        snprintf(c->name, sizeof(c->name), "%s", iam);
+        snprintf(c->addr, sizeof(c->addr), "%s", r[fresh[k]].addr);
+        if (conn_open(c)) { g_nconns++; LOGI("adding server address %s", c->addr); }
+    }
+}
+
 int main(int argc, char **argv) {
     bool        do_boot   = false;
     bool        do_daemon = false;
@@ -1323,6 +1881,7 @@ int main(int argc, char **argv) {
 
     config_t cfg;
     config_load(&cfg, cfgpath);
+    probe_init(&g_probe, &cfg);
 
     if (do_daemon) {
         daemonize();
@@ -1405,9 +1964,24 @@ int main(int argc, char **argv) {
     pthread_t tid;
     pthread_create(&tid, NULL, monitor_thread, &tctx);
 
+    pthread_t ptid; bool probe_running = false;
+    if (g_probe_enabled) {
+        int prc = pthread_create(&ptid, NULL, probe_thread, &g_probe);
+        if (prc == 0) probe_running = true;
+        else LOGW("pthread_create(probe): %s", strerror(prc));
+    }
+
+    /* Re-resolution at the DNS TTL; detached, the process exit reaps it */
+    dns_ctx_t dns_ctx = { hosts, nhost, cfg.hb_port, af_filter };
+    pthread_t dtid;
+    int drc = pthread_create(&dtid, NULL, dns_thread, &dns_ctx);
+    if (drc == 0) pthread_detach(dtid);
+    else LOGW("pthread_create(dns): %s", strerror(drc));
+
     /* Main heartbeat + receive loop */
     time_t last_hb = 0;
     while (g_running) {
+        dns_apply(&conn_id, iam, cfg.hb_port);
         time_t now = time(NULL);
         if (now - last_hb >= cfg.interval) {
             for (int i = 0; i < g_nconns; i++) {
@@ -1415,6 +1989,7 @@ int main(int argc, char **argv) {
                 kv_set_int(&hb, "acks",     g_conns[i].ack_count);
                 kv_set_dbl(&hb, "rtt",      g_conns[i].rtt);
                 kv_set_int(&hb, "interval", cfg.interval);
+                if (g_probe_enabled) probe_flags(&g_probe, &hb);
                 conn_send(&g_conns[i], "HTB", &hb);
             }
             last_hb = now;
@@ -1449,6 +2024,10 @@ int main(int argc, char **argv) {
         conn_send(primary, "HTB", &sh);
         usleep(300000);
     }
+    /* Joined after the shutdown HTB goes out: the probe thread is never cancelled (it can be
+     * inside popen), so joining it can block for a few seconds — long enough to lose the
+     * clean-shutdown notification to an impatient supervisor's SIGKILL. */
+    if (probe_running) pthread_join(ptid, NULL);
     for (int i = 0; i < g_nconns; i++) conn_close(&g_conns[i]);
     if (g_use_syslog) closelog();
     if (g_restart) { LOGI("restarting..."); execv(argv[0], argv); }

scripts/c/test_probe.c

@@ -0,0 +1,294 @@
+/* Unit tests for the connectivity-probe helpers in hbc_mini.c.
+ * Build and run:  make test        (live check of this host:  ./test_probe --live)
+ * The source file is included directly so its static functions can be exercised; its main()
+ * is renamed out of the way.
+ */
+#define main hbc_mini_main
+#include "hbc_mini.c"
+#undef main
+
+static int g_run = 0, g_fail = 0;
+#define CHECK(cond) do { g_run++; if (!(cond)) { g_fail++; \
+    fprintf(stderr, "FAIL %s:%d: %s\n", __FILE__, __LINE__, #cond); } } while (0)
+#define CHECK_STR(a, b) do { g_run++; if (strcmp((a), (b)) != 0) { g_fail++; \
+    fprintf(stderr, "FAIL %s:%d: \"%s\" != \"%s\"\n", __FILE__, __LINE__, (a), (b)); } } while (0)
+
+static const char PROC_V4[] =
+    "Iface\tDestination\tGateway \tFlags\tRefCnt\tUse\tMetric\tMask\t\tMTU\tWindow\tIRTT\n"
+    "wlan0\t00000000\t0100A8C0\t0003\t0\t0\t600\t00000000\t0\t0\t0\n"
+    "eth0\t00000000\t0101A8C0\t0003\t0\t0\t100\t00000000\t0\t0\t0\n"
+    "wlan0\t0000A8C0\t00000000\t0001\t0\t0\t600\t00FFFFFF\t0\t0\t0\n";
+
+static const char PROC_V6[] =
+    "00000000000000000000000000000000 00 00000000000000000000000000000000 00 "
+    "fe800000000000000000000000000001 00000400 00000001 00000000 00000003 wlan0\n"
+    "00000000000000000000000000000000 00 00000000000000000000000000000000 00 "
+    "00000000000000000000000000000000 ffffffff 00000001 00000000 00200200 lo\n"
+    "fe800000000000000000000000000000 40 00000000000000000000000000000000 00 "
+    "00000000000000000000000000000000 00000100 00000001 00000000 00000001 wlan0\n";
+
+static const char ROUTE_GET[] =
+    "   route to: 0.0.0.0\ndestination: 0.0.0.0\n       mask: 0.0.0.0\n"
+    "    gateway: 192.0.2.1\n        fib: 0\n  interface: vtnet0\n";
+
+static const char LINUX_PING[] =
+    "PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.\n"
+    "64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=11.2 ms\n\n"
+    "--- 1.1.1.1 ping statistics ---\n"
+    "1 packets transmitted, 1 received, 0% packet loss, time 0ms\n"
+    "rtt min/avg/max/mdev = 11.234/11.234/11.234/0.000 ms\n";
+
+static const char FREEBSD_PING[] =
+    "PING 1.1.1.1 (1.1.1.1): 56 data bytes\n"
+    "64 bytes from 1.1.1.1: icmp_seq=0 ttl=59 time=4.006 ms\n\n"
+    "--- 1.1.1.1 ping statistics ---\n"
+    "1 packets transmitted, 1 packets received, 0.0% packet loss, 1 packets out of wait time\n"
+    "round-trip min/avg/max/stddev = 4.006/4.006/4.006/0.000 ms\n";
+
+static void test_gateway_parsers(void) {
+    char gw[64];
+    gw_parse_proc_v4(PROC_V4, gw, sizeof(gw));
+    CHECK_STR(gw, "192.168.1.1");                    /* eth0, metric 100 beats wlan0, 600 */
+    gw_parse_proc_v4("Iface\tDestination\n", gw, sizeof(gw));
+    CHECK_STR(gw, "");
+    gw_parse_proc_v6(PROC_V6, gw, sizeof(gw));
+    CHECK_STR(gw, "fe80::1");                        /* reject and non-default lines skipped */
+    gw_parse_route_get(ROUTE_GET, gw, sizeof(gw));
+    CHECK_STR(gw, "192.0.2.1");
+    gw_parse_route_get("route: writing to routing socket: not in table\n", gw, sizeof(gw));
+    CHECK_STR(gw, "");
+}
+
+static void test_ping_parser(void) {
+    CHECK(ping_parse_rtt(LINUX_PING) > 11.23 && ping_parse_rtt(LINUX_PING) < 11.24);
+    CHECK(ping_parse_rtt(FREEBSD_PING) > 4.0 && ping_parse_rtt(FREEBSD_PING) < 4.01);
+    CHECK(ping_parse_rtt("1 packets transmitted, 0 received, 100% packet loss\n") == -1.0);
+    CHECK(ping_parse_rtt("sh: 1: ping: not found\n") == -2.0);
+    CHECK(ping_parse_rtt("ping: command not found\n") == -2.0);
+}
+
+/* ---- scriptable network for probe_step ------------------------------------------------ */
+static struct {
+    char   addr[2][INET6_ADDRSTRLEN];   /* "" = no route */
+    char   gw[2][INET6_ADDRSTRLEN];
+    double rtt[2];                      /* -1 no reply, -2 ping unavailable */
+    int    unsupported[2];
+    int    npings[2];
+} net;
+
+static int fake_local(int af, const char *t, char *out, size_t n) {
+    int f = af == AF_INET6; (void)t;
+    if (net.unsupported[f]) return -1;
+    if (!net.addr[f][0]) return 0;
+    snprintf(out, n, "%s", net.addr[f]);
+    return 1;
+}
+static double fake_ping(int af, const char *t) { int f = af == AF_INET6; (void)t; net.npings[f]++; return net.rtt[f]; }
+static void fake_gw(int af, char *out, size_t n) { snprintf(out, n, "%s", net.gw[af == AF_INET6]); }
+static const probe_ops_t FAKE = { fake_local, fake_ping, fake_gw };
+
+static void net_reset(void) {
+    memset(&net, 0, sizeof(net));
+    snprintf(net.addr[0], INET6_ADDRSTRLEN, "192.0.2.5");
+    snprintf(net.addr[1], INET6_ADDRSTRLEN, "2001:db8::5");
+    snprintf(net.gw[0], INET6_ADDRSTRLEN, "192.0.2.1");
+    snprintf(net.gw[1], INET6_ADDRSTRLEN, "fe80::1");
+    net.rtt[0] = 10.0; net.rtt[1] = 20.0;
+}
+
+static void probe_reset(probe_t *p) {
+    memset(p, 0, sizeof(*p));
+    snprintf(p->target[0], sizeof(p->target[0]), "1.1.1.1");
+    snprintf(p->target[1], sizeof(p->target[1]), "2606:4700:4700::1111");
+    p->probe_interval = 60; p->retry_interval = 10; p->cheap_gateway = true;
+    p->fam[0].present = p->fam[1].present = PROBE_UNKNOWN;
+    p->fam[0].rtt = p->fam[1].rtt = -1;
+    pthread_mutex_init(&p->mu, NULL);
+}
+
+static void step_both(probe_t *p, double now) {
+    probe_step(p, &p->fam[0], 0, &FAKE, now);
+    probe_step(p, &p->fam[1], 1, &FAKE, now);
+}
+
+static void test_present_with_details(void) {
+    net_reset(); probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    CHECK(p.fam[0].present == 1 && p.fam[1].present == 1);
+    CHECK_STR(p.fam[0].addr, "192.0.2.5");
+    CHECK_STR(p.fam[0].gateway, "192.0.2.1");
+    CHECK(p.fam[0].rtt == 10.0);
+    kvdict_t d; kv_clear(&d);
+    probe_flags(&p, &d);
+    CHECK_STR(kv_get(&d, "p4") ? kv_get(&d, "p4") : "(null)", "1");
+    CHECK_STR(kv_get(&d, "p6") ? kv_get(&d, "p6") : "(null)", "1");
+    char js[512];
+    CHECK(probe_metric_json(&p, js, sizeof(js)) > 0);
+    CHECK(strstr(js, "\"ipv6\":{\"present\":true,\"addr\":\"2001:db8::5\",\"gateway\":\"fe80::1\"") != NULL);
+}
+
+static void test_no_route_is_absent_and_wastes_no_ping(void) {
+    net_reset(); net.addr[1][0] = '\0'; probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    CHECK(p.fam[1].present == 0 && net.npings[1] == 0 && net.npings[0] == 1);
+    kvdict_t d; kv_clear(&d);
+    probe_flags(&p, &d);
+    CHECK_STR(kv_get(&d, "p6") ? kv_get(&d, "p6") : "(null)", "0");
+    char js[512];
+    probe_metric_json(&p, js, sizeof(js));
+    CHECK(strstr(js, "\"ipv6\":{\"present\":false,\"addr\":null,\"gateway\":null,\"rtt\":null}") != NULL);
+}
+
+static void test_debounce_and_rate_limit(void) {
+    net_reset(); probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    step_both(&p, 1000);                                  /* cached: no second ping */
+    CHECK(net.npings[0] == 1 && net.npings[1] == 1);
+    net.rtt[1] = -1;
+    step_both(&p, 1060);                                  /* first failure: still present */
+    CHECK(p.fam[1].present == 1);
+    step_both(&p, 1070);                                  /* retry interval elapsed: flips */
+    CHECK(p.fam[1].present == 0);
+    net.rtt[1] = 25;
+    step_both(&p, 1080);                                  /* first success recovers */
+    CHECK(p.fam[1].present == 1);
+}
+
+static void test_first_failure_is_unknown_and_omitted(void) {
+    net_reset(); net.rtt[0] = -1; probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    CHECK(p.fam[0].present == PROBE_UNKNOWN);
+    kvdict_t d; kv_clear(&d);
+    probe_flags(&p, &d);
+    CHECK(kv_get(&d, "p4") == NULL && kv_get(&d, "p6") != NULL);
+    char js[512];
+    probe_metric_json(&p, js, sizeof(js));
+    CHECK(strstr(js, "\"ipv4\":{\"present\":null") != NULL);
+}
+
+static void test_address_or_gateway_change_repings(void) {
+    net_reset(); probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    snprintf(net.addr[0], INET6_ADDRSTRLEN, "198.51.100.7");
+    step_both(&p, 1000);
+    CHECK(net.npings[0] == 2);
+    snprintf(net.gw[0], INET6_ADDRSTRLEN, "192.0.2.254");
+    step_both(&p, 1000);
+    CHECK(net.npings[0] == 3);
+}
+
+static void test_route_loss_then_return_pings_immediately(void) {
+    net_reset(); probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    net.addr[1][0] = '\0';
+    step_both(&p, 1000);
+    CHECK(p.fam[1].present == 0);
+    snprintf(net.addr[1], INET6_ADDRSTRLEN, "2001:db8::5");
+    step_both(&p, 1000);
+    CHECK(net.npings[1] == 2 && p.fam[1].present == 1);
+}
+
+static void test_missing_ping_stays_unknown_even_after_route_loss(void) {
+    net_reset(); net.rtt[0] = -2; probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    step_both(&p, 1600);
+    CHECK(p.fam[0].present == PROBE_UNKNOWN && net.npings[0] == 1);
+    net.addr[0][0] = '\0';
+    step_both(&p, 1700);
+    CHECK(p.fam[0].present == 0);                         /* no route is definitive */
+    snprintf(net.addr[0], INET6_ADDRSTRLEN, "192.0.2.5");
+    step_both(&p, 1800);
+    CHECK(p.fam[0].present == PROBE_UNKNOWN && net.npings[0] == 1);
+}
+
+static void test_unsupported_family_is_unknown(void) {
+    net_reset(); net.unsupported[1] = 1; probe_t p; probe_reset(&p);
+    step_both(&p, 1000);
+    CHECK(p.fam[1].present == PROBE_UNKNOWN && p.fam[0].present == 1);
+}
+
+static int live_mode(void) {
+    static const probe_ops_t sys_ops = { sys_local_address, sys_ping, sys_gateway };
+    probe_t p; probe_reset(&p);
+    p.cheap_gateway = false;
+    snprintf(p.target[0], sizeof(p.target[0]), "1.1.1.1");
+    for (int round = 0; round < 3; round++) {
+        double now = mono_ts();
+        for (int f = 0; f < 2; f++) probe_step(&p, &p.fam[f], f, &sys_ops, now);
+        sleep(round < 2 ? 11 : 0);
+    }
+    char js[512];
+    probe_metric_json(&p, js, sizeof(js));
+    kvdict_t d; kv_clear(&d);
+    probe_flags(&p, &d);
+    printf("%s\np4=%s p6=%s\n", js, kv_get(&d, "p4") ? kv_get(&d, "p4") : "-",
+           kv_get(&d, "p6") ? kv_get(&d, "p6") : "-");
+    return 0;
+}
+
+/* ---- server address re-resolution (dns_apply) ---- */
+
+static void dns_conn(int i, int af, const char *addr) {
+    memset(&g_conns[i], 0, sizeof(conn_t));
+    g_conns[i].conn_id = i + 1; g_conns[i].af = af; g_conns[i].port = 5;
+    snprintf(g_conns[i].addr, sizeof(g_conns[i].addr), "%s", addr);
+    conn_open(&g_conns[i]);
+}
+
+static void dns_push(int n, const dns_addr_t *a) {
+    memcpy(g_dns.addrs, a, n * sizeof(*a)); g_dns.n = n; g_dns.ready = true;
+}
+
+static void test_dns_apply_follows_changes(void) {
+    int id = 3;
+    dns_conn(0, AF_INET, "1.1.1.1"); dns_conn(1, AF_INET6, "2001:db8::1"); g_nconns = 2;
+
+    dns_addr_t moved[2] = {{AF_INET, "9.9.9.9"}, {AF_INET6, "2001:db8::9"}};
+    dns_push(2, moved); dns_apply(&id, "me", 5);
+    CHECK(g_nconns == 2);
+    CHECK_STR(g_conns[0].addr, "9.9.9.9");
+    CHECK_STR(g_conns[1].addr, "2001:db8::9");
+
+    /* the first connection's address vanishes and nothing replaces it: it survives, the other goes */
+    dns_addr_t v6only[1] = {{AF_INET6, "2001:db8::9"}};
+    dns_push(1, v6only); dns_apply(&id, "me", 5);
+    CHECK(g_nconns == 1);
+    CHECK_STR(g_conns[0].addr, "2001:db8::9");
+    CHECK(g_conns[0].af == AF_INET6);
+
+    dns_addr_t grown[2] = {{AF_INET6, "2001:db8::9"}, {AF_INET, "5.5.5.5"}};
+    dns_push(2, grown); dns_apply(&id, "me", 5);
+    CHECK(g_nconns == 2);
+    CHECK_STR(g_conns[1].addr, "5.5.5.5");
+    CHECK(g_conns[1].conn_id == 3);
+
+    dns_push(2, grown); dns_apply(&id, "me", 5);        /* unchanged: nothing happens */
+    CHECK(g_nconns == 2 && id == 4);
+    for (int i = 0; i < g_nconns; i++) conn_close(&g_conns[i]);
+    g_nconns = 0;
+}
+
+static void test_dns_ttl_unknown_for_ip_literals(void) {
+    CHECK(dns_host_ttl("192.0.2.1", 0) == -1);
+    CHECK(dns_host_ttl("2001:db8::1", 0) == -1);
+    const char *h[] = {"192.0.2.1"}; dns_ctx_t x = {h, 1, 5, 0};
+    CHECK(dns_refresh_delay(&x) == DNS_DEFAULT_TTL);
+}
+
+int main(int argc, char **argv) {
+    if (argc > 1 && !strcmp(argv[1], "--live")) return live_mode();
+    test_gateway_parsers();
+    test_ping_parser();
+    test_present_with_details();
+    test_no_route_is_absent_and_wastes_no_ping();
+    test_debounce_and_rate_limit();
+    test_first_failure_is_unknown_and_omitted();
+    test_address_or_gateway_change_repings();
+    test_route_loss_then_return_pings_immediately();
+    test_missing_ping_stays_unknown_even_after_route_loss();
+    test_unsupported_family_is_unknown();
+    test_dns_apply_follows_changes();
+    test_dns_ttl_unknown_for_ip_literals();
+    printf("%d checks, %d failed\n", g_run, g_fail);
+    return g_fail ? 1 : 0;
+}

scripts/hbc_mini.py

@@ -10,18 +10,25 @@ Built-in plugins (always available):
   os_info         — OS name, kernel, distro, Python version
   ping_monitor    — ICMP ping RTT to configured hosts
   nagios_runner   — run Nagios-compatible check scripts
+  network_monitor — per-interface TX/RX bytes/s (Linux); per-protocol connectivity (all platforms)
 
 Linux-only plugins (via /proc, no extra tools needed):
   cpu_monitor     — CPU %, load average, core count
   memory_monitor  — RAM and swap usage
-  network_monitor — per-interface TX/RX bytes/s
 
 Cross-platform (via df subprocess):
   disk_monitor    — per-mount usage and capacity
+
+Presence probe (built in): reports per protocol (IPv4/IPv6) whether the host has internet
+connectivity, as p4/p6 flags on every heartbeat, so hbd stops treating a missing IPv6 (or
+IPv4) path as an outage. Config, in the network_monitor section of ~/.hbc.json:
+  probe_enabled (default true), probe_v4 (default 1.1.1.1), probe_v6 (default
+  2606:4700:4700::1111; IP literals only), probe_interval (default 60 s)
 """
 
 import argparse
 import asyncio
+import ipaddress
 import json
 import logging
 import os
@@ -30,18 +37,20 @@ import re
 import shutil
 import signal
 import socket
+import struct
 import subprocess
 import sys
 import time
 import zlib
 from abc import ABC, abstractmethod
 from collections import defaultdict
+from dataclasses import asdict, dataclass
 from logging.handlers import SysLogHandler
 from pathlib import Path
-from typing import Any, Dict, List, Optional, Tuple
+from typing import Any, Awaitable, Callable, Dict, List, Optional, Set, Tuple
 
 # updated by scripts/bumpminor.sh
-__version__ = "5.4.3"
+__version__ = "5.4.7"
 
 # ---------------------------------------------------------------------------
 # Protocol  (mirrors hbd/common/proto.py)
@@ -637,6 +646,315 @@ class DiskMonitorPlugin(MonitorPlugin):
         return {"partitions": partitions} if partitions else {}
 
 
+# ---------------------------------------------------------------------------
+# Connectivity probe — per-protocol internet presence (IPv4 / IPv6)
+#
+# Tier 1 (every tier1_interval s, sends no packets): a UDP connect() to the probe address
+# shows whether a route exists and which source address would be used. Tier 2 (every
+# probe_interval s, sooner after a failure or a change of address or gateway): one ICMP echo,
+# only when tier 1 found a route. A family is absent after no route at all or fail_limit
+# consecutive failed echoes, present after one success, and unknown (flag omitted) when it
+# cannot be probed. The result rides on every HTB as p4/p6 and, when network_monitor runs, as
+# its `connectivity` metric. Mirrors hbd/client/connectivity.py.
+# ---------------------------------------------------------------------------
+
+_FAMILIES = ("ipv4", "ipv6")
+_FLAG_KEYS = {"ipv4": "p4", "ipv6": "p6"}
+_DEFAULT_PROBES = {"ipv4": "1.1.1.1", "ipv6": "2606:4700:4700::1111"}
+_BSD_PLATFORMS = ("darwin", "freebsd", "netbsd", "openbsd", "dragonfly")
+
+
+class _ProbeUnavailable(Exception):
+    """The address family cannot be probed on this host (e.g. no IPv6 support)."""
+
+
+class _PingUnavailable(Exception):
+    """The system ping binary could not be executed."""
+
+
+def _parse_proc_route_v4(text: str) -> Optional[str]:
+    """Default gateway from /proc/net/route (lowest metric wins)."""
+    best: Optional[Tuple[int, str]] = None
+    for line in text.splitlines()[1:]:
+        f = line.split()
+        if len(f) < 8 or f[1] != "00000000" or not int(f[3], 16) & 0x2:
+            continue
+        metric = int(f[6])
+        if best is None or metric < best[0]:
+            # the kernel prints the raw network-order u32 in host order: use the host byte order
+            best = (metric, socket.inet_ntoa(int(f[2], 16).to_bytes(4, sys.byteorder)))
+    return best[1] if best else None
+
+
+def _parse_proc_route_v6(text: str) -> Optional[str]:
+    """Default gateway from /proc/net/ipv6_route (lowest metric wins)."""
+    best: Optional[Tuple[int, str]] = None
+    for line in text.splitlines():
+        f = line.split()
+        if len(f) < 10:
+            continue
+        dest, plen, nexthop, metric, flags, iface = f[0], f[1], f[4], f[5], f[8], f[9]
+        if dest != "0" * 32 or plen != "00" or nexthop == "0" * 32 or iface == "lo":
+            continue
+        fl = int(flags, 16)
+        if not fl & 0x2 or fl & 0x200:            # need RTF_GATEWAY, skip RTF_REJECT
+            continue
+        m = int(metric, 16)
+        if best is None or m < best[0]:
+            best = (m, socket.inet_ntop(socket.AF_INET6, bytes.fromhex(nexthop)))
+    return best[1] if best else None
+
+
+def _parse_route_get(text: str) -> Optional[str]:
+    """Gateway from `route -n get [-inet6] default` (macOS and the BSDs)."""
+    m = re.search(r"^\s*gateway:\s*(\S+)", text, re.MULTILINE)
+    return m.group(1) if m else None
+
+
+def _parse_windows_route(text: str, family: str) -> Optional[str]:
+    """Default gateway from `route print -4` / `route print -6` (lowest metric wins)."""
+    best: Optional[Tuple[int, str]] = None
+    for line in text.splitlines():
+        if family == "ipv4":
+            f = line.split()
+            if not (len(f) >= 5 and f[0] == "0.0.0.0" and f[1] == "0.0.0.0" and f[2] != "On-link"):
+                continue
+            cand = (int(f[4]), f[2])
+        else:
+            m = re.match(r"^\s*\d+\s+(\d+)\s+::/0\s+(\S+)\s*$", line)
+            if not m or m.group(2) == "On-link":
+                continue
+            cand = (int(m.group(1)), m.group(2))
+        if best is None or cand[0] < best[0]:
+            best = cand
+    return best[1] if best else None
+
+
+async def _run_cmd(cmd: List[str], timeout: float = 5.0) -> str:
+    proc = await asyncio.create_subprocess_exec(
+        *cmd, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.DEVNULL)
+    try:
+        out, _ = await asyncio.wait_for(proc.communicate(), timeout=timeout)
+    except asyncio.TimeoutError:
+        proc.kill()
+        await proc.wait()
+        raise
+    return out.decode(errors="replace")
+
+
+async def _default_gateway(family: str) -> Optional[str]:
+    """Default gateway for "ipv4"/"ipv6", or None on any failure (informational only)."""
+    v4 = family == "ipv4"
+    try:
+        if sys.platform.startswith("linux"):
+            with open("/proc/net/route" if v4 else "/proc/net/ipv6_route") as fh:
+                text = fh.read()
+            return _parse_proc_route_v4(text) if v4 else _parse_proc_route_v6(text)
+        if sys.platform.startswith(_BSD_PLATFORMS):
+            cmd = ["route", "-n", "get"] + ([] if v4 else ["-inet6"]) + ["default"]
+            return _parse_route_get(await _run_cmd(cmd))
+        if sys.platform == "win32":
+            text = await _run_cmd(["route", "print", "-4" if v4 else "-6"])
+            return _parse_windows_route(text, family)
+    except Exception:
+        return None
+    return None
+
+
+def _probe_ping_command(host: str, family: str) -> List[str]:
+    """Command for ONE echo request with a ~2 s reply wait."""
+    v4 = family == "ipv4"
+    if sys.platform == "win32":
+        return ["ping", "-n", "1", "-w", "2000", "-4" if v4 else "-6", host]
+    if sys.platform.startswith(_BSD_PLATFORMS):
+        # BSD/macOS -W is in milliseconds, and IPv6 is reached through ping6.
+        return ["ping", "-c", "1", "-W", "2000", host] if v4 else ["ping6", "-c", "1", host]
+    return ["ping", "-c", "1", "-W", "2", "-4" if v4 else "-6", host]
+
+
+def _parse_ping_rtt(output: str) -> Optional[float]:
+    """Average RTT in ms from ping output, or None when there was no reply."""
+    m = re.search(r"(?:rtt|round-trip)\s+min/avg/max/\S+\s*=\s*([\d.]+)/([\d.]+)/([\d.]+)", output)
+    if m:
+        return float(m.group(2))
+    if "TTL=" in output:                           # Windows reply line (also when localized)
+        m = re.search(r"[=<]\s*(\d+)\s*ms", output)
+        return float(m.group(1)) if m else 0.0
+    return None
+
+
+async def _ping_rtt(target: str, family: str) -> Optional[float]:
+    """One ICMP echo; RTT in ms, or None if no reply. Raises _PingUnavailable without ping."""
+    try:
+        proc = await asyncio.create_subprocess_exec(
+            *_probe_ping_command(target, family),
+            stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE)
+    except FileNotFoundError as e:
+        raise _PingUnavailable(str(e))
+    try:
+        out, _ = await asyncio.wait_for(proc.communicate(), timeout=6)
+    except asyncio.TimeoutError:
+        proc.kill()
+        await proc.wait()
+        return None
+    return _parse_ping_rtt(out.decode(errors="replace"))
+
+
+def _local_address(family: str, target: str) -> Optional[str]:
+    """Source address the kernel would use to reach target, or None if there is no route.
+
+    A UDP connect() sends nothing. Raises _ProbeUnavailable if the family is unsupported.
+    """
+    af = socket.AF_INET if family == "ipv4" else socket.AF_INET6
+    try:
+        sock = socket.socket(af, socket.SOCK_DGRAM)
+    except OSError as e:
+        raise _ProbeUnavailable(str(e))
+    try:
+        sock.connect((target, 9))
+        return str(sock.getsockname()[0])
+    except OSError:
+        return None
+    finally:
+        sock.close()
+
+
+@dataclass
+class _FamilyState:
+    present: Optional[bool] = None      # None = unknown (probe cannot run yet/at all)
+    addr: Optional[str] = None
+    gateway: Optional[str] = None
+    rtt: Optional[float] = None
+
+
+class _Connectivity:
+    def __init__(
+        self,
+        probes: Optional[Dict[str, str]] = None,
+        probe_interval: float = 60.0,
+        retry_interval: float = 10.0,
+        tier1_interval: float = 5.0,
+        fail_limit: int = 2,
+        local_address_fn: Callable[[str, str], Optional[str]] = _local_address,
+        ping_fn: Callable[[str, str], Awaitable[Optional[float]]] = _ping_rtt,
+        gateway_fn: Callable[[str], Awaitable[Optional[str]]] = _default_gateway,
+        clock: Callable[[], float] = time.monotonic,
+    ):
+        self.probes = dict(_DEFAULT_PROBES)
+        self.probes.update(probes or {})
+        self.probe_interval = probe_interval
+        self.retry_interval = retry_interval
+        self.tier1_interval = tier1_interval
+        self.fail_limit = fail_limit
+        self._local_address = local_address_fn
+        self._ping = ping_fn
+        self._gateway_fn = gateway_fn
+        self._clock = clock
+        # Reading the gateway is a cheap /proc read on Linux but a subprocess elsewhere, so
+        # elsewhere it is refreshed only when a tier-2 probe is due.
+        self._cheap_gateway = sys.platform.startswith("linux")
+        self.states: Dict[str, _FamilyState] = {f: _FamilyState() for f in _FAMILIES}
+        self._fails: Dict[str, int] = {f: 0 for f in _FAMILIES}
+        self._next_ping: Dict[str, float] = {f: 0.0 for f in _FAMILIES}
+        self._last_key: Dict[str, Optional[Tuple[Optional[str], Optional[str]]]] = {
+            f: None for f in _FAMILIES}
+        self._ping_disabled: Set[str] = set()
+
+    @classmethod
+    def from_config(cls, cfg: Optional[Dict[str, Any]]) -> "_Connectivity":
+        cfg = cfg or {}
+        probes = {}
+        if cfg.get("probe_v4"):
+            probes["ipv4"] = str(cfg["probe_v4"])
+        if cfg.get("probe_v6"):
+            probes["ipv6"] = str(cfg["probe_v6"])
+        return cls(probes=probes, probe_interval=float(cfg.get("probe_interval", 60)))
+
+    # ---- readers (cheap; safe to call on every heartbeat) ---------------------------
+    def snapshot(self) -> Dict[str, _FamilyState]:
+        return {f: _FamilyState(**asdict(s)) for f, s in self.states.items()}
+
+    def flags(self) -> Dict[str, bool]:
+        """HTB flags for families whose presence is known."""
+        return {_FLAG_KEYS[f]: s.present for f, s in self.states.items() if s.present is not None}
+
+    def as_metric(self) -> Dict[str, Dict[str, Any]]:
+        return {f: asdict(s) for f, s in self.states.items()}
+
+    # ---- probing ----------------------------------------------------------------------
+    async def refresh(self) -> None:
+        for fam in _FAMILIES:
+            try:
+                await self._refresh_family(fam)
+            except Exception:
+                logging.getLogger("hbc.probe").exception("connectivity probe failed for %s", fam)
+
+    async def run(self) -> None:
+        while _running:
+            await self.refresh()
+            await _sleep(self.tier1_interval)
+
+    async def _gateway(self, fam: str) -> Optional[str]:
+        try:
+            return await self._gateway_fn(fam)
+        except Exception:
+            return None
+
+    async def _refresh_family(self, fam: str) -> None:
+        st = self.states[fam]
+        try:
+            addr = self._local_address(fam, self.probes[fam])
+        except _ProbeUnavailable:
+            st.present = None
+            return
+
+        if addr is None:                       # tier 1: no route => absent immediately
+            self._fails[fam] = 0
+            self._next_ping[fam] = 0.0
+            self._last_key[fam] = None
+            st.present, st.addr, st.gateway, st.rtt = False, None, None, None
+            return
+
+        now = self._clock()
+        gw = await self._gateway(fam) if self._cheap_gateway else st.gateway
+        key = (addr, gw)
+        due = key != self._last_key[fam] or now >= self._next_ping[fam]
+        if due and not self._cheap_gateway:
+            gw = await self._gateway(fam)
+            key = (addr, gw)
+        st.addr, st.gateway = addr, gw
+        self._last_key[fam] = key
+        if fam in self._ping_disabled:
+            st.present = None                  # cannot probe: unknown, never absent, while a route exists
+        elif due:
+            await self._tier2(fam, st, now)
+
+    async def _tier2(self, fam: str, st: _FamilyState, now: float) -> None:
+        try:
+            rtt = await self._ping(self.probes[fam], fam)
+        except _PingUnavailable:
+            logging.getLogger("hbc.probe").warning("ping unavailable; %s presence stays unknown", fam)
+            self._ping_disabled.add(fam)
+            st.present = None
+            return
+        except (asyncio.TimeoutError, OSError):
+            rtt = None
+        if rtt is None:
+            self._fails[fam] += 1
+            self._next_ping[fam] = now + self.retry_interval
+            if self._fails[fam] >= self.fail_limit:
+                st.present, st.rtt = False, None
+        else:
+            self._fails[fam] = 0
+            self._next_ping[fam] = now + self.probe_interval
+            st.present, st.rtt = True, rtt
+
+
+# The running probe (None when probe_enabled is false); shared by the heartbeat sender and
+# the network_monitor plugin. Set in _async_main.
+_probe: Optional[_Connectivity] = None
+
+
 # ---------------------------------------------------------------------------
 # Plugin: network_monitor  (Linux /proc/net/dev)
 # ---------------------------------------------------------------------------
@@ -658,7 +976,7 @@ def _read_net_dev() -> Dict[str, Tuple[int, int]]:
 
 class NetworkMonitorPlugin(MonitorPlugin):
     name = "network_monitor"
-    description = "Network I/O rates via /proc/net/dev (Linux)"
+    description = "Network I/O rates via /proc/net/dev (Linux) and per-protocol internet connectivity"
     interval = 300
 
     def __init__(self, config: Optional[Dict[str, Any]] = None):
@@ -667,19 +985,25 @@ class NetworkMonitorPlugin(MonitorPlugin):
         self.interval = cfg.get("interval", 300)
         self.skip_ifaces: set = set(cfg.get("skip_interfaces", ["lo"]))
         self._prev: Optional[Tuple[float, Dict[str, Tuple[int, int]]]] = None
+        self._have_dev = False
 
     async def initialize(self) -> bool:
-        if platform.system() != "Linux":
-            self.skip_reason = "Linux only (/proc/net/dev not available)"
+        dev = _read_net_dev() if platform.system() == "Linux" else {}
+        self._have_dev = bool(dev)
+        if not self._have_dev and _probe is None:
+            self.skip_reason = "no /proc/net/dev (Linux only) and the connectivity probe is disabled"
             return False
-        dev = _read_net_dev()
-        if not dev:
-            self.skip_reason = "/proc/net/dev not readable"
-            return False
-        self._prev = (time.time(), dev)  # prime first delta
+        if self._have_dev:
+            self._prev = (time.time(), dev)  # prime first delta
         return True
 
     async def _collect_metrics(self) -> Dict[str, Any]:
+        data: Dict[str, Any] = await self._collect_interfaces() if self._have_dev else {}
+        if _probe is not None:
+            data["connectivity"] = _probe.as_metric()
+        return data
+
+    async def _collect_interfaces(self) -> Dict[str, Any]:
         now = time.time()
         curr = _read_net_dev()
         if not curr or self._prev is None:
@@ -842,6 +1166,14 @@ class AsyncConnection:
             self._transport.close()
             self._transport = None
 
+    def set_target(self, addr: str, af: int):
+        """Re-point at a new server address; the socket is reopened on the next send if the family changed."""
+        if af != self.af:
+            self.close()
+            self.af = af
+        self.addr = addr
+        self._log = logging.getLogger(f"hbc.conn.{addr}")
+
     def _handle_ack(self, msg: Dict[str, Any], now: float):
         rtt = (now - self.lastsend) * 1000.0
         self.rtts.append(rtt)
@@ -866,6 +1198,207 @@ class AsyncConnection:
         self.lastsend = time.time()
 
 
+# ---------------------------------------------------------------------------
+# Re-resolution of the server hosts at the TTL of their A/AAAA records
+# getaddrinfo() hides the TTL, so a minimal stub-resolver query is made only to learn it; the
+# addresses still come from getaddrinfo(). TTL unknown (IP literal, /etc/hosts name, no
+# resolv.conf, timeout) -> _DNS_DEFAULT_TTL.
+# ---------------------------------------------------------------------------
+
+log = logging.getLogger("hbc.dns")
+
+_DNS_DEFAULT_TTL = 300       # used when the real TTL is unknown
+_DNS_MIN_TTL = 30            # a TTL of 0 must not turn into a resolution loop
+_DNS_FAILED_RETRY = 60       # re-try delay after a failed resolution
+_DNS_QUERY_TIMEOUT = 2.0
+
+_DNS_TYPE_A, _DNS_TYPE_CNAME, _DNS_TYPE_AAAA = 1, 5, 28
+
+
+def _dns_parse_resolv_conf(path: str = "/etc/resolv.conf") -> Tuple[List[str], List[str]]:
+    """Return (nameservers, search domains)."""
+    servers: List[str] = []
+    search: List[str] = []
+    try:
+        with open(path) as f:
+            for line in f:
+                parts = line.split()
+                if len(parts) < 2:
+                    continue
+                if parts[0] == "nameserver":
+                    servers.append(parts[1].split("%")[0])
+                elif parts[0] in ("search", "domain"):
+                    search = parts[1:]
+    except OSError:
+        pass
+    return servers, search
+
+
+def _dns_build_query(name: str, qtype: int, qid: int) -> bytes:
+    out = struct.pack("!HHHHHH", qid, 0x0100, 1, 0, 0, 0)
+    for label in name.rstrip(".").split("."):
+        out += bytes([len(label)]) + label.encode("idna")
+    return out + b"\x00" + struct.pack("!HH", qtype, 1)
+
+
+def _dns_skip_name(buf: bytes, off: int) -> int:
+    while True:
+        n = buf[off]
+        if n == 0:
+            return off + 1
+        if n & 0xC0:
+            return off + 2
+        off += n + 1
+
+
+def _dns_min_answer_ttl(resp: bytes, qid: int, qtype: int) -> Optional[int]:
+    """Smallest TTL among the answer RRs (CNAMEs included), or None if there is no qtype answer."""
+    try:
+        rid, flags, qd, an = struct.unpack("!HHHH", resp[:8])
+        if rid != qid or flags & 0x000F:
+            return None
+        off = 12
+        for _ in range(qd):
+            off = _dns_skip_name(resp, off) + 4
+        ttls: List[int] = []
+        found = False
+        for _ in range(an):
+            off = _dns_skip_name(resp, off)
+            rtype, _cls, ttl, rdlen = struct.unpack("!HHIH", resp[off:off + 10])
+            off += 10 + rdlen
+            if rtype in (qtype, _DNS_TYPE_CNAME):
+                ttls.append(ttl)
+            found = found or rtype == qtype
+        return min(ttls) if found else None
+    except (struct.error, IndexError):
+        return None
+
+
+def _dns_query_ttl(server: str, name: str, qtype: int) -> Optional[int]:
+    qid = struct.unpack("!H", os.urandom(2))[0]
+    try:
+        af = socket.AF_INET6 if ":" in server else socket.AF_INET
+        with socket.socket(af, socket.SOCK_DGRAM) as s:
+            s.settimeout(_DNS_QUERY_TIMEOUT)
+            s.sendto(_dns_build_query(name, qtype, qid), (server, 53))
+            return _dns_min_answer_ttl(s.recv(4096), qid, qtype)
+    except (OSError, UnicodeError):
+        return None
+
+
+def _dns_lookup_ttl(host: str, af_filter: int = 0) -> Optional[int]:
+    """Blocking. Smallest A/AAAA TTL for host, or None when it cannot be determined."""
+    try:
+        ipaddress.ip_address(host)
+        return None
+    except ValueError:
+        pass
+    servers, search = _dns_parse_resolv_conf()
+    if host.endswith("."):
+        names = [host]
+    elif "." in host:
+        names = [host] + [f"{host}.{d}" for d in search]
+    else:
+        names = [f"{host}.{d}" for d in search] + [host]
+    qtypes = [t for t, af in ((_DNS_TYPE_A, socket.AF_INET), (_DNS_TYPE_AAAA, socket.AF_INET6))
+              if af_filter in (0, af)]
+    for name in names:
+        ttls = []
+        for qtype in qtypes:
+            for server in servers:
+                ttl = _dns_query_ttl(server, name, qtype)
+                if ttl is not None:
+                    ttls.append(ttl)
+                    break
+        if ttls:
+            return min(ttls)
+    return None
+
+
+def _dns_refresh_delay(ttls: List[Optional[int]]) -> int:
+    """Seconds until the next resolution: the smallest known TTL, floored at _DNS_MIN_TTL."""
+    known = [t for t in ttls if t is not None]
+    return max(min(known), _DNS_MIN_TTL) if known else _DNS_DEFAULT_TTL
+
+
+def _dns_resolve(hosts: List[str], port: int, af_filter: int) -> Optional[List[Tuple[int, str]]]:
+    """Blocking. Every (family, address) the hosts resolve to, deduplicated, or None if any lookup fails."""
+    found: List[Tuple[int, str]] = []
+    for host in hosts:
+        try:
+            infos = socket.getaddrinfo(host, port, af_filter, 0, socket.SOL_UDP)
+        except socket.gaierror as e:
+            log.warning("cannot resolve %s: %s", host, e)
+            return None
+        for info in infos:
+            entry = (info[0], str(info[4][0]))
+            if entry not in found:
+                found.append(entry)
+    return found or None
+
+
+def _dns_retarget(connections: list, resolved: List[Tuple[int, str]]) -> Tuple[List[Tuple[int, str]], list]:
+    """Point the connections at the resolved addresses, in place.
+
+    A connection whose address is gone is re-pointed at an address that is new, in list order, so
+    the first connection (the one plugin data uses) survives as long as anything resolves.
+    Returns (new addresses still needing a connection, connections to drop).
+    """
+    current = {c.addr for c in connections}
+    wanted = {addr for _, addr in resolved}
+    new = [(af, addr) for af, addr in resolved if addr not in current]
+    gone = [c for c in connections if c.addr not in wanted]
+    paired = min(len(gone), len(new))
+    for conn, (af, addr) in zip(gone, new):
+        log.info("server address %s -> %s", conn.addr, addr)
+        conn.set_target(addr, af)
+    drop = gone[paired:]
+    if connections and connections[0] in drop:
+        # plugin data goes out on the first connection: hand it a surviving address and drop that one instead
+        keep = next(c for c in connections if c not in gone)
+        connections[0].set_target(keep.addr, keep.af)
+        drop.remove(connections[0])
+        drop.append(keep)
+    return new[paired:], drop
+
+
+async def _dns_refresher(connections: List["AsyncConnection"], hosts: List[str], port: int, af_filter: int,
+                         name: str, interval: int, probe: Optional[_Connectivity], next_id: int):
+    """Re-resolve the server hosts every TTL seconds and follow address changes.
+
+    A failed lookup leaves the connections untouched. Addresses that disappear are re-pointed at
+    new ones where possible; the rest are retired, and surplus new ones get a connection.
+    """
+    loop = asyncio.get_running_loop()
+
+    def ttl_delay() -> int:
+        return _dns_refresh_delay([_dns_lookup_ttl(h, af_filter) for h in hosts])
+
+    delay = await loop.run_in_executor(None, ttl_delay)
+    while _running:
+        await _sleep(delay)
+        if not _running:
+            break
+        resolved = await loop.run_in_executor(None, _dns_resolve, hosts, port, af_filter)
+        if resolved is None:
+            delay = _DNS_FAILED_RETRY
+            continue
+        new, gone = _dns_retarget(connections, resolved)
+        for conn in gone:
+            if len(connections) > 1:
+                log.info("dropping server address %s", conn.addr)
+                conn._dead = True
+                conn.close()
+                connections.remove(conn)
+        for af, addr in new:
+            conn = AsyncConnection(next_id, addr, port, af, name)
+            next_id += 1
+            log.info("adding server address %s", addr)
+            connections.append(conn)
+            _active_tasks.append(asyncio.create_task(_heartbeat_sender(conn, interval, probe)))
+        delay = await loop.run_in_executor(None, ttl_delay)
+
+
 # ---------------------------------------------------------------------------
 # Server command handlers
 # ---------------------------------------------------------------------------
@@ -940,15 +1473,21 @@ async def _handle_update(conn: AsyncConnection):
 # Heartbeat sender
 # ---------------------------------------------------------------------------
 
-async def _heartbeat_sender(conn: AsyncConnection, interval: int):
+def _build_heartbeat_msg(conn: AsyncConnection, interval: int,
+                         probe: Optional[_Connectivity] = None) -> Dict[str, Any]:
+    """HTB payload; carries p4/p6 presence flags when the probe knows them."""
+    msg: Dict[str, Any] = {"acks": conn.ackcount, "rtt": conn.rtts[-1], "interval": interval}
+    if probe is not None:
+        msg.update(probe.flags())
+    return msg
+
+
+async def _heartbeat_sender(conn: AsyncConnection, interval: int,
+                            probe: Optional[_Connectivity] = None):
     log = logging.getLogger("hbc.hb")
     while _running:
         try:
-            await conn.sendto({
-                "acks": conn.ackcount,
-                "rtt": conn.rtts[-1],
-                "interval": interval,
-            })
+            await conn.sendto(_build_heartbeat_msg(conn, interval, probe))
         except Exception as e:
             log.error("send: %s", e)
         await _sleep(interval)
@@ -1056,7 +1595,7 @@ def _reconfigure_syslog(level: int):
 # ---------------------------------------------------------------------------
 
 async def _async_main(args, cfg: Dict[str, Any]) -> int:
-    global _running, _shutdown_event, _active_tasks, send_shutdown
+    global _running, _shutdown_event, _active_tasks, send_shutdown, _probe
     _running = True
     _shutdown_event = asyncio.Event()
     _active_tasks = []
@@ -1113,6 +1652,10 @@ async def _async_main(args, cfg: Dict[str, Any]) -> int:
                 c.close()
             return 0
 
+    # Per-protocol internet presence probe (shared with the network_monitor plugin)
+    net_cfg = dict((cfg.get("plugins") or {}).get("network_monitor") or cfg.get("network_monitor") or {})
+    _probe = _Connectivity.from_config(net_cfg) if net_cfg.get("probe_enabled", True) else None
+
     plugins = await _load_plugins(cfg)
 
     loop = asyncio.get_event_loop()
@@ -1126,8 +1669,13 @@ async def _async_main(args, cfg: Dict[str, Any]) -> int:
 
     loop.add_signal_handler(signal.SIGHUP, _sighup)
 
+    if _probe is not None:
+        _active_tasks.append(asyncio.create_task(_probe.run()))
     for conn in connections:
-        _active_tasks.append(asyncio.create_task(_heartbeat_sender(conn, interval)))
+        _active_tasks.append(asyncio.create_task(_heartbeat_sender(conn, interval, _probe)))
+
+    _active_tasks.append(asyncio.create_task(
+        _dns_refresher(connections, args.hosts, port, af_filter, iam, interval, _probe, conn_id)))
 
     if plugins and connections:
         _active_tasks.append(asyncio.create_task(_plugin_collector(connections[0], plugins)))

tests/test_client_dnsttl.py

@@ -0,0 +1,119 @@
+"""TTL-driven re-resolution of the server hostnames in the full client."""
+import socket
+import struct
+
+from hbd.client import dnsttl
+
+
+def _rr(rtype, ttl, rdata):
+    return b"\xc0\x0c" + struct.pack("!HHIH", rtype, 1, ttl, len(rdata)) + rdata
+
+
+def _response(qid, qtype, answers, rcode=0):
+    q = dnsttl.build_query("hbd.example.com", qtype, qid)[12:]
+    return struct.pack("!HHHHHH", qid, 0x8180 | rcode, 1, len(answers), 0, 0) + q + b"".join(answers)
+
+
+def test_min_ttl_covers_cname_and_address():
+    resp = _response(7, dnsttl.TYPE_A, [_rr(dnsttl.TYPE_CNAME, 60, b"\x01x\x00"),
+                                        _rr(dnsttl.TYPE_A, 300, b"\x01\x02\x03\x04")])
+    assert dnsttl.min_answer_ttl(resp, 7, dnsttl.TYPE_A) == 60
+
+
+def test_no_answer_of_requested_type_is_unknown():
+    assert dnsttl.min_answer_ttl(_response(7, dnsttl.TYPE_AAAA, []), 7, dnsttl.TYPE_AAAA) is None
+
+
+def test_error_rcode_and_foreign_id_are_unknown():
+    ans = [_rr(dnsttl.TYPE_A, 300, b"\x01\x02\x03\x04")]
+    assert dnsttl.min_answer_ttl(_response(7, 1, ans, rcode=3), 7, 1) is None
+    assert dnsttl.min_answer_ttl(_response(7, 1, ans), 8, 1) is None
+
+
+def test_truncated_response_is_unknown():
+    assert dnsttl.min_answer_ttl(b"\x00\x07\x81", 7, 1) is None
+
+
+def test_refresh_delay():
+    assert dnsttl.refresh_delay([300, 120, None]) == 120
+    assert dnsttl.refresh_delay([0]) == dnsttl.MIN_TTL
+    assert dnsttl.refresh_delay([None]) == dnsttl.DEFAULT_TTL
+
+
+def test_ip_literal_has_no_ttl():
+    assert dnsttl.lookup_ttl("192.0.2.1") is None
+    assert dnsttl.lookup_ttl("2001:db8::1") is None
+
+
+class _C:
+    def __init__(self, addr, af=socket.AF_INET):
+        self.addr, self.af = addr, af
+
+    def set_target(self, addr, af):
+        self.addr, self.af = addr, af
+
+
+V4, V6 = socket.AF_INET, socket.AF_INET6
+
+
+def test_retarget_follows_address_change_in_place():
+    first, second = _C("1.1.1.1"), _C("2001:db8::1", V6)
+    new, gone = dnsttl.retarget([first, second], [(V4, "9.9.9.9"), (V6, "2001:db8::9")])
+    assert (new, gone) == ([], [])
+    assert (first.addr, second.addr) == ("9.9.9.9", "2001:db8::9")
+
+
+def test_retarget_unchanged_is_a_noop():
+    conns = [_C("1.1.1.1")]
+    assert dnsttl.retarget(conns, [(V4, "1.1.1.1")]) == ([], [])
+
+
+def test_retarget_reports_surplus_new_and_gone():
+    a, b = _C("1.1.1.1"), _C("2.2.2.2")
+    new, gone = dnsttl.retarget([a, b], [(V4, "3.3.3.3")])
+    assert new == [] and gone == [b] and a.addr == "3.3.3.3"
+    new, gone = dnsttl.retarget([a], [(V4, "3.3.3.3"), (V4, "4.4.4.4")])
+    assert new == [(V4, "4.4.4.4")] and gone == []
+
+
+def test_retarget_never_drops_the_first_connection():
+    a, b = _C("1.1.1.1"), _C("2.2.2.2")
+    new, gone = dnsttl.retarget([a, b], [(V4, "2.2.2.2")])
+    assert new == [] and gone == [b] and a.addr == "2.2.2.2"
+
+
+def test_resolve_failure_returns_none(monkeypatch):
+    def boom(*a):
+        raise socket.gaierror("nope")
+    monkeypatch.setattr(socket, "getaddrinfo", boom)
+    assert dnsttl.resolve(["x.invalid"], 1, 0) is None
+
+
+def test_refresher_follows_a_change_and_adds_a_family(monkeypatch):
+    import asyncio
+    from hbd.client import main as hbc
+
+    answers = [[(V4, "9.9.9.9"), (V6, "2001:db8::9")], None, [(V4, "9.9.9.9")]]
+    monkeypatch.setattr(dnsttl, "resolve", lambda *a: answers.pop(0) if len(answers) > 1 else answers[0])
+    monkeypatch.setattr(dnsttl, "lookup_ttl", lambda *a: None)
+    monkeypatch.setattr(dnsttl, "DEFAULT_TTL", 0.01)
+    monkeypatch.setattr(dnsttl, "FAILED_RETRY", 0.01)
+
+    async def idle(*a):
+        await asyncio.sleep(10)
+    monkeypatch.setattr(hbc, "heartbeat_sender", idle)
+
+    async def go():
+        hbc.shutdown_event = asyncio.Event()
+        hbc.active_tasks = []
+        conns = [hbc.AsyncConnection(1, "1.1.1.1", 5, V4, "me")]
+        task = asyncio.create_task(hbc.dns_refresher(conns, ["h"], 5, 0, "me", 10, None, 2))
+        await asyncio.sleep(0.2)
+        hbc.shutdown_event.set()
+        await task
+        for t in hbc.active_tasks:
+            t.cancel()
+        return conns
+
+    conns = asyncio.run(go())
+    assert [(c.conn_id, c.addr) for c in conns] == [(1, "9.9.9.9")]

tests/test_client_heartbeat.py

@@ -0,0 +1,28 @@
+from hbd.client.main import build_heartbeat_msg
+
+
+class _Conn:
+    ackcount = 3
+    rtts = [0, 12.5]
+
+
+class _Conn2:
+    def __init__(self, flags):
+        self._flags = flags
+
+    def flags(self):
+        return self._flags
+
+
+def test_heartbeat_without_connectivity_is_unchanged():
+    assert build_heartbeat_msg(_Conn(), 60) == {"acks": 3, "rtt": 12.5, "interval": 60}
+
+
+def test_heartbeat_carries_presence_flags():
+    msg = build_heartbeat_msg(_Conn(), 60, _Conn2({"p4": True, "p6": False}))
+    assert msg == {"acks": 3, "rtt": 12.5, "interval": 60, "p4": True, "p6": False}
+
+
+def test_heartbeat_omits_unknown_flags():
+    msg = build_heartbeat_msg(_Conn(), 60, _Conn2({}))
+    assert "p4" not in msg and "p6" not in msg

tests/test_configio.py

@@ -1,5 +1,3 @@
-import glob
-import os
 import pytest
 from hbd.server import configio
 

tests/test_connectivity.py

@@ -0,0 +1,173 @@
+import asyncio
+
+from hbd.client.connectivity import Connectivity, ProbeUnavailable
+from hbd.client.pingutil import PingUnavailable
+
+
+class Net:
+    """Scriptable network: addr None = no route, 'unavailable' = family unsupported."""
+
+    def __init__(self):
+        self.addr = {"ipv4": "192.0.2.5", "ipv6": "2001:db8::5"}
+        self.gw = {"ipv4": "192.0.2.1", "ipv6": "fe80::1"}
+        self.rtt = {"ipv4": 10.0, "ipv6": 20.0}   # None = ping fails, 'unavailable' = no ping binary
+        self.pings = []
+        self.now = 1000.0
+
+    def local(self, fam, target):
+        a = self.addr[fam]
+        if a == "unavailable":
+            raise ProbeUnavailable("no such family")
+        return a
+
+    async def ping(self, target, fam):
+        self.pings.append(fam)
+        r = self.rtt[fam]
+        if r == "unavailable":
+            raise PingUnavailable("no ping")
+        return r
+
+    async def gateway(self, fam):
+        return self.gw[fam]
+
+    def clock(self):
+        return self.now
+
+
+def make(net, **kw):
+    c = Connectivity(local_address_fn=net.local, ping_fn=net.ping, gateway_fn=net.gateway,
+                     clock=net.clock, **kw)
+    c._cheap_gateway = True
+    return c
+
+
+def refresh(c):
+    asyncio.run(c.refresh())
+
+
+def test_present_with_addr_gateway_rtt():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    s = c.snapshot()["ipv4"]
+    assert (s.present, s.addr, s.gateway, s.rtt) == (True, "192.0.2.5", "192.0.2.1", 10.0)
+    assert c.flags() == {"p4": True, "p6": True}
+    assert c.as_metric()["ipv6"]["gateway"] == "fe80::1"
+
+
+def test_no_route_is_immediately_absent():
+    net = Net()
+    net.addr["ipv6"] = None
+    c = make(net)
+    refresh(c)
+    assert c.flags() == {"p4": True, "p6": False}
+    assert net.pings == ["ipv4"]          # no ping wasted on a family without a route
+    assert c.snapshot()["ipv6"].addr is None
+
+
+def test_ping_is_debounced_and_rate_limited():
+    net = Net()
+    c = make(net, probe_interval=60.0, retry_interval=10.0)
+    refresh(c)
+    refresh(c)                            # same instant: cached, no second ping
+    assert net.pings == ["ipv4", "ipv6"]
+    net.now += 60
+    net.rtt["ipv6"] = None                # first failure: still present (debounce)
+    refresh(c)
+    assert c.flags()["p6"] is True
+    net.now += 10                         # retry_interval elapsed: second failure flips it
+    refresh(c)
+    assert c.flags()["p6"] is False
+
+
+def test_first_probe_failing_once_is_unknown():
+    net = Net()
+    net.rtt["ipv4"] = None
+    c = make(net)
+    refresh(c)
+    assert "p4" not in c.flags() and c.flags()["p6"] is True
+
+
+def test_recovers_on_first_success():
+    net = Net()
+    net.rtt["ipv6"] = None
+    c = make(net, retry_interval=10.0)
+    refresh(c)
+    net.now += 10
+    refresh(c)
+    assert c.flags()["p6"] is False
+    net.rtt["ipv6"] = 25.0
+    net.now += 10
+    refresh(c)
+    assert c.flags()["p6"] is True
+
+
+def test_address_change_invalidates_cache():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    net.addr["ipv4"] = "198.51.100.7"     # moved networks, no time has passed
+    refresh(c)
+    assert net.pings.count("ipv4") == 2
+    assert c.snapshot()["ipv4"].addr == "198.51.100.7"
+
+
+def test_gateway_change_invalidates_cache():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    net.gw["ipv4"] = "192.0.2.254"
+    refresh(c)
+    assert net.pings.count("ipv4") == 2
+
+
+def test_route_loss_then_return_pings_immediately():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    net.addr["ipv6"] = None
+    refresh(c)
+    net.addr["ipv6"] = "2001:db8::5"
+    refresh(c)
+    assert net.pings.count("ipv6") == 2 and c.flags()["p6"] is True
+
+
+def test_missing_ping_binary_leaves_presence_unknown_and_stops_retrying():
+    net = Net()
+    net.rtt["ipv4"] = "unavailable"
+    c = make(net)
+    refresh(c)
+    net.now += 600
+    refresh(c)
+    assert "p4" not in c.flags()
+    assert net.pings.count("ipv4") == 1
+
+
+def test_unsupported_family_is_unknown():
+    net = Net()
+    net.addr["ipv6"] = "unavailable"
+    c = make(net)
+    refresh(c)
+    assert "p6" not in c.flags() and c.flags()["p4"] is True
+
+
+def test_from_config():
+    c = Connectivity.from_config({"probe_v4": "9.9.9.9", "probe_interval": 30})
+    assert c.probes["ipv4"] == "9.9.9.9"
+    assert c.probes["ipv6"] == "2606:4700:4700::1111"
+    assert c.probe_interval == 30.0
+
+
+def test_ping_less_family_stays_unknown_after_route_returns():
+    net = Net()
+    net.rtt["ipv4"] = "unavailable"
+    c = make(net)
+    refresh(c)
+    assert "p4" not in c.flags()
+    net.addr["ipv4"] = None
+    refresh(c)
+    assert c.flags()["p4"] is False       # no route is definitive
+    net.addr["ipv4"] = "192.0.2.5"
+    refresh(c)
+    assert "p4" not in c.flags()
+    assert net.pings.count("ipv4") == 1   # missing binary is not retried

tests/test_dns.py

@@ -1,9 +1,8 @@
-import time
-import queue
+import asyncio
 import unittest
 from unittest.mock import patch, MagicMock
 
-import hbd.dns as dns
+import hbd.server.dns as dns
 
 
 class TestDNS(unittest.TestCase):
@@ -17,7 +16,7 @@ class TestDNS(unittest.TestCase):
         p = dns.create_nsupdate_payload("host", "2001:db8::1", "example")
         self.assertIn("update add host.dy.example 5 AAAA 2001:db8::1", p)
 
-    @patch("hbd.dns.Popen")
+    @patch("hbd.server.dns.Popen")
     def test_nsupdate_success(self, mock_popen):
         proc = MagicMock()
         proc.communicate.return_value = (b"status: NOERROR", None)
@@ -41,7 +40,7 @@ class TestDNS(unittest.TestCase):
             stderr=dns.STDOUT,
         )
 
-    @patch("hbd.dns.Popen")
+    @patch("hbd.server.dns.Popen")
     def test_nsupdate_failure(self, mock_popen):
         proc = MagicMock()
         proc.communicate.return_value = (b"some error", None)
@@ -57,83 +56,7 @@ class TestDNS(unittest.TestCase):
         self.assertIsNotNone(err)
         self.assertIn("some error", err)
 
-    def test_dnsupdatethread_processes_queue(self):
-        # patch nsupdate to succeed
-        with patch("hbd.dns.nsupdate", return_value=None):
-            logs = []
-
-            def log(h, m):
-                logs.append((h, m))
-
-            emails = []
-
-            def email(s, m):
-                emails.append((s, m))
-
-            class FakeHost:
-                dnsQ = queue.Queue()
-
-            class FakeHbd:
-                Host = FakeHost
-
-            # start the thread (daemon) that processes the queue
-            t = dns.start_dns_thread(
-                FakeHbd, {"dyndomains": ["example"]}, log=log, email=email
-            )
-            self.assertTrue(t.is_alive())
-
-            # enqueue one item and wait for it to be processed (polling with timeout)
-            FakeHbd.Host.dnsQ.put(("testhost", "1.2.3.4"))
-
-            for _ in range(30):
-                if logs:
-                    break
-                time.sleep(0.1)
-
-            self.assertTrue(logs, "dnsupdatethread did not call log")
-            self.assertTrue(
-                any("changed address" in m or "DNS updated" in m for (_h, m) in logs)
-            )
-
-    def test_dnsupdatethread_calls_email_on_failure(self):
-        # patch nsupdate to fail with an error message
-        with patch("hbd.dns.nsupdate", return_value="error: failed"):
-            logs = []
-
-            def log(h, m):
-                logs.append((h, m))
-
-            emails = []
-
-            def email(s, m):
-                emails.append((s, m))
-
-            class FakeHost:
-                dnsQ = queue.Queue()
-
-            class FakeHbd:
-                Host = FakeHost
-
-            dns.start_dns_thread(
-                FakeHbd, {"dyndomains": ["example"]}, log=log, email=email
-            )
-            # enqueue and wait for the email to be sent
-            FakeHbd.Host.dnsQ.put(("testhost", "1.2.3.4"))
-
-            for _ in range(30):
-                if emails:
-                    break
-                time.sleep(0.1)
-
-            self.assertTrue(emails, "dnsupdatethread did not call email on failure")
-            self.assertTrue(
-                any(
-                    "nsupdate failed" in s or "nsupdate failed" in m or "error" in m
-                    for (s, m) in emails
-                )
-            )
-
-    @patch("hbd.dns.Popen")
+    @patch("hbd.server.dns.Popen")
     def test_nsupdate_raises_oserror(self, mock_popen):
         mock_popen.side_effect = OSError("noexec")
         err = dns.nsupdate(
@@ -147,5 +70,37 @@ class TestDNS(unittest.TestCase):
         self.assertIn("execution failed", err)
 
 
+class TestDNSWorker(unittest.IsolatedAsyncioTestCase):
+
+    async def _run_worker(self, nsupdate_result):
+        """Feed one update through dns_update_worker; returns the (name, level, message) log calls."""
+        logs = []
+        queue = asyncio.Queue()
+        await queue.put(("testhost", "1.2.3.4"))
+        await queue.put(None)
+        with patch("hbd.server.dns.nsupdate", return_value=nsupdate_result):
+            await dns.dns_update_worker(
+                None, {"dyndomains": ["example"]}, async_queue=queue,
+                log=lambda name, level, msg: logs.append((name, level, msg)),
+            )
+        return logs
+
+    async def test_worker_logs_successful_update(self):
+        logs = await self._run_worker(None)
+        self.assertEqual(len(logs), 1)
+        name, level, msg = logs[0]
+        self.assertEqual((name, level), ("testhost", "INFO"))
+        self.assertIn("DNS updated testhost.dy.example", msg)
+        self.assertIn("1.2.3.4", msg)
+
+    async def test_worker_logs_error_on_failed_update(self):
+        logs = await self._run_worker("error: failed")
+        self.assertEqual(len(logs), 1)
+        name, level, msg = logs[0]
+        self.assertEqual((name, level), ("testhost", "ERROR"))
+        self.assertIn("DNS update failed", msg)
+        self.assertIn("error: failed", msg)
+
+
 if __name__ == "__main__":
     unittest.main()

tests/test_gateway.py

@@ -0,0 +1,75 @@
+import asyncio
+import struct
+
+from hbd.client import gateway
+
+PROC_V4 = (
+    "Iface\tDestination\tGateway \tFlags\tRefCnt\tUse\tMetric\tMask\t\tMTU\tWindow\tIRTT\n"
+    "wlan0\t00000000\t0100A8C0\t0003\t0\t0\t600\t00000000\t0\t0\t0\n"
+    "eth0\t00000000\t0101A8C0\t0003\t0\t0\t100\t00000000\t0\t0\t0\n"
+    "wlan0\t0000A8C0\t00000000\t0001\t0\t0\t600\t00FFFFFF\t0\t0\t0\n"
+)
+
+PROC_V6 = (
+    "00000000000000000000000000000000 00 00000000000000000000000000000000 00 "
+    "fe800000000000000000000000000001 00000400 00000001 00000000 00000003 wlan0\n"
+    "00000000000000000000000000000000 00 00000000000000000000000000000000 00 "
+    "00000000000000000000000000000000 ffffffff 00000001 00000000 00200200 lo\n"
+    "fe800000000000000000000000000000 40 00000000000000000000000000000000 00 "
+    "00000000000000000000000000000000 00000100 00000001 00000000 00000001 wlan0\n"
+)
+
+MACOS = """   route to: default
+destination: default
+       mask: default
+    gateway: 192.168.1.1
+  interface: en0
+"""
+
+WIN_V4 = """IPv4 Route Table
+Active Routes:
+Network Destination        Netmask          Gateway       Interface  Metric
+          0.0.0.0          0.0.0.0      192.168.1.1    192.168.1.50     25
+          0.0.0.0          0.0.0.0        10.0.0.1        10.0.0.9     50
+        127.0.0.0        255.0.0.0         On-link         127.0.0.1    331
+"""
+
+WIN_V6 = """IPv6 Route Table
+Active Routes:
+ If Metric Network Destination      Gateway
+ 12    281 ::/0                     fe80::1
+  1    331 ::1/128                  On-link
+"""
+
+
+def test_proc_v4_picks_lowest_metric_gateway():
+    assert gateway.parse_proc_route_v4(PROC_V4) == "192.168.1.1"  # eth0, metric 100
+
+
+def test_proc_v4_none_without_default_route():
+    assert gateway.parse_proc_route_v4(PROC_V4.splitlines()[0] + "\n") is None
+
+
+def test_proc_v6_skips_reject_and_non_default():
+    assert gateway.parse_proc_route_v6(PROC_V6) == "fe80::1"
+
+
+def test_macos():
+    assert gateway.parse_macos_route(MACOS) == "192.168.1.1"
+    assert gateway.parse_macos_route("route: writing to routing socket: not in table") is None
+
+
+def test_windows_v4_lowest_metric():
+    assert gateway.parse_windows_route(WIN_V4, "ipv4") == "192.168.1.1"
+
+
+def test_windows_v6():
+    assert gateway.parse_windows_route(WIN_V6, "ipv6") == "fe80::1"
+
+
+def test_default_gateway_never_raises_on_parser_error(monkeypatch):
+    """default_gateway catches all exceptions; never raises even if parser fails."""
+    monkeypatch.setattr("sys.platform", "linux")
+    monkeypatch.setattr(gateway, "parse_proc_route_v4", lambda _: (_ for _ in ()).throw(struct.error))
+    result = asyncio.run(gateway.default_gateway("ipv4"))
+    assert result is None

tests/test_handle_datagram.py

@@ -1,5 +1,5 @@
-from hbd.udp import handle_datagram, parse_message
-from hbd.proto import dicttos
+from hbd.server.udp import handle_datagram, parse_message
+from hbd.common.proto import dicttos, stodict
 
 
 class FakeTransport:
@@ -17,7 +17,7 @@ def dummy_noop(*a, **k):
 def test_handle_cmd_sends_command():
     ftr = FakeTransport()
     # prepare ctx linking to hbdclass
-    import hbdclass
+    from hbd.server import hbdclass
 
     ctx = {
         "config": {"watchhosts": []},
@@ -34,7 +34,7 @@ def test_handle_cmd_sends_command():
     # create host by sending initial heartbeat
     msg = parse_message(dicttos("HTB", {"name": "cmdhost", "interval": 10}))
     handle_datagram(msg, ("127.0.0.1", 50000), ftr, ctx)
-    assert ftr.sent[0][0] == b"ACK"
+    assert ftr.sent[0][0].startswith(b"!ACK:")
 
     # queue a CMD for the host and send another heartbeat; expect command sent
     h = hbdclass.Host.hosts["cmdhost"]
@@ -42,8 +42,6 @@ def test_handle_cmd_sends_command():
     ftr.sent.clear()
     msg2 = parse_message(dicttos("HTB", {"name": "cmdhost", "interval": 10}))
     handle_datagram(msg2, ("127.0.0.1", 50000), ftr, ctx)
-    # should have sent ACK and the command; last send should be non-empty
-    assert len(ftr.sent) >= 1
-    # the command for ??0 will be sent as raw cmd string
-    # so at least one send contains b'doit' or similar
-    assert any(b"doit" in s[0] for s in ftr.sent)
+    # the ACK and the queued command are both sent; the command is a compressed CMD packet
+    decoded = [stodict(data) for data, _addr in ftr.sent]
+    assert any(d["ID"] == "CMD" and d["cmd"] == "doit" for d in decoded)

tests/test_hbc_mini_dnsttl.py

@@ -0,0 +1,72 @@
+"""TTL-driven re-resolution in the standalone scripts/hbc_mini.py."""
+import asyncio
+import importlib.util
+import pathlib
+import socket
+import struct
+import sys
+
+_PATH = pathlib.Path(__file__).resolve().parents[1] / "scripts" / "hbc_mini.py"
+_SPEC = importlib.util.spec_from_file_location("hbc_mini", _PATH)
+mini = importlib.util.module_from_spec(_SPEC)
+sys.modules["hbc_mini"] = mini
+_SPEC.loader.exec_module(mini)
+
+V4, V6 = socket.AF_INET, socket.AF_INET6
+
+
+def _rr(rtype, ttl, rdata):
+    return b"\xc0\x0c" + struct.pack("!HHIH", rtype, 1, ttl, len(rdata)) + rdata
+
+
+def _response(qid, qtype, answers, rcode=0):
+    q = mini._dns_build_query("hbd.example.com", qtype, qid)[12:]
+    return struct.pack("!HHHHHH", qid, 0x8180 | rcode, 1, len(answers), 0, 0) + q + b"".join(answers)
+
+
+def test_min_ttl_covers_cname_and_address():
+    resp = _response(7, 1, [_rr(5, 60, b"\x01x\x00"), _rr(1, 300, b"\x01\x02\x03\x04")])
+    assert mini._dns_min_answer_ttl(resp, 7, 1) == 60
+
+
+def test_unknown_ttl_cases():
+    assert mini._dns_min_answer_ttl(_response(7, 28, []), 7, 28) is None
+    assert mini._dns_min_answer_ttl(_response(7, 1, [_rr(1, 9, b"1234")], rcode=3), 7, 1) is None
+    assert mini._dns_min_answer_ttl(b"\x00\x07\x81", 7, 1) is None
+    assert mini._dns_lookup_ttl("192.0.2.1") is None
+
+
+def test_refresh_delay():
+    assert mini._dns_refresh_delay([300, 120, None]) == 120
+    assert mini._dns_refresh_delay([0]) == mini._DNS_MIN_TTL
+    assert mini._dns_refresh_delay([None]) == mini._DNS_DEFAULT_TTL
+
+
+def test_refresher_follows_a_change_and_adds_a_family(monkeypatch):
+    answers = [[(V4, "9.9.9.9"), (V6, "2001:db8::9")], None, [(V4, "9.9.9.9")]]
+    monkeypatch.setattr(mini, "_dns_resolve", lambda *a: answers.pop(0) if len(answers) > 1 else answers[0])
+    monkeypatch.setattr(mini, "_dns_lookup_ttl", lambda *a: None)
+    monkeypatch.setattr(mini, "_DNS_DEFAULT_TTL", 0.01)
+    monkeypatch.setattr(mini, "_DNS_FAILED_RETRY", 0.01)
+
+    async def idle(*a):
+        await asyncio.sleep(10)
+    monkeypatch.setattr(mini, "_heartbeat_sender", idle)
+
+    async def go():
+        mini._running = True
+        mini._shutdown_event = asyncio.Event()
+        mini._active_tasks = []
+        conns = [mini.AsyncConnection(1, "1.1.1.1", 5, V4, "me")]
+        task = asyncio.create_task(mini._dns_refresher(conns, ["h"], 5, 0, "me", 10, None, 2))
+        await asyncio.sleep(0.2)
+        mini._running = False
+        mini._shutdown_event.set()
+        await task
+        for t in mini._active_tasks:
+            t.cancel()
+        return conns
+
+    conns = asyncio.run(go())
+    assert [(c.conn_id, c.addr) for c in conns] == [(1, "9.9.9.9")]
+    assert conns[0].af == V4

tests/test_hbc_mini_probe.py

@@ -0,0 +1,381 @@
+"""Connectivity probe in the standalone scripts/hbc_mini.py (stdlib-only single-file client)."""
+import asyncio
+import importlib.util
+import pathlib
+import sys
+
+import pytest
+
+_PATH = pathlib.Path(__file__).resolve().parents[1] / "scripts" / "hbc_mini.py"
+_SPEC = importlib.util.spec_from_file_location("hbc_mini", _PATH)
+mini = importlib.util.module_from_spec(_SPEC)
+sys.modules["hbc_mini"] = mini          # dataclasses resolve their module by name
+_SPEC.loader.exec_module(mini)
+
+PROC_V4 = (
+    "Iface\tDestination\tGateway \tFlags\tRefCnt\tUse\tMetric\tMask\t\tMTU\tWindow\tIRTT\n"
+    "wlan0\t00000000\t0100A8C0\t0003\t0\t0\t600\t00000000\t0\t0\t0\n"
+    "eth0\t00000000\t0101A8C0\t0003\t0\t0\t100\t00000000\t0\t0\t0\n"
+    "wlan0\t0000A8C0\t00000000\t0001\t0\t0\t600\t00FFFFFF\t0\t0\t0\n"
+)
+PROC_V6 = (
+    "00000000000000000000000000000000 00 00000000000000000000000000000000 00 "
+    "fe800000000000000000000000000001 00000400 00000001 00000000 00000003 wlan0\n"
+    "00000000000000000000000000000000 00 00000000000000000000000000000000 00 "
+    "00000000000000000000000000000000 ffffffff 00000001 00000000 00200200 lo\n"
+    "fe800000000000000000000000000000 40 00000000000000000000000000000000 00 "
+    "00000000000000000000000000000000 00000100 00000001 00000000 00000001 wlan0\n"
+)
+ROUTE_GET = """   route to: 0.0.0.0
+destination: 0.0.0.0
+       mask: 0.0.0.0
+    gateway: 192.0.2.1
+        fib: 0
+  interface: vtnet0
+"""
+WIN_V4 = """IPv4 Route Table
+Active Routes:
+Network Destination        Netmask          Gateway       Interface  Metric
+          0.0.0.0          0.0.0.0      192.168.1.1    192.168.1.50     25
+          0.0.0.0          0.0.0.0        10.0.0.1        10.0.0.9     50
+        127.0.0.0        255.0.0.0         On-link         127.0.0.1    331
+"""
+WIN_V6 = """IPv6 Route Table
+Active Routes:
+ If Metric Network Destination      Gateway
+ 12    281 ::/0                     fe80::1
+  1    331 ::1/128                  On-link
+"""
+LINUX_PING = """PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
+64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=11.2 ms
+
+--- 1.1.1.1 ping statistics ---
+1 packets transmitted, 1 received, 0% packet loss, time 0ms
+rtt min/avg/max/mdev = 11.234/11.234/11.234/0.000 ms
+"""
+FREEBSD_PING = """PING 1.1.1.1 (1.1.1.1): 56 data bytes
+64 bytes from 1.1.1.1: icmp_seq=0 ttl=59 time=4.006 ms
+
+--- 1.1.1.1 ping statistics ---
+1 packets transmitted, 1 packets received, 0.0% packet loss, 1 packets out of wait time
+round-trip min/avg/max/stddev = 4.006/4.006/4.006/0.000 ms
+"""
+WIN_PING = """Pinging 1.1.1.1 with 32 bytes of data:
+Reply from 1.1.1.1: bytes=32 time=12ms TTL=57
+
+Ping statistics for 1.1.1.1:
+    Packets: Sent = 1, Received = 1, Lost = 0 (0% loss),
+"""
+LOST_PING = """--- 1.1.1.1 ping statistics ---
+1 packets transmitted, 0 received, 100% packet loss, time 0ms
+"""
+
+
+# ---- gateway parsers ---------------------------------------------------------------
+
+def test_proc_v4_lowest_metric_wins():
+    assert mini._parse_proc_route_v4(PROC_V4) == "192.168.1.1"      # eth0, metric 100
+
+
+def test_proc_v4_none_without_default_route():
+    assert mini._parse_proc_route_v4(PROC_V4.splitlines()[0] + "\n") is None
+
+
+def test_proc_v6_skips_reject_and_non_default():
+    assert mini._parse_proc_route_v6(PROC_V6) == "fe80::1"
+
+
+def test_route_get_bsd_and_macos():
+    assert mini._parse_route_get(ROUTE_GET) == "192.0.2.1"
+    assert mini._parse_route_get("route: writing to routing socket: not in table") is None
+
+
+def test_windows_routes():
+    assert mini._parse_windows_route(WIN_V4, "ipv4") == "192.168.1.1"
+    assert mini._parse_windows_route(WIN_V6, "ipv6") == "fe80::1"
+
+
+def test_default_gateway_never_raises(monkeypatch):
+    monkeypatch.setattr(sys, "platform", "linux")
+    monkeypatch.setattr(mini, "_parse_proc_route_v4", lambda t: 1 / 0)
+    assert asyncio.run(mini._default_gateway("ipv4")) is None
+
+
+# ---- ping ---------------------------------------------------------------------------
+
+@pytest.mark.parametrize("platform,family,expected", [
+    ("linux", "ipv4", ["ping", "-c", "1", "-W", "2", "-4", "h"]),
+    ("linux", "ipv6", ["ping", "-c", "1", "-W", "2", "-6", "h"]),
+    ("freebsd15", "ipv4", ["ping", "-c", "1", "-W", "2000", "h"]),
+    ("freebsd15", "ipv6", ["ping6", "-c", "1", "h"]),
+    ("darwin", "ipv4", ["ping", "-c", "1", "-W", "2000", "h"]),
+    ("darwin", "ipv6", ["ping6", "-c", "1", "h"]),
+    ("win32", "ipv4", ["ping", "-n", "1", "-w", "2000", "-4", "h"]),
+    ("win32", "ipv6", ["ping", "-n", "1", "-w", "2000", "-6", "h"]),
+])
+def test_probe_ping_command(monkeypatch, platform, family, expected):
+    monkeypatch.setattr(sys, "platform", platform)
+    assert mini._probe_ping_command("h", family) == expected
+
+
+def test_parse_ping_rtt():
+    assert mini._parse_ping_rtt(LINUX_PING) == 11.234
+    assert mini._parse_ping_rtt(FREEBSD_PING) == 4.006
+    assert mini._parse_ping_rtt(WIN_PING) == 12.0
+    assert mini._parse_ping_rtt(LOST_PING) is None
+    assert mini._parse_ping_rtt("ping: connect: Network is unreachable") is None
+
+
+# ---- probe state machine ------------------------------------------------------------
+
+class Net:
+    """Scriptable network: addr None = no route, 'unavailable' = family unsupported."""
+
+    def __init__(self):
+        self.addr = {"ipv4": "192.0.2.5", "ipv6": "2001:db8::5"}
+        self.gw = {"ipv4": "192.0.2.1", "ipv6": "fe80::1"}
+        self.rtt = {"ipv4": 10.0, "ipv6": 20.0}   # None = ping fails, 'unavailable' = no ping binary
+        self.pings = []
+        self.now = 1000.0
+
+    def local(self, fam, target):
+        a = self.addr[fam]
+        if a == "unavailable":
+            raise mini._ProbeUnavailable("no such family")
+        return a
+
+    async def ping(self, target, fam):
+        self.pings.append(fam)
+        r = self.rtt[fam]
+        if r == "unavailable":
+            raise mini._PingUnavailable("no ping")
+        return r
+
+    async def gateway(self, fam):
+        return self.gw[fam]
+
+    def clock(self):
+        return self.now
+
+
+def make(net, **kw):
+    c = mini._Connectivity(local_address_fn=net.local, ping_fn=net.ping, gateway_fn=net.gateway,
+                           clock=net.clock, **kw)
+    c._cheap_gateway = True
+    return c
+
+
+def refresh(c):
+    asyncio.run(c.refresh())
+
+
+def test_present_with_addr_gateway_rtt():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    s = c.snapshot()["ipv4"]
+    assert (s.present, s.addr, s.gateway, s.rtt) == (True, "192.0.2.5", "192.0.2.1", 10.0)
+    assert c.flags() == {"p4": True, "p6": True}
+    assert c.as_metric()["ipv6"]["gateway"] == "fe80::1"
+
+
+def test_no_route_is_immediately_absent():
+    net = Net()
+    net.addr["ipv6"] = None
+    c = make(net)
+    refresh(c)
+    assert c.flags() == {"p4": True, "p6": False}
+    assert net.pings == ["ipv4"]
+    assert c.snapshot()["ipv6"].addr is None
+
+
+def test_ping_is_debounced_and_rate_limited():
+    net = Net()
+    c = make(net, probe_interval=60.0, retry_interval=10.0)
+    refresh(c)
+    refresh(c)
+    assert net.pings == ["ipv4", "ipv6"]
+    net.now += 60
+    net.rtt["ipv6"] = None
+    refresh(c)
+    assert c.flags()["p6"] is True
+    net.now += 10
+    refresh(c)
+    assert c.flags()["p6"] is False
+
+
+def test_first_probe_failing_once_is_unknown():
+    net = Net()
+    net.rtt["ipv4"] = None
+    c = make(net)
+    refresh(c)
+    assert "p4" not in c.flags() and c.flags()["p6"] is True
+
+
+def test_recovers_on_first_success():
+    net = Net()
+    net.rtt["ipv6"] = None
+    c = make(net, retry_interval=10.0)
+    refresh(c)
+    net.now += 10
+    refresh(c)
+    assert c.flags()["p6"] is False
+    net.rtt["ipv6"] = 25.0
+    net.now += 10
+    refresh(c)
+    assert c.flags()["p6"] is True
+
+
+def test_address_change_invalidates_cache():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    net.addr["ipv4"] = "198.51.100.7"
+    refresh(c)
+    assert net.pings.count("ipv4") == 2
+    assert c.snapshot()["ipv4"].addr == "198.51.100.7"
+
+
+def test_gateway_change_invalidates_cache():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    net.gw["ipv4"] = "192.0.2.254"
+    refresh(c)
+    assert net.pings.count("ipv4") == 2
+
+
+def test_route_loss_then_return_pings_immediately():
+    net = Net()
+    c = make(net)
+    refresh(c)
+    net.addr["ipv6"] = None
+    refresh(c)
+    net.addr["ipv6"] = "2001:db8::5"
+    refresh(c)
+    assert net.pings.count("ipv6") == 2 and c.flags()["p6"] is True
+
+
+def test_missing_ping_binary_leaves_presence_unknown_and_stops_retrying():
+    net = Net()
+    net.rtt["ipv4"] = "unavailable"
+    c = make(net)
+    refresh(c)
+    net.now += 600
+    refresh(c)
+    assert "p4" not in c.flags()
+    assert net.pings.count("ipv4") == 1
+
+
+def test_ping_less_family_stays_unknown_after_route_returns():
+    net = Net()
+    net.rtt["ipv4"] = "unavailable"
+    c = make(net)
+    refresh(c)
+    assert "p4" not in c.flags()
+    net.addr["ipv4"] = None
+    refresh(c)
+    assert c.flags()["p4"] is False              # no route is definitive
+    net.addr["ipv4"] = "192.0.2.5"
+    refresh(c)
+    assert "p4" not in c.flags() and net.pings.count("ipv4") == 1
+
+
+def test_unsupported_family_is_unknown():
+    net = Net()
+    net.addr["ipv6"] = "unavailable"
+    c = make(net)
+    refresh(c)
+    assert "p6" not in c.flags() and c.flags()["p4"] is True
+
+
+def test_from_config():
+    c = mini._Connectivity.from_config({"probe_v4": "9.9.9.9", "probe_interval": 30})
+    assert c.probes["ipv4"] == "9.9.9.9"
+    assert c.probes["ipv6"] == "2606:4700:4700::1111"
+    assert c.probe_interval == 30.0
+
+
+def test_local_address_on_this_host_never_raises():
+    # Real UDP connect(): either an address string, None (no route) or _ProbeUnavailable.
+    for fam, target in (("ipv4", "1.1.1.1"), ("ipv6", "2606:4700:4700::1111")):
+        try:
+            r = mini._local_address(fam, target)
+        except mini._ProbeUnavailable:
+            continue
+        assert r is None or isinstance(r, str)
+
+
+# ---- heartbeat message and plugin wiring ---------------------------------------------
+
+class _Conn:
+    ackcount = 3
+    rtts = [0.0, 12.5]
+
+
+class _FlagsProbe:
+    def __init__(self, flags):
+        self._flags = flags
+
+    def flags(self):
+        return self._flags
+
+
+def test_heartbeat_without_probe_is_unchanged():
+    assert mini._build_heartbeat_msg(_Conn(), 60) == {"acks": 3, "rtt": 12.5, "interval": 60}
+
+
+def test_heartbeat_carries_presence_flags():
+    msg = mini._build_heartbeat_msg(_Conn(), 60, _FlagsProbe({"p4": True, "p6": False}))
+    assert msg == {"acks": 3, "rtt": 12.5, "interval": 60, "p4": True, "p6": False}
+
+
+def test_heartbeat_omits_unknown_flags():
+    msg = mini._build_heartbeat_msg(_Conn(), 60, _FlagsProbe({}))
+    assert "p4" not in msg and "p6" not in msg
+
+
+def test_flags_survive_the_wire_encoding():
+    msg = mini._build_heartbeat_msg(_Conn(), 60, _FlagsProbe({"p4": True, "p6": False}))
+    out = mini._stodict(mini._dicttos("HTB", dict(msg, name="h", id=1)))
+    assert out["p4"] == 1 and out["p6"] == 0
+
+
+class _FakeProbe:
+    def as_metric(self):
+        return {"ipv4": {"present": True, "addr": "192.0.2.5", "gateway": "192.0.2.1", "rtt": 9.0},
+                "ipv6": {"present": False, "addr": None, "gateway": None, "rtt": None}}
+
+
+def test_plugin_reports_connectivity_alongside_interfaces(monkeypatch):
+    monkeypatch.setattr(mini, "_probe", _FakeProbe())
+    monkeypatch.setattr(mini.platform, "system", lambda: "Linux")
+    plugin = mini.NetworkMonitorPlugin({})
+    assert asyncio.run(plugin.initialize()) is True
+    data = asyncio.run(plugin._collect_metrics())
+    assert data["connectivity"]["ipv4"]["present"] is True
+    assert data["connectivity"]["ipv6"]["present"] is False
+
+
+def test_plugin_probe_only_mode_on_non_linux(monkeypatch):
+    monkeypatch.setattr(mini, "_probe", _FakeProbe())
+    monkeypatch.setattr(mini.platform, "system", lambda: "FreeBSD")
+    plugin = mini.NetworkMonitorPlugin({})
+    assert asyncio.run(plugin.initialize()) is True
+    assert list(asyncio.run(plugin._collect_metrics())) == ["connectivity"]
+
+
+def test_plugin_still_skips_on_non_linux_without_probe(monkeypatch):
+    monkeypatch.setattr(mini, "_probe", None)
+    monkeypatch.setattr(mini.platform, "system", lambda: "FreeBSD")
+    plugin = mini.NetworkMonitorPlugin({})
+    assert asyncio.run(plugin.initialize()) is False
+    assert plugin.skip_reason
+
+
+def test_plugin_omits_connectivity_without_probe(monkeypatch):
+    monkeypatch.setattr(mini, "_probe", None)
+    monkeypatch.setattr(mini.platform, "system", lambda: "Linux")
+    plugin = mini.NetworkMonitorPlugin({})
+    assert asyncio.run(plugin.initialize()) is True
+    assert "connectivity" not in asyncio.run(plugin._collect_metrics())

tests/test_host_status.py

@@ -0,0 +1,120 @@
+import time
+
+import pytest
+
+from hbd.server import hbdclass
+from hbd.server.hbdclass import Connection as C
+
+
+def _host(name):
+    hbdclass.Host.hosts.pop(name, None)
+    return hbdclass.Host(name)
+
+
+def _conn(host, afam, state):
+    addr = "192.0.2.1" if afam == "IPv4" else "2001:db8::1"
+    conn, _ = host.conndata(0, addr, None, 1000.0)
+    conn.state = state
+    return conn
+
+
+CASES = [
+    # v4 state, v6 state, presence flags, expected status
+    (C.UP, C.UP, {}, "up"),
+    (C.UP, C.OVERDUE, {}, "simplex"),                       # both present (legacy) => simplex
+    (C.UP, C.OVERDUE, {"IPv6": True}, "simplex"),
+    (C.UP, C.OVERDUE, {"IPv6": False}, "up"),               # v6 not present => not monitored
+    (C.OVERDUE, C.OVERDUE, {}, "overdue"),
+    (C.OVERDUE, C.ABSENT, {"IPv6": False}, "overdue"),      # only present family is overdue
+    (C.UP, C.ABSENT, {"IPv6": False}, "up"),
+    (C.DOWN, C.UP, {}, "down"),
+    (C.UNKNOWN, C.UNKNOWN, {}, "unknown"),
+    (C.OVERDUE, C.UNKNOWN, {}, "overdue"),
+    (C.UP, C.UNKNOWN, {}, "up"),
+    (C.OVERDUE, C.UP, {"IPv6": False}, "simplex"),          # HTB arriving => present despite flag
+]
+
+
+@pytest.mark.parametrize("s4,s6,flags,expected", CASES)
+def test_status_matrix(s4, s6, flags, expected):
+    h = _host("status-matrix")
+    _conn(h, "IPv4", s4)
+    _conn(h, "IPv6", s6)
+    for afam, present in flags.items():
+        h.set_presence(afam, present, time.time())
+    assert h.status() == expected
+
+
+def test_status_single_family_host():
+    h = _host("status-v4only")
+    _conn(h, "IPv4", C.UP)
+    assert h.status() == "up"
+    h.connections["IPv4"].state = C.OVERDUE
+    assert h.status() == "overdue"
+
+
+def test_status_no_connections_is_unknown():
+    assert _host("status-empty").status() == "unknown"
+
+
+def test_family_present_rules():
+    h = _host("fp")
+    v6 = _conn(h, "IPv6", C.OVERDUE)
+    assert h.family_present("IPv6") is True            # never reported => present
+    h.set_presence("IPv6", False, 1.0)
+    assert h.family_present("IPv6") is False
+    v6.state = C.UP                                    # real traffic overrides the flag
+    assert h.family_present("IPv6") is True
+    assert h.flag_present("IPv6") is False
+
+
+def test_stateinfo_and_statedict_expose_status():
+    h = _host("status-exposed")
+    _conn(h, "IPv4", C.UP)
+    assert h.stateinfo()["status"] == "up"
+    assert h.statedict()["status"] == "up"
+
+
+def test_absent_connection_statedict_is_neutral():
+    h = _host("absent-dict")
+    conn = _conn(h, "IPv6", C.ABSENT)
+    conn.rtts.append(42.0)
+    d = conn.statedict()
+    assert d["state"] == "absent" and d["rtt"] == "" and d["rttstate"] == "absent"
+
+
+def test_presence_survives_pickle_and_old_pickles_get_default():
+    import pickle
+    h = _host("pickled")
+    h.set_presence("IPv6", False, 5.0)
+    h2 = pickle.loads(pickle.dumps(h))
+    assert h2.presence == {"IPv6": {"present": False, "ts": 5.0}}
+    del h2.__dict__["presence"]                        # simulate a pre-feature pickle
+    h2.fixup()
+    assert h2.presence == {}
+
+
+def _all_absent(name, s4, s6=None):
+    h = _host(name)
+    _conn(h, "IPv4", s4)
+    h.set_presence("IPv4", False, time.time())
+    if s6 is not None:
+        _conn(h, "IPv6", s6)
+        h.set_presence("IPv6", False, time.time())
+    return h
+
+
+def test_all_absent_single_family_overdue_is_overdue():
+    assert _all_absent("abs-single", C.OVERDUE).status() == "overdue"
+
+
+def test_all_absent_dual_stack_one_absent_one_overdue_is_overdue():
+    assert _all_absent("abs-dual-od", C.ABSENT, C.OVERDUE).status() == "overdue"
+
+
+def test_all_absent_dual_stack_v6_up_is_up():
+    assert _all_absent("abs-dual-up", C.ABSENT, C.UP).status() == "up"
+
+
+def test_all_absent_all_unknown_is_unknown():
+    assert _all_absent("abs-unknown", C.UNKNOWN, C.UNKNOWN).status() == "unknown"

tests/test_http_config_api.py

@@ -1,5 +1,4 @@
 """Tests for the config read/write API helpers in http.py."""
-import pytest
 from hbd.server import http
 
 

tests/test_http_users_me.py

@@ -1,5 +1,4 @@
 """Tests for PUT /api/0/users/me logic."""
-import pytest
 from hbd.server import users as users_mod
 
 
@@ -87,7 +86,6 @@ def test_put_users_me_notification_channels(tmp_path):
 
 def test_visible_channels_excludes_private_from_others():
     """Private channels owned by another user must not appear in the visible set."""
-    from hbd.server import settings as settings_mod
 
     config = {
         "notification_channels": {
@@ -105,7 +103,7 @@ def test_visible_channels_excludes_private_from_others():
             self.admin = admin
 
     alice = FakeUser("alice")
-    bob   = FakeUser("bob")
+    bob = FakeUser("bob")
     admin = FakeUser("admin", admin=True)
 
     # Simulate _visible_channels_for_user logic (mirrors http.py implementation)
@@ -119,5 +117,5 @@ def test_visible_channels_excludes_private_from_others():
         }
 
     assert visible(alice) == {"public_ch", "alice_priv"}
-    assert visible(bob)   == {"public_ch", "bob_priv"}
+    assert visible(bob) == {"public_ch", "bob_priv"}
     assert visible(admin) == {"public_ch", "alice_priv", "bob_priv"}

tests/test_nagios_runner.py

@@ -1,15 +1,11 @@
 import asyncio
 import logging
-import os
 import stat
 
-from hbd.client.plugins.nagios_runner import (
-    NagiosRunnerPlugin,
-    NAGIOS_OK,
-    NAGIOS_WARNING,
-    NAGIOS_CRITICAL,
-    NAGIOS_UNKNOWN,
-)
+from hbd.client.plugins.nagios_runner import NagiosRunnerPlugin, NAGIOS_UNKNOWN
+
+# Nagios plugin exit codes (the module itself only needs UNKNOWN)
+NAGIOS_OK, NAGIOS_WARNING, NAGIOS_CRITICAL = 0, 1, 2
 
 
 def test_no_commands_sets_skip_reason():

tests/test_network_monitor_connectivity.py

@@ -0,0 +1,26 @@
+import asyncio
+
+from hbd.client import connectivity
+from hbd.client.plugins.network_monitor import NetworkMonitorPlugin
+
+
+class _FakeShared:
+    def as_metric(self):
+        return {"ipv4": {"present": True, "addr": "192.0.2.5", "gateway": "192.0.2.1", "rtt": 9.0},
+                "ipv6": {"present": False, "addr": None, "gateway": None, "rtt": None}}
+
+
+def test_collect_includes_connectivity_when_probe_running():
+    connectivity.set_shared(_FakeShared())
+    try:
+        data = asyncio.run(NetworkMonitorPlugin()._collect_metrics())
+    finally:
+        connectivity.set_shared(None)
+    assert data["connectivity"]["ipv4"]["present"] is True
+    assert data["connectivity"]["ipv6"]["present"] is False
+
+
+def test_collect_omits_connectivity_without_probe():
+    connectivity.set_shared(None)
+    data = asyncio.run(NetworkMonitorPlugin()._collect_metrics())
+    assert "connectivity" not in data

tests/test_notification_channels_api.py

@@ -1,5 +1,4 @@
 """Tests for notification channel CRUD via configio helpers and visibility logic."""
-import pytest
 from hbd.server import configio, settings as settings_mod
 
 

tests/test_oauth.py

@@ -37,7 +37,6 @@ def reset_users_dict():
     users_mod.users = original
 
 
-
 def test_make_state_returns_unique_tokens():
     s1 = oauth.make_state()
     s2 = oauth.make_state()
@@ -124,7 +123,6 @@ def test_provision_oauth_user_survives_config_reload():
     assert "oauthonly" in users_mod.users
 
 
-
 # ---------------------------------------------------------------------------
 # Integration-style tests: callback logic chain
 # ---------------------------------------------------------------------------

tests/test_pingutil.py

@@ -0,0 +1,60 @@
+import sys
+
+import pytest
+
+from hbd.client import pingutil
+
+LINUX_OK = """PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
+64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=11.2 ms
+
+--- 1.1.1.1 ping statistics ---
+1 packets transmitted, 1 received, 0% packet loss, time 0ms
+rtt min/avg/max/mdev = 11.234/11.234/11.234/0.000 ms
+"""
+
+MACOS_OK = """PING 1.1.1.1 (1.1.1.1): 56 data bytes
+64 bytes from 1.1.1.1: icmp_seq=0 ttl=57 time=12.3 ms
+
+--- 1.1.1.1 ping statistics ---
+1 packets transmitted, 1 packets received, 0.0% packet loss
+round-trip min/avg/max/stddev = 12.3/12.3/12.3/0.000 ms
+"""
+
+LOST = """--- 1.1.1.1 ping statistics ---
+1 packets transmitted, 0 received, 100% packet loss, time 0ms
+"""
+
+
+def test_parse_linux():
+    r = pingutil.parse_ping_output(LINUX_OK)
+    assert r == {"rtt_min": 11.234, "rtt_avg": 11.234, "rtt_max": 11.234, "loss": 0.0}
+
+
+def test_parse_macos():
+    r = pingutil.parse_ping_output(MACOS_OK)
+    assert r["rtt_avg"] == 12.3 and r["loss"] == 0.0
+
+
+def test_parse_lost():
+    r = pingutil.parse_ping_output(LOST)
+    assert r["rtt_avg"] == float("inf") and r["loss"] == 100.0
+
+
+def test_parse_garbage_is_unreachable():
+    r = pingutil.parse_ping_output("ping: connect: Network is unreachable")
+    assert r["rtt_avg"] == float("inf") and r["loss"] == 100.0
+
+
+@pytest.mark.parametrize("platform,family,expected", [
+    ("linux", None, ["ping", "-c", "3", "-W", "5", "h"]),
+    ("linux", 6, ["ping", "-c", "3", "-W", "5", "-6", "h"]),
+    ("linux", 4, ["ping", "-c", "3", "-W", "5", "-4", "h"]),
+    ("win32", None, ["ping", "-n", "3", "-w", "5000", "h"]),
+    ("win32", 6, ["ping", "-n", "3", "-w", "5000", "-6", "h"]),
+    ("darwin", None, ["ping", "-c", "3", "-W", "5", "h"]),
+    ("darwin", 4, ["ping", "-c", "3", "-W", "5000", "h"]),
+    ("darwin", 6, ["ping6", "-c", "3", "h"]),
+])
+def test_ping_command(monkeypatch, platform, family, expected):
+    monkeypatch.setattr(sys, "platform", platform)
+    assert pingutil.ping_command("h", 3, 5, family) == expected

tests/test_proto.py

@@ -1,19 +1,19 @@
-from hbd.proto import dicttos, stodict, oldmtodict
+from hbd.common.proto import dicttos, stodict, oldmtodict
 
 
-def test_dicttos_and_stodict_uncompressed():
+def test_dicttos_and_stodict_roundtrip():
     msg = dicttos("HTB", {"name": "host.example", "interval": 10})
+    assert msg.startswith(b"!HTB:")
     d = stodict(msg)
-    assert d["ID"].startswith("HTB")
+    assert d["ID"] == "HTB"
     assert d["name"] == "host.example"
     assert d["interval"] == 10
 
 
-def test_dicttos_and_stodict_compressed():
-    msg = dicttos("ACK", {"time": 12345}, compress=True)
-    d = stodict(msg)
-    # for compressed the original code included the colon in ID slice; ensure no crash
-    assert "ID" in d
+def test_dicttos_and_stodict_ack():
+    d = stodict(dicttos("ACK", {"time": 12345}))
+    assert d["ID"] == "ACK"
+    assert d["time"] == 12345
 
 
 def test_oldmtodict():

tests/test_settings_sections.py

@@ -1,4 +1,3 @@
-import pytest
 from hbd.server import settings as settings_mod
 
 CFG = {

tests/test_threshold.py

@@ -13,13 +13,12 @@ Tests cover:
 """
 
 import sys
-import time
 from pathlib import Path
 
 # Add parent directory to path for imports
 sys.path.insert(0, str(Path(__file__).parent.parent))
 
-from hbd.threshold import (
+from hbd.server.threshold import (  # noqa: E402
     ThresholdChecker,
     ThresholdConfig,
     AlertLevel,
@@ -31,37 +30,37 @@ from hbd.threshold import (
 def test_threshold_config_basic():
     """Test basic threshold configuration."""
     print("Test 1: Basic threshold configuration...")
-    
+
     config = ThresholdConfig(
         metric_path="cpu_monitor.cpu_percent",
         warning=80.0,
         critical=90.0,
         operator=">",
     )
-    
+
     # Test below warning
     result = config.evaluate(50.0)
     assert result == AlertLevel.OK, f"Expected OK, got {result}"
-    
+
     # Test at warning
     result = config.evaluate(80.0)
     assert result == AlertLevel.OK, f"Expected OK at boundary, got {result}"
-    
+
     # Test above warning but below critical
     result = config.evaluate(85.0)
     assert result == AlertLevel.WARNING, f"Expected WARNING, got {result}"
-    
+
     # Test above critical
     result = config.evaluate(95.0)
     assert result == AlertLevel.CRITICAL, f"Expected CRITICAL, got {result}"
-    
+
     print("  ✓ Basic threshold configuration works")
 
 
 def test_threshold_operators():
     """Test all comparison operators."""
     print("\nTest 2: Comparison operators...")
-    
+
     # Greater than operator
     config_gt = ThresholdConfig(
         metric_path="test.metric",
@@ -71,7 +70,7 @@ def test_threshold_operators():
     )
     assert config_gt.evaluate(85.0) == AlertLevel.WARNING
     assert config_gt.evaluate(75.0) == AlertLevel.OK
-    
+
     # Less than operator (for inverse thresholds like available memory)
     config_lt = ThresholdConfig(
         metric_path="memory.available_mb",
@@ -82,7 +81,7 @@ def test_threshold_operators():
     assert config_lt.evaluate(800) == AlertLevel.WARNING, "Should warn when below 1000"
     assert config_lt.evaluate(400) == AlertLevel.CRITICAL, "Should be critical when below 500"
     assert config_lt.evaluate(1500) == AlertLevel.OK, "Should be OK when above 1000"
-    
+
     # Greater than or equal
     config_gte = ThresholdConfig(
         metric_path="test.metric",
@@ -91,7 +90,7 @@ def test_threshold_operators():
     )
     assert config_gte.evaluate(80.0) == AlertLevel.WARNING
     assert config_gte.evaluate(79.9) == AlertLevel.OK
-    
+
     # Less than or equal
     config_lte = ThresholdConfig(
         metric_path="test.metric",
@@ -100,14 +99,14 @@ def test_threshold_operators():
     )
     assert config_lte.evaluate(20.0) == AlertLevel.WARNING
     assert config_lte.evaluate(20.1) == AlertLevel.OK
-    
+
     print("  ✓ All comparison operators work correctly")
 
 
 def test_hysteresis():
     """Test hysteresis to prevent flapping."""
     print("\nTest 3: Hysteresis...")
-    
+
     config = ThresholdConfig(
         metric_path="cpu_monitor.cpu_percent",
         warning=80.0,
@@ -115,87 +114,87 @@ def test_hysteresis():
         operator=">",
         hysteresis=0.1,  # 10% hysteresis
     )
-    
+
     # Start at OK, go to WARNING
     result = config.evaluate_with_hysteresis(85.0, AlertLevel.OK)
     assert result == AlertLevel.WARNING, "Should enter WARNING state"
-    
+
     # Try to recover with insufficient improvement (within hysteresis)
     # Warning threshold is 80, hysteresis is 10%, so need to go below 80 - 8 = 72
     result = config.evaluate_with_hysteresis(77.0, AlertLevel.WARNING)
     assert result == AlertLevel.WARNING, "Should stay in WARNING (hysteresis)"
-    
+
     # Recover with sufficient improvement
     result = config.evaluate_with_hysteresis(70.0, AlertLevel.WARNING)
     assert result == AlertLevel.OK, "Should recover to OK"
-    
+
     # Test critical hysteresis
     result = config.evaluate_with_hysteresis(95.0, AlertLevel.WARNING)
     assert result == AlertLevel.CRITICAL, "Should escalate to CRITICAL"
-    
+
     # Try to recover from critical with insufficient improvement
     # Critical threshold is 90, hysteresis is 10%, so need to go below 90 - 9 = 81
     result = config.evaluate_with_hysteresis(85.0, AlertLevel.CRITICAL)
     assert result == AlertLevel.CRITICAL, "Should stay CRITICAL (hysteresis, still above 81)"
-    
+
     # Sufficient improvement to drop from CRITICAL (below 81)
     result = config.evaluate_with_hysteresis(75.0, AlertLevel.CRITICAL)
     assert result == AlertLevel.OK, "Should drop to OK (below warning threshold)"
-    
+
     # Now test dropping from CRITICAL to WARNING
     result = config.evaluate_with_hysteresis(95.0, AlertLevel.OK)
     assert result == AlertLevel.CRITICAL, "Should go to CRITICAL"
-    
+
     # Drop to between warning and critical, but still in hysteresis zone
     result = config.evaluate_with_hysteresis(82.0, AlertLevel.CRITICAL)
     assert result == AlertLevel.CRITICAL, "Should stay CRITICAL (in hysteresis)"
-    
+
     # Drop below critical hysteresis but still above warning threshold
     # At 80.1, we're above WARNING (80) so should evaluate to WARNING
     result = config.evaluate_with_hysteresis(80.5, AlertLevel.CRITICAL)
     assert result == AlertLevel.WARNING, "Should drop to WARNING when below critical hysteresis"
-    
+
     print("  ✓ Hysteresis prevents flapping")
 
 
 def test_alert_state():
     """Test alert state tracking."""
     print("\nTest 4: Alert state tracking...")
-    
+
     alert = AlertState("cpu_monitor.cpu_percent")
-    
+
     # Initial state
     assert alert.level == AlertLevel.OK
     assert alert.notification_count == 0
-    
+
     # Update to WARNING - should trigger notification
     changed = alert.update(AlertLevel.WARNING, 85.0)
-    assert changed == True, "State change should return True"
+    assert changed is True, "State change should return True"
     assert alert.level == AlertLevel.WARNING
     assert alert.last_value == 85.0
-    
+
     # Update with same level - no notification
     changed = alert.update(AlertLevel.WARNING, 86.0)
-    assert changed == False, "No state change should return False"
+    assert changed is False, "No state change should return False"
     assert alert.last_value == 86.0
-    
+
     # Escalate to CRITICAL
     changed = alert.update(AlertLevel.CRITICAL, 95.0)
-    assert changed == True, "Escalation should trigger notification"
+    assert changed is True, "Escalation should trigger notification"
     assert alert.level == AlertLevel.CRITICAL
-    
+
     # Recover to OK
     changed = alert.update(AlertLevel.OK, 50.0)
-    assert changed == True, "Recovery should trigger notification"
+    assert changed is True, "Recovery should trigger notification"
     assert alert.level == AlertLevel.OK
-    
+
     print("  ✓ Alert state tracking works correctly")
 
 
 def test_threshold_checker_parsing():
     """Test parsing threshold configuration from YAML structure."""
     print("\nTest 5: Configuration parsing...")
-    
+
     config = {
         "thresholds": {
             "cpu_monitor": {
@@ -239,9 +238,9 @@ def test_threshold_checker_parsing():
             },
         }
     }
-    
+
     checker = ThresholdChecker(config)
-    
+
     # Verify thresholds were parsed
     assert "cpu_monitor.cpu_percent" in checker.thresholds
     assert "cpu_monitor.load_1min" in checker.thresholds
@@ -249,18 +248,27 @@ def test_threshold_checker_parsing():
     assert "memory_monitor.available_mb" in checker.thresholds
     assert "disk_monitor./.percent" in checker.thresholds
     assert "disk_monitor./home.percent" in checker.thresholds
-    
+
     # Verify operators were parsed correctly
     assert checker.thresholds["cpu_monitor.cpu_percent"].operator == ComparisonOperator.GT
     assert checker.thresholds["memory_monitor.available_mb"].operator == ComparisonOperator.LT
-    
+
     print(f"  ✓ Parsed {len(checker.thresholds)} thresholds correctly")
 
 
-def test_check_plugin_data():
+def _checker_with_notifications(config, notifications, monkeypatch):
+    """ThresholdChecker with no grace period that records notifications instead of sending them."""
+    monkeypatch.setattr(
+        ThresholdChecker, "_send_notification",
+        lambda self, host, lvl, message, *a: notifications.append(f"[{lvl}] {host} {message}"),
+    )
+    return ThresholdChecker({"grace": 0, **config})
+
+
+def test_check_plugin_data(monkeypatch):
     """Test checking plugin data against thresholds."""
     print("\nTest 6: Plugin data checking...")
-    
+
     config = {
         "thresholds": {
             "cpu_monitor": {
@@ -275,57 +283,53 @@ def test_check_plugin_data():
             },
         }
     }
-    
+
     notifications = []
-    
-    def notification_callback(msg):
-        notifications.append(msg)
-    
-    checker = ThresholdChecker(config, notification_callback=notification_callback)
+    checker = _checker_with_notifications(config, notifications, monkeypatch)
     alert_states = {}
-    
+
     # First check - OK
     plugin_data = {
         "cpu_percent": 50.0,
         "load_1min": 2.0,
     }
-    
+
     state_changes = checker.check_plugin_data(
         host_name="testhost",
         plugin_name="cpu_monitor",
         data=plugin_data,
         alert_states=alert_states,
     )
-    
+
     assert len(state_changes) == 0, "No thresholds violated, no state changes"
     assert len(notifications) == 0, "No notifications should be sent"
-    
+
     # Second check - WARNING
     plugin_data = {
         "cpu_percent": 85.0,
         "load_1min": 2.0,
     }
-    
+
     state_changes = checker.check_plugin_data(
         host_name="testhost",
         plugin_name="cpu_monitor",
         data=plugin_data,
         alert_states=alert_states,
     )
-    
+
     assert len(state_changes) == 1, "One metric should change state"
     assert state_changes[0][0] == "cpu_monitor.cpu_percent"
     assert state_changes[0][2] == AlertLevel.WARNING
     assert len(notifications) == 1, "One notification should be sent"
     assert "WARNING" in notifications[0]
     assert "testhost" in notifications[0]
-    
+
     # Third check - CRITICAL
     plugin_data = {
         "cpu_percent": 95.0,
         "load_1min": 9.0,
     }
-    
+
     notifications.clear()
     state_changes = checker.check_plugin_data(
         host_name="testhost",
@@ -333,16 +337,16 @@ def test_check_plugin_data():
         data=plugin_data,
         alert_states=alert_states,
     )
-    
+
     assert len(state_changes) == 2, "Two metrics should change state"
     assert len(notifications) == 2, "Two notifications should be sent"
-    
+
     # Fourth check - Recovery
     plugin_data = {
         "cpu_percent": 50.0,
         "load_1min": 1.0,
     }
-    
+
     notifications.clear()
     state_changes = checker.check_plugin_data(
         host_name="testhost",
@@ -350,18 +354,18 @@ def test_check_plugin_data():
         data=plugin_data,
         alert_states=alert_states,
     )
-    
+
     assert len(state_changes) == 2, "Two metrics should recover"
     assert len(notifications) == 2, "Two recovery notifications"
-    assert any("RECOVERED" in n for n in notifications), "Should have recovery notification"
-    
+    assert any("RECOVER" in n for n in notifications), "Should have recovery notification"
+
     print("  ✓ Plugin data checking and notifications work")
 
 
-def test_nested_metrics():
+def test_nested_metrics(monkeypatch):
     """Test checking nested metrics like disk partitions."""
     print("\nTest 7: Nested metrics (partitions)...")
-    
+
     config = {
         "thresholds": {
             "disk_monitor": {
@@ -382,11 +386,11 @@ def test_nested_metrics():
             },
         }
     }
-    
+
     notifications = []
-    checker = ThresholdChecker(config, notification_callback=lambda m: notifications.append(m))
+    checker = _checker_with_notifications(config, notifications, monkeypatch)
     alert_states = {}
-    
+
     plugin_data = {
         "partitions": {
             "/": {
@@ -399,26 +403,26 @@ def test_nested_metrics():
             },
         },
     }
-    
+
     state_changes = checker.check_plugin_data(
         host_name="testhost",
         plugin_name="disk_monitor",
         data=plugin_data,
         alert_states=alert_states,
     )
-    
+
     assert len(state_changes) == 1, "One partition should trigger alert"
     assert "/home" in state_changes[0][0], "Should be /home partition"
     assert state_changes[0][2] == AlertLevel.WARNING
     assert len(notifications) == 1
-    
+
     print("  ✓ Nested metric checking works")
 
 
 def test_alert_summary():
     """Test getting alert summaries."""
     print("\nTest 8: Alert summaries...")
-    
+
     config = {
         "thresholds": {
             "cpu_monitor": {
@@ -430,31 +434,31 @@ def test_alert_summary():
             },
         }
     }
-    
+
     checker = ThresholdChecker(config)
     alert_states = {}
-    
+
     # Create some alert states
     plugin_data = {
         "cpu_percent": 85.0,  # WARNING
         "load_1min": 9.0,     # CRITICAL
     }
     checker.check_plugin_data("testhost", "cpu_monitor", plugin_data, alert_states)
-    
+
     plugin_data = {
         "percent": 96.0,  # CRITICAL
     }
     checker.check_plugin_data("testhost", "memory_monitor", plugin_data, alert_states)
-    
+
     # Get summary
     summary = checker.get_alert_summary(alert_states)
     assert summary["warning"] == 1, "Should have 1 warning"
     assert summary["critical"] == 2, "Should have 2 critical"
-    
+
     # Get active alerts
     active = checker.get_active_alerts(alert_states)
     assert len(active) == 3, "Should have 3 active alerts"
-    
+
     print("  ✓ Alert summaries work correctly")
 
 
@@ -463,7 +467,7 @@ def run_all_tests():
     print("=" * 70)
     print("THRESHOLD SYSTEM TEST SUITE")
     print("=" * 70)
-    
+
     try:
         test_threshold_config_basic()
         test_threshold_operators()
@@ -473,12 +477,12 @@ def run_all_tests():
         test_check_plugin_data()
         test_nested_metrics()
         test_alert_summary()
-        
+
         print("\n" + "=" * 70)
         print("✓ ALL TESTS PASSED")
         print("=" * 70)
         return 0
-    
+
     except AssertionError as e:
         print(f"\n✗ TEST FAILED: {e}")
         import traceback

tests/test_udp.py

@@ -1,5 +1,5 @@
-from hbd.udp import parse_message
-from hbd.proto import dicttos
+from hbd.server.udp import parse_message
+from hbd.common.proto import dicttos
 
 
 def test_parse_message_uncompressed():

tests/test_udp_presence.py

@@ -0,0 +1,250 @@
+"""Host-level overdue alerting driven by client-reported protocol presence."""
+import asyncio
+import time
+
+from hbd.common.proto import dicttos
+from hbd.server import hbdclass, udp
+from hbd.server.hbdclass import Connection as C
+from hbd.server.udp import handle_datagram, parse_message
+
+
+class _FakeTransport:
+    def sendto(self, data, addr):
+        pass
+
+
+def _ctx():
+    return {"config": {}, "hbdclass": hbdclass, "msg_to_websockets": None,
+            "DEBUG": 0, "verbose": False}
+
+
+def _fresh(name):
+    hbdclass.Host.hosts.pop(name, None)
+
+
+def _send(name, addr, ctx, **fields):
+    msg = parse_message(dicttos("HTB", {"name": name, "id": 0, **fields}))
+    handle_datagram(msg, (addr, 50000), _FakeTransport(), ctx)
+
+
+def _events(monkeypatch):
+    events = []
+    monkeypatch.setattr(udp, "eventlog", lambda u, level, msg, **kw: events.append((level, msg)))
+    return events
+
+
+def _dual_stack(name, ctx):
+    _fresh(name)
+    _send(name, "127.0.0.1", ctx)
+    _send(name, "::1", ctx)
+    return hbdclass.Host.hosts[name]
+
+
+def test_flags_are_stored_on_host():
+    ctx = _ctx()
+    _fresh("pres-store")
+    _send("pres-store", "127.0.0.1", ctx, p4=1, p6=0)
+    h = hbdclass.Host.hosts["pres-store"]
+    assert h.presence["IPv4"]["present"] is True
+    assert h.presence["IPv6"]["present"] is False
+
+
+def test_no_flags_leaves_presence_unknown():
+    ctx = _ctx()
+    _fresh("pres-legacy")
+    _send("pres-legacy", "127.0.0.1", ctx)
+    assert hbdclass.Host.hosts["pres-legacy"].presence == {}
+
+
+def test_absent_flag_moves_overdue_family_to_absent_and_clears_alert(monkeypatch):
+    _events(monkeypatch)
+    ctx = _ctx()
+    h = _dual_stack("pres-absent", ctx)
+    v6 = h.connections["IPv6"]
+    v6.state = C.OVERDUE
+    udp._set_connectivity_alert(h, "IPv6", "CRITICAL")
+    _send("pres-absent", "127.0.0.1", ctx, p4=1, p6=0)
+    assert v6.state == C.ABSENT
+    assert "connectivity.IPv6" not in h.alert_states
+    assert h.status() == "up"
+
+
+def test_absent_flag_does_not_touch_family_that_is_up():
+    ctx = _ctx()
+    h = _dual_stack("pres-up-stays", ctx)
+    _send("pres-up-stays", "127.0.0.1", ctx, p4=1, p6=0)
+    assert h.connections["IPv6"].state == C.UP          # traffic overrides the flag
+
+
+def test_present_again_arms_grace_timer():
+    ctx = _ctx()
+
+    async def scenario():
+        _fresh("pres-back")
+        _send("pres-back", "127.0.0.1", ctx, interval=10)
+        _send("pres-back", "::1", ctx, interval=10)
+        h = hbdclass.Host.hosts["pres-back"]
+        v6 = h.connections["IPv6"]
+        v6.cancel_overdue_timer()
+        v6.state = C.ABSENT
+        _send("pres-back", "127.0.0.1", ctx, interval=10, p6=1)
+        assert v6.state == C.UNKNOWN and v6.overdue_timer is not None
+        for c in h.connections.values():
+            c.cancel_overdue_timer()
+
+    asyncio.run(scenario())
+
+
+def test_one_family_overdue_is_simplex_all_overdue_is_critical(monkeypatch):
+    events = _events(monkeypatch)
+    ctx = _ctx()
+    h = _dual_stack("pres-simplex", ctx)
+    on_overdue, _ = udp._make_timer_callbacks("pres-simplex", h, ctx)
+
+    asyncio.run(on_overdue(h.connections["IPv6"]))
+    assert h.status() == "simplex"
+    assert ("INFO", "IPv6 overdue (simplex)") in events
+    assert not any(level == "CRITICAL" for level, _ in events)
+    assert "connectivity.IPv6" not in h.alert_states
+
+    asyncio.run(on_overdue(h.connections["IPv4"]))
+    assert h.status() == "overdue"
+    assert [e for e in events if e[0] == "CRITICAL"] == [("CRITICAL", "IPv4, IPv6 overdue")]
+    assert {"connectivity.IPv4", "connectivity.IPv6"} <= set(h.alert_states)
+
+
+def test_timer_expiry_on_absent_family_is_silent(monkeypatch):
+    events = _events(monkeypatch)
+    ctx = _ctx()
+    h = _dual_stack("pres-quiet", ctx)
+    _send("pres-quiet", "127.0.0.1", ctx, p4=1, p6=0)   # v6 still UP => left alone
+    events.clear()
+    on_overdue, _ = udp._make_timer_callbacks("pres-quiet", h, ctx)
+    asyncio.run(on_overdue(h.connections["IPv6"]))
+    assert h.connections["IPv6"].state == C.ABSENT
+    assert events == []
+    assert h.status() == "up"
+
+
+def test_recovery_from_simplex_is_info_only(monkeypatch):
+    events = _events(monkeypatch)
+    ctx = _ctx()
+    h = _dual_stack("pres-rec-simplex", ctx)
+    v6 = h.connections["IPv6"]
+    v6.state, v6.statetime = C.OVERDUE, time.time() - 100
+    events.clear()
+    _send("pres-rec-simplex", "::1", ctx)
+    assert v6.state == C.UP
+    assert [e[0] for e in events] == ["INFO"] and "back after" in events[0][1]
+
+
+def test_recovery_from_overdue_is_recover_and_clears_all_alerts(monkeypatch):
+    events = _events(monkeypatch)
+    ctx = _ctx()
+    h = _dual_stack("pres-rec-overdue", ctx)
+    for afam in ("IPv4", "IPv6"):
+        c = h.connections[afam]
+        c.state, c.statetime = C.OVERDUE, time.time() - 100
+        udp._set_connectivity_alert(h, afam, "CRITICAL")
+    events.clear()
+    _send("pres-rec-overdue", "127.0.0.1", ctx)
+    assert [e[0] for e in events] == ["RECOVER"]
+    assert h.status() == "simplex"
+    assert "connectivity.IPv6" not in h.alert_states     # left overdue => all cleared
+    assert "connectivity.IPv4" not in h.alert_states
+
+
+def test_restore_skips_absent_and_sets_no_alert_for_simplex():
+    ctx = _ctx()
+
+    async def scenario():
+        h = _dual_stack("pres-restore", ctx)
+        h.connections["IPv6"].state = C.OVERDUE          # v4 UP + v6 OVERDUE => simplex
+        udp.restore_connection_timers(hbdclass, ctx)
+        assert "connectivity.IPv6" not in h.alert_states
+        h.connections["IPv6"].cancel_overdue_timer()
+        h.connections["IPv6"].state = C.ABSENT
+        h.set_presence("IPv6", False, time.time())
+        udp.restore_connection_timers(hbdclass, ctx)
+        assert h.connections["IPv6"].state == C.ABSENT
+        for c in h.connections.values():
+            c.cancel_overdue_timer()
+
+    asyncio.run(scenario())
+
+
+def _past_drop(conn):
+    conn.state, conn.statetime = C.OVERDUE, time.time() - (udp.DROPOVERDUE + 10)
+
+
+def test_restore_past_drop_window_keeps_alert_single_family():
+    ctx = _ctx()
+
+    async def scenario():
+        _fresh("pres-drop-single")
+        _send("pres-drop-single", "127.0.0.1", ctx)
+        h = hbdclass.Host.hosts["pres-drop-single"]
+        _past_drop(h.connections["IPv4"])
+        udp.restore_connection_timers(hbdclass, ctx)
+        assert h.connections["IPv4"].state == C.UNKNOWN
+        assert "connectivity.IPv4" in h.alert_states
+
+    asyncio.run(scenario())
+
+
+def test_restore_past_drop_window_keeps_alerts_dual_family():
+    ctx = _ctx()
+
+    async def scenario():
+        h = _dual_stack("pres-drop-dual", ctx)
+        for c in h.connections.values():
+            _past_drop(c)
+        udp.restore_connection_timers(hbdclass, ctx)
+        assert {"connectivity.IPv4", "connectivity.IPv6"} <= set(h.alert_states)
+
+    asyncio.run(scenario())
+
+
+def test_v4_only_host_overdue_is_single_critical(monkeypatch):
+    events = _events(monkeypatch)
+    ctx = _ctx()
+    _fresh("pres-v4only")
+    _send("pres-v4only", "127.0.0.1", ctx)
+    h = hbdclass.Host.hosts["pres-v4only"]
+    on_overdue, _ = udp._make_timer_callbacks("pres-v4only", h, ctx)
+    asyncio.run(on_overdue(h.connections["IPv4"]))
+    assert [e for e in events if e[0] == "CRITICAL"] == [("CRITICAL", "IPv4 overdue")]
+    assert "connectivity.IPv4" in h.alert_states
+    asyncio.run(on_overdue(h.connections["IPv4"]))
+    assert len([e for e in events if e[0] == "CRITICAL"]) == 1
+
+
+def test_all_families_absent_v4_only_host_still_alerts_when_it_dies(monkeypatch):
+    events = _events(monkeypatch)
+    ctx = _ctx()
+    _fresh("pres-allabsent")
+    _send("pres-allabsent", "127.0.0.1", ctx, p4=0)
+    h = hbdclass.Host.hosts["pres-allabsent"]
+    on_overdue, _ = udp._make_timer_callbacks("pres-allabsent", h, ctx)
+    asyncio.run(on_overdue(h.connections["IPv4"]))
+    assert [e for e in events if e[0] == "CRITICAL"] == [("CRITICAL", "IPv4 overdue")]
+    assert h.connections["IPv4"].state == C.OVERDUE
+    assert h.status() == "overdue"
+    assert "connectivity.IPv4" in h.alert_states
+
+
+def test_last_present_family_is_not_parked_when_other_is_absent(monkeypatch):
+    events = _events(monkeypatch)
+    ctx = _ctx()
+    h = _dual_stack("pres-lastfam", ctx)
+    _send("pres-lastfam", "127.0.0.1", ctx, p4=0, p6=0)
+    events.clear()
+    on_overdue, _ = udp._make_timer_callbacks("pres-lastfam", h, ctx)
+    asyncio.run(on_overdue(h.connections["IPv4"]))
+    assert h.connections["IPv4"].state == C.ABSENT
+    assert not any(level == "CRITICAL" for level, _ in events)
+    assert h.status() == "up"
+    asyncio.run(on_overdue(h.connections["IPv6"]))
+    assert [e for e in events if e[0] == "CRITICAL"] == [("CRITICAL", "IPv6 overdue")]
+    assert h.connections["IPv6"].state == C.OVERDUE
+    assert h.status() == "overdue"

tests/test_udp_stray_datagrams.py

@@ -0,0 +1,40 @@
+"""Datagrams that are not heartbeat traffic (internet scanners) must not create hosts."""
+from hbd.common.proto import dicttos
+from hbd.server import hbdclass
+from hbd.server.udp import handle_datagram, parse_message
+
+
+class _FakeTransport:
+    sent = 0
+
+    def sendto(self, data, addr):
+        _FakeTransport.sent += 1
+
+
+def _ctx():
+    return {"config": {}, "hbdclass": hbdclass, "msg_to_websockets": None,
+            "DEBUG": 0, "verbose": False}
+
+
+def _handle(raw):
+    handle_datagram(parse_message(raw), ("192.0.2.9", 50000), _FakeTransport(), _ctx())
+
+
+def test_sip_invite_does_not_create_a_host():
+    hbdclass.Host.hosts.pop("unknown", None)
+    _handle(b"INVITE sip:10000@192.0.2.1:50003 SIP/2.0\r\nTo: <sip:10000@192.0.2.1>\r\n\r\n")
+    assert "unknown" not in hbdclass.Host.hosts
+
+
+def test_nameless_heartbeat_does_not_create_a_host_or_get_acked():
+    hbdclass.Host.hosts.pop("unknown", None)
+    _FakeTransport.sent = 0
+    _handle(dicttos("HTB", {"id": 0}))
+    assert "unknown" not in hbdclass.Host.hosts
+    assert _FakeTransport.sent == 0
+
+
+def test_named_heartbeat_still_creates_its_host():
+    hbdclass.Host.hosts.pop("stray-ok", None)
+    _handle(dicttos("HTB", {"name": "stray-ok", "id": 0}))
+    assert "stray-ok" in hbdclass.Host.hosts

tox.ini

@@ -21,5 +21,5 @@ commands =
     mypy hbd
 
 [flake8]
-max-line-length = 111
+max-line-length = 165
 extend-ignore = E203