Compare commits

...

9 Commits

Author SHA1 Message Date
Andreas Wrede e6436fc236 version 5.1.5
Release / release (push) Successful in 5s
2026-04-30 13:55:21 -04:00
Andreas Wrede c5ce41762e feat: update hbc via hb_install.sh instead of code patching
Server now sends a bare UPD command; client runs hb_install.sh to
reinstall from the package registry, then restarts. hb_install.sh
also copies itself alongside hbc on client installs.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-30 13:55:15 -04:00
Andreas Wrede 26ca0c095f install.sh --> hb_innstall.sh 2026-04-30 09:54:48 -04:00
Andreas Wrede 1eecd67594 update docu 2026-04-30 09:19:11 -04:00
Andreas Wrede caf3c2c0ac don't error exit on pip insttalled test 2026-04-30 09:16:22 -04:00
Andreas Wrede 9af4006097 version 5.1.4
Release / release (push) Successful in 6s
2026-04-30 08:12:15 -04:00
Andreas Wrede ddf7067d13 feat: redesign Plugin Metrics page as Host Overview
Replace pill-tab plugin view with an accordion layout that shows key
metrics (CPU%, MEM%, top disk%, net delta, nagios status) at a glance
in each host card header. Plugin sections expand as structured tables.

- Rename page to "Host Overview" (URL /plugins unchanged)
- Three-wave parallel data loading: glance plugins on host expand,
  on-demand fetch for filesystem_info and extras
- Per-plugin table renderers with inline percent bars and threshold
  colour coding
- Add escHtml() for XSS-safe rendering of all field values
- Remove stale planning docs (REFACTORING.md, hbd/Plan.md)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-30 08:12:07 -04:00
andreas 505353a8a8 Update CLAUDE. md 2026-04-29 21:20:28 -04:00
andreas 0402d33c71 Add CLAUDE. md 2026-04-29 21:18:21 -04:00
11 changed files with 978 additions and 1201 deletions
+4
View File
@@ -0,0 +1,4 @@
1. Don't assume. Don't hide confusion. Surface tradeoffs.
2. Minimum code that solves the problem. Nothing speculative.
3. Touch only what you must. Clean up only your own mess.
4. Define success criteria. Loop until verified.
+1 -1
View File
@@ -377,7 +377,7 @@ This project now declares its dependencies in `pyproject.toml`. Instead
of the old `requirements.txt` flow, install the package into a virtualenv of the old `requirements.txt` flow, install the package into a virtualenv
using `pip`: using `pip`:
See `scripts/install.sh` for a way to install. See `scripts/hb_install.sh` for a way to install.
Run the daemon (example): Run the daemon (example):
-234
View File
@@ -1,234 +0,0 @@
# HBD/HBC Separation Refactoring
## Overview
The heartbeat monitoring system has been refactored into a modular package structure with separate client and server components. This allows users to install only what they need and provides clear separation of concerns.
## New Package Structure
```
hbd/
├── __init__.py # Main package (minimal)
├── client/ # HBC - System monitoring client
│ ├── __init__.py
│ ├── main.py # Entry point (was hbc.py)
│ ├── config.py # Client-specific configuration
│ ├── plugin.py # Plugin framework
│ ├── threshold.py # Threshold checking
│ └── plugins/ # Monitoring plugins
│ ├── cpu_monitor.py
│ ├── disk_monitor.py
│ ├── memory_monitor.py
│ ├── network_monitor.py
│ ├── filesystem_info.py
│ ├── os_info.py
│ └── nagios_runner.py
├── server/ # HBD - Heartbeat daemon/server
│ ├── __init__.py
│ ├── main.py # Server runtime (was server.py)
│ ├── cli.py # Command-line interface
│ ├── config.py # Server-specific configuration
│ ├── http.py # HTTP/REST API
│ ├── ws.py # WebSocket server
│ ├── udp.py # UDP heartbeat listener
│ ├── dns.py # DNS update functionality
│ ├── notify.py # Notification handlers
│ ├── monitor.py # Host monitoring
│ ├── hbdclass.py # Host class definitions
│ ├── journal.py # Message journaling
│ ├── templates/ # Jinja2 web templates
│ └── static/ # Web UI assets
└── common/ # Shared utilities
├── __init__.py
├── proto.py # Protocol encoding/decoding
└── utils.py # Common utilities
## Configuration Files
### Client Configuration (hbd/client/config.py)
Client-specific defaults:
- `hb_port`: Port where hbd servers listen (default: 50003)
- `interval`: Heartbeat interval in seconds (default: 10)
- `plugins`: Per-plugin configuration
- `thresholds`: Threshold configuration for monitoring
### Server Configuration (hbd/server/config.py)
Server-specific defaults:
- `hb_port`: Port to listen for heartbeats (default: 50003)
- `hbd_port`: HTTP API port (default: 50004)
- `ws_port`: WebSocket port (default: 50005)
- `logfile`: Log file path
- `pushsrv`, `pushover_token`, etc.: Notification settings
- `watchhosts`, `dyndnshosts`: Host monitoring
- `smtpserver`, etc.: Email settings
- `journal_*`: Message journaling settings
## Installation Options
### Install Core Only (minimal, PyYAML only)
```bash
pip install hbd
```
### Install Client Only (for monitoring)
```bash
pip install hbd[client]
# Installs: PyYAML, psutil
```
### Install Server Only (for daemon)
```bash
pip install hbd[server]
# Installs: PyYAML, websockets, mattermostdriver, aiohttp, Jinja2
```
### Install Everything
```bash
pip install hbd[all]
# Installs all dependencies for both client and server
```
### Development Installation
```bash
pip install -e ".[dev]"
# Includes all dependencies plus testing/linting tools
```
## Command-Line Interfaces
### HBC (Client)
```bash
hbc [options] host1 [host2 ...]
# Entry point: hbd.client.main:main
# Location: hbd/client/main.py
```
### HBD (Server)
```bash
hbd [options]
# Entry point: hbd.server.cli:main
# Location: hbd/server/cli.py → hbd/server/main.py
```
## Import Changes
### Client Code
```python
# Old imports
from .config import load_config
from .proto import dicttos, stodict
from .plugin import PluginRegistry
# New imports
from .config import load_config # Still in client/
from ..common.proto import dicttos # Moved to common/
from .plugin import PluginRegistry # Still in client/
```
### Server Code
```python
# Old imports
from .config import load_config
from .proto import stodict
from .threshold import AlertLevel
# New imports
from .config import load_config # Server-specific config
from ..common.proto import stodict # Moved to common/
from ..client.threshold import AlertLevel # Client module
```
### Plugin Code
```python
# Old import
from hbd.plugin import MonitorPlugin
# New import
from hbd.client.plugin import MonitorPlugin
```
## Benefits
1. **Modular Installation**: Install only what you need
- Client-only systems don't need web server dependencies
- Server-only systems don't need psutil
2. **Clearer Architecture**: Explicit separation of concerns
- Client: System monitoring and data collection
- Server: Heartbeat reception, web UI, notifications
- Common: Shared protocol and utilities
3. **Independent Evolution**: Client and server can evolve separately
- Different release cycles possible
- Clear API boundaries via common/
4. **Smaller Footprint**: Reduced dependency installation
- Client: ~1 dependency (psutil)
- Server: ~4 dependencies (websockets, aiohttp, Jinja2, mattermostdriver)
## Migration Guide
### For Existing Installations
1. **Reinstall the package**:
```bash
pip install -e ".[all]" # For development
# or
pip install hbd[all] # For production
```
2. **Configuration files remain unchanged**:
- Both client and server read from `~/.hb.yaml`
- All existing config keys are supported in both configs
- Server has additional keys (journal, websocket, email, etc.)
- Client has minimal keys (interval, plugins, thresholds)
3. **Commands remain the same**:
- `hbc` command works identically
- `hbd` command works identically
### For New Deployments
1. **Client-only system** (monitoring host):
```bash
pip install hbd[client]
hbc server1.example.com server2.example.com
```
2. **Server-only system** (monitoring daemon):
```bash
pip install hbd[server]
hbd -c /etc/hbd.yaml -f
```
3. **Combined system** (dev/test):
```bash
pip install hbd[all]
```
## Testing
All imports and entry points have been tested and validated:
- ✅ Package imports work correctly
- ✅ `hbc` command entry point functional
- ✅ `hbd` command entry point functional
- ✅ Optional dependencies properly configured
- ✅ All internal imports updated
## Files Archived
The following files were renamed to avoid conflicts:
- `hbd/config.py` → `hbd/config.py.old` (split into client/server configs)
- `hbd/hbc_old.py` → `hbd/hbc_old.py.bak` (backup file)
## Next Steps
1. Test client functionality with a monitoring host
2. Test server functionality with web UI and notifications
3. Update documentation (README.md) with new structure
4. Consider publishing to PyPI with new structure
5. Update any deployment scripts/Dockerfiles to use optional dependencies
-21
View File
@@ -1,21 +0,0 @@
Plan the following changes, ask questions to clarify before implementing
Re-factor the notification system:
- use available libraries for pushover, matrix, email and sms notifications.
- notifications have a title/subject: alert_type (recover/warning/critical), a body (info from threshold check) and a link to the host plugin metrix page
- define a list of notification channels for each user
- notifications are dispatched to users that are listed as managers for the host
1 - correct
2 - for now channels are defined globaly
3 - matrix-nio)sounds good, homeserver URL, access token, room ID per channel?
4 - use the REST api provided by https://voip.ms/api/v1/rest.php
5 - The page does not exist yet, point at the host tab in the /plugins
6 - per-channel minimum severity is a good idea, go fo it
7 - yes
1 - use base_url, there might not have been any incoming requests yet
2 - use same asyncio loop for matrix-nio
3 - for now, just silently do nothing
+1 -1
View File
@@ -14,4 +14,4 @@ Install options:
""" """
__all__ = ["__version__"] __all__ = ["__version__"]
__version__ = "5.1.3" __version__ = "5.1.5"
+27 -31
View File
@@ -14,7 +14,6 @@ import signal
import socket import socket
import sys import sys
import time import time
from hashlib import md5
from logging.handlers import SysLogHandler from logging.handlers import SysLogHandler
from pathlib import Path from pathlib import Path
from typing import Dict, List, Optional from typing import Dict, List, Optional
@@ -204,48 +203,45 @@ async def handle_command(conn: AsyncConnection, msg: dict):
await conn.sendto(response) await conn.sendto(response)
async def handle_update(conn: AsyncConnection, msg: dict): async def handle_update(conn: AsyncConnection, _msg: dict): # pyright: ignore[reportUnusedParameter]
"""Handle self-update from server.""" """Handle self-update by running hb_install.sh."""
import codecs
import shutil import shutil
logger = logging.getLogger("hbc.update") logger = logging.getLogger("hbc.update")
try: installer = shutil.which("hb_install.sh")
code = codecs.decode(msg["code"], "base64").decode() if installer is None:
csum = msg["csum"] candidate = Path(sys.argv[0]).parent / "hb_install.sh"
except Exception as e: if candidate.exists():
error = f"Missing code/csum: {e}" installer = str(candidate)
if installer is None:
error = "hb_install.sh not found in PATH or alongside hbc"
logger.error(error) logger.error(error)
await conn.sendto({"service": "update", "msg": error}) await conn.sendto({"service": "update", "msg": error})
return return
# Verify checksum logger.info(f"Running installer: {installer}")
m = md5() try:
m.update(code.encode()) proc = await asyncio.create_subprocess_exec(
if m.hexdigest() != csum: installer, "client",
error = "Checksum mismatch" stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.STDOUT,
)
out, _ = await asyncio.wait_for(proc.communicate(), timeout=120)
except asyncio.TimeoutError:
error = "Installer timed out"
logger.error(error)
await conn.sendto({"service": "update", "msg": error})
return
except Exception as e:
error = f"Installer failed: {e}"
logger.error(error) logger.error(error)
await conn.sendto({"service": "update", "msg": error}) await conn.sendto({"service": "update", "msg": error})
return return
# Backup current file if proc.returncode != 0:
fn = sys.argv[0] error = f"Installer exited {proc.returncode}: {out.decode().strip()}"
ofn = f"{fn}.sav"
try:
shutil.copy2(fn, ofn)
except Exception as e:
error = f"Backup failed: {e}"
logger.error(error)
await conn.sendto({"service": "update", "msg": error})
return
# Write new code
try:
with open(fn, "w") as fh:
fh.write(code)
except Exception as e:
error = f"Write failed: {e}"
logger.error(error) logger.error(error)
await conn.sendto({"service": "update", "msg": error}) await conn.sendto({"service": "update", "msg": error})
return return
+6 -11
View File
@@ -210,15 +210,11 @@ async def start(
return err return err
qa = request.rel_url.query qa = request.rel_url.query
uname = urllib.parse.unquote(qa.get("h", "")) uname = urllib.parse.unquote(qa.get("h", ""))
ucode = qa.get("c") if not uname:
if not ucode or not uname: return web.Response(status=400, text="need h= argument")
return web.Response(status=400, text="need h= and c= arguments")
if uname != "All" and uname not in hbdclass.Host.hosts: if uname != "All" and uname not in hbdclass.Host.hosts:
return web.Response(status=400, text=f"h={uname} not found") return web.Response(status=400, text=f"h={uname} not found")
if uname != "All": names = [uname] if uname != "All" else list(hbdclass.Host.hosts)
names = [uname]
else:
names = [n for n in hbdclass.Host.hosts]
out = [] out = []
for n in names: for n in names:
host = hbdclass.Host.hosts[n] host = hbdclass.Host.hosts[n]
@@ -227,8 +223,7 @@ async def start(
continue continue
op_err = None op_err = None
try: try:
r = {"csum": None, "code": ucode} host.cmds.append(("UPD", {}))
host.cmds.append(("UPD", r))
except Exception as e: except Exception as e:
op_err = str(e) op_err = str(e)
out.append(f"update started for {n}: {op_err if op_err else 'OK'}") out.append(f"update started for {n}: {op_err if op_err else 'OK'}")
@@ -520,8 +515,8 @@ async def start(
tmpl = env.get_template("plugins.html") tmpl = env.get_template("plugins.html")
body = tmpl.render( body = tmpl.render(
title="Plugin Metrics - Heartbeat", title="Host Overview - Heartbeat",
header="Plugin Metrics", header="Host Overview",
hosts=hosts_with_plugins, hosts=hosts_with_plugins,
current_user=current_user.to_dict() if current_user else None, current_user=current_user.to_dict() if current_user else None,
active_page="plugins", active_page="plugins",
+1 -1
View File
@@ -4,7 +4,7 @@
</button> </button>
<div class="nav-links" id="nav-links"> <div class="nav-links" id="nav-links">
<a href="/live"{% if active_page == "live" %} class="active"{% endif %}>Live Dashboard</a> <a href="/live"{% if active_page == "live" %} class="active"{% endif %}>Live Dashboard</a>
<a href="/plugins"{% if active_page == "plugins" %} class="active"{% endif %}>Plugin Metrics</a> <a href="/plugins"{% if active_page == "plugins" %} class="active"{% endif %}>Host Overview</a>
<a href="/alerts"{% if active_page == "alerts" %} class="active"{% endif %}>Alerts</a> <a href="/alerts"{% if active_page == "alerts" %} class="active"{% endif %}>Alerts</a>
{% if current_user and current_user.admin %} {% if current_user and current_user.admin %}
<a href="/settings"{% if active_page == "settings" %} class="active"{% endif %}>Settings</a> <a href="/settings"{% if active_page == "settings" %} class="active"{% endif %}>Settings</a>
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "hbd" name = "hbd"
version = "5.1.3" version = "5.1.5"
description = "Heartbeat monitoring system — client (hbc) and server (hbd)" description = "Heartbeat monitoring system — client (hbc) and server (hbd)"
readme = "README.md" readme = "README.md"
requires-python = ">=3.11" requires-python = ">=3.11"
+16 -7
View File
@@ -1,6 +1,6 @@
#!/bin/sh #!/bin/sh
# install the heartbeat client, hbc. The server is installed when the arg 'server' is passed # Helper script to install the heartbeat tools. By default, it will only
# install the heartbeat client, hbc. The server is installed when the arg 'server' is passed # install the heartbeat client, hbc. The server is installed when the arg 'server' is passed
# to the script. The script will install the heartbeat tools in a python # to the script. The script will install the heartbeat tools in a python
# virtual environment in ~/venvs/hbd. The hbd and hbc commands will be # virtual environment in ~/venvs/hbd. The hbd and hbc commands will be
@@ -9,17 +9,20 @@
# reused. The script will also remove any existing symlinks for hbd and hbc # reused. The script will also remove any existing symlinks for hbd and hbc
# in ~/bin before creating new ones. # in ~/bin before creating new ones.
# hbd/hbc from wheel and create symlinks for hbd and hbc in ~/bin
set -e set -e
what=$1 what=$1
on_ha=0 on_ha=0
[ -z "$what" ] && what="client" [ -z "$what" ] && what="client"
if [ -d /homeassistant ]; then if [ -d /homeassistant ]; then
echo "cannot install in HA, run \"docker exec -it homeassistant $0 $@\"" echo "cannot install in HA, running \"docker exec homeassistant $0 $@\""
exit 1 docker exec homeassistant $0 $@
rc=$?
if [ $rc -ne 0 ]; then
echo "Failed to install heartbeat in HA, please check the logs for more details"
exit 1
fi
exit 0
fi fi
if [ -d /config ]; then if [ -d /config ]; then
echo "Installing on HA" echo "Installing on HA"
@@ -46,8 +49,11 @@ fi
echo "Installing heartbeat $what" echo "Installing heartbeat $what"
if [ ! -d $venv/hbd ]; then if [ ! -d $venv/hbd ]; then
set +e
python3 -m pip --version > /dev/null 2>&1 python3 -m pip --version > /dev/null 2>&1
if [ $? -ne 0 ]; then rc=$?
set -e
if [ $rc -ne 0 ]; then
# truenas does not have pip installed by default, so we need to fetch get-pip.py and install pip # truenas does not have pip installed by default, so we need to fetch get-pip.py and install pip
echo "pip is not installed, fetching get-pip.py and installing pip" echo "pip is not installed, fetching get-pip.py and installing pip"
arg="--without-pip" arg="--without-pip"
@@ -78,6 +84,9 @@ if [ "$what" = "server" ]; then
else else
rm -f $where/hbc rm -f $where/hbc
ln -sf $(which hbc) $where/hbc ln -sf $(which hbc) $where/hbc
rm -f $where/hb_install.sh
cp "$0" $where/hb_install.sh
chmod +x $where/hb_install.sh
if [ $on_ha -eq 1 ]; then if [ $on_ha -eq 1 ]; then
echo "restarting hbc " echo "restarting hbc "
job=$(grep run_hbc configuration.yaml | sed 's/run_hbc://') job=$(grep run_hbc configuration.yaml | sed 's/run_hbc://')