Compare commits

...
67 Commits
Author SHA1 Message Date
andreas f838920f7c version 5.4.3
Release / release (push) Successful in 1m16s
2026-08-17 13:54:02 -04:00
andreasandClaude Sonnet 5 45d11a66b5 fix: fix RTT y-axis label clipping, add x-axis tickmarks, enlarge charts
Y-axis labels on the RTT chart were truncated above 99ms because the
left margin was a fixed guess; it's now sized to the widest tick
label. The x-axis only showed start/end timestamps; added evenly
spaced intermediate tickmarks and labels. Plot area enlarged 15%.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-17 13:53:57 -04:00
andreas a42d783c0c version 5.4.2
Release / release (push) Successful in 1m10s
2026-08-17 09:24:23 -04:00
andreasandClaude Sonnet 5 18e33656c0 fix: preserve chart history across reconnects, show gaps for missing data
Only wipe real (non-RTT) plugin data on an actual client reboot (boot
flag), not on every ordinary OVERDUE/DOWN -> UP recovery. A transient
network blip no longer erases CPU/memory/etc. history.

Also split the shared time-series chart into separate line/area segments
wherever the gap between samples is much larger than the typical spacing,
so missing data (host overdue, or history that simply hasn't accumulated
across a drop) renders as a visual gap instead of an interpolated line.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEimzMv4Q5EjFg3hoiZ69T
2026-08-17 09:13:49 -04:00
andreas 4ac5a34a43 version 5.4.1
Release / release (push) Successful in 1m19s
2026-08-17 08:16:25 -04:00
andreasandClaude Sonnet 5 6d39c5be47 fix: correct empty-state copy on Host Overview page
Since RTT history keys make host.plugin_data non-empty after the very
first heartbeat, the "no hosts" empty state can now only trigger when
zero hosts have ever connected, not when hosts exist but haven't sent
plugin metrics yet (that in-between state no longer exists). Update
the copy to describe the actual current trigger condition.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEimzMv4Q5EjFg3hoiZ69T
2026-08-17 08:07:23 -04:00
andreasandClaude Sonnet 5 cdb2183989 fix: stop synthetic rtt_* keys from masking real plugin-data checks
host.plugin_data now always holds rtt_ipv4/rtt_ipv6 history after the
first heartbeat, which broke two pre-existing behaviors that assumed
plugin_data reflects only real client-collected data:

- The request_update ACK gate in handle_datagram() never fired again
  after a host's first heartbeat, so stale OS/agent-version info was
  never refreshed after a reconnect (only a client restart fixed it).
- plugin_data.clear() on every UP transition wiped rtt_* history too,
  so a flaky host could never accumulate a useful RTT graph.

Add _is_rtt_key()/_has_real_plugin_data() helpers so both spots treat
rtt_* keys as synthetic: the gate now looks only at real plugin data,
and the recovery clear only drops real plugin keys, preserving RTT
history across reconnects.

Also fixes two pre-existing E127 continuation-indent flake8 issues in
tests/test_udp_rtt_history.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEimzMv4Q5EjFg3hoiZ69T
2026-08-17 08:07:19 -04:00
andreasandClaude Sonnet 5 41c78e8e34 feat: live-refresh Connectivity section and RTT charts every 30s
Add fetchHostInfo() call to the 30-second auto-refresh setInterval loop
for expanded host cards. This ensures the Connectivity table and RTT
charts update in real-time without requiring the user to collapse and
re-expand the host card.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEimzMv4Q5EjFg3hoiZ69T
2026-08-17 07:44:14 -04:00
andreas acf1eb8804 feat: render RTT history chart per connection in Connectivity section 2026-08-17 07:39:17 -04:00
andreasandClaude Sonnet 5 49b631f02d refactor: extract shared time-series chart renderer from CPU chart
Pulls the CPU chart's SVG-drawing logic into a parameterized
renderTimeSeriesChart(elId, pts, opts) helper (yDomain, colorFor,
clipId, unitSuffix), keeping renderCpuChart's signature and behavior
unchanged. Task 5 will reuse this for the RTT chart.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEimzMv4Q5EjFg3hoiZ69T
2026-08-17 07:33:22 -04:00
andreas 8204737ea6 fix: use the latest RTT sample, not the oldest, in host info API 2026-08-17 07:28:36 -04:00
andreas 5b2fc8fd46 fix: hide synthetic rtt history keys from plugin accordion list 2026-08-17 07:25:17 -04:00
andreasandClaude Sonnet 5 70c3e93684 fix: remove extra blank line to pass flake8 E303 check
Removed extraneous blank line after conndata() try/except block
that was left during code relocation. Fixes E303 linting error.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEimzMv4Q5EjFg3hoiZ69T
2026-08-17 07:22:33 -04:00
andreasandClaude Sonnet 5 52968201fa feat: record RTT history per heartbeat for charting
Capture RTT on every heartbeat using the existing plugin_data history
mechanism, recording under rtt_ipv4 or rtt_ipv6 keys. Maintains 100-sample
retention per host and address family.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEimzMv4Q5EjFg3hoiZ69T
2026-08-17 07:18:01 -04:00
andreasandClaude Sonnet 5 a7b698e6b9 fix: clear flap state when a host is dropped
flap._state persists at module level keyed by (host, service) and only
clears via a RECOVER-triggered quiet window. A host dropped mid-flap with
no RECOVER ever received leaves ok_since permanently None, so the
flapping flag could never clear on its own.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-07 08:35:10 -04:00
andreas 136b10239f update 2026-08-07 07:30:41 -04:00
Andreas WredeandClaude Sonnet 5 1e46f33648 fix: correct plugin import path in docs, wire up orphaned footer template
docs/PLUGIN_DEVELOPMENT.md referenced a nonexistent hbd/plugin.py and
hbd/plugins/ directory; the real module is hbd/client/plugin.py with
plugins under hbd/client/plugins/, as README.md and the actual code
already use. Anyone following the doc would hit ModuleNotFoundError.

foot.html was never included by any template (confirmed via git grep
and template audit) despite being actively maintained; wired it into
about.html. menu.html was fully commented out and unreferenced, so
removed it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 09:51:36 -04:00
andreas 4c3a63e645 update claude guideline 2026-07-31 09:22:57 -04:00
Andreas Wrede 1f4dd7cb1f version 5.4.0
Release / release (push) Successful in 44s
2026-07-23 13:17:33 -07:00
Andreas WredeandClaude Opus 4.8 ae3f2fc70f feat: gate remote command execution behind allow_remote_command
CMD packets arrive as unauthenticated UDP datagrams, yet every hbc client
executed the shell command they carry without any opt-in. Add an
allow_remote_command config key, default false: when off, the command is
logged and refused with "Refused: allow_remote_command is false" (visible in
the server event log under the command service), and subprocess is never
reached. When on, the client warns at startup that it will execute CMD
packets.

Applied to all four clients that handle CMD — hbc, hbc_windows.py,
hbc_mini.py, and the C hbc_mini — since gating only one leaves the others
wide open. The C client reads the same key from ~/.hbc.json and needs a
rebuild to pick it up. UPD (self-update) is unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:10:41 -07:00
Andreas WredeandClaude Opus 4.8 4414967bdc feat: flapping detection suppresses notification storms
A (host, service) pair is flapping once it exceeds flap_count warning or
critical notifications within flap_interval minutes. The notification that
trips the state carries "Now flapping!! No more messages!" and every later
one is dropped, including RECOVER. The state ends silently flap_interval
minutes after a RECOVER, provided no further alert arrived meanwhile.

Hooked into notify.send_notification, the single choke point for channel
delivery, so only outbound notifications are suppressed — eventlog keeps
recording, leaving the journal and /log with the full history of the flap.

Threshold alerts key on their metric path, so a flapping disk check cannot
silence a CPU alert; connectivity, boot and shutdown events key on the host
itself. State lives at module level in flap.py and is never pickled.

Flapping pairs surface in Host.stateinfo() and render as an amber badge on
the live dashboard. Config: flap_count (5), flap_interval (10 minutes),
0 in either disables; both editable on the settings page.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 09:53:46 -07:00
Andreas Wrede e3b0e5041f add: info on deplyment 2026-07-23 09:51:32 -07:00
andreasandClaude Fable 5 92982c0ca7 fix: harden events log — non-dict ring entries, off-loop journal reads, no default-config singleton
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 15:51:34 -04:00
andreasandClaude Fable 5 00fdbcaa5b fix: skip journal scheduling when the event loop is not running
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 15:39:42 -04:00
andreasandClaude Fable 5 600e68b509 feat: remove log section from live dashboard (moved to /log)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 15:29:39 -04:00
andreasandClaude Fable 5 b26e853a1c fix: dedup log rows between API seed and websocket tail
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 15:25:01 -04:00
andreasandClaude Fable 5 36adad5b3b feat: dedicated /log page with journal-backed history and nav item
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 15:20:17 -04:00
andreasandClaude Fable 5 7591268a89 feat: GET /api/0/log — paged, filtered events journal API
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 15:15:46 -04:00
andreasandClaude Fable 5 7afc06a329 fix: journal startup/shutdown lifecycle events (fired outside journal lifetime)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 15:11:30 -04:00
andreasandClaude Fable 5 a3f303e6ba feat: eventlog writes to dedicated events journal; init/backfill/close wiring
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 15:03:48 -04:00
andreasandClaude Fable 5 16c2922bea feat: events journal read path with filters and backward pagination
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 14:57:32 -04:00
andreasandClaude Fable 5 0343eb4a24 fix: drop typing names not yet used from journal.py import
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 14:53:24 -04:00
andreasandClaude Fable 5 fb5570155b feat: events journal write primitives (log_event, backfill, get_events_journal)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 14:47:42 -04:00
andreasandClaude Fable 5 8d268adf70 feat: Live Dashboard redesign — 6 columns, hover details, last-alert column
Replaces the 11-column table with severity-sorted record rows: host,
combined status chip, latency, alert chips, time-in-state, and the most
recent alert message. Per-family address/state/latency/last-change move
to a hover card on the status chip and permanently to a new Connectivity
table in the Host Overview info section (host info API now includes
connections). Last-alert is seeded from /api/0/alerts and kept fresh
from the event stream; the event log below uses the same row idiom with
its filters intact. WS reconnect state shows in the toolbar instead of
a modal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-10 13:45:03 -04:00
andreasandClaude Fable 5 5d9f161706 feat: extend the record-row design system to Host Overview, Alerts, and About
Extract the settings redesign's tokens and components into shared
static/hbd-ui.css + hbd-ui.js and dedupe settings.html against them.
About becomes yaml-key sections with kv rows; Alerts gets stat tiles,
chip filters, and alert record rows (same fetch/ack logic); Host
Overview keeps its DOM and live-update JS but is re-skinned to the
token system with a page toolbar. Live Dashboard intentionally
untouched pending its own redesign.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-10 09:37:58 -04:00
andreasandClaude Fable 5 067feacd59 fix: settings toolbar sticks below the site nav instead of scrolling away
Sticky offsets (toolbar, rail, mobile chips, anchor margins) derive from
the measured nav/toolbar heights via CSS variables, since the fixed nav's
height varies with viewport width.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-10 09:19:21 -04:00
andreasandClaude Fable 5 c564fafe73 fix: settings page not scrollable — restore html/body overflow override
static/style.css pins html/body to height:100% with overflow:hidden for
the dashboard pages; the redesign dropped the counter-override the old
template carried.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-10 08:20:21 -04:00
andreasandClaude Fable 5 8eeae48964 feat: unified record-row redesign of the settings page
One idiom for every section: mono identifier, badges, fact chips, inline
accordion editor. Scalar config sections merge into one grouped 'server'
section. Sidebar becomes YAML-key nav (mobile: horizontal chips); sticky
toolbar carries pending-changes state and Publish. Staging, publish,
channel CRUD, and permission gating are unchanged — same payload shapes
and endpoints.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-10 08:08:58 -04:00
andreas 880dc0e33f version 5.3.12
Release / release (push) Successful in 47s
2026-07-09 20:43:59 -04:00
andreasandClaude Fable 5 f06b2ef9e8 fix: users loop clobbered requesting username in settings filtering
The users-section loop reused 'username' as its loop variable, overwriting
the requesting user's name so host/threshold filtering compared against the
last user in the config. Rename to 'uname' and add a regression test with a
user that is not last in the users dict.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-09 18:48:05 -04:00
andreasandClaude Fable 5 447b9574a3 fix: type annotations for mypy parity with master
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-09 17:52:02 -04:00
andreasandClaude Fable 5 e98cca22f7 docs: ownership rules and settings access for owners/managers
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-09 17:50:24 -04:00
andreasandClaude Fable 5 5fe069a1a2 feat: retire per-channel private flag on profile page
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-09 16:53:55 -04:00
andreasandClaude Fable 5 ca2868a888 feat: settings UI renders per-user edit rights and owner controls
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-09 16:52:44 -04:00
andreasandClaude Fable 5 ddb1b2b875 feat: settings page accessible to all authenticated users
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-09 16:50:01 -04:00
andreasandClaude Fable 5 39c4d45bd7 feat: per-user filtering of settings sections
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-09 16:48:58 -04:00
andreasandClaude Fable 5 07fefab861 feat: scoped non-admin saves through POST /api/0/config
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-09 16:46:47 -04:00
andreasandClaude Fable 5 6b4524b30d feat: threshold form payload carries per-config owner
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-09 16:46:10 -04:00
andreasandClaude Fable 5 a149a1e285 feat: channel ownership rule — owner-presence means private, admin promote/demote
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-09 16:45:21 -04:00
andreasandClaude Fable 5 dd939ab86e feat: scoped threshold-config merge for non-admin saves
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-09 16:43:56 -04:00
andreasandClaude Fable 5 5c6f462081 feat: scoped host merge for non-admin config saves
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-09 16:43:08 -04:00
andreasandClaude Fable 5 98b50dbbf4 feat: ownership visibility helpers for config entities
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfPpSpccTWBfZg1FTveyaU
2026-07-09 16:42:00 -04:00
andreasandClaude Opus 4.8 960822918a fix: don't re-announce boot/shutdown on SIGHUP restart
Setting args.boot = False after sending the boot message was dead code: the
SIGHUP restart re-execs via os.execv(sys.argv[0], sys.argv) with the original
argv, so the re-launched process re-parsed -b and announced a boot again. The
exiting process also sent a spurious shutdown (send_shutdown armed by -b), so
each config reload looked like a host reboot to the server.

Strip -b/--boot from the argv passed to execv, and skip the shutdown message
in cleanup() when dorestart is set. A real SIGTERM/SIGINT shutdown still sends
shutdown as before.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 15:46:09 -04:00
andreas a376ebba8d version 5.3.11
Release / release (push) Successful in 48s
2026-06-26 11:39:23 -04:00
andreasandClaude Opus 4.8 f603ef9bd2 fix: declare plugin interval so stale data waits two full intervals
The server inferred each plugin's collection interval from the gap between
the last two received PLG samples, then expired data at interval * 3. After
an outage this guess was wrong: the request_update re-send of collect-once
InfoPlugins produced two close samples, yielding a tiny inferred interval
that purged permanent info data minutes after recovery.

Clients now declare each plugin's interval in the PLG message (_interval).
The server uses it directly (from the first post-recovery sample), expiring
at interval * 3 so live data survives at least two full intervals; interval
0 marks collect-once InfoPlugins that never go stale. Legacy clients omit
the field and fall back to the previous inferred-gap behavior.

Adds _interval to all three clients: hbc, hbc_mini.py, hbc_mini.c.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 11:37:48 -04:00
andreasandClaude Opus 4.8 731545dd0e fix: cap event buffer and replay only recent messages on dashboard connect
data.msgs grew without bound (eventlog appended forever, also persisted in
the pickle), and ws.py replayed the entire history to every dashboard
client on connect - one JSON frame per message - making page reloads
progressively sluggish. Cap the buffer in eventlog() (configurable via
msg_buffer_size, default 500) and slice the WebSocket replay to the last
30, matching the window the HTTP render already uses.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 11:37:40 -04:00
andreasandClaude Opus 4.8 356c77b0b8 fix: strip plugin timers when pickling Host to prevent save failure
Host.plugin_timers holds asyncio TimerHandle objects (and their lambda
callbacks) which are not picklable, causing the 5-minute state save to
fail with "Can't pickle local object reset_plugin_timer.<locals>.<lambda>".
Add Host.__getstate__ to reset plugin_timers to {} before pickling,
mirroring Connection.__getstate__; timers are recreated on the next PLG.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 11:37:35 -04:00
andreasandClaude Sonnet 4.6 6282077fe0 fix: correct zero-safe pathconf checks and connectivity prefix match
- Use `is not None` for pathconf values so 0 is not silently dropped
- Broaden connectivity prefix check to catch bare "connectivity" key

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 13:07:54 -04:00
Andreas WredeandClaude Sonnet 4.6 ddd857173b fix: address security vulnerabilities from audit
- Path traversal: confine avatar file serving to avatar_dir (defaults to
  config file directory); validate on both read and write
- UDP owner injection: server-configured owner now takes precedence over
  UDP-supplied value, matching the documented intent
- Open redirect: reject non-relative next= values after login
- Stored XSS: enable Jinja2 autoescape on all template environments;
  add escHtml() helper in live.html and apply to all innerHTML sinks
  sourced from network data (host names, addrs, states, log messages)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 13:06:05 -04:00
Andreas WredeandClaude Sonnet 4.6 f46f725d12 feat: add Windows hbc client with PyInstaller spec and NSSM install script
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 07:53:57 -04:00
Andreas WredeandClaude Sonnet 4.6 3da6976b53 fix: don't purge connectivity/rtt alerts in purge_stale_alerts
These entries are set by the connection state machine, not by threshold
config, so they have no threshold entry and were being deleted on every
startup. Guard them explicitly so overdue/down alerts survive the purge.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 14:45:47 -04:00
Andreas WredeandClaude Sonnet 4.6 3a0c48e32b fix: restore connectivity alerts for overdue/unknown/down hosts on startup
restore_connection_timers now calls _set_connectivity_alert("CRITICAL")
for DOWN, OVERDUE, and UNKNOWN connections, ensuring alerts are present
even if hbd was shut down before the transition callbacks recorded them.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 14:40:04 -04:00
Andreas WredeandClaude Sonnet 4.6 cf6e19704f fix: clear plugin data and timers on connection UP transition
Moves the plugin-state purge from the boot flag to the UP transition,
so stale history and alerts are cleared on any reconnect (reboot, or
recovery from overdue/unknown) not just detected reboots.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 14:35:58 -04:00
Andreas WredeandClaude Sonnet 4.6 b0addd7c67 feat: clear alerts for individual plugin metrics that disappear between samples
When a PLG message arrives with fewer keys than the previous sample,
alert states for the missing metrics are removed immediately. Handles
nagios checks removed from configuration while the runner plugin continues.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 11:32:38 -04:00
Andreas WredeandClaude Sonnet 4.6 32680d34a4 feat: show alerts for all hosts on Alerts page, not just watched
Notifications are still gated by host.watched; only the listing changes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 11:24:33 -04:00
Andreas WredeandClaude Sonnet 4.6 a7abdcb5c5 fix: restore host link from Dashboard to Host Overview
live.html used host.raw_name which stateinfo() never included — the
hash was always empty. Use host.name (the raw hostname stateinfo()
does include). Also exclude plugin_timers from stateinfo() to prevent
asyncio handles from breaking jsons().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 11:15:27 -04:00
Andreas WredeandClaude Sonnet 4.6 7bab15ae52 fix: don't set stale timer until two plugin samples establish real interval
Avoids false-stale firing for slow plugins (e.g. nagios_runner at 300 s)
when the heartbeat interval is much shorter. On the first sample cancel
any leftover timer; arm the 3× stale timer only after the second sample.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 09:00:09 -04:00
52 changed files with 6193 additions and 3233 deletions
+1
View File
@@ -16,3 +16,4 @@ uv.lock
.superpowers/
rndc-key
docs/superpowers/
graphify-out/
+100
View File
@@ -2,6 +2,106 @@
All notable changes to this project are documented here, organized by release.
## [5.4.3]
### Fixed
- fix RTT y-axis label clipping, add x-axis tickmarks, enlarge charts
---
## [5.4.2]
### Fixed
- preserve chart history across reconnects, show gaps for missing data
---
## [5.4.1]
### Added
- live-refresh Connectivity section and RTT charts every 30s
- render RTT history chart per connection in Connectivity section
- record RTT history per heartbeat for charting
### Fixed
- correct empty-state copy on Host Overview page
- stop synthetic rtt_* keys from masking real plugin-data checks
- use the latest RTT sample, not the oldest, in host info API
- hide synthetic rtt history keys from plugin accordion list
- remove extra blank line to pass flake8 E303 check
- clear flap state when a host is dropped
- correct plugin import path in docs, wire up orphaned footer template
---
## [5.4.0]
### Added
- gate remote command execution behind allow_remote_command
- flapping detection suppresses notification storms
- remove log section from live dashboard (moved to /log)
- dedicated /log page with journal-backed history and nav item
- GET /api/0/log — paged, filtered events journal API
- eventlog writes to dedicated events journal; init/backfill/close wiring
- events journal read path with filters and backward pagination
- events journal write primitives (log_event, backfill, get_events_journal)
- Live Dashboard redesign — 6 columns, hover details, last-alert column
- extend the record-row design system to Host Overview, Alerts, and About
- unified record-row redesign of the settings page
### Fixed
- harden events log — non-dict ring entries, off-loop journal reads, no default-config singleton
- skip journal scheduling when the event loop is not running
- dedup log rows between API seed and websocket tail
- journal startup/shutdown lifecycle events (fired outside journal lifetime)
- drop typing names not yet used from journal.py import
- settings toolbar sticks below the site nav instead of scrolling away
- settings page not scrollable — restore html/body overflow override
---
## [5.3.12]
### Added
- retire per-channel private flag on profile page
- settings UI renders per-user edit rights and owner controls
- settings page accessible to all authenticated users
- per-user filtering of settings sections
- scoped non-admin saves through POST /api/0/config
- threshold form payload carries per-config owner
- channel ownership rule — owner-presence means private, admin promote/demote
- scoped threshold-config merge for non-admin saves
- scoped host merge for non-admin config saves
- ownership visibility helpers for config entities
### Fixed
- users loop clobbered requesting username in settings filtering
- type annotations for mypy parity with master
- don't re-announce boot/shutdown on SIGHUP restart
---
## [5.3.11]
### Added
- add Windows hbc client with PyInstaller spec and NSSM install script
- clear alerts for individual plugin metrics that disappear between samples
- show alerts for all hosts on Alerts page, not just watched
### Fixed
- declare plugin interval so stale data waits two full intervals
- cap event buffer and replay only recent messages on dashboard connect
- strip plugin timers when pickling Host to prevent save failure
- correct zero-safe pathconf checks and connectivity prefix match
- address security vulnerabilities from audit
- don't purge connectivity/rtt alerts in purge_stale_alerts
- restore connectivity alerts for overdue/unknown/down hosts on startup
- clear plugin data and timers on connection UP transition
- restore host link from Dashboard to Host Overview
- don't set stale timer until two plugin samples establish real interval
---
## [5.3.10]
### Added
+94 -1
View File
@@ -1,4 +1,97 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Working principles
1. Don't assume. Don't hide confusion. Surface tradeoffs.
2. Minimum code that solves the problem. Nothing speculative.
3. Touch only what you must. Clean up only your own mess.
4. Define success criteria. Loop until verified.
4. Define success criteria. Loop until verified.
## Commands
```bash
# Run tests
pytest -q # all tests
pytest tests/test_threshold.py # single file
pytest -k test_name # single test
# Lint and type check
tox -e lint # flake8 over hbd/ and tests/
tox -e mypy # mypy over hbd/
# Install for development
pip install -e ".[all,dev]"
# Run server
python -m hbd.server.cli serve -c .hb.yaml -f -v
# Generate a password hash
hbd passwd <username>
```
Tests live in `tests/` (pytest, imported as a package). A second directory `test/` contains only TLS fixtures (`.pem` files), not test code.
Line length: 111 characters (`black`, `flake8`, both configured).
## Architecture
The system has two components: `hbc` (client) and `hbd` (server), both in the `hbd` Python package.
```
hbd/
common/ # proto.py (encode/decode), utils.py
client/ # hbc: plugins/, main.py, config.py, plugin.py
server/ # hbd: all server modules
```
### Server module map
| Module | Role |
|---|---|
| `cli.py` | Argument parsing, entry point, daemon startup |
| `main.py` | Asyncio runtime: starts UDP, HTTP, WebSocket; pickle save/load |
| `hbdclass.py` | `Host` and `Connection` domain objects; DNS queue |
| `udp.py` | UDP datagram listener; processes `HTB` and `PLG` messages; sets/resets overdue timers |
| `http.py` | aiohttp web server; all REST API routes (`/api/0/`) and page routes |
| `ws.py` | WebSocket broadcast to connected dashboard clients |
| `notify.py` | Notification dispatch (Pushover, email, Mattermost, Matrix, Signal, SMS); `eventlog()` is the single choke-point for all alert events |
| `threshold.py` | Threshold evaluation against plugin metrics; state transitions (OK/WARNING/CRITICAL/UNKNOWN) |
| `monitor.py` | Timer cleanup on shutdown (reachability is event-driven via timers in `udp.py`, not polled) |
| `data.py` | Shared in-memory message ring buffer |
| `journal.py` | JSONL message journal with size-based rotation |
| `users.py` | Session management, password hashing (PBKDF2), role checks |
| `config.py` | Config loading and defaults |
| `configio.py` | Config file read/write via `ruamel.yaml` (preserves comments) |
| `settings.py` | Settings sections for the web UI settings page |
| `dns.py` | `nsupdate` integration for dynamic DNS |
| `oauth.py` | OAuth2 login (Gitea) |
### Key data flows
**Heartbeat received:** `udp.py` decodes the `HTB` datagram → updates `Connection` state in `hbdclass.py` → resets overdue asyncio timer → broadcasts via `ws.py``notify.py` fires connectivity alerts on state change.
**Plugin data received:** `udp.py` decodes `PLG` datagram → stores on `Host``threshold.py` evaluates against configured thresholds → `notify.py` fires threshold alerts on state transitions.
**State persistence:** `Host.hosts` dict + `data.msgs` ring + active sessions are pickled every 5 minutes and on clean shutdown. Asyncio timers are stripped before pickling (`Connection.__getstate__`).
**Config reload:** SIGHUP → `configio.py` re-reads YAML → live-updates hosts, thresholds, users, notification channels. Port/cert/pickle/journal changes require a full restart.
### Client plugin system
Plugins in `hbd/client/plugins/` subclass `InfoPlugin` (collected once, on demand) or `MonitorPlugin` (periodic). `initialize()` returns `False` to self-disable. Data is sent as `PLG` UDP messages.
`hbc_mini.py` (scripts/) and `hbc_mini.c` (scripts/c/) are standalone single-file clients with no external dependencies.
### Protocol
All UDP messages: `!<ID>: <zlib-compressed key=value payload>`. Encoding in `hbd/common/proto.py`. Lists/dicts encoded as JSON with `@` prefix; booleans as `1`/`0`.
### Web UI
Jinja2 templates in `hbd/server/templates/`. Static assets in `hbd/server/static/`. Live pages (`/live`, `/plugins`) use WebSocket connections for real-time push.
### CI
Gitea Actions workflow at `.gitea/workflows/release.yml`.
+33 -2
View File
@@ -20,7 +20,7 @@ A lightweight UDP-based host monitoring system. Monitored hosts run a client (`h
└────────────────────┘ └────────────────────────────┘
```
**Package:** `hbd` v5.3.10
**Package:** `hbd` v5.4.3
**Python:** 3.11+
### Subpackages
@@ -116,6 +116,11 @@ dyndomains:
# Threshold alert re-notification interval (seconds)
threshold_renotify_interval: 3600
# Flap detection — silence a service/host after flap_count warning or critical
# notifications within flap_interval minutes (flap_count: 0 disables)
flap_count: 5
flap_interval: 10
# Notification channels
notification_channels:
pushover_ops:
@@ -207,6 +212,9 @@ hb_port: 50003 # Server UDP port
interval: 10 # Heartbeat interval (seconds)
owner: alice # Optional: claim ownership of this host
allow_remote_command: false # Execute shell commands sent by the server in CMD packets.
# Default false — see Remote command execution below.
plugins:
cpu_monitor:
interval: 300 # Override collection interval
@@ -237,6 +245,27 @@ plugins:
- If a connection fails to open at startup, IPv6 connections are dropped after 3 consecutive failures. IPv4 connections retry indefinitely.
- In daemon mode (`-d`), all log output goes to syslog (`LOG_DAEMON` facility).
### Remote command execution
The server can queue a shell command for a host, delivered in a `CMD` packet. Because
heartbeat packets are unauthenticated UDP, the client only runs those commands when the
host opts in:
```yaml
allow_remote_command: true
```
With the default `false`, the command is logged and refused, and the client replies
`Refused: allow_remote_command is false` — visible in the server's event log under the
`command` service, so a queued command never fails silently. When enabled, `hbc` logs a
warning at startup naming the risk. `SIGHUP` re-execs the client, so a config change takes
effect on reload.
All four clients enforce this: `hbc`, `hbc_windows.py`, `hbc_mini.py`, and the C
`hbc_mini` (which reads the same key from `~/.hbc.json` and needs a rebuild to pick the
change up). It does not gate `UPD` (self-update via `hb_install.sh`), which remains
ungated.
---
## UDP Protocol
@@ -254,7 +283,7 @@ Payload format: `key=value;key=value;...`
| `HTB` | client → server | Heartbeat (name, timestamp, RTT, acks, interval) |
| `PLG` | client → server | Plugin data (plugin name + metrics) |
| `ACK` | server → client | Acknowledgment |
| `CMD` | server → client | Execute a shell command on the client |
| `CMD` | server → client | Execute a shell command on the client (requires `allow_remote_command`) |
| `UPD` | server → client | Trigger self-update via `hb_install.sh` |
Value encoding:
@@ -400,6 +429,8 @@ hosts:
Notifications are sent on state transitions (OK → WARNING, WARNING → CRITICAL, CRITICAL → OK). De-escalations (CRITICAL → WARNING) do not trigger a notification. Ongoing alerts generate a re-notification every `threshold_renotify_interval` seconds (default: 3600). Alerts can be acknowledged via the web UI or API to suppress re-notifications.
A service or host that exceeds `flap_count` warning/critical notifications within `flap_interval` minutes is marked **flapping**: the tripping notification carries `Now flapping!! No more messages!` and further notifications are suppressed until it stays OK for `flap_interval` minutes. Flapping hosts are badged on the live dashboard; the event log keeps recording throughout. See [docs/NOTIFICATIONS.md](docs/NOTIFICATIONS.md#flap-detection).
### RTT thresholds
The server measures heartbeat round-trip time and supports RTT thresholds using the same format:
+15
View File
@@ -406,6 +406,21 @@ Potential improvements for future versions:
- Journal file encryption
- Signed journal entries
## Events journal
Alert/connectivity events shown on the `/log` page are written to a second,
dedicated journal (one JSON object per line, same rotation mechanism):
| Key | Default | Meaning |
|---|---|---|
| `events_journal_file` | `events.journal` | Filename inside `journal_dir` |
| `events_journal_max_size` | 10 MB | Rotation threshold |
| `events_journal_max_backups` | 10 | Rotated files kept |
`journal_dir` and `journal_enabled` are shared with the message journal.
On startup, if the events journal file is empty, it is seeded once from the
pickled in-memory message ring so history survives the upgrade.
## See Also
- [Configuration Guide](../hbd/config.py) - Full configuration options
+54 -13
View File
@@ -9,6 +9,7 @@ Notifications are dispatched to the **owner and managers** of a host, each via t
```
Alert event (udp.py / threshold.py)
└─ notify.send_notification(host_name, Notification)
├─ flap.observe(host, service, level) → pass | trip | suppress
├─ look up host.owner + host.managers
├─ for each user → user.notification_channels
└─ for each channel → _dispatch_to_channel (filtered by min_level)
@@ -19,6 +20,7 @@ Every notification carries:
- **body** — detail message (metric value, threshold, duration)
- **url** — link to the plugin metrics page (`{base_url}/plugins#{hostname}`)
- **level** — `RECOVER | WARNING | CRITICAL | INFO`
- **service** — flap-detection key within the host (empty = the host itself)
## Configuration
@@ -32,15 +34,16 @@ base_url: https://hbd.example.com
### Channel definitions
Channels are defined under `notification_channels`. Each entry specifies a delivery type and its credentials. Two optional metadata fields control visibility:
Channels are defined under `notification_channels`. Each entry specifies a delivery type and its credentials. Ownership is the single visibility signal:
| Field | Default | Description |
|---|---|---|
| `owner` | *(absent)* | Username who created/owns this channel. Absent = admin-created. |
| `private` | `false` | When `true`, only the owner can see and select this channel. |
| `owner` | *(absent)* | Owning username. Present = private to that user; absent = global. |
| `min_level` | `WARNING` | Minimum alert level this channel receives. |
**Admin-created channels** (set in the config file or via the admin settings UI) are public by default — all users can select them:
(The former `private` flag is retired; leftover `private` keys are ignored and dropped on the next edit.)
**Global channels** (no `owner`; set in the config file or by an admin) can be selected by all users but edited only by admins:
```yaml
notification_channels:
@@ -90,7 +93,7 @@ notification_channels:
username: heartbeat-bot
```
**User-created channels** are written by authenticated users through the API or their profile page. They carry an `owner` field and optionally `private: true`:
**User-created channels** are written by authenticated users through the API, their profile page, or the settings page. They carry an `owner` field and are private to that user:
```yaml
notification_channels:
@@ -99,17 +102,18 @@ notification_channels:
type: pushover
token: personal-token
user: personal-key
owner: alice # created by alice
private: true # only alice can see this channel
owner: alice # private to alice
```
### Channel visibility
| Channel | Who can see / select it |
|---|---|
| No `private` field (or `private: false`) | All users |
| `private: true` | Only the `owner` |
| Any channel | Admins always see everything |
| Channel | Who can see / select it | Who can edit it |
|---|---|---|
| No `owner` (global) | All users | Admins |
| `owner` set (private) | Only the `owner` | The owner |
| Any channel | Admins always see everything | Admins |
Admins can **promote** a private channel to global by clearing its owner on the settings page, or **demote** a global channel by assigning an owner.
### Users with notification channels
@@ -266,6 +270,43 @@ min_level: WARNING
Reminder notifications (re-notify) are sent only for CRITICAL level alerts.
## Flap detection
A check that toggles between OK and alerting produces a notification per swing. Flap
detection silences it after the first few.
A **`(host, service)`** pair is flapping once it exceeds `flap_count` WARNING/CRITICAL
notifications within `flap_interval` minutes. Threshold alerts key on their metric path,
so a flapping disk check does not silence an unrelated CPU alert; connectivity, boot and
shutdown events key on the host itself.
```yaml
flap_count: 5 # notifications within the window that trip flapping (0 disables)
flap_interval: 10 # minutes — both the counting window and the quiet window
```
Lifecycle:
| Event | Effect |
|---|---|
| Alerts 1..`flap_count` within the window | Delivered normally |
| Alert `flap_count + 1` | Delivered with ` Now flapping!! No more messages!` appended to the body |
| Every notification after that | Dropped — including RECOVER and INFO |
| RECOVER while flapping | Dropped, and starts the `flap_interval` quiet window |
| WARNING/CRITICAL during the quiet window | Restarts the quiet window; still flapping |
| Quiet window elapses | Flapping ends **silently** — no notification |
Only outbound notifications are suppressed. `notify.eventlog` keeps recording every event,
so the journal and the `/log` page retain the full history of the flap.
Flapping pairs appear in each host's `stateinfo()` under `flapping` (a list of service
keys; `""` means the host itself) and render as an amber **flapping** badge next to the
host name on the live dashboard, with the affected services in its tooltip.
State lives at module level in `hbd/server/flap.py` and is never pickled — a server restart
starts every check with a clean slate. Hosts with `watch: false` never reach the
notification path, so they never flap.
## API reference
### `send_notification(host_name, notif) -> dict`
@@ -299,7 +340,7 @@ Called once at startup from `main.py`. Pass the running asyncio event loop so Ma
- Check that the host has an `owner` or `managers` set
- Check that users have `notification_channels` listed
- Check that the channel names in user config match keys under `notification_channels:`
- If a user can't select a channel, check whether it is `private: true` and owned by someone else
- If a user can't select a channel, check whether it has an `owner` other than that user
**min_level filtering too aggressive:**
- Default is `WARNING` — both WARNING and CRITICAL are sent
+10 -10
View File
@@ -21,7 +21,7 @@ Heartbeat's plugin system is designed to be simple yet powerful. Plugins are Pyt
### Key Concepts
- **Plugin Registry**: Central registry that manages all loaded plugins
- **Plugin Loader**: Automatically discovers and loads plugins from the `hbd/plugins/` directory
- **Plugin Loader**: Automatically discovers and loads plugins from the `hbd/client/plugins/` directory
- **Plugin Types**: InfoPlugin (static data) and MonitorPlugin (periodic metrics)
- **Async/Await**: All plugin methods are async for non-blocking operation
@@ -64,7 +64,7 @@ Decide whether your plugin collects static information (InfoPlugin) or dynamic m
### Step 2: Create Plugin File
Create a new Python file in `hbd/plugins/` directory:
Create a new Python file in `hbd/client/plugins/` directory:
```python
"""
@@ -82,7 +82,7 @@ try:
except ImportError:
psutil = None
from hbd.plugin import MonitorPlugin # or InfoPlugin
from hbd.client.plugin import MonitorPlugin # or InfoPlugin
logger = logging.getLogger(__name__)
@@ -193,7 +193,7 @@ from pathlib import Path
# Add parent directory to path
sys.path.insert(0, str(Path(__file__).parent))
from hbd.plugins.my_awesome_plugin import MyAwesomePlugin
from hbd.client.plugins.my_awesome_plugin import MyAwesomePlugin
async def test():
# Create plugin instance
@@ -224,7 +224,7 @@ Understanding the plugin lifecycle helps you implement plugins correctly:
```
1. Plugin Discovery
└─> Loader scans hbd/plugins/ directory
└─> Loader scans hbd/client/plugins/ directory
└─> Finds Python files (except those starting with _)
└─> Imports modules
@@ -378,7 +378,7 @@ Document your plugin thoroughly:
### Example 1: Simple InfoPlugin
```python
from hbd.plugin import InfoPlugin
from hbd.client.plugin import InfoPlugin
import platform
class SimpleInfoPlugin(InfoPlugin):
@@ -406,7 +406,7 @@ plugin = SimpleInfoPlugin
### Example 2: MonitorPlugin with State
```python
from hbd.plugin import MonitorPlugin
from hbd.client.plugin import MonitorPlugin
import time
class CounterPlugin(MonitorPlugin):
@@ -442,7 +442,7 @@ plugin = CounterPlugin
### Example 3: Plugin with External Command
```python
from hbd.plugin import MonitorPlugin
from hbd.client.plugin import MonitorPlugin
import asyncio
class CommandPlugin(MonitorPlugin):
@@ -561,7 +561,7 @@ python -m hbd.hbc -c test_config.yaml --verbose
## Further Reading
- [Plugin Framework Source](../hbd/plugin.py) - Core plugin implementation
- [Built-in Plugins](../hbd/plugins/) - Examples of working plugins
- [Plugin Framework Source](../hbd/client/plugin.py) - Core plugin implementation
- [Built-in Plugins](../hbd/client/plugins/) - Examples of working plugins
- [Nagios Integration](NAGIOS_INTEGRATION.md) - Running external plugins
- [Configuration Guide](../hbd/config_example.yaml) - Full configuration reference
+10 -1
View File
@@ -19,6 +19,15 @@ Users are defined in the server config file. Each host can have an **owner**, ze
`admin` is a flag on the user, not a per-host role. An admin user has owner-level access on every host without being listed as owner/manager/monitor.
### Settings page access
All authenticated users may open `/settings`. Admins see every section; non-admins see only **Notification Channels**, **Hosts**, and **Threshold Configurations**, filtered to global items plus what they own or manage:
- **Owners** may add hosts, delete their hosts, edit host settings, and change access lists (managers/monitors, ownership transfer).
- **Managers** may edit host settings (watch, dyndns, channel and threshold assignments) but not access lists, and may not delete hosts.
- Anyone may create private notification channels and threshold configs (owned by them) and assign them — or global ones — to their hosts.
- Admins promote a private channel/threshold config to global by clearing its owner, or demote by assigning one.
---
## Configuration
@@ -200,7 +209,7 @@ Update the current user's profile. All fields are optional — send only what yo
```json
{ "notification_channels": ["pushover_ops", "email_ops"] }
```
Only channels visible to the user (public + own private) are accepted; others are silently dropped.
Only channels visible to the user (global + own) are accepted; others are silently dropped.
**Change password:**
```json
+1 -1
View File
@@ -14,4 +14,4 @@ Install options:
"""
__all__ = ["__version__"]
__version__ = "5.3.10"
__version__ = "5.4.3"
+3
View File
@@ -19,6 +19,9 @@ CLIENT_DEFAULTS = {
# Host identity
"owner": None, # Optional username to set as this host's owner on the server
# Security
"allow_remote_command": False, # Execute shell commands received in CMD packets from the server
# Runtime flags
"foreground": False,
"verbose": False,
+33 -13
View File
@@ -37,6 +37,10 @@ dorestart = False
shutdown_event: Optional[asyncio.Event] = None
active_tasks: List[asyncio.Task] = []
# Set from config in async_main. Off by default: a CMD packet is an unauthenticated
# UDP datagram, so executing one must be opted into per host.
allow_remote_command = False
class AsyncConnection:
"""Async UDP connection to a heartbeat server."""
@@ -183,16 +187,25 @@ class HeartbeatProtocol(asyncio.DatagramProtocol):
async def handle_command(conn: AsyncConnection, msg: dict):
"""Execute a command received from server."""
"""Execute a command received from server, if allow_remote_command is set."""
import subprocess
cmd = msg.get("cmd", "")
if not cmd:
return
logger = logging.getLogger("hbc.command")
if not allow_remote_command:
logger.warning(f"Refused command (allow_remote_command is false): {cmd}")
await conn.sendto({
"service": "command",
"msg": "Refused: allow_remote_command is false",
})
return
logger.info(f"Executing command: {cmd}")
try:
result = subprocess.check_output(
cmd, shell=True, stderr=subprocess.STDOUT, timeout=30
@@ -356,7 +369,8 @@ async def _info_plugin_refresh_loop(conn: AsyncConnection, info_plugins: List):
try:
data = await plugin.collect()
if data:
await conn.sendto({"plugin": plugin.name, **data}, "PLG")
await conn.sendto(
{"plugin": plugin.name, **data, "_interval": plugin.interval}, "PLG")
logger.info(f"Resent {plugin.name} data")
except Exception as e:
logger.error(f"Error re-collecting {plugin.name}: {e}", exc_info=True)
@@ -377,8 +391,8 @@ async def plugin_collector(conn: AsyncConnection, registry: PluginRegistry):
try:
data = await plugin.collect()
if data:
# Create PLG message with plugin name
plugin_msg = {"plugin": plugin.name, **data}
# Create PLG message with plugin name and declared interval
plugin_msg = {"plugin": plugin.name, **data, "_interval": plugin.interval}
await conn.sendto(plugin_msg, "PLG")
logger.info(f"Sent {plugin.name} data")
except Exception as e:
@@ -430,7 +444,7 @@ async def plugin_collector_interval(
data = await plugin.collect()
if data:
# Don't use encode_plugin_data - create dict directly
plugin_msg = {"plugin": plugin.name, **data}
plugin_msg = {"plugin": plugin.name, **data, "_interval": plugin.interval}
await conn.sendto(plugin_msg, "PLG")
logger.debug(f"Sent {plugin.name} data")
except asyncio.CancelledError:
@@ -485,7 +499,8 @@ async def cleanup(connections: List[AsyncConnection]):
logger.info("Cleaning up connections")
target = next((c for c in connections if c.transport), connections[0] if connections else None)
if target and send_shutdown:
# A SIGHUP restart is not a host shutdown, so don't announce one.
if target and send_shutdown and not dorestart:
try:
await target.sendto({"shutdown": 1, "acks": target.ackcount})
except Exception as e:
@@ -499,11 +514,12 @@ async def cleanup(connections: List[AsyncConnection]):
async def async_main(args, config):
"""Async main function."""
global running, shutdown_event, active_tasks, send_shutdown
global running, shutdown_event, active_tasks, send_shutdown, allow_remote_command
# Create shutdown event
shutdown_event = asyncio.Event()
active_tasks = []
allow_remote_command = bool(config.get("allow_remote_command", False))
logger = logging.getLogger("hbc.main")
@@ -517,6 +533,8 @@ async def async_main(args, config):
interval = config.get("interval", INTERVAL)
logger.info(f"hbc {__version__} on {iam} -> {hb_hosts} port={hb_port}, interval={interval}s")
if allow_remote_command:
logger.warning("allow_remote_command is true — CMD packets from the server will be executed")
af_filter = (socket.AF_INET if getattr(args, "ipv4_only", False)
else socket.AF_INET6 if getattr(args, "ipv6_only", False)
@@ -563,7 +581,6 @@ async def async_main(args, config):
boot_msg = {}
if args.boot:
boot_msg["boot"] = 1
args.boot = False # Clear boot flag so we don't send it again in main loop
send_shutdown = True
if args.message:
boot_msg["service"] = "service"
@@ -792,7 +809,10 @@ def main(argv=None):
# Handle restart
if dorestart:
logging.info("Restarting...")
os.execv(sys.argv[0], sys.argv)
# Drop -b/--boot so the re-exec'd process doesn't re-announce a boot;
# a SIGHUP restart is not a host reboot.
restart_argv = [sys.argv[0]] + [a for a in sys.argv[1:] if a not in ("-b", "--boot")]
os.execv(restart_argv[0], restart_argv)
sys.exit(exit_code)
+3 -3
View File
@@ -127,15 +127,15 @@ class FilesystemInfoPlugin(InfoPlugin):
try:
# Maximum filename length
max_name = os.pathconf(partition.mountpoint, 'PC_NAME_MAX')
if max_name:
if max_name is not None:
fs_info['maxfile'] = max_name
except (OSError, ValueError):
pass
try:
# Maximum path length
max_path = os.pathconf(partition.mountpoint, 'PC_PATH_MAX')
if max_path:
if max_path is not None:
fs_info['maxpath'] = max_path
except (OSError, ValueError):
pass
+4
View File
@@ -30,6 +30,10 @@ SERVER_DEFAULTS = {
"grace": 2, # Grace period (extra seconds before notifying after a missed heartbeat)
"threshold_renotify_interval": 3600, # Seconds between threshold re-notifications
# Flap detection (0 in either key disables it)
"flap_count": 5, # Warning/critical notifications within flap_interval that trip flapping
"flap_interval": 10, # Minutes: the counting window, and the quiet window after an OK
# User management
"users": {}, # username -> {full_name, avatar, password, admin, notification_channels}
"default_owner": None, # Username that owns hosts with no explicit owner
+168
View File
@@ -0,0 +1,168 @@
"""Ownership rules for config-file entities (hosts, channels, threshold configs).
Rule: an entry whose config dict has a non-empty ``owner`` is private to that
owner — visible, usable, and editable only by the owner (and admins). An
entry with no ``owner`` is global: usable by everyone, editable by admins only.
The scoped-merge helpers implement the non-admin save path for
``POST /api/0/config``: the caller's visible subset of a section is replaced
by the submitted payload; everything else is preserved untouched.
"""
from typing import Any, Dict
class ScopedMergeError(Exception):
"""A non-admin payload violated an ownership rule; message names the entry."""
def is_global(cfg: Any) -> bool:
"""True when *cfg* has no owner (usable by everyone)."""
return not (isinstance(cfg, dict) and cfg.get("owner"))
def user_can_use(cfg: Any, username: str) -> bool:
"""True when *username* may use/see this entry (global or own)."""
return is_global(cfg) or cfg.get("owner") == username
def _as_list(value: Any) -> list:
if value is None:
return []
if isinstance(value, str):
return [value]
return list(value)
def user_hosts(hosts_cfg: Any, username: str) -> Dict[str, Any]:
"""Subset of *hosts_cfg* where *username* is owner or manager."""
result = {}
for name, cfg in (hosts_cfg or {}).items():
if not isinstance(cfg, dict):
continue
if cfg.get("owner") == username or username in _as_list(cfg.get("managers")):
result[name] = cfg
return result
def user_channels(channels_cfg: Any, username: str) -> Dict[str, Any]:
"""Subset of channels usable by *username*: global + own."""
return {
name: cfg for name, cfg in (channels_cfg or {}).items()
if isinstance(cfg, dict) and user_can_use(cfg, username)
}
def user_threshold_configs(threshold_cfgs: Any, username: str) -> Dict[str, Any]:
"""Subset of threshold configs usable by *username*: global + own."""
return {
name: cfg for name, cfg in (threshold_cfgs or {}).items()
if isinstance(cfg, dict) and user_can_use(cfg, username)
}
_HOST_SETTING_KEYS = ("watch", "dyndns", "notification_channels", "threshold_config")
_HOST_ACCESS_KEYS = ("owner", "managers", "monitors")
_HOST_ALLOWED_KEYS = frozenset(_HOST_SETTING_KEYS) | frozenset(_HOST_ACCESS_KEYS)
def _check_added_assignments(host_name: str, old_cfg: Any, new_cfg: dict,
key: str, usable: set) -> None:
"""Every value under *key* not already on the host must be in *usable*."""
old_vals = set(_as_list((old_cfg or {}).get(key)))
for v in _as_list(new_cfg.get(key)):
if v not in old_vals and v not in usable:
raise ScopedMergeError(
f"host {host_name!r}: {key} entry {v!r} is not available to you")
def merge_hosts_scoped(existing: Any, payload: Any, username: str,
channels_cfg: Any, threshold_cfgs: Any) -> Dict[str, Any]:
"""Return a new hosts section with *username*'s visible subset replaced by *payload*.
Hosts the user cannot see are preserved untouched. Visible hosts missing
from the payload are deleted (owners only). Managers may change setting
keys but not access keys and may not delete. New hosts get their owner
forced to *username*. Newly added channel/threshold assignments must be
global or owned by the user. Violations raise ScopedMergeError.
"""
existing = existing or {}
payload = payload or {}
visible = user_hosts(existing, username)
usable_channels = set(user_channels(channels_cfg, username))
usable_tcs = set(user_threshold_configs(threshold_cfgs, username)) | {"default"}
result: Dict[str, Any] = {n: c for n, c in existing.items() if n not in visible}
for name, entry in payload.items():
if not isinstance(entry, dict):
raise ScopedMergeError(f"host {name!r}: invalid entry")
unknown = set(entry) - _HOST_ALLOWED_KEYS
if unknown:
raise ScopedMergeError(
f"host {name!r}: fields not permitted: {', '.join(sorted(unknown))}")
if name in existing and name not in visible:
raise ScopedMergeError(f"host {name!r}: you are not an owner or manager")
old = visible.get(name)
if old is None:
new_cfg = dict(entry)
new_cfg["owner"] = username
else:
new_cfg = dict(old)
for key in _HOST_SETTING_KEYS:
if key in entry:
new_cfg[key] = entry[key]
else:
new_cfg.pop(key, None)
if old.get("owner") == username:
for key in _HOST_ACCESS_KEYS:
if key in entry:
new_cfg[key] = entry[key]
else:
new_cfg.pop(key, None)
else:
if "owner" in entry and (entry.get("owner") or None) != (old.get("owner") or None):
raise ScopedMergeError(
f"host {name!r}: only the owner may change ownership")
for key in ("managers", "monitors"):
if key in entry and set(_as_list(entry[key])) != set(_as_list(old.get(key))):
raise ScopedMergeError(
f"host {name!r}: only the owner may change {key}")
_check_added_assignments(name, old, new_cfg, "notification_channels", usable_channels)
_check_added_assignments(name, old, new_cfg, "threshold_config", usable_tcs)
result[name] = new_cfg
for name, old in visible.items():
if name not in payload and old.get("owner") != username:
raise ScopedMergeError(f"host {name!r}: only the owner may delete a host")
return result
def merge_threshold_configs_scoped(existing: Any, payload: Any,
username: str) -> Dict[str, Any]:
"""Return a new threshold_configs section with the user's own configs
replaced by *payload*.
Global and foreign-owned configs are preserved and may not appear in the
payload ('default' included). Own configs missing from the payload are
deleted. Every payload entry gets its owner forced to *username*.
"""
existing = existing or {}
payload = payload or {}
result: Dict[str, Any] = {
n: c for n, c in existing.items()
if not (isinstance(c, dict) and c.get("owner") == username)
}
for name, entry in payload.items():
if name == "default":
raise ScopedMergeError("threshold config 'default' is global and admin-managed")
old = existing.get(name)
if old is not None and (not isinstance(old, dict) or old.get("owner") != username):
raise ScopedMergeError(f"threshold config {name!r}: not owned by you")
new_cfg = dict(entry) if isinstance(entry, dict) else {}
new_cfg["owner"] = username
result[name] = new_cfg
return result
+1
View File
@@ -19,6 +19,7 @@ def _make_yaml() -> YAML:
_SERVER_KEYS = [
"hbd_port", "hbd_host", "ws_port", "wss_port", "hb_port",
"interval", "grace", "base_url", "threshold_renotify_interval",
"flap_count", "flap_interval",
"logfile", "pidfile", "pickfile", "journal_enabled", "journal_dir",
"journal_max_size", "journal_max_backups", "default_owner",
"default_threshold_config",
+130
View File
@@ -0,0 +1,130 @@
"""Flap detection: silence checks that toggle faster than they are useful.
A ``(host, service)`` pair is *flapping* once it exceeds ``flap_count``
WARNING/CRITICAL notifications within ``flap_interval`` minutes. The
notification that trips the threshold is delivered with ``FLAP_MARKER``
appended; every notification for that pair afterwards is dropped. The state
clears silently ``flap_interval`` minutes after a RECOVER, provided no further
WARNING/CRITICAL arrived in the meantime.
Only outbound notifications are affected — ``notify.eventlog`` keeps recording
every event, so the journal and ``/log`` retain the full history of the flap.
State lives here at module level rather than on ``Host`` so it is never
pickled: a restart starts every check with a clean slate.
"""
import logging
import time
logger = logging.getLogger(__name__)
FLAP_MARKER = "Now flapping!! No more messages!"
# Actions returned by observe()
PASS = "pass" # deliver unchanged
TRIP = "trip" # deliver with FLAP_MARKER appended
SUPPRESS = "suppress" # drop
_count = 0 # flap_count: alerts within the window needed to trip
_window = 0.0 # flap_interval in seconds
# {(host, service): {"events": [ts, ...], "flapping": bool, "ok_since": float|None}}
_state: dict = {}
def setup(cfg) -> None:
"""Read flap_count / flap_interval from *cfg* (also called on reload)."""
global _count, _window
_count = int(cfg.get("flap_count", 0) or 0)
_window = float(cfg.get("flap_interval", 0) or 0) * 60.0
def _enabled() -> bool:
return _count > 0 and _window > 0
def _label(key) -> str:
host, service = key
return f"{host}/{service}" if service else host
def _sweep(st: dict, now: float) -> bool:
"""Clear the flapping flag once the post-RECOVER quiet window has elapsed."""
if st["flapping"] and st["ok_since"] is not None and now - st["ok_since"] >= _window:
st["flapping"] = False
st["ok_since"] = None
st["events"].clear()
return True
return False
def observe(host: str, service: str, level: str) -> str:
"""Record a notification for *host*/*service* and return what to do with it.
Returns PASS, TRIP or SUPPRESS.
"""
if not _enabled():
return PASS
now = time.time()
key = (host or "", service or "")
level = (level or "").upper()
st = _state.get(key)
if st is not None and _sweep(st, now):
logger.info("flapping cleared for %s", _label(key))
if level in ("WARNING", "CRITICAL"):
if st is None:
st = _state[key] = {"events": [], "flapping": False, "ok_since": None}
# An alert inside the quiet window means it never settled.
st["ok_since"] = None
st["events"] = [t for t in st["events"] if now - t < _window]
st["events"].append(now)
if st["flapping"]:
return SUPPRESS
if len(st["events"]) > _count:
st["flapping"] = True
logger.info(
"flapping detected for %s (%d alerts in %.0f min)",
_label(key), len(st["events"]), _window / 60,
)
return TRIP
return PASS
if st is None or not st["flapping"]:
return PASS
if level == "RECOVER":
# Starts the quiet window; the state ends silently when it elapses.
st["ok_since"] = now
return SUPPRESS
def clear_host(host: str) -> None:
"""Discard all flap state for *host* (called when a host is dropped).
A dropped host may be mid-flap with no RECOVER ever received, in which
case ``ok_since`` stays ``None`` and ``_sweep`` can never clear it on its
own — the state would otherwise persist forever.
"""
for key in [k for k in _state if k[0] == host]:
del _state[key]
def flapping_services(host: str) -> list:
"""Return the services of *host* that are currently flapping.
An empty string in the result means the host itself (connectivity, boot,
shutdown) rather than a named service.
"""
if not _enabled():
return []
now = time.time()
flapping = []
for (h, service), st in _state.items():
if h != host:
continue
_sweep(st, now)
if st["flapping"]:
flapping.append(service)
return sorted(flapping)
+13 -1
View File
@@ -304,6 +304,14 @@ class Host:
self.managers: list = [] # usernames with manager role
self.monitors: list = [] # usernames with monitor role
def __getstate__(self):
"""Prepare Host for pickling by excluding non-serializable timer objects."""
state = self.__dict__.copy()
# asyncio TimerHandles (and their lambda callbacks) can't be pickled.
# They're recreated when the next PLG arrives after unpickling.
state['plugin_timers'] = {}
return state
def statedict(self):
d = {}
d["raw_name"] = self.name
@@ -367,7 +375,7 @@ class Host:
def stateinfo(self):
ddict = {}
for d in self.__dict__:
if d in ["alert_states", "plugin_data"]:
if d in ["alert_states", "plugin_data", "plugin_timers"]:
continue
if d == "connections":
cl = []
@@ -420,6 +428,10 @@ class Host:
ddict["alert_critical_unacked"] = critical_unacked
ddict["alert_critical_acked"] = critical_acked
# Flap detection state (module-level in flap.py, never pickled)
from . import flap
ddict["flapping"] = flap.flapping_services(self.name)
# User access
ddict["owner"] = getattr(self, "owner", None)
ddict["managers"] = list(getattr(self, "managers", []))
+186 -46
View File
@@ -14,12 +14,15 @@ import logging
from aiohttp import web
import jinja2
from . import data
from . import flap as flap_mod
from . import notify as notify_mod
from . import settings as settings_mod
from . import users as users_mod
from . import oauth as oauth_mod
from . import ws as ws_mod
from . import configio as configio_mod
from . import journal as journal_mod
from . import config_access
logger = logging.getLogger(__name__)
@@ -27,19 +30,25 @@ eventlog = notify_mod.eventlog
def _build_threshold_configs_from_form(form_data: dict) -> dict:
"""Convert form-submitted flat threshold data to nested threshold_configs YAML structure.
"""Convert form-submitted threshold data to the nested threshold_configs structure.
Input: {config_name: {metric_path: {warning, critical, operator, hysteresis, enabled, count, display}}}
Output: {config_name: {thresholds: {plugin: {metric: {warning, critical, ...}}}}}
Input: {config_name: {owner?: str, metrics: {metric_path: {warning, critical, ...}}}}
Output: {config_name: {owner?: str, thresholds: {plugin: {metric: {...}}}}}
"""
result = {}
for config_name, metrics in form_data.items():
for config_name, cfg in form_data.items():
if not isinstance(cfg, dict):
continue
metrics = cfg.get("metrics")
if not isinstance(metrics, dict):
continue
thresholds = {}
thresholds: dict = {}
for metric_path, values in metrics.items():
_insert_threshold_metric(thresholds, metric_path, values)
result[config_name] = {"thresholds": thresholds}
entry = {"thresholds": thresholds}
if cfg.get("owner"):
entry["owner"] = cfg["owner"]
result[config_name] = entry
return result
@@ -201,6 +210,16 @@ def _mask_config_for_api(config) -> dict:
return result
def _visible_plugin_names(plugin_data: dict) -> list:
"""Plugin names to render as accordion sections.
Excludes synthetic rtt_ipv4 / rtt_ipv6 history streams (written by
udp.py for charting) which aren't real client plugins and have no
accordion renderer.
"""
return [p for p in plugin_data.keys() if not p.startswith("rtt_")]
def _build_host_info(host, threshold_checker=None) -> dict:
"""Assemble the info payload for GET /api/0/hosts/{hostname}/info."""
hbc_version = None
@@ -251,12 +270,25 @@ def _build_host_info(host, threshold_checker=None) -> dict:
key=lambda x: x["metric"],
)
connections = [
{
"family": getattr(conn, "afam", family),
"addr": getattr(conn, "addr", ""),
"state": getattr(conn, "state", ""),
"rtt": (getattr(conn, "rtts", None) or [None])[-1],
"statetime": getattr(conn, "statetime", None),
"lastbeat": getattr(conn, "lastbeat", None),
}
for family, conn in sorted(host.connections.items())
]
return {
"owner": getattr(host, "owner", None),
"managers": list(getattr(host, "managers", [])),
"hbc_version": hbc_version,
"hbc_type": hbc_type,
"last_packet": last_packet,
"connections": connections,
"thresholds": thresholds,
}
@@ -342,6 +374,50 @@ async def start(
lst = data.msgs[-30:]
return web.json_response(lst)
async def api_log(request):
"""Paged, filtered read of the events journal (newest first)."""
user, err = _require_auth(request)
if err:
return err
qa = request.rel_url.query
try:
limit = max(1, min(int(qa.get("limit", "100")), 1000))
except ValueError:
limit = 100
before = None
if qa.get("before"):
try:
before = float(qa["before"])
except ValueError:
return web.json_response({"error": "invalid before"}, status=400)
host_f = qa.get("host") or None
level_f = qa.get("level") or None
q_f = qa.get("q") or None
def visible(ev):
h = ev.get("host")
return not h or ws_mod._user_can_see_host(user, h)
ej = journal_mod.get_events_journal(config)
if ej.enabled and ej.journal_path.is_file():
# File scan can be large; keep it off the loop that services UDP/WS
events, more = await asyncio.get_running_loop().run_in_executor(
None,
lambda: journal_mod.read_events(
ej.journal_dir, ej.journal_file,
limit=limit, before=before, host=host_f, level=level_f, q=q_f,
predicate=visible,
),
)
else:
# Journal disabled or not yet written: serve the in-memory ring
events, more = journal_mod.filter_events(
reversed(data.msgs),
limit=limit, before=before, host=host_f, level=level_f, q=q_f,
predicate=visible,
)
return web.json_response({"events": events, "more": more})
async def cmd(request):
user, err = _require_auth(request)
if err:
@@ -374,6 +450,7 @@ async def start(
return web.json_response({"error": "Forbidden"}, status=403)
eventlog(uname, "INFO", "dropped")
del hbdclass.Host.hosts[uname]
flap_mod.clear_host(uname)
return web.Response(text="Done")
async def register(request):
@@ -424,7 +501,7 @@ async def start(
# Resolve templates directory relative to the hbd package
pkg_dir = os.path.dirname(__file__)
templates_dir = config.get("templates_dir", os.path.join(pkg_dir, "templates"))
env = jinja2.Environment(loader=jinja2.FileSystemLoader(templates_dir))
env = jinja2.Environment(loader=jinja2.FileSystemLoader(templates_dir), autoescape=True)
host = config.get("hb_host", "localhost")
extra_scripts = config.get("http_extra_scripts", "")
host = request.host # includes port if non-standard
@@ -446,7 +523,6 @@ async def start(
for h in sorted(hbdclass.Host.hosts)
if _can_operate_host(current_user, hbdclass.Host.hosts[h])
],
messages=data.msgs[-30:],
current_user=current_user.to_dict() if current_user else None,
active_page="live",
)
@@ -597,8 +673,6 @@ async def start(
all_alerts = []
for hostname, host in hbdclass.Host.hosts.items():
if not host.watched:
continue
if not _can_view_host(user, host):
continue
if threshold_checker:
@@ -692,7 +766,7 @@ async def start(
current_user, _ = _require_auth_redirect(request)
pkg_dir = os.path.dirname(__file__)
templates_dir = config.get("templates_dir", os.path.join(pkg_dir, "templates"))
env = jinja2.Environment(loader=jinja2.FileSystemLoader(templates_dir))
env = jinja2.Environment(loader=jinja2.FileSystemLoader(templates_dir), autoescape=True)
# Collect all hosts with plugin data (filtered by visibility)
hosts_with_plugins = []
@@ -703,7 +777,7 @@ async def start(
if host.plugin_data:
hosts_with_plugins.append({
"name": hostname,
"plugins": list(host.plugin_data.keys()),
"plugins": _visible_plugin_names(host.plugin_data),
"is_owner": _can_own_host(current_user, host),
"owner": host.owner,
})
@@ -723,7 +797,7 @@ async def start(
current_user, _ = _require_auth_redirect(request)
pkg_dir = os.path.dirname(__file__)
templates_dir = config.get("templates_dir", os.path.join(pkg_dir, "templates"))
env = jinja2.Environment(loader=jinja2.FileSystemLoader(templates_dir))
env = jinja2.Environment(loader=jinja2.FileSystemLoader(templates_dir), autoescape=True)
tmpl = env.get_template("alerts.html")
body = tmpl.render(
@@ -734,6 +808,27 @@ async def start(
)
return web.Response(text=body, content_type="text/html")
async def log_page(request):
"""Render the event log page."""
current_user, _ = _require_auth_redirect(request)
pkg_dir = os.path.dirname(__file__)
templates_dir = config.get("templates_dir", os.path.join(pkg_dir, "templates"))
env = jinja2.Environment(loader=jinja2.FileSystemLoader(templates_dir), autoescape=True)
host = request.host # includes port if non-standard
forwarded_proto = request.headers.get("X-Forwarded-Proto", "")
is_secure = request.secure or forwarded_proto.lower() == "https"
scheme = "wss" if is_secure else "ws"
heartbeat_ws_url = f"{scheme}://{host}/ws"
tmpl = env.get_template("log.html")
body = tmpl.render(
title="Log - Heartbeat",
header="Log",
heartbeat_ws_url=heartbeat_ws_url,
current_user=current_user.to_dict() if current_user else None,
active_page="log",
)
return web.Response(text=body, content_type="text/html")
# -------------------------------------------------------------------------
# Auth endpoints
# -------------------------------------------------------------------------
@@ -780,6 +875,8 @@ async def start(
token = users_mod.create_session(username)
eventlog("hbd", "INFO", f"Login: {username} via password")
redirect_to = request.rel_url.query.get("next", "/")
if not redirect_to.startswith("/"):
redirect_to = "/"
resp = web.HTTPFound(redirect_to)
resp.set_cookie(
SESSION_COOKIE,
@@ -891,6 +988,13 @@ async def start(
if not target_user.avatar_is_local():
return web.Response(status=404, text="No local avatar configured")
path = target_user.avatar
avatar_dir = config.get("avatar_dir") or (
os.path.dirname(os.path.realpath(_config_path)) if _config_path else None
)
if not avatar_dir:
return web.Response(status=403, text="Local avatars not configured")
if not os.path.realpath(path).startswith(os.path.realpath(avatar_dir) + os.sep):
return web.Response(status=403, text="Forbidden")
if not os.path.isfile(path):
return web.Response(status=404, text="Avatar file not found")
# Infer content-type from extension
@@ -994,7 +1098,7 @@ async def start(
current_user, _ = _require_auth_redirect(request)
pkg_dir = os.path.dirname(__file__)
templates_dir = config.get("templates_dir", os.path.join(pkg_dir, "templates"))
env = jinja2.Environment(loader=jinja2.FileSystemLoader(templates_dir))
env = jinja2.Environment(loader=jinja2.FileSystemLoader(templates_dir), autoescape=True)
# Build host access summary for this user.
# Merge live hosts with config-only hosts (not yet seen) so the profile
@@ -1044,7 +1148,7 @@ async def start(
"name": name,
"type": cfg.get("type", ""),
"owner": cfg.get("owner"),
"private": bool(cfg.get("private", False)),
"private": not config_access.is_global(cfg),
}
for name, cfg in visible_channels.items()
if isinstance(cfg, dict)
@@ -1078,7 +1182,7 @@ async def start(
current_user, _ = _require_auth_redirect(request)
pkg_dir = os.path.dirname(__file__)
templates_dir = config.get("templates_dir", os.path.join(pkg_dir, "templates"))
env = jinja2.Environment(loader=jinja2.FileSystemLoader(templates_dir))
env = jinja2.Environment(loader=jinja2.FileSystemLoader(templates_dir), autoescape=True)
from hbd import __version__ as hbd_version
uptime_secs = int(time.time() - _start_epoch)
@@ -1112,19 +1216,18 @@ async def start(
return web.Response(text=body, content_type="text/html")
# -------------------------------------------------------------------------
# Settings page (admin only)
# Settings page
# -------------------------------------------------------------------------
async def settings_page(request):
"""GET /settings — read-only view of the current server configuration."""
"""GET /settings — server configuration; non-admins see only what they own or manage."""
current_user, _ = _require_auth_redirect(request)
if current_user and not current_user.admin:
raise web.HTTPForbidden(reason="Admin access required")
pkg_dir = os.path.dirname(__file__)
templates_dir = config.get("templates_dir", os.path.join(pkg_dir, "templates"))
env = jinja2.Environment(loader=jinja2.FileSystemLoader(templates_dir))
env = jinja2.Environment(loader=jinja2.FileSystemLoader(templates_dir), autoescape=True)
tmpl = env.get_template("settings.html")
settings_data = settings_mod.get_settings_data(config, threshold_checker=threshold_checker)
settings_data = settings_mod.get_settings_data(
config, threshold_checker=threshold_checker, user=current_user)
body = tmpl.render(
title="Settings - Heartbeat",
sections=settings_data["sections"],
@@ -1274,12 +1377,16 @@ async def start(
return web.json_response({"backups": backups})
async def api_config_post(request):
"""POST /api/0/config — publish staged changes to .hb.yaml. Admin only."""
"""POST /api/0/config — publish staged changes to .hb.yaml.
Admins may write any section. Non-admins may submit only 'hosts' and
'thresholds'; their payload is merged into the config scoped to the
entries they own or manage (see hbd.server.config_access).
"""
user, err = _require_auth(request)
if err:
return err
if user and not user.admin:
return web.json_response({"error": "Forbidden"}, status=403)
is_admin = user is None or user.admin
if not _config_path:
return web.json_response({"error": "Config path not available"}, status=503)
try:
@@ -1290,6 +1397,13 @@ async def start(
if not isinstance(payload, dict):
return web.json_response({"error": "Invalid JSON"}, status=400)
if not is_admin:
extra = set(payload) - {"hosts", "thresholds"}
if extra:
return web.json_response(
{"error": f"Not permitted to edit: {', '.join(sorted(extra))}"},
status=403)
try:
data = configio_mod.read_roundtrip(_config_path)
@@ -1338,18 +1452,36 @@ async def start(
if "thresholds" in payload:
tc = payload["thresholds"]
if isinstance(tc, str):
if not is_admin:
return web.json_response({"error": "Forbidden"}, status=403)
configio_mod.apply_yaml_section(data, "thresholds", tc)
elif isinstance(tc, dict):
data["threshold_configs"] = _build_threshold_configs_from_form(tc)
built = _build_threshold_configs_from_form(tc)
if is_admin:
data["threshold_configs"] = built
else:
data["threshold_configs"] = config_access.merge_threshold_configs_scoped(
data.get("threshold_configs") or {}, built, user.username)
if "hosts" in payload:
h = payload["hosts"]
if isinstance(h, dict):
configio_mod.apply_structured_section(data, "hosts", h)
else:
if is_admin:
configio_mod.apply_structured_section(data, "hosts", h)
else:
merged = config_access.merge_hosts_scoped(
dict(data.get("hosts") or {}), h, user.username,
data.get("notification_channels") or {},
data.get("threshold_configs") or {})
configio_mod.apply_structured_section(data, "hosts", merged)
elif is_admin:
configio_mod.apply_yaml_section(data, "hosts", h)
else:
return web.json_response({"error": "Forbidden"}, status=403)
configio_mod.write_config(_config_path, data)
except config_access.ScopedMergeError as exc:
return web.json_response({"error": str(exc)}, status=403)
except Exception as exc:
logger.error("Config write failed: %s", exc)
return web.json_response({"error": str(exc)}, status=500)
@@ -1400,19 +1532,13 @@ async def start(
# -------------------------------------------------------------------------
def _visible_channels_for_user(user):
"""Return {name: cfg} of channels visible to user (public + own private)."""
"""Return {name: cfg} of channels visible to user (global + own)."""
all_channels = config.get("notification_channels") or {}
if user is None:
return {}
if user.admin:
return dict(all_channels)
visible = {}
for name, cfg in all_channels.items():
if not isinstance(cfg, dict):
continue
if not cfg.get("private") or cfg.get("owner") == user.username:
visible[name] = cfg
return visible
return config_access.user_channels(all_channels, user.username)
def _build_channel_response(ch_name, ch_cfg):
"""Serialize a channel config dict for the API response."""
@@ -1436,7 +1562,7 @@ async def start(
"type": ch_type,
"type_label": settings_mod._CHANNEL_TYPE_LABELS.get(ch_type, ch_type.title()),
"owner": ch_cfg.get("owner"),
"private": bool(ch_cfg.get("private", False)),
"private": not config_access.is_global(ch_cfg),
"min_level": ch_cfg.get("min_level", "WARNING"),
"fields": fields,
}
@@ -1501,9 +1627,12 @@ async def start(
if body.get("min_level"):
channel_cfg["min_level"] = body["min_level"]
channel_cfg["owner"] = user.username
if body.get("private"):
channel_cfg["private"] = True
if user.admin:
owner = (body.get("owner") or "").strip()
if owner:
channel_cfg["owner"] = owner
else:
channel_cfg["owner"] = user.username
try:
disk_data = configio_mod.read_roundtrip(_config_path)
@@ -1568,12 +1697,12 @@ async def start(
if body.get("min_level"):
channel_cfg["min_level"] = body["min_level"]
if owner is not None:
if user.admin:
new_owner = (body.get("owner") or "").strip() if "owner" in body else (owner or "")
if new_owner:
channel_cfg["owner"] = new_owner
elif owner is not None:
channel_cfg["owner"] = owner
if "private" in body:
channel_cfg["private"] = bool(body["private"])
elif existing_on_disk.get("private"):
channel_cfg["private"] = True
configio_mod.apply_channel(disk_data, ch_name, channel_cfg)
configio_mod.write_config(_config_path, disk_data)
@@ -1661,7 +1790,16 @@ async def start(
if "full_name" in body:
user_entry["full_name"] = str(body["full_name"])
if "avatar" in body:
user_entry["avatar"] = str(body["avatar"])
avatar_val = str(body["avatar"])
if avatar_val.startswith("/"):
avatar_dir = config.get("avatar_dir") or (
os.path.dirname(os.path.realpath(_config_path)) if _config_path else None
)
if not avatar_dir:
return web.json_response({"error": "Local avatars not configured"}, status=400)
if not os.path.realpath(avatar_val).startswith(os.path.realpath(avatar_dir) + os.sep):
return web.json_response({"error": "Avatar path outside allowed directory"}, status=400)
user_entry["avatar"] = avatar_val
if "notification_channels" in body:
visible = _visible_channels_for_user(user)
user_entry["notification_channels"] = [
@@ -1718,6 +1856,7 @@ async def start(
web.get("/api/0/hosts", api_hosts),
web.get("/api/0/alert_summary", api_alert_summary),
web.get("/api/0/messages", api_messages),
web.get("/api/0/log", api_log),
web.get("/api/0/hosts/{hostname}/plugins", api_host_plugins),
web.get("/api/0/hosts/{hostname}/plugins/{plugin_name}", api_host_plugin_detail),
web.get("/api/0/hosts/{hostname}/alerts", api_host_alerts),
@@ -1733,6 +1872,7 @@ async def start(
web.get("/live", live),
web.get("/plugins", plugins_page),
web.get("/alerts", alerts_page),
web.get("/log", log_page),
web.get("/about", about_page),
web.get("/profile", profile_page),
web.get("/settings", settings_page),
+136 -4
View File
@@ -11,7 +11,7 @@ import os
import asyncio
from datetime import datetime
from pathlib import Path
from typing import Dict, Any, Optional
from typing import Dict, Any, Optional, Callable, Iterable, List, Tuple, Union
logger = logging.getLogger(__name__)
@@ -149,10 +149,35 @@ class MessageJournal:
self._current_size += len(json_bytes)
logger.debug(f"Logged message from {addr[0]}: {msg.get('ID', 'UNKNOWN')}")
except Exception as e:
logger.error(f"Error writing to journal: {e}")
async def log_event(self, event: Dict[str, Any]):
"""Write a caller-provided dict verbatim as one JSONL line (with rotation)."""
if not self.enabled or not self._initialized:
return
async with self._lock:
try:
line = json.dumps(event, separators=(',', ':')) + '\n'
nbytes = len(line.encode('utf-8'))
if self._current_size + nbytes > self.max_size:
await self._rotate()
if self._file_handle:
self._file_handle.write(line)
self._file_handle.flush()
self._current_size += nbytes
except Exception as e:
logger.error(f"Error writing event to journal: {e}")
async def backfill(self, events: Iterable[Dict[str, Any]]):
"""One-time seed: write *events* only if the journal file is currently empty."""
if not self.enabled or not self._initialized or self._current_size > 0:
return
for ev in events:
if isinstance(ev, dict):
await self.log_event(ev)
async def _rotate(self):
"""
Rotate the journal file.
@@ -309,6 +334,88 @@ class MessageJournal:
}
def _iter_journal_events(journal_dir: Union[str, Path], journal_file: str) -> Iterable[Dict[str, Any]]:
"""Yield event dicts from the journal files, newest event first.
Reads the current file, then rotated backups newest-to-oldest (backup
names embed rotation timestamps, so filename sort is chronological).
"""
dirp = Path(journal_dir)
files = [dirp / journal_file]
files.extend(sorted(dirp.glob(journal_file + '.*'), reverse=True))
for f in files:
if not f.is_file():
continue
try:
lines = f.read_text(encoding='utf-8', errors='replace').splitlines()
except OSError as e:
logger.warning(f"Cannot read journal file {f}: {e}")
continue
for line in reversed(lines):
try:
ev = json.loads(line)
except ValueError:
continue
if isinstance(ev, dict):
yield ev
def filter_events(
events: Iterable[Dict[str, Any]],
limit: int = 100,
before: Optional[float] = None,
host: Optional[str] = None,
level: Optional[str] = None,
q: Optional[str] = None,
predicate: Optional[Callable[[Dict[str, Any]], bool]] = None,
) -> Tuple[List[Dict[str, Any]], bool]:
"""Filter an iterable of event dicts already ordered newest-first.
Returns (events, more): up to *limit* matching events, and whether at
least one further matching event exists beyond the limit.
"""
host_l = host.lower() if host else None
level_l = level.lower() if level else None
q_l = q.lower() if q else None
out: List[Dict[str, Any]] = []
for ev in events:
if not isinstance(ev, dict):
continue
ts = ev.get('ts')
if before is not None and (not isinstance(ts, (int, float)) or ts >= before):
continue
if host_l and host_l not in str(ev.get('host') or '').lower():
continue
if level_l and str(ev.get('level') or '').lower() != level_l:
continue
if q_l and q_l not in str(ev.get('message') or '').lower():
continue
if predicate is not None and not predicate(ev):
continue
if len(out) >= limit:
return out, True
out.append(ev)
return out, False
def read_events(
journal_dir: Union[str, Path],
journal_file: str = 'events.journal',
*,
limit: int = 100,
before: Optional[float] = None,
host: Optional[str] = None,
level: Optional[str] = None,
q: Optional[str] = None,
predicate: Optional[Callable[[Dict[str, Any]], bool]] = None,
) -> Tuple[List[Dict[str, Any]], bool]:
"""Read filtered events newest-first from the events journal files."""
return filter_events(
_iter_journal_events(journal_dir, journal_file),
limit=limit, before=before, host=host, level=level, q=q, predicate=predicate,
)
# Global journal instance
_journal_instance: Optional[MessageJournal] = None
@@ -332,7 +439,7 @@ def get_journal(config: Optional[Dict[str, Any]] = None) -> MessageJournal:
async def log_message(msg: Dict[str, Any], addr: tuple, timestamp: Optional[float] = None):
"""
Convenience function to log a message using the global journal.
Args:
msg: Parsed message dictionary
addr: Source address (ip, port) tuple
@@ -340,3 +447,28 @@ async def log_message(msg: Dict[str, Any], addr: tuple, timestamp: Optional[floa
"""
journal = get_journal()
await journal.log_message(msg, addr, timestamp)
# Global events journal instance (human-readable event log, written by notify.eventlog)
_events_journal_instance: Optional[MessageJournal] = None
def get_events_journal(config: Optional[Dict[str, Any]] = None) -> MessageJournal:
"""Get or create the global events journal instance.
Uses the events_journal_* config keys; shares journal_dir and
journal_enabled with the raw-datagram journal.
"""
global _events_journal_instance
if _events_journal_instance is None:
cfg = config or {}
_events_journal_instance = MessageJournal(
{
'journal_dir': cfg.get('journal_dir', '/var/log/heartbeat'),
'journal_file': cfg.get('events_journal_file', 'events.journal'),
'journal_max_size': cfg.get('events_journal_max_size', 10 * 1024 * 1024),
'journal_max_backups': cfg.get('events_journal_max_backups', 10),
'journal_enabled': cfg.get('journal_enabled', True),
}
)
return _events_journal_instance
+31 -4
View File
@@ -160,11 +160,20 @@ async def _run_async(config, config_path=None):
from . import threshold as threshold_mod
notify_mod.setup(config, loop=loop)
# Initialize message journal
msg_journal = journal_mod.get_journal(config)
await msg_journal.initialize()
# Initialize events journal (human-readable event log for the /log page)
events_journal = journal_mod.get_events_journal(config)
await events_journal.initialize()
if data.msgs:
# One-time seed on upgrade: only writes when the journal file is empty
await events_journal.backfill(data.msgs)
notify_mod.eventlog(None, "INFO", f"hbd version {__version__} starting up")
# Initialize threshold checker
threshold_checker = threshold_mod.ThresholdChecker(
config=config,
@@ -379,6 +388,26 @@ async def _run_async(config, config_path=None):
except Exception as e:
logger.warning("Error closing message journal: %s", e)
# Journal the shutdown event now; run()'s shutdown eventlog fires after this journal closes
try:
await events_journal.log_event(
{
"ts": time.time(),
"host": None,
"level": "INFO",
"service": None,
"message": f"hbd version {__version__} shutdown",
}
)
except Exception as e:
logger.debug("Could not journal shutdown event: %s", e)
# Close events journal
try:
await events_journal.close()
except Exception as e:
logger.warning("Error closing events journal: %s", e)
# Signal DNS worker to exit and await it
try:
if "dns_task" in locals() and dns_task:
@@ -487,8 +516,6 @@ def run(config, config_path=None):
except Exception as e:
logger.warning("Failed to write pidfile %s: %s", pidfile, e)
eventlog(None, "INFO", f"hbd version {__version__} starting up")
if config_path:
logger.info(f"Config file: {config_path} (reload with SIGHUP)")
else:
+39 -1
View File
@@ -25,6 +25,7 @@ from dataclasses import dataclass, field
from typing import Optional
from . import data
from . import flap as flap_mod
from . import ws as ws_mod
logger = logging.getLogger(__name__)
@@ -33,6 +34,7 @@ msg_to_websockets = ws_mod.broadcast
# Module-level state set via setup()
_config: dict = {}
_loop: Optional[asyncio.AbstractEventLoop] = None
# Tracks which channels fired a WARNING/CRITICAL per host.
# {host_name: set of channel_names} — used to route RECOVER to the same channels.
@@ -62,6 +64,7 @@ class Notification:
body: str # detail message
level: str # RECOVER | WARNING | CRITICAL | INFO
url: str = "" # link to plugin metrics page
service: str = "" # flap-detection key within the host ("" = the host itself)
# ---------------------------------------------------------------------------
@@ -70,14 +73,18 @@ class Notification:
def setup(cfg: dict, loop: Optional[asyncio.AbstractEventLoop] = None):
"""Initialize notifier from configuration dict."""
global _config
global _config, _loop
_config = dict(cfg)
flap_mod.setup(_config)
if loop is not None:
_loop = loop
def reload_config(cfg: dict):
"""Reload notification configuration on SIGHUP."""
global _config
_config = dict(cfg)
flap_mod.setup(_config)
logger.info("Notification configuration reloaded")
@@ -114,6 +121,11 @@ def eventlog(host, lvl, m, service=None):
"message": m,
}
data.msgs.append(msg)
# Cap the in-memory buffer so it doesn't grow without bound; this list is
# replayed to every dashboard client on connect and persisted in the pickle.
cap = _config.get("msg_buffer_size", 500)
if cap and len(data.msgs) > cap:
del data.msgs[:-cap]
s = f"{time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(ts))} {lvl} "
if host:
s += f"{host} "
@@ -126,6 +138,21 @@ def eventlog(host, lvl, m, service=None):
except Exception as e:
logger.warning("failed to write to logfile: %s", e)
msg_to_websockets("message", msg)
_journal_event(msg)
def _journal_event(msg: dict):
"""Schedule an async write of *msg* to the events journal (no-op without a loop)."""
if _loop is None or not _loop.is_running():
return
from . import journal as journal_mod
ej = journal_mod._events_journal_instance
if ej is None or not ej.enabled:
return
try:
asyncio.run_coroutine_threadsafe(ej.log_event(msg), _loop)
except Exception as e:
logger.warning("failed to schedule events journal write: %s", e)
# ---------------------------------------------------------------------------
@@ -416,11 +443,22 @@ async def send_notification(host_name: str, notif: Notification) -> dict:
notification_channels, and dispatches. Silently does nothing if
no users are configured.
Flap detection runs first: once *host_name*/*notif.service* is flapping the
notification is dropped, and the one that trips the state carries
``flap.FLAP_MARKER``.
Returns a dict of {channel_name: bool} results.
"""
from . import users as users_mod
from . import hbdclass
action = flap_mod.observe(host_name, notif.service, notif.level)
if action == flap_mod.SUPPRESS:
logger.debug("flapping: suppressed %s notification for %s", notif.level, host_name)
return {}
if action == flap_mod.TRIP:
notif.body = f"{notif.body} {flap_mod.FLAP_MARKER}"
if not users_mod.users_enabled():
return {}
+61 -16
View File
@@ -21,6 +21,8 @@ editable bool Reserved for future use — currently always False
sensitive bool True when the raw value must never be shown
"""
from . import config_access
# Credential field names that should always be masked.
_SECRET_KEYS = frozenset({
"password", "token", "user_key", "api_key", "secret",
@@ -140,9 +142,14 @@ def _sanitize_channel(name, cfg):
# Public API
# ---------------------------------------------------------------------------
def get_settings_sections(config: dict, threshold_checker=None) -> list:
def get_settings_sections(config: dict, threshold_checker=None, user=None) -> list:
"""Return ordered list of setting sections for the settings page.
*user* is an object with ``username``/``admin`` attributes, or None for
the unauthenticated (admin-equivalent) view. Non-admins get only the
channels/hosts/thresholds sections, filtered to global items plus what
they own or manage.
Each section:
{
"title": str,
@@ -162,6 +169,9 @@ def get_settings_sections(config: dict, threshold_checker=None) -> list:
"sensitive": bool,
}
"""
is_admin = user is None or getattr(user, "admin", False)
username: str = getattr(user, "username", "") or ""
def field(key, label, ftype, description="", editable=False, sensitive=False):
raw = config.get(key)
if sensitive:
@@ -200,6 +210,8 @@ def get_settings_sections(config: dict, threshold_checker=None) -> list:
for ch_name, ch_cfg in sorted((config.get("notification_channels") or {}).items()):
if not isinstance(ch_cfg, dict):
continue
if not is_admin and not config_access.user_can_use(ch_cfg, username):
continue
ch_type = ch_cfg.get("type", "")
fields = []
for k, v in ch_cfg.items():
@@ -219,18 +231,18 @@ def get_settings_sections(config: dict, threshold_checker=None) -> list:
"type": ch_type,
"type_label": _CHANNEL_TYPE_LABELS.get(ch_type, ch_type.title()),
"owner": ch_cfg.get("owner"),
"private": bool(ch_cfg.get("private", False)),
"editable": is_admin or ch_cfg.get("owner") == username,
"min_level": ch_cfg.get("min_level", "WARNING"),
"fields": fields,
})
# ---- Users (show metadata only, never password hashes) ----------------
users_list = []
for username, attrs in (config.get("users") or {}).items():
for uname, attrs in (config.get("users") or {}).items():
if not isinstance(attrs, dict):
continue
users_list.append({
"username": username,
"username": uname,
"full_name": attrs.get("full_name", ""),
"admin": bool(attrs.get("admin", False)),
"avatar": attrs.get("avatar", ""),
@@ -252,9 +264,14 @@ def get_settings_sections(config: dict, threshold_checker=None) -> list:
}
threshold_config_list = []
raw_threshold_cfgs = config.get("threshold_configs") or {}
if threshold_checker is not None:
if threshold_checker.threshold_configs:
for cfg_name, cfg_metrics in sorted(threshold_checker.threshold_configs.items()):
raw_cfg = raw_threshold_cfgs.get(cfg_name)
tc_owner = raw_cfg.get("owner") if isinstance(raw_cfg, dict) else None
if not is_admin and tc_owner and tc_owner != username:
continue
# For the default config use the merged effective set;
# for named overrides use only the explicitly defined metrics
# (threshold_raw_configs) so inherited defaults are not repeated.
@@ -266,25 +283,37 @@ def get_settings_sections(config: dict, threshold_checker=None) -> list:
[_tc_to_row(tc) for tc in display_metrics.values()],
key=lambda m: m["metric"],
)
threshold_config_list.append({"name": cfg_name, "metrics": metrics})
threshold_config_list.append({
"name": cfg_name,
"metrics": metrics,
"owner": tc_owner,
"editable": is_admin or (tc_owner is not None and tc_owner == username),
})
elif threshold_checker.thresholds:
metrics = sorted(
[_tc_to_row(tc) for tc in threshold_checker.thresholds.values()],
key=lambda m: m["metric"],
)
threshold_config_list.append({"name": "default", "metrics": metrics})
threshold_config_list.append({"name": "default", "metrics": metrics,
"owner": None, "editable": is_admin})
# ---- Hosts summary ----------------------------------------------------
hosts_list = []
for hname, hcfg in sorted((config.get("hosts") or {}).items()):
if not isinstance(hcfg, dict):
continue
managers = hcfg.get("managers", [])
if isinstance(managers, str):
managers = [managers]
if not is_admin and hcfg.get("owner") != username and username not in managers:
continue
hosts_list.append({
"name": hname,
"watch": bool(hcfg.get("watch", True)),
"dyndns": bool(hcfg.get("dyndns", False)),
"owner": hcfg.get("owner", ""),
"managers": hcfg.get("managers", []),
"is_owner": is_admin or hcfg.get("owner") == username,
"managers": managers,
"monitors": hcfg.get("monitors", []),
"threshold_configs": (
list(v) if isinstance(v := hcfg.get("threshold_config"), list)
@@ -309,7 +338,7 @@ def get_settings_sections(config: dict, threshold_checker=None) -> list:
"logo": pattrs.get("logo", ""),
})
return [
sections: list = [
{
"id": "network",
"title": "Network",
@@ -357,6 +386,12 @@ def get_settings_sections(config: dict, threshold_checker=None) -> list:
"Extra seconds to wait after a missed heartbeat before sending notifications.", editable=True),
field("threshold_renotify_interval", "Re-notify interval", "duration",
"How often to re-send notifications for ongoing threshold alerts.", editable=True),
field("flap_count", "Flap count", "number",
"Warning/critical notifications within the flap interval that mark a "
"service as flapping and silence it. 0 disables flap detection.", editable=True),
field("flap_interval", "Flap interval", "number",
"Minutes: the window flap count is measured over, and how long a service "
"must stay OK before flapping ends.", editable=True),
field("autosave_interval", "Autosave interval", "duration",
"How often the server saves its state to disk."),
field("base_url", "Base URL", "text",
@@ -483,16 +518,26 @@ def get_settings_sections(config: dict, threshold_checker=None) -> list:
},
]
if not is_admin:
sections = [s for s in sections if s["id"] in ("channels", "hosts", "thresholds")]
for s in sections:
s["fields"] = []
return sections
def get_settings_data(config: dict, threshold_checker=None) -> dict:
def get_settings_data(config: dict, threshold_checker=None, user=None) -> dict:
"""Return sections list + auxiliary data for the settings template."""
sections = get_settings_sections(config, threshold_checker=threshold_checker)
all_channel_names = sorted((config.get("notification_channels") or {}).keys())
all_usernames = sorted((config.get("users") or {}).keys())
all_threshold_configs = sorted((config.get("threshold_configs") or {}).keys())
sections = get_settings_sections(config, threshold_checker=threshold_checker, user=user)
is_admin = user is None or getattr(user, "admin", False)
username: str = getattr(user, "username", "") or ""
channels = config.get("notification_channels") or {}
threshold_cfgs = config.get("threshold_configs") or {}
if not is_admin:
channels = config_access.user_channels(channels, username)
threshold_cfgs = config_access.user_threshold_configs(threshold_cfgs, username)
return {
"sections": sections,
"all_channel_names": all_channel_names,
"all_usernames": all_usernames,
"all_threshold_configs": all_threshold_configs,
"all_channel_names": sorted(channels.keys()),
"all_usernames": sorted((config.get("users") or {}).keys()),
"all_threshold_configs": sorted(threshold_cfgs.keys()),
}
+155
View File
@@ -0,0 +1,155 @@
/* hbd-ui.css — shared design system for hbd pages.
*
* Green-biased neutrals, teal accent, mono identifiers. Pages link this
* after static/style.css and build on the tokens + components below.
* Introduced with the settings-page redesign; see settings.html for the
* fullest use of the row/editor idiom.
*/
:root {
--st-paper: #f4f6f4;
--st-surface: #ffffff;
--st-surface-2: #eef1ee;
--st-ink: #1d2420;
--st-muted: #5f6a63;
--st-faint: #8a948d;
--st-line: #dde3de;
--st-line-soft: #e8ece8;
--st-accent: #0e7280;
--st-accent-ink: #ffffff;
--st-accent-soft: #e0eef0;
--st-ok: #3f9c5a;
--st-ok-soft: #e2f2e7;
--st-warn: #d9982a;
--st-warn-soft: #f9efdc;
--st-crit: #d64545;
--st-crit-soft: #f9e3e3;
--st-badge-owner: #ede7f6; --st-badge-owner-ink: #5e35b1;
--st-badge-mgr: #e3f2fd; --st-badge-mgr-ink: #1565c0;
--st-mono: ui-monospace, "SF Mono", "Cascadia Mono", Menlo, Consolas, monospace;
}
html[data-theme="dark"] {
--st-paper: #131816;
--st-surface: #1b211e;
--st-surface-2: #222925;
--st-ink: #e4eae6;
--st-muted: #9aa69e;
--st-faint: #6e7a72;
--st-line: #2c342f;
--st-line-soft: #252c28;
--st-accent: #4cc4d4;
--st-accent-ink: #0d2326;
--st-accent-soft: #143b40;
--st-ok: #5cba77;
--st-ok-soft: #1c2f22;
--st-warn: #e0ad4e;
--st-warn-soft: #33290f;
--st-crit: #e06c6c;
--st-crit-soft: #3a1a1a;
--st-badge-owner: #33294a; --st-badge-owner-ink: #b39ddb;
--st-badge-mgr: #1b3350; --st-badge-mgr-ink: #90caf9;
}
/* ── sticky toolbar under the site nav ── */
.st-toolbar {
position: sticky; top: var(--nav-h, 48px); z-index: 90;
display: flex; align-items: center; gap: 12px; flex-wrap: wrap;
padding: 9px 20px;
background: var(--st-surface); border-bottom: 1px solid var(--st-line);
}
.st-toolbar .brand { font-family: var(--st-mono); font-size: 15px; font-weight: 700; letter-spacing: -.02em; color: var(--st-ink); }
.st-toolbar .brand .tld { color: var(--st-accent); }
.st-toolbar .hintline { font-size: 12px; color: var(--st-faint); }
.st-toolbar .spacer { flex: 1; }
/* ── ECG pulse divider ── */
.pulse { display: block; width: 100%; height: 14px; }
.pulse polyline { fill: none; stroke: var(--st-accent); stroke-width: 1.5; opacity: .55; }
/* ── buttons ── */
.btn {
font: 600 13px/1 inherit; border: 1px solid var(--st-line); border-radius: 6px;
background: var(--st-surface); color: var(--st-ink); padding: 7px 14px; cursor: pointer;
}
.btn:focus-visible, a:focus-visible { outline: 2px solid var(--st-accent); outline-offset: 2px; }
.btn.primary { background: var(--st-accent); border-color: var(--st-accent); color: var(--st-accent-ink); }
.btn.quiet { border-color: transparent; background: transparent; color: var(--st-muted); }
.btn.small { padding: 4px 10px; font-size: 12px; }
.btn.danger { color: var(--st-crit); }
.btn:disabled { opacity: .6; cursor: default; }
/* ── section headers as config keys ── */
section.st { margin-top: 26px; scroll-margin-top: calc(var(--bars-h, 94px) + 12px); }
.sec-head { display: flex; align-items: baseline; gap: 10px; margin-bottom: 8px; flex-wrap: wrap; }
.sec-head h2 { margin: 0; font-family: var(--st-mono); font-size: 15px; font-weight: 700; color: var(--st-ink); }
.sec-head h2 .colon { color: var(--st-accent); }
.sec-head .count { font-family: var(--st-mono); font-size: 11.5px; color: var(--st-faint); }
.sec-head .sub { font-size: 12px; color: var(--st-muted); }
.sec-head .add { margin-left: auto; }
/* ── record list + row ── */
.list { background: var(--st-surface); border: 1px solid var(--st-line); border-radius: 8px; overflow: hidden; }
.row {
display: grid; grid-template-columns: 190px minmax(0,1fr) auto;
gap: 4px 14px; align-items: center;
padding: 8px 14px; border-bottom: 1px solid var(--st-line-soft);
transition: background .12s ease;
}
@media (prefers-reduced-motion: reduce) { .row { transition: none; } }
.row:last-child { border-bottom: none; }
.row:hover { background: var(--st-surface-2); }
.row .id { font-family: var(--st-mono); font-size: 13px; font-weight: 600; color: var(--st-ink); display: flex; align-items: center; gap: 8px; min-width: 0; }
.row .id .name { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.row .act { display: flex; gap: 4px; justify-content: flex-end; }
.dot { width: 8px; height: 8px; border-radius: 50%; flex: none; }
.dot.ok { background: var(--st-ok); }
.dot.warn { background: var(--st-warn); }
.dot.crit { background: var(--st-crit); }
.dot.off { background: transparent; border: 1.5px solid var(--st-faint); }
.facts { display: flex; flex-wrap: wrap; gap: 4px 6px; align-items: center; min-width: 0; }
.chip {
display: inline-flex; align-items: center; gap: 4px;
font-family: var(--st-mono); font-size: 11px; line-height: 1.3;
padding: 3px 7px; border-radius: 4px;
background: var(--st-surface-2); color: var(--st-muted);
}
.chip.owner { background: var(--st-badge-owner); color: var(--st-badge-owner-ink); }
.chip.mgr { background: var(--st-badge-mgr); color: var(--st-badge-mgr-ink); }
.chip.level { background: transparent; border: 1px solid var(--st-line); }
.chip.private { background: transparent; border: 1px dashed var(--st-badge-owner-ink); color: var(--st-badge-owner-ink); }
.chip.ok { background: var(--st-ok-soft); color: var(--st-ok); }
.chip.warn { background: var(--st-warn-soft); color: var(--st-warn); }
.chip.crit { background: var(--st-crit-soft); color: var(--st-crit); }
.chip .k { color: var(--st-faint); }
.chip .w { color: var(--st-warn); font-weight: 700; }
.chip .c { color: var(--st-crit); font-weight: 700; }
/* ── key/value rows (server groups, about page) ── */
.kv .row { grid-template-columns: 230px minmax(0,1fr); padding: 6px 14px; }
.group {
display: flex; align-items: center; gap: 10px;
padding: 7px 14px 5px; font-family: var(--st-mono); font-size: 11px; color: var(--st-faint);
background: var(--st-surface); border-bottom: 1px solid var(--st-line-soft);
}
.group .gbtn { margin-left: auto; }
.kv .val { font-family: var(--st-mono); font-size: 12.5px; color: var(--st-muted); min-width: 0; }
.kv .val a { color: var(--st-accent); text-decoration: none; }
.kv .val a:hover { text-decoration: underline; }
.kv .desc { display: block; font-size: 11px; color: var(--st-faint); margin-top: 2px; }
.val-masked { color: var(--st-faint); letter-spacing: 2px; }
.note { margin: 8px 2px 0; font-size: 12px; color: var(--st-faint); }
/* ── handheld defaults for rows ── */
@media (max-width: 760px) {
.st-toolbar { padding: 8px 10px; }
.st-toolbar .hintline { display: none; }
.row { grid-template-columns: minmax(0,1fr) auto; padding: 9px 12px; }
.row .id { grid-row: 1; grid-column: 1; }
.row .act { grid-row: 1; grid-column: 2; }
.row .facts { grid-column: 1 / -1; }
.kv .row { grid-template-columns: 1fr; }
section.st { scroll-margin-top: calc(var(--bars-h, 94px) + 50px); }
}
+16
View File
@@ -0,0 +1,16 @@
/* hbd-ui.js — shared helpers for pages using hbd-ui.css.
*
* Keeps sticky toolbars stuck just below the fixed site nav, whose height
* varies with viewport width and wrapping.
*/
function _setStickyOffsets() {
const nav = document.querySelector('.nav');
const tb = document.querySelector('.st-toolbar');
const navH = nav ? nav.offsetHeight : 48;
const barsH = navH + (tb ? tb.offsetHeight : 46);
document.documentElement.style.setProperty('--nav-h', navH + 'px');
document.documentElement.style.setProperty('--bars-h', barsH + 'px');
}
window.addEventListener('resize', _setStickyOffsets);
document.addEventListener('DOMContentLoaded', _setStickyOffsets);
+96 -173
View File
@@ -1,190 +1,113 @@
<!DOCTYPE html>
<html>
{% include 'head.html' %}
<link rel="stylesheet" href="/static/hbd-ui.css">
<script src="/static/hbd-ui.js"></script>
<style>
html, body { overflow: visible; }
html, body { height: auto; overflow: visible; }
body { background: var(--st-paper); padding: 60px 0 0; }
.container {
max-width: 700px;
margin: 0 auto;
.about-main { max-width: 700px; margin: 0 auto; padding: 0 20px 60px; }
.hero {
display: flex; align-items: baseline; gap: 14px; flex-wrap: wrap;
padding: 26px 2px 0;
}
h1 {
color: #333;
margin-bottom: 4px;
font-size: 1.5em;
.hero .hb-logo {
font-family: var(--st-mono); font-size: 30px; font-weight: 700;
letter-spacing: -.03em; color: var(--st-ink);
}
.subtitle {
color: #666;
margin-bottom: 24px;
font-size: 0.9em;
}
.section {
background: #fff;
border-radius: 8px;
box-shadow: 0 1px 6px rgba(0,0,0,0.1);
padding: 20px 24px;
margin-bottom: 20px;
}
.section h2 {
font-size: 1em;
font-weight: 700;
color: #333;
margin: 0 0 16px;
padding-bottom: 10px;
border-bottom: 1px solid #eee;
text-transform: uppercase;
letter-spacing: 0.5px;
}
.info-row {
display: flex;
align-items: baseline;
padding: 8px 0;
border-bottom: 1px solid #f5f5f5;
font-size: 0.9em;
}
.info-row:last-child { border-bottom: none; }
.info-label {
width: 160px;
flex-shrink: 0;
color: #666;
font-size: 0.88em;
}
.info-value {
color: #222;
word-break: break-all;
}
.info-value a {
color: #0066cc;
text-decoration: none;
}
.info-value a:hover { text-decoration: underline; }
.version-badge {
display: inline-block;
padding: 3px 12px;
background: #e8f0fe;
color: #1a73e8;
border-radius: 12px;
font-size: 1.00em;
font-weight: 600;
font-family: monospace;
}
.hb-logo {
font-size: 2.5em;
font-weight: 700;
color: #0066cc;
letter-spacing: -1px;
margin-bottom: 6px;
}
.hb-tagline {
color: #555;
font-size: 0.95em;
}
.logo-section {
display: flex;
align-items: center;
gap: 20px;
padding: 8px 0 4px;
}
.logo-text { flex: 1; }
/* ── Dark mode ── */
html[data-theme="dark"] h1 { color: var(--text); }
html[data-theme="dark"] .subtitle { color: var(--text-sec); }
html[data-theme="dark"] .section { background: var(--surface); box-shadow: 0 1px 6px var(--shadow); }
html[data-theme="dark"] .section h2 { color: var(--text); border-bottom-color: var(--border); }
html[data-theme="dark"] .info-row { border-bottom-color: var(--border-4); }
html[data-theme="dark"] .info-label { color: var(--text-sec); }
html[data-theme="dark"] .info-value { color: var(--text); }
html[data-theme="dark"] .info-value a { color: var(--link); }
html[data-theme="dark"] .hb-logo { color: var(--link); }
html[data-theme="dark"] .hb-tagline { color: var(--text-sec); }
html[data-theme="dark"] .version-badge { background: #1a3255; color: #60a5fa; }
.hero .hb-logo .tld { color: var(--st-accent); }
.hero .version { font-family: var(--st-mono); font-size: 13px; color: var(--st-accent);
background: var(--st-accent-soft); border-radius: 999px; padding: 3px 12px; }
.hero .tagline { flex-basis: 100%; font-size: 13px; color: var(--st-muted); }
</style>
<body>
{% include 'nav.html' %}
<div class="container">
<h1>{{ header }}</h1>
<p class="subtitle">Heartbeat monitoring system</p>
<div class="section">
<div class="logo-section">
<div class="logo-text">
<div class="hb-logo">Heartbeat</div>
<div class="hb-tagline">Lightweight host monitoring over UDP</div>
</div>
<span class="version-badge">v{{ hbd_version }}</span>
</div>
</div>
<div class="section">
<h2>Version</h2>
<div class="info-row">
<span class="info-label">Server version</span>
<span class="info-value">{{ hbd_version }}</span>
</div>
<div class="info-row">
<span class="info-label">Python</span>
<span class="info-value">{{ python_version }}</span>
</div>
<div class="info-row">
<span class="info-label">License</span>
<span class="info-value">MIT</span>
</div>
</div>
<div class="section">
<h2>Runtime</h2>
<div class="info-row">
<span class="info-label">Host</span>
<span class="info-value">{{ server_hostname }}</span>
</div>
<div class="info-row">
<span class="info-label">Started</span>
<span class="info-value">{{ start_time_str }}</span>
</div>
<div class="info-row">
<span class="info-label">Uptime</span>
<span class="info-value" id="uptime-value">{{ uptime_str }}</span>
</div>
<div class="info-row">
<span class="info-label">Hosts monitored</span>
<span class="info-value">{{ host_count }}</span>
</div>
</div>
<div class="section">
<h2>Contact &amp; Source</h2>
<div class="info-row">
<span class="info-label">Author</span>
<span class="info-value">Andreas Wrede</span>
</div>
<div class="info-row">
<span class="info-label">Email</span>
<span class="info-value"><a href="mailto:aew.hbd@wrede.ca">aew.hbd@wrede.ca</a></span>
</div>
<div class="info-row">
<span class="info-label">Repository</span>
<span class="info-value"><a href="https://git.wrede.ca/andreas/heartbeat" target="_blank" rel="noopener">git.wrede.ca/andreas/heartbeat</a></span>
</div>
</div>
<div class="st-toolbar">
<span class="brand">hbd<span class="tld">·about</span></span>
<span class="hintline">heartbeat monitoring system</span>
</div>
<svg class="pulse" viewBox="0 0 1200 14" preserveAspectRatio="none" aria-hidden="true">
<polyline points="0,10 340,10 352,10 358,3 364,13 370,1 378,12 384,10 560,10 572,10 578,3 584,13 590,1 598,12 604,10 1200,10"/>
</svg>
<div class="about-main">
<div class="hero">
<span class="hb-logo">heart<span class="tld">beat</span></span>
<span class="version">v{{ hbd_version }}</span>
<span class="tagline">Lightweight host monitoring over UDP</span>
</div>
<section class="st">
<div class="sec-head">
<h2>version<span class="colon">:</span></h2>
</div>
<div class="list kv">
<div class="row">
<span class="id"><span class="name">server</span></span>
<span class="val">{{ hbd_version }}</span>
</div>
<div class="row">
<span class="id"><span class="name">python</span></span>
<span class="val">{{ python_version }}</span>
</div>
<div class="row">
<span class="id"><span class="name">license</span></span>
<span class="val">MIT</span>
</div>
</div>
</section>
<section class="st">
<div class="sec-head">
<h2>runtime<span class="colon">:</span></h2>
</div>
<div class="list kv">
<div class="row">
<span class="id"><span class="name">host</span></span>
<span class="val">{{ server_hostname }}</span>
</div>
<div class="row">
<span class="id"><span class="name">started</span></span>
<span class="val">{{ start_time_str }}</span>
</div>
<div class="row">
<span class="id"><span class="name">uptime</span></span>
<span class="val" id="uptime-value">{{ uptime_str }}</span>
</div>
<div class="row">
<span class="id"><span class="name">hosts_monitored</span></span>
<span class="val">{{ host_count }}</span>
</div>
</div>
</section>
<section class="st">
<div class="sec-head">
<h2>contact<span class="colon">:</span></h2>
</div>
<div class="list kv">
<div class="row">
<span class="id"><span class="name">author</span></span>
<span class="val">Andreas Wrede</span>
</div>
<div class="row">
<span class="id"><span class="name">email</span></span>
<span class="val"><a href="mailto:aew.hbd@wrede.ca">aew.hbd@wrede.ca</a></span>
</div>
<div class="row">
<span class="id"><span class="name">repository</span></span>
<span class="val"><a href="https://git.wrede.ca/andreas/heartbeat" target="_blank" rel="noopener">git.wrede.ca/andreas/heartbeat</a></span>
</div>
</div>
</section>
</div>
{% include 'foot.html' %}
<script>
(function() {
+142 -493
View File
@@ -1,376 +1,97 @@
<!DOCTYPE html>
<html>
{% include 'head.html' %}
<link rel="stylesheet" href="/static/hbd-ui.css">
<script src="/static/hbd-ui.js"></script>
<style>
html, body { height: auto; overflow: visible; }
body { background: var(--st-paper); padding: 60px 0 0; }
html, body {
height: auto;
overflow-y: auto;
}
.alerts-main { max-width: 1100px; margin: 0 auto; padding: 0 20px 60px; }
.container {
max-width: 1400px;
margin: 0 auto;
/* summary tiles */
.stats { display: flex; gap: 10px; flex-wrap: wrap; padding-top: 18px; }
.stat {
flex: 1 1 140px; display: flex; align-items: baseline; gap: 10px;
background: var(--st-surface); border: 1px solid var(--st-line); border-radius: 8px;
padding: 10px 14px;
}
.stat .num { font-family: var(--st-mono); font-size: 24px; font-weight: 700; font-variant-numeric: tabular-nums; }
.stat .lbl { font-family: var(--st-mono); font-size: 11.5px; color: var(--st-faint); }
.stat.crit .num { color: var(--st-crit); }
.stat.warn .num { color: var(--st-warn); }
.stat.ok .num { color: var(--st-ok); }
h1 { color: #333; margin-bottom: 5px; margin-top: 15px; font-size: 1.5em; }
/* filter bar */
.filterbar { display: flex; gap: 6px; align-items: center; flex-wrap: wrap; margin: 14px 0 0; }
.filterbar .fbtn {
font-family: var(--st-mono); font-size: 12px; cursor: pointer;
color: var(--st-muted); background: var(--st-surface); border: 1px solid var(--st-line);
border-radius: 999px; padding: 5px 12px;
}
.filterbar .fbtn.active { background: var(--st-accent-soft); border-color: var(--st-accent); color: var(--st-accent); font-weight: 600; }
.filterbar input {
font: 12px var(--st-mono); color: var(--st-ink);
background: var(--st-surface); border: 1px solid var(--st-line); border-radius: 999px;
padding: 5px 12px; min-width: 170px;
}
.filterbar input.invalid { border-color: var(--st-crit); }
.filterbar .upd { margin-left: auto; font-size: 11.5px; color: var(--st-faint); }
.subtitle {
color: #666;
margin-bottom: 30px;
}
/* alert row specifics */
.row.alert-crit { border-left: 3px solid var(--st-crit); padding-left: 11px; }
.row.alert-warn { border-left: 3px solid var(--st-warn); padding-left: 11px; }
.row.acked { opacity: .55; border-left-style: dashed; }
.row .id a { color: inherit; text-decoration: none; }
.row .id a:hover { color: var(--st-accent); }
.acked-chip { color: var(--st-ok); font-size: 12px; white-space: nowrap; }
.summary-cards {
display: flex;
flex-wrap: wrap;
gap: 10px;
margin-bottom: 16px;
.empty {
padding: 36px 14px; text-align: center; color: var(--st-faint); font-size: 13px;
}
.summary-card {
background: white;
border-radius: 6px;
padding: 6px 14px;
box-shadow: 0 1px 4px rgba(0,0,0,0.1);
display: flex;
align-items: center;
gap: 8px;
border-left: 4px solid #ddd;
.empty .big { font-size: 26px; color: var(--st-ok); display: block; margin-bottom: 6px; }
.errorbox {
background: var(--st-crit-soft); color: var(--st-crit);
border-radius: 8px; padding: 14px 16px; font-size: 13px;
}
.summary-card.critical { border-left-color: #ea1e0f; }
.summary-card.warning { border-left-color: #ff9800; }
.summary-card.ok { border-left-color: #4caf50; }
.summary-number {
font-size: 1.4em;
font-weight: bold;
line-height: 1;
}
.summary-number.critical { color: #ea1e0f; }
.summary-number.warning { color: #ff9800; }
.summary-number.ok { color: #4caf50; }
.summary-label {
color: #666;
font-size: 1.00em;
}
.filters {
background: white;
border-radius: 8px;
padding: 15px;
margin-bottom: 20px;
box-shadow: 0 2px 8px rgba(0,0,0,0.1);
display: flex;
gap: 15px;
align-items: center;
}
.filter-label {
font-weight: bold;
color: #555;
}
.filter-button {
padding: 8px 16px;
border: 2px solid #ddd;
background: white;
border-radius: 20px;
cursor: pointer;
transition: all 0.2s;
font-size: 0.9em;
}
.filter-button:hover {
border-color: #2196f3;
}
.filter-button.active {
background: #2196f3;
color: white;
border-color: #2196f3;
}
.filter-input {
padding: 7px 12px;
border: 2px solid #ddd;
border-radius: 20px;
font-size: 0.9em;
outline: none;
width: 200px;
transition: border-color 0.2s;
}
.filter-input:focus {
border-color: #2196f3;
}
.filter-input.invalid {
border-color: #f44336;
}
.alerts-container {
background: white;
border-radius: 8px;
padding: 20px;
box-shadow: 0 2px 8px rgba(0,0,0,0.1);
}
.alert-item {
border-left: 5px solid #ddd;
padding: 15px;
margin-bottom: 15px;
background: #fafafa;
border-radius: 4px;
display: flex;
justify-content: space-between;
align-items: center;
transition: all 0.2s;
}
.alert-item.acknowledged {
opacity: 0.8;
background: #f0f0f0;
}
.alert-item:hover {
box-shadow: 0 2px 8px rgba(0,0,0,0.1);
transform: translateX(5px);
}
.alert-item.critical {
border-left-color: #f44336;
background: #ffebee;
}
.alert-item.warning {
border-left-color: #ff9800;
background: #fff3e0;
}
.alert-item.unknown {
border-left-color: #9e9e9e;
background: #f5f5f5;
}
.alert-main {
flex: 1;
}
.alert-header {
display: flex;
align-items: center;
gap: 15px;
margin-bottom: 8px;
}
.alert-level {
padding: 4px 12px;
border-radius: 12px;
font-size: 0.75em;
font-weight: bold;
text-transform: uppercase;
letter-spacing: 0.5px;
}
.alert-level.critical {
background: #f44336;
color: white;
}
.alert-level.warning {
background: #ff9800;
color: white;
}
.alert-level.unknown {
background: #9e9e9e;
color: white;
}
.alert-hostname {
font-weight: bold;
color: #0066cc;
font-size: 1.1em;
text-decoration: none;
}
.alert-hostname:hover {
text-decoration: underline;
}
.alert-metric {
color: #0066cc;
font-size: 1.1em;
font-weight: normal;
}
.alert-details {
display: flex;
gap: 20px;
color: #666;
font-size: 0.9em;
}
.alert-value {
font-weight: bold;
color: #333;
}
.alert-duration {
color: #999;
font-size: 1.00em;
}
.alert-actions {
display: flex;
flex-direction: column;
gap: 8px;
margin-left: 15px;
}
.acknowledge-btn {
padding: 8px 16px;
background: #2196f3;
color: white;
border: none;
border-radius: 4px;
cursor: pointer;
font-size: 1.00em;
transition: all 0.2s;
white-space: nowrap;
}
.acknowledge-btn:hover {
background: #1976d2;
transform: scale(1.05);
}
.acknowledge-btn:disabled {
background: #ccc;
cursor: not-allowed;
transform: none;
}
.acknowledged-badge {
padding: 4px 8px;
background: #4caf50;
color: white;
border-radius: 4px;
font-size: 0.75em;
text-align: center;
white-space: nowrap;
}
.no-alerts {
text-align: center;
padding: 60px 20px;
color: #999;
}
.no-alerts-icon {
font-size: 4em;
margin-bottom: 20px;
}
.loading {
text-align: center;
padding: 40px;
color: #666;
}
.error {
background: #ffebee;
border-left: 4px solid #f44336;
padding: 20px;
margin: 20px 0;
border-radius: 4px;
color: #c62828;
}
.refresh-info {
text-align: center;
color: #999;
font-size: 1.00em;
margin-top: 20px;
padding-top: 20px;
border-top: 1px solid #e0e0e0;
}
.last-update {
color: #666;
font-size: 0.9em;
text-align: right;
margin-bottom: 15px;
}
/* ── Dark mode ── */
html[data-theme="dark"] h1 { color: var(--text); }
html[data-theme="dark"] .subtitle { color: var(--text-sec); }
html[data-theme="dark"] .summary-card { background: var(--surface); }
html[data-theme="dark"] .summary-label { color: var(--text-sec); }
html[data-theme="dark"] .filters { background: var(--surface); }
html[data-theme="dark"] .filter-label { color: var(--text-sec); }
html[data-theme="dark"] .filter-button { background: var(--surface-2); border-color: var(--border); color: var(--text); }
html[data-theme="dark"] .filter-button.active { background: #2196f3; color: #fff; border-color: #2196f3; }
html[data-theme="dark"] .filter-input { background: var(--input-bg); border-color: var(--input-border); color: var(--text); }
html[data-theme="dark"] .alerts-container { background: var(--surface); }
html[data-theme="dark"] .alert-item { background: var(--surface-2); }
html[data-theme="dark"] .alert-item.acknowledged { background: var(--surface-3); }
html[data-theme="dark"] .alert-item.critical { background: #2e0a0a; border-left-color: #f44336; }
html[data-theme="dark"] .alert-item.warning { background: #2e1a00; border-left-color: #ff9800; }
html[data-theme="dark"] .alert-item.unknown { background: var(--surface-2); }
html[data-theme="dark"] .alert-hostname { color: var(--link); }
html[data-theme="dark"] .alert-details { color: var(--text-sec); }
html[data-theme="dark"] .alert-value { color: var(--text); }
html[data-theme="dark"] .alert-duration { color: var(--text-muted); }
html[data-theme="dark"] .last-update { color: var(--text-sec); }
html[data-theme="dark"] .refresh-info { color: var(--text-muted); border-top-color: var(--border); }
html[data-theme="dark"] .no-alerts,
html[data-theme="dark"] .loading { color: var(--text-muted); }
</style>
<body>
{% include 'nav.html' %}
<div class="container">
<h1>{{ header }}</h1>
<p class="subtitle">Real-time monitoring alerts and threshold violations</p>
<div class="st-toolbar">
<span class="brand">hbd<span class="tld">·alerts</span></span>
<span class="hintline">threshold violations and reachability — refreshes every 15 s</span>
</div>
<svg class="pulse" viewBox="0 0 1200 14" preserveAspectRatio="none" aria-hidden="true">
<polyline points="0,10 340,10 352,10 358,3 364,13 370,1 378,12 384,10 560,10 572,10 578,3 584,13 590,1 598,12 604,10 1200,10"/>
</svg>
<div class="summary-cards" id="summary-cards">
<div class="summary-card critical">
<div class="summary-label">Critical</div>
<div class="summary-number critical" id="critical-count">-</div>
</div>
<div class="summary-card warning">
<div class="summary-label">Warning</div>
<div class="summary-number warning" id="warning-count">-</div>
</div>
<div class="summary-card ok">
<div class="summary-label">Total Hosts</div>
<div class="summary-number ok" id="host-count">-</div>
</div>
<div class="alerts-main">
<div class="stats">
<div class="stat crit"><span class="num" id="critical-count"></span><span class="lbl">critical</span></div>
<div class="stat warn"><span class="num" id="warning-count"></span><span class="lbl">warning</span></div>
<div class="stat ok"><span class="num" id="host-count"></span><span class="lbl">hosts</span></div>
</div>
<div class="filters">
<span class="filter-label">Show:</span>
<button class="filter-button active" onclick="filterAlerts('all')">All</button>
<button class="filter-button" onclick="filterAlerts('critical')">Critical Only</button>
<button class="filter-button" onclick="filterAlerts('warning')">Warning Only</button>
<input id="host-filter" class="filter-input" type="text" placeholder="host filter (regex)" oninput="onHostFilterInput(this)">
<div class="filterbar">
<button class="fbtn active" onclick="filterAlerts('all', this)">all</button>
<button class="fbtn" onclick="filterAlerts('critical', this)">critical</button>
<button class="fbtn" onclick="filterAlerts('warning', this)">warning</button>
<input id="host-filter" type="text" placeholder="host filter (regex)" oninput="onHostFilterInput(this)">
<span class="upd">updated <span id="last-update-time">never</span></span>
</div>
<div class="alerts-container">
<div class="last-update">Last updated: <span id="last-update-time">Never</span></div>
<div id="alerts-list">
<div class="loading">Loading alerts...</div>
<section class="st" id="alerts">
<div class="sec-head">
<h2>alerts<span class="colon">:</span></h2>
<span class="count" id="alert-count"></span>
</div>
<div class="refresh-info">
Auto-refreshing every 15 seconds
<div class="list" id="alerts-list">
<div class="empty">Loading alerts…</div>
</div>
</div>
</section>
</div>
<script>
@@ -378,207 +99,138 @@
let allAlerts = [];
let hostFilterRe = null;
function escHtml(s) {
return String(s).replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;').replace(/"/g,'&quot;').replace(/'/g,'&#39;');
}
async function loadAlerts() {
try {
const response = await fetch('/api/0/alerts');
if (!response.ok) {
throw new Error(`HTTP ${response.status}`);
}
if (!response.ok) throw new Error(`HTTP ${response.status}`);
const data = await response.json();
allAlerts = data.alerts;
// Update summary cards
document.getElementById('critical-count').textContent = data.summary.critical || 0;
document.getElementById('warning-count').textContent = data.summary.warning || 0;
document.getElementById('host-count').textContent = data.host_count || 0;
// Update last update time
document.getElementById('last-update-time').textContent = new Date().toLocaleTimeString();
// Render alerts
renderAlerts(allAlerts);
} catch (error) {
document.getElementById('alerts-list').innerHTML =
`<div class="error">Failed to load alerts: ${error.message}</div>`;
document.getElementById('alerts-list').innerHTML =
`<div class="errorbox">Failed to load alerts: ${escHtml(error.message)}. Retrying automatically.</div>`;
}
}
function renderAlerts(alerts) {
const container = document.getElementById('alerts-list');
// Filter alerts based on current filter
let filteredAlerts = alerts;
let filtered = alerts;
if (currentFilter !== 'all') {
filteredAlerts = filteredAlerts.filter(alert =>
alert.level.toLowerCase() === currentFilter
);
filtered = filtered.filter(a => a.level.toLowerCase() === currentFilter);
}
if (hostFilterRe) {
filteredAlerts = filteredAlerts.filter(alert => hostFilterRe.test(alert.hostname));
filtered = filtered.filter(a => hostFilterRe.test(a.hostname));
}
if (filteredAlerts.length === 0) {
if (currentFilter === 'all' && alerts.length === 0) {
container.innerHTML = `
<div class="no-alerts">
<div class="no-alerts-icon">✓</div>
<h2>All Systems Normal</h2>
<p>No active alerts at this time</p>
</div>
`;
} else {
container.innerHTML = `
<div class="no-alerts">
<p>No ${currentFilter} alerts</p>
</div>
`;
}
document.getElementById('alert-count').textContent =
filtered.length === alerts.length ? alerts.length : `${filtered.length} of ${alerts.length}`;
if (filtered.length === 0) {
container.innerHTML = (currentFilter === 'all' && !hostFilterRe && alerts.length === 0)
? `<div class="empty"><span class="big">✓</span>All systems normal — no active alerts.</div>`
: `<div class="empty">No matching alerts.</div>`;
return;
}
let html = '';
for (const alert of filteredAlerts) {
html += renderAlert(alert);
}
container.innerHTML = html;
container.innerHTML = filtered.map(renderAlert).join('');
}
function renderAlert(alert) {
const level = alert.level.toLowerCase();
const cls = level === 'critical' ? 'crit' : (level === 'warning' ? 'warn' : 'level');
const duration = getDuration(alert.since);
const acknowledged = alert.acknowledged || false;
// Use formatted message if available, otherwise build from individual fields
let valueText = `Value: <span class="alert-value">${formatValue(alert.last_value)}</span>`;
const acked = alert.acknowledged || false;
const metric = (alert.metric_path.includes('.')
? alert.metric_path.slice(alert.metric_path.indexOf('.') + 1)
: alert.metric_path).replace(/_status_code$/, '');
let chips = `<span class="chip ${cls}">${escHtml(alert.level)}</span>`;
chips += `<span class="chip"><span class="k">metric</span> ${escHtml(metric)}</span>`;
if (alert.formatted_message) {
valueText += ` <span class="threshold-info">${alert.formatted_message}</span>`;
} else if (alert.threshold_value !== undefined && alert.threshold_value !== null && alert.operator) {
valueText += ` <span class="threshold-info">(threshold: ${alert.operator} ${formatValue(alert.threshold_value)})</span>`;
chips += `<span class="chip">${escHtml(alert.formatted_message)}</span>`;
} else {
chips += `<span class="chip"><span class="k">value</span> ${escHtml(formatValue(alert.last_value))}</span>`;
if (alert.threshold_value !== undefined && alert.threshold_value !== null && alert.operator) {
chips += `<span class="chip level">${escHtml(alert.operator)} ${escHtml(formatValue(alert.threshold_value))}</span>`;
}
}
if (alert.recovery_threshold !== undefined && alert.recovery_threshold !== null) {
const recOp = (alert.operator === '>' || alert.operator === '>=') ? '<' : '>';
valueText += ` <span class="threshold-info" style="color:#888">(recovers ${recOp} ${formatValue(alert.recovery_threshold)})</span>`;
chips += `<span class="chip level">recovers ${recOp} ${escHtml(formatValue(alert.recovery_threshold))}</span>`;
}
// Build actions section
let actionsHtml = '';
if (acknowledged) {
actionsHtml = `
<div class="alert-actions">
<div class="acknowledged-badge">✓ Acknowledged</div>
</div>
`;
} else {
actionsHtml = `
<div class="alert-actions">
<button class="acknowledge-btn" onclick="acknowledgeAlert('${alert.hostname}', '${alert.metric_path}', event)">
Acknowledge
</button>
</div>
`;
}
chips += `<span class="chip"><span class="k">for</span> ${duration}</span>`;
const act = acked
? `<span class="acked-chip">✓ acknowledged</span>`
: `<button class="btn small" onclick="acknowledgeAlert('${escHtml(alert.hostname)}', '${escHtml(alert.metric_path)}', event)">Acknowledge</button>`;
return `
<div class="alert-item ${level} ${acknowledged ? 'acknowledged' : ''}">
<div class="alert-main">
<div class="alert-header">
<span class="alert-level ${level}">${alert.level}</span>
<a class="alert-hostname" href="/plugins#${alert.hostname}">${alert.hostname}</a>
<span class="alert-metric">${(alert.metric_path.includes('.') ? alert.metric_path.slice(alert.metric_path.indexOf('.') + 1) : alert.metric_path).replace(/_status_code$/, '')}</span>
</div>
<div class="alert-details">
<span>${valueText}</span>
<span class="alert-duration">Active for ${duration}</span>
</div>
</div>
${actionsHtml}
</div>
`;
<div class="row alert-${cls === 'level' ? 'warn' : cls}${acked ? ' acked' : ''}">
<span class="id">
<span class="dot ${cls === 'crit' ? 'crit' : 'warn'}"></span>
<a class="name" href="/plugins#${encodeURIComponent(alert.hostname)}">${escHtml(alert.hostname)}</a>
</span>
<span class="facts">${chips}</span>
<span class="act">${act}</span>
</div>`;
}
function formatValue(value) {
if (typeof value === 'number') {
if (value > 1000) {
return value.toLocaleString();
}
if (value > 1000) return value.toLocaleString();
return value.toFixed(2);
}
return value;
}
function getDuration(timestamp) {
const now = Date.now() / 1000;
const seconds = Math.floor(now - timestamp);
if (seconds < 60) {
return `${seconds}s`;
} else if (seconds < 3600) {
return `${Math.floor(seconds / 60)}m`;
} else if (seconds < 86400) {
const hours = Math.floor(seconds / 3600);
const minutes = Math.floor((seconds % 3600) / 60);
return `${hours}h ${minutes}m`;
} else {
const days = Math.floor(seconds / 86400);
const hours = Math.floor((seconds % 86400) / 3600);
return `${days}d ${hours}h`;
const seconds = Math.floor(Date.now() / 1000 - timestamp);
if (seconds < 60) return `${seconds}s`;
if (seconds < 3600) return `${Math.floor(seconds / 60)}m`;
if (seconds < 86400) {
return `${Math.floor(seconds / 3600)}h ${Math.floor((seconds % 3600) / 60)}m`;
}
return `${Math.floor(seconds / 86400)}d ${Math.floor((seconds % 86400) / 3600)}h`;
}
function filterAlerts(filter) {
function filterAlerts(filter, btn) {
currentFilter = filter;
// Update active button
document.querySelectorAll('.filter-button').forEach(btn => {
btn.classList.remove('active');
});
event.target.classList.add('active');
// Re-render with new filter
document.querySelectorAll('.filterbar .fbtn').forEach(b => b.classList.remove('active'));
btn.classList.add('active');
renderAlerts(allAlerts);
}
async function acknowledgeAlert(hostname, metricPath, event) {
// Prevent event bubbling
if (event) {
event.stopPropagation();
}
// Disable the button
if (event) event.stopPropagation();
const button = event.target;
button.disabled = true;
button.textContent = 'Acknowledging...';
button.textContent = 'Acknowledging';
try {
const response = await fetch('/api/0/alerts/acknowledge', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
hostname: hostname,
metric_path: metricPath,
}),
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({hostname: hostname, metric_path: metricPath}),
});
if (!response.ok) {
throw new Error(`HTTP ${response.status}`);
}
if (!response.ok) throw new Error(`HTTP ${response.status}`);
const result = await response.json();
// Update the alert in our local data
const alert = allAlerts.find(a => a.hostname === hostname && a.metric_path === metricPath);
if (alert) {
alert.acknowledged = true;
alert.acknowledged_at = result.acknowledged_at;
const a = allAlerts.find(x => x.hostname === hostname && x.metric_path === metricPath);
if (a) {
a.acknowledged = true;
a.acknowledged_at = result.acknowledged_at;
}
// Re-render alerts
renderAlerts(allAlerts);
} catch (error) {
alert(`Failed to acknowledge alert: ${error.message}`);
button.disabled = false;
@@ -603,10 +255,8 @@
renderAlerts(allAlerts);
}
// Auto-refresh every 15 seconds
setInterval(loadAlerts, 15000);
// Initialise filter from URL query string (?filter=...)
(function () {
const param = new URLSearchParams(window.location.search).get('filter');
if (param) {
@@ -616,7 +266,6 @@
}
})();
// Initial load
loadAlerts();
</script>
</body>
File diff suppressed because it is too large Load Diff
+228
View File
@@ -0,0 +1,228 @@
<!DOCTYPE html>
<html>
{% include 'head.html' %}
<link rel="stylesheet" href="/static/hbd-ui.css">
<script src="/static/hbd-ui.js"></script>
<style>
html, body { height: auto; overflow: visible; }
body { background: var(--st-paper); padding: 60px 0 0; }
.log-main { max-width: 1100px; margin: 0 auto; padding: 0 20px 60px; }
.st-toolbar input, .st-toolbar select {
font: 12px var(--st-mono); color: var(--st-ink);
background: var(--st-surface); border: 1px solid var(--st-line); border-radius: 999px;
padding: 5px 12px;
}
.live-dot { display: inline-flex; align-items: center; gap: 6px; font-size: 12px; font-weight: 600; color: var(--st-ok); }
.live-dot::before { content: ""; width: 7px; height: 7px; border-radius: 50%; background: var(--st-ok); }
.live-dot.reconnecting { color: var(--st-crit); }
.live-dot.reconnecting::before { background: var(--st-crit); }
.logrow {
display: grid; grid-template-columns: 130px 74px 150px minmax(0,1fr);
gap: 4px 14px; align-items: baseline;
padding: 4px 14px; border-bottom: 1px solid var(--st-line-soft);
font-family: var(--st-mono); font-size: 11.5px;
}
.logrow:last-child { border-bottom: none; }
.logrow .t { color: var(--st-faint); font-variant-numeric: tabular-nums; white-space: nowrap; }
.logrow .h { color: var(--st-ink); font-weight: 600; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.logrow .m { color: var(--st-muted); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.empty { padding: 24px 14px; text-align: center; color: var(--st-faint); font-size: 12.5px; }
.loadmore {
display: block; margin: 14px auto 0; cursor: pointer;
font-family: var(--st-mono); font-size: 12px; color: var(--st-muted);
background: var(--st-surface); border: 1px solid var(--st-line); border-radius: 999px;
padding: 6px 16px;
}
.loadmore:hover { color: var(--st-accent); border-color: var(--st-accent); }
.loadmore:disabled { opacity: .5; cursor: default; }
@media (max-width: 760px) {
.logrow { grid-template-columns: 66px minmax(0,1fr); }
.logrow .lvlc, .logrow .h { display: none; }
}
</style>
<body>
{% include 'nav.html' %}
<div class="st-toolbar">
<span class="brand">hbd<span class="tld">·log</span></span>
<span class="hintline">connectivity and alert events — full journal history</span>
<span class="spacer"></span>
<input type="text" id="filter-host" placeholder="host">
<select id="filter-level">
<option value="">level</option>
<option value="info">INFO</option>
<option value="warning">WARNING</option>
<option value="critical">CRITICAL</option>
<option value="recover">RECOVER</option>
<option value="unknown">UNKNOWN</option>
</select>
<input type="text" id="filter-msg" placeholder="message">
<span class="live-dot reconnecting" id="ws-status">connecting…</span>
</div>
<svg class="pulse" viewBox="0 0 1200 14" preserveAspectRatio="none" aria-hidden="true">
<polyline points="0,10 340,10 352,10 358,3 364,13 370,1 378,12 384,10 560,10 572,10 578,3 584,13 590,1 598,12 604,10 1200,10"/>
</svg>
<div class="log-main">
<section class="st">
<div class="sec-head">
<h2>log<span class="colon">:</span></h2>
<span class="count" id="log-count"></span>
</div>
<div class="list" id="messages">
<div class="empty">Loading events…</div>
</div>
<button class="loadmore" id="load-older" style="display:none">load older</button>
</section>
</div>
<script>
var PAGE_SIZE = 200;
var oldestTs = null; // ts of the oldest loaded row (pagination cursor)
var seq = 0; // guards against out-of-order fetch responses
var newestTs = null; // ts of the newest loaded row (dedup guard for the WS tail)
function escHtml(s) {
return String(s).replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;').replace(/"/g,'&quot;').replace(/'/g,'&#39;');
}
function fmtClock(ts) {
var d = new Date(ts * 1000);
function p(n) { return n < 10 ? '0' + n : '' + n; }
var t = p(d.getHours()) + ':' + p(d.getMinutes()) + ':' + p(d.getSeconds());
return (d.toDateString() === new Date().toDateString())
? t
: d.getFullYear() + '-' + p(d.getMonth() + 1) + '-' + p(d.getDate()) + ' ' + t;
}
function logRowHtml(msg) {
var lvl = (msg.level || 'INFO');
var lc = lvl.toLowerCase();
var chipCls = lc === 'critical' ? 'crit' : (lc === 'warning' ? 'warn' : (lc === 'recover' ? 'ok' : ''));
var short = lc === 'critical' ? 'CRIT' : (lc === 'warning' ? 'WARN' : escHtml(lvl));
return '<div class="logrow">'
+ '<span class="t">' + fmtClock(msg.ts) + '</span>'
+ '<span class="lvlc"><span class="chip ' + chipCls + '">' + short + '</span></span>'
+ '<span class="h">' + escHtml(msg.host || '') + (msg.service ? ' · ' + escHtml(msg.service) : '') + '</span>'
+ '<span class="m">' + escHtml(msg.message || '') + '</span>'
+ '</div>';
}
function currentFilters() {
return {
host: document.getElementById('filter-host').value.trim(),
level: document.getElementById('filter-level').value,
q: document.getElementById('filter-msg').value.trim()
};
}
function matchesFilters(msg) {
var f = currentFilters();
if (f.host && !(msg.host || '').toLowerCase().includes(f.host.toLowerCase())) return false;
if (f.level && (msg.level || '').toLowerCase() !== f.level) return false;
if (f.q && !(msg.message || '').toLowerCase().includes(f.q.toLowerCase())) return false;
return true;
}
function updateCount() {
var n = document.querySelectorAll('#messages .logrow').length;
document.getElementById('log-count').textContent = n ? n + ' events loaded' : '';
}
async function fetchLog(reset) {
var mySeq = ++seq;
var box = document.getElementById('messages');
var btn = document.getElementById('load-older');
btn.disabled = true;
var f = currentFilters();
var params = new URLSearchParams();
params.set('limit', String(PAGE_SIZE));
if (f.host) params.set('host', f.host);
if (f.level) params.set('level', f.level);
if (f.q) params.set('q', f.q);
if (!reset && oldestTs != null) params.set('before', String(oldestTs));
var data;
try {
var r = await fetch('/api/0/log?' + params.toString());
if (!r.ok) throw new Error('HTTP ' + r.status);
data = await r.json();
} catch (e) {
if (mySeq === seq && reset) {
box.innerHTML = '<div class="empty">Failed to load events: ' + escHtml(e.message) + '</div>';
}
btn.disabled = false;
return;
}
if (mySeq !== seq) return; // a newer request superseded this one
if (reset) { box.innerHTML = ''; oldestTs = null; }
var html = '';
data.events.forEach(function (ev) {
html += logRowHtml(ev);
if (ev.ts != null && (oldestTs == null || ev.ts < oldestTs)) oldestTs = ev.ts;
if (ev.ts != null && (newestTs == null || ev.ts > newestTs)) newestTs = ev.ts;
});
box.insertAdjacentHTML('beforeend', html);
if (!box.children.length) box.innerHTML = '<div class="empty">No events found.</div>';
btn.style.display = data.more ? '' : 'none';
btn.disabled = false;
updateCount();
}
function onFilterChange() { fetchLog(true); }
function debounce(fn, ms) {
var t;
return function () { clearTimeout(t); t = setTimeout(fn, ms); };
}
document.getElementById('filter-host').addEventListener('input', debounce(onFilterChange, 300));
document.getElementById('filter-msg').addEventListener('input', debounce(onFilterChange, 300));
document.getElementById('filter-level').addEventListener('change', onFilterChange);
document.getElementById('load-older').addEventListener('click', function () { fetchLog(false); });
// ---- live tail over the shared websocket --------------------------------
function setWsStatus(ok) {
var el = document.getElementById('ws-status');
el.classList.toggle('reconnecting', !ok);
el.textContent = ok ? 'live' : 'reconnecting…';
}
function WS_Connect() {
if (!("WebSocket" in window)) return;
var ws_hbd = new WebSocket("{{ heartbeat_ws_url }}");
ws_hbd.onopen = function () {
setWsStatus(true);
ws_hbd.send("heartbeat_web");
};
ws_hbd.onmessage = function (event) {
var state = JSON.parse(event.data);
// history replays are covered by the API seed; only tail new events
if (state.type !== "message" || state.history) return;
var msg = state.data;
if (!matchesFilters(msg)) return;
if (msg.ts != null && newestTs != null && msg.ts <= newestTs) return; // already loaded via the API
if (msg.ts != null && (newestTs == null || msg.ts > newestTs)) newestTs = msg.ts;
var box = document.getElementById('messages');
var placeholder = box.querySelector('.empty');
if (placeholder) placeholder.remove();
box.insertAdjacentHTML('afterbegin', logRowHtml(msg));
updateCount();
};
ws_hbd.onclose = function () {
setWsStatus(false);
setTimeout(WS_Connect, 3000);
};
}
// ---- boot ----------------------------------------------------------------
fetchLog(true);
WS_Connect();
</script>
</body>
</html>
-2
View File
@@ -1,2 +0,0 @@
<!-- <label for="drawer-toggle" id="drawer-toggle-label"></label>
s<header>{{ header }}</header> -->
+3 -2
View File
@@ -6,12 +6,13 @@
<a href="/live"{% if active_page == "live" %} class="active"{% endif %}>Live Dashboard</a>
<a href="/plugins"{% if active_page == "plugins" %} class="active"{% endif %}>Host Overview</a>
<a href="/alerts"{% if active_page == "alerts" %} class="active"{% endif %}>Alerts</a>
{% if current_user and current_user.admin %}
<a href="/log"{% if active_page == "log" %} class="active"{% endif %}>Log</a>
{% if current_user %}
<a href="/settings"{% if active_page == "settings" %} class="active"{% endif %}>Settings</a>
{% endif %}
<a href="/about"{% if active_page == "about" %} class="active"{% endif %}>About</a>
</div>
{% if current_user and current_user.admin %}
{% if current_user %}
<button id="nav-publish-btn" class="nav-publish-btn" onclick="navPublishConfig()" style="display:none" title="Publish pending config changes to .hb.yaml">&#9888; Publish Config</button>
{% endif %}
<div class="nav-pie" title="Host alert status">
+317 -369
View File
@@ -2,38 +2,30 @@
<html>
{% include 'head.html' %}
<link rel="stylesheet" href="/static/hbd-ui.css">
<script src="/static/hbd-ui.js"></script>
<style>
body { overflow: hidden; }
body { overflow: hidden; background: var(--st-paper); }
/* This page scrolls inside .container, not the body — the toolbar is
always in view, and sticky misbehaves inside an overflow:hidden body. */
.st-toolbar { position: static; }
.container {
max-width: 1400px;
margin: 0 auto;
max-height: calc(100vh - 120px);
max-height: calc(100vh - var(--bars-h, 94px) - 30px);
overflow-y: auto;
padding-right: 10px;
padding: 0 10px 20px;
}
h1 {
color: #333;
margin-bottom: 5px;
margin-top: 15px;
font-size: 1.5em;
}
.subtitle {
color: #666;
margin-bottom: 15px;
font-size: 0.9em;
}
/* ── Host cards ─────────────────────────────────────────────── */
/* ── Host cards: one record list per host ───────────────────── */
.host-card {
background: white;
border-radius: 6px;
padding: 0;
margin-bottom: 10px;
box-shadow: 0 1px 4px rgba(0,0,0,0.1);
background: var(--st-surface);
border: 1px solid var(--st-line);
border-radius: 8px;
margin-bottom: 8px;
}
.host-header {
@@ -42,115 +34,74 @@
gap: 10px;
cursor: pointer;
user-select: none;
padding: 10px 15px;
border-radius: 6px;
padding: 8px 14px;
border-radius: 8px;
}
.host-header:hover { background: var(--st-surface-2); border-radius: 8px 8px 0 0; }
.host-card.collapsed .host-header:hover { border-radius: 8px; }
.host-header:hover { background: #f9f9f9; border-radius: 6px 6px 0 0; }
.host-card.collapsed .host-header:hover { border-radius: 6px; }
.host-left {
display: flex;
align-items: center;
gap: 8px;
flex-shrink: 0;
}
.host-left { display: flex; align-items: center; gap: 8px; flex-shrink: 0; }
.collapse-icon {
font-size: 1em;
color: #888;
font-size: .85em;
color: var(--st-faint);
transition: transform 0.2s;
min-width: 16px;
min-width: 14px;
}
@media (prefers-reduced-motion: reduce) { .collapse-icon, .acc-icon { transition: none; } }
.host-card.collapsed .collapse-icon { transform: rotate(-90deg); }
.host-name {
font-size: 1.05em;
font-weight: bold;
color: #333;
font-family: var(--st-mono);
font-size: 13.5px;
font-weight: 600;
color: var(--st-ink);
white-space: nowrap;
}
/* ── Glance strip ───────────────────────────────────────────── */
.glance-strip {
display: flex;
align-items: center;
gap: 8px;
flex: 1;
flex-wrap: wrap;
padding: 0 12px;
}
.glance-strip { display: flex; align-items: center; gap: 5px; flex: 1; flex-wrap: wrap; padding: 0 8px; }
.glance-chip {
font-size: 0.78em;
padding: 2px 9px;
border-radius: 10px;
font-weight: 500;
white-space: nowrap;
background: #e8f5e9;
color: #2e7d32;
display: inline-flex; align-items: center;
font-family: var(--st-mono); font-size: 11px; line-height: 1.3;
padding: 3px 7px; border-radius: 4px; white-space: nowrap;
background: var(--st-ok-soft); color: var(--st-ok);
}
.glance-chip.warn { background: #fff3e0; color: #e65100; }
.glance-chip.crit { background: #ffebee; color: #b71c1c; }
.glance-chip.neutral { background: #f5f5f5; color: #555; }
.glance-loading { font-size: 0.8em; color: #bbb; font-style: italic; }
.glance-chip.warn { background: var(--st-warn-soft); color: var(--st-warn); }
.glance-chip.crit { background: var(--st-crit-soft); color: var(--st-crit); }
.glance-chip.neutral { background: var(--st-surface-2); color: var(--st-muted); }
.glance-loading { font-size: 11.5px; color: var(--st-faint); font-style: italic; }
/* ── Host right zone ────────────────────────────────────────── */
.host-right {
display: flex;
align-items: center;
gap: 10px;
flex-shrink: 0;
}
.host-right { display: flex; align-items: center; gap: 8px; flex-shrink: 0; }
.nagios-badge {
font-size: 0.75em;
font-weight: bold;
padding: 2px 10px;
border-radius: 10px;
background: #9e9e9e;
color: white;
text-transform: uppercase;
white-space: nowrap;
font-family: var(--st-mono); font-size: 10.5px; font-weight: 700;
padding: 3px 9px; border-radius: 4px;
background: var(--st-surface-2); color: var(--st-muted);
text-transform: uppercase; white-space: nowrap;
}
.nagios-badge.ok { background: #4caf50; }
.nagios-badge.warning { background: #ff9800; }
.nagios-badge.critical { background: #f44336; }
.nagios-badge.ok { background: var(--st-ok-soft); color: var(--st-ok); }
.nagios-badge.warning { background: var(--st-warn-soft); color: var(--st-warn); }
.nagios-badge.critical { background: var(--st-crit-soft); color: var(--st-crit); }
.os-label {
font-size: 0.75em;
color: #999;
white-space: nowrap;
max-width: 200px;
overflow: hidden;
text-overflow: ellipsis;
font-size: 11px; color: var(--st-faint); white-space: nowrap;
max-width: 200px; overflow: hidden; text-overflow: ellipsis;
}
.host-action-btn {
font-size: 0.75em;
font-weight: bold;
padding: 3px 10px;
border-radius: 4px;
border: none;
cursor: pointer;
text-decoration: none;
white-space: nowrap;
font: 600 12px/1 inherit;
padding: 4px 10px; border-radius: 6px;
border: 1px solid var(--st-line);
background: var(--st-surface);
cursor: pointer; text-decoration: none; white-space: nowrap;
}
.host-action-btn.update-btn {
background: #e3f2fd;
color: #1565c0;
}
.host-action-btn.update-btn:hover { background: #bbdefb; }
.host-action-btn.delete-btn {
background: #ffebee;
color: #c62828;
}
.host-action-btn.delete-btn:hover { background: #ffcdd2; }
.host-action-btn.update-btn { color: var(--st-accent); }
.host-action-btn.update-btn:hover { background: var(--st-accent-soft); border-color: var(--st-accent); }
.host-action-btn.delete-btn { color: var(--st-crit); }
.host-action-btn.delete-btn:hover { background: var(--st-crit-soft); border-color: var(--st-crit); }
/* ── Action result toast ───────────────────────────────────── */
#action-toast {
@@ -158,8 +109,8 @@
bottom: 24px;
left: 50%;
transform: translateX(-50%) translateY(20px);
background: #323232;
color: #fff;
background: var(--st-ink);
color: var(--st-paper);
padding: 12px 22px;
border-radius: 6px;
font-size: 0.9em;
@@ -171,301 +122,176 @@
z-index: 9000;
white-space: pre-wrap;
}
#action-toast.show {
opacity: 1;
transform: translateX(-50%) translateY(0);
}
#action-toast.error { background: #c62828; }
#action-toast.show { opacity: 1; transform: translateX(-50%) translateY(0); }
#action-toast.error { background: var(--st-crit); color: #fff; }
/* ── Host body ──────────────────────────────────────────────── */
.host-body {
padding: 8px 15px 12px;
border-top: 1px solid #f0f0f0;
}
.host-body { padding: 8px 14px 12px; border-top: 1px solid var(--st-line-soft); }
.host-card.collapsed .host-body { display: none; }
/* ── Plugin accordions ──────────────────────────────────────── */
.plugin-accordion {
border: 1px solid #e8e8e8;
border-radius: 4px;
border: 1px solid var(--st-line-soft);
border-radius: 6px;
margin-bottom: 5px;
overflow: hidden;
}
.plugin-acc-header {
display: flex;
align-items: center;
gap: 10px;
padding: 7px 12px;
cursor: pointer;
background: #fafafa;
display: flex; align-items: center; gap: 10px;
padding: 6px 12px; cursor: pointer;
background: var(--st-surface-2);
user-select: none;
}
.plugin-acc-header:hover { background: var(--st-accent-soft); }
.plugin-acc-header:hover { background: #f0f4ff; }
.acc-icon {
font-size: 0.7em;
color: #999;
transition: transform 0.15s;
min-width: 12px;
}
.acc-icon { font-size: 0.7em; color: var(--st-faint); transition: transform 0.15s; min-width: 12px; }
.plugin-accordion:not(.collapsed) .acc-icon { transform: rotate(90deg); }
.plugin-label {
font-weight: 600;
font-size: 1.00em;
color: #444;
font-family: var(--st-mono);
font-weight: 600; font-size: 12.5px;
color: var(--st-ink);
min-width: 140px;
}
.plugin-summary {
font-size: 0.82em;
color: #888;
flex: 1;
}
.plugin-summary { font-family: var(--st-mono); font-size: 11.5px; color: var(--st-faint); flex: 1; }
.plugin-accordion.collapsed .plugin-acc-body { display: none; }
.plugin-acc-body { padding: 10px 12px; }
/* ── Tables ─────────────────────────────────────────────────── */
.data-table {
width: 100%;
border-collapse: collapse;
font-size: 1.00em;
background: #fff;
box-shadow: 0 1px 3px rgba(0,0,0,0.08);
border-radius: 4px;
font-size: 12.5px;
background: var(--st-surface);
border: 1px solid var(--st-line-soft);
border-radius: 6px;
overflow: hidden;
margin-bottom: 10px;
font-variant-numeric: tabular-nums;
}
.data-table thead { background: #2196f3; color: white; }
.data-table thead { background: var(--st-surface-2); }
.data-table th {
padding: 7px 10px;
text-align: left;
font-weight: 600;
text-transform: uppercase;
font-size: 0.75em;
letter-spacing: 0.4px;
font-family: var(--st-mono);
font-size: 10.5px; text-transform: uppercase; letter-spacing: .05em;
color: var(--st-faint); font-weight: 600;
text-align: left; padding: 5px 8px;
border-bottom: 1px solid var(--st-line);
}
.data-table th.num { text-align: right; }
.data-table th.center { text-align: center; }
.data-table td {
/* padding: 6px 10px; */
border-top: 1px solid #e8e8e8;
color: #333;
padding: 4px 8px;
border-top: 1px solid var(--st-line-soft);
color: var(--st-ink);
}
.data-table td.num {
text-align: right;
font-family: 'Courier New', monospace;
font-size: 0.95em;
}
.data-table td.num { text-align: right; font-family: var(--st-mono); }
.data-table td.center { text-align: center; }
.data-table td.key { color: #666; font-weight: 500; width: 38%; }
.data-table td.key { color: var(--st-muted); font-weight: 500; width: 38%; }
.data-table tbody tr:hover { background: var(--st-surface-2); }
.data-table tbody tr:nth-child(even) { background: #fafafa; }
.data-table tbody tr:hover { background: #f0f4ff; }
.iface-name { font-weight: bold; color: #2196f3; }
.iface-name { font-family: var(--st-mono); font-weight: 600; color: var(--st-accent); }
/* ── Percent bars ───────────────────────────────────────────── */
.bar-wrap {
display: flex;
align-items: center;
gap: 8px;
}
.bar-wrap { display: flex; align-items: center; gap: 8px; min-width: 120px; }
.bar-track {
display: inline-block;
width: 70px;
height: 6px;
background: #e0e0e0;
border-radius: 3px;
vertical-align: middle;
flex-shrink: 0;
flex: 1; height: 7px; border-radius: 4px;
background: var(--st-line-soft); overflow: hidden; min-width: 60px;
}
.bar-fill {
height: 6px;
border-radius: 3px;
background: #4caf50;
max-width: 100%;
}
.bar-fill.warn { background: #ff9800; }
.bar-fill.crit { background: #f44336; }
.bar-fill { height: 100%; border-radius: 4px; background: var(--st-ok); }
.bar-fill.warn { background: var(--st-warn); }
.bar-fill.crit { background: var(--st-crit); }
/* ── Disk two-table layout ──────────────────────────────────── */
.flex-tables {
display: flex;
gap: 14px;
flex-wrap: wrap;
}
.flex-tables { display: flex; gap: 14px; flex-wrap: wrap; }
.flex-tables > div { flex: 1 1 380px; }
.table-section-label {
font-size: 0.78em;
font-weight: 600;
text-transform: uppercase;
color: #888;
letter-spacing: 0.4px;
margin-bottom: 4px;
font-family: var(--st-mono);
font-size: 11px; font-weight: 600; color: var(--st-faint);
text-transform: uppercase; letter-spacing: .05em;
margin: 4px 0 6px;
}
/* ── Status / misc ──────────────────────────────────────────── */
.status-up { color: var(--st-ok); font-weight: bold; }
.status-down { color: var(--st-crit); font-weight: bold; }
.status-up { color: #4caf50; font-weight: bold; }
.status-down { color: #f44336; font-weight: bold; }
.pct-ok { color: var(--st-ok); font-weight: bold; }
.pct-warn { color: var(--st-warn); font-weight: bold; }
.pct-crit { color: var(--st-crit); font-weight: bold; }
.pct-ok { color: #2e7d32; font-weight: bold; }
.pct-warn { color: #e65100; font-weight: bold; }
.pct-crit { color: #b71c1c; font-weight: bold; }
.check-ok { background: var(--st-ok-soft); }
.check-warning { background: var(--st-warn-soft); }
.check-critical { background: var(--st-crit-soft); }
.check-unknown { background: var(--st-surface-2); }
.check-ok { background: #e8f5e9; }
.check-warning { background: #fff3e0; }
.check-critical { background: #ffebee; }
.check-unknown { background: #f5f5f5; }
.check-status-ok { color: var(--st-ok); font-weight: bold; }
.check-status-warning { color: var(--st-warn); font-weight: bold; }
.check-status-critical { color: var(--st-crit); font-weight: bold; }
.check-status-unknown { color: var(--st-faint); font-weight: bold; }
.check-status-ok { color: #2e7d32; font-weight: bold; }
.check-status-warning { color: #e65100; font-weight: bold; }
.check-status-critical { color: #b71c1c; font-weight: bold; }
.check-status-unknown { color: #777; font-weight: bold; }
.check-output { font-size: 0.9em; color: #555; }
.check-output { font-size: 0.9em; color: var(--st-muted); }
.timestamp {
color: #bbb;
font-size: 0.75em;
margin-top: 8px;
padding-top: 6px;
border-top: 1px solid #f0f0f0;
text-align: right;
}
.no-data {
text-align: center;
padding: 20px;
color: #aaa;
font-style: italic;
font-size: 0.9em;
}
.loading {
text-align: center;
padding: 12px;
color: #aaa;
font-size: 1.00em;
font-size: 11px; color: var(--st-faint);
margin-top: 6px; padding-top: 6px;
border-top: 1px solid var(--st-line-soft);
}
.no-data { text-align: center; padding: 40px; color: var(--st-faint); }
.loading { text-align: center; padding: 12px; color: var(--st-faint); font-style: italic; }
.error {
background: #ffebee;
border-left: 3px solid #f44336;
padding: 8px 12px;
margin: 8px 0;
border-radius: 3px;
color: #c62828;
font-size: 1.00em;
background: var(--st-crit-soft);
border-left: 3px solid var(--st-crit);
padding: 12px 16px; margin: 10px 0;
border-radius: 6px; color: var(--st-crit);
}
/* ── Scrollbar ──────────────────────────────────────────────── */
.container::-webkit-scrollbar { width: 8px; }
.container::-webkit-scrollbar-track { background: #f1f1f1; border-radius: 4px; }
.container::-webkit-scrollbar-thumb { background: #ccc; border-radius: 4px; }
.container::-webkit-scrollbar-thumb:hover { background: #999; }
.container::-webkit-scrollbar-track { background: var(--st-surface-2); border-radius: 4px; }
.container::-webkit-scrollbar-thumb { background: var(--st-line); border-radius: 4px; }
.container::-webkit-scrollbar-thumb:hover { background: var(--st-faint); }
/* ── Host info section ──────────────────────────────────────────────────── */
/* ── Host info section ──────────────────────────────────────── */
.host-info-section {
padding: 12px 16px;
background: #fafafa;
border-bottom: 1px solid #e0e0e0;
font-size: 1.00em;
background: var(--st-surface-2);
border-radius: 6px;
padding: 8px 12px;
margin-bottom: 8px;
font-size: 12.5px;
}
.info-meta {
display: grid;
grid-template-columns: max-content 1fr;
gap: 3px 14px;
margin-bottom: 10px;
}
.info-label { font-weight: 600; color: #555; white-space: nowrap; }
.info-value { color: #222; }
.info-meta { display: flex; flex-wrap: wrap; gap: 4px 18px; }
.info-label { font-family: var(--st-mono); font-size: 11px; font-weight: 600; color: var(--st-faint); white-space: nowrap; }
.info-value { color: var(--st-ink); }
.info-thresholds-title {
font-weight: 600;
color: #555;
margin-bottom: 6px;
font-family: var(--st-mono);
font-size: 11px; font-weight: 600; color: var(--st-faint);
text-transform: uppercase; letter-spacing: .05em;
margin: 8px 0 4px;
}
.info-note { color: #888; font-style: italic; }
.info-loading { color: #bbb; font-style: italic; }
.threshold-covers { font-size: 1.00em; color: #777; font-style: italic; }
/* ── Dark mode ── */
html[data-theme="dark"] h1 { color: var(--text); }
html[data-theme="dark"] .subtitle { color: var(--text-sec); }
html[data-theme="dark"] .host-card { background: var(--surface); }
html[data-theme="dark"] .host-header:hover { background: var(--surface-2); }
html[data-theme="dark"] .host-name { color: var(--text); }
html[data-theme="dark"] .collapse-icon,
html[data-theme="dark"] .acc-icon { color: var(--text-muted); }
html[data-theme="dark"] .host-body { border-top-color: var(--border-3); }
html[data-theme="dark"] .plugin-accordion { border-color: var(--border); }
html[data-theme="dark"] .plugin-acc-header { background: var(--surface-2); }
html[data-theme="dark"] .plugin-acc-header:hover { background: var(--surface-3); }
html[data-theme="dark"] .plugin-label { color: var(--text-2); }
html[data-theme="dark"] .plugin-summary { color: var(--text-muted); }
html[data-theme="dark"] .data-table { background: var(--surface); }
html[data-theme="dark"] .data-table td { border-top-color: var(--border); color: var(--text); }
html[data-theme="dark"] .data-table td.key { color: var(--text-sec); }
html[data-theme="dark"] .data-table tbody tr:nth-child(even) { background: var(--surface-2); }
html[data-theme="dark"] .data-table tbody tr:hover { background: #1e3a5f; }
html[data-theme="dark"] .bar-track { background: var(--border); }
html[data-theme="dark"] .table-section-label { color: var(--text-muted); }
html[data-theme="dark"] .no-data,
html[data-theme="dark"] .loading { color: var(--text-dim); }
html[data-theme="dark"] .timestamp { color: var(--text-dim); border-top-color: var(--border-3); }
html[data-theme="dark"] .glance-chip.neutral { background: var(--surface-3); color: var(--text-sec); }
html[data-theme="dark"] .os-label { color: var(--text-muted); }
html[data-theme="dark"] .host-info-section { background: var(--surface-2); border-bottom-color: var(--border); }
html[data-theme="dark"] .info-label { color: var(--text-3); }
html[data-theme="dark"] .info-value { color: var(--text); }
html[data-theme="dark"] .info-thresholds-title { color: var(--text-3); }
html[data-theme="dark"] .info-note,
html[data-theme="dark"] .info-loading,
html[data-theme="dark"] .threshold-covers { color: var(--text-muted); }
html[data-theme="dark"] .check-ok { background: #0d2e17; }
html[data-theme="dark"] .check-warning { background: #2e1a00; }
html[data-theme="dark"] .check-critical { background: #2e0a0a; }
html[data-theme="dark"] .check-unknown { background: var(--surface-2); }
html[data-theme="dark"] .check-output { color: var(--text-sec); }
html[data-theme="dark"] .container::-webkit-scrollbar-track { background: var(--surface-2); }
html[data-theme="dark"] .container::-webkit-scrollbar-thumb { background: var(--border); }
.info-note { color: var(--st-faint); font-style: italic; }
.info-loading { color: var(--st-faint); font-style: italic; }
.threshold-covers { font-size: 11.5px; color: var(--st-faint); font-style: italic; }
</style>
<body>
{% include 'nav.html' %}
<div class="st-toolbar">
<span class="brand">hbd<span class="tld">·overview</span></span>
<span class="hintline">per-host metrics — expand a host for plugin details</span>
</div>
<svg class="pulse" viewBox="0 0 1200 14" preserveAspectRatio="none" aria-hidden="true">
<polyline points="0,10 340,10 352,10 358,3 364,13 370,1 378,12 384,10 560,10 572,10 578,3 584,13 590,1 598,12 604,10 1200,10"/>
</svg>
<div class="container">
<h1>{{ header }}</h1>
<p class="subtitle">Per-host system metrics — expand a host to see plugin details</p>
{% if not hosts %}
<div class="no-data">
<p>No hosts with plugin data available</p>
<p style="font-size:0.9em;margin-top:10px;">Hosts will appear here once they start sending plugin metrics</p>
<p>No hosts have connected yet</p>
<p style="font-size:0.9em;margin-top:10px;">Hosts will appear here once they start sending heartbeats</p>
</div>
{% else %}
<div id="hosts-container">
@@ -618,6 +444,34 @@
<span class="info-label">Last Packet</span><span class="info-value">${lastPkt}</span>
</div>`;
if (data.connections && data.connections.length) {
html += `<div class="info-thresholds-title">Connectivity</div>`;
for (const c of data.connections) {
html += `<div class="info-note">${escHtml(c.family)} &middot; ${escHtml(c.addr || '—')} RTT</div>
<div id="rtt-chart-${hostname}-${escHtml(c.family)}" style="margin-bottom:8px;"></div>`;
}
html += `<table class="data-table"><thead><tr>
<th>Family</th><th>Address</th><th>State</th>
<th class="num">RTT</th><th>Last Change</th><th>Last Packet</th>
</tr></thead><tbody>`;
for (const c of data.connections) {
const st = escHtml(c.state || '—');
const stCls = c.state === 'up' ? 'status-up' : (c.state ? 'status-down' : '');
const rtt = (c.state === 'up' && c.rtt) ? Math.round(c.rtt) + ' ms' : '—';
const chg = c.statetime ? new Date(c.statetime * 1000).toLocaleString() : '—';
const seen = c.lastbeat ? new Date(c.lastbeat * 1000).toLocaleString() : '—';
html += `<tr>
<td class="key">${escHtml(c.family)}</td>
<td>${escHtml(c.addr || '—')}</td>
<td><span class="${stCls}">${st}</span></td>
<td class="num">${rtt}</td>
<td>${chg}</td>
<td>${seen}</td>
</tr>`;
}
html += `</tbody></table>`;
}
if (data.thresholds === null) {
html += `<div class="info-note">Threshold alerting not configured.</div>`;
} else if (data.thresholds.length === 0) {
@@ -645,6 +499,15 @@
}
el.innerHTML = html;
if (data.connections && data.connections.length) {
const rttThresholds = (data.thresholds || []).find(t => t.metric === 'rtt') || null;
for (const c of data.connections) {
fetchRttHistory(hostname, c.family)
.then(samples => renderRttChart(hostname, c.family, samples, rttThresholds))
.catch(() => {});
}
}
}
async function fetchHostGlance(hostname) {
@@ -959,84 +822,164 @@
return json.samples || [];
}
function renderCpuChart(hostname, samples) {
const el = document.getElementById(`cpu-chart-${hostname}`);
if (!el || !samples.length) return;
function renderTimeSeriesChart(elId, pts, opts) {
const el = document.getElementById(elId);
if (!el || pts.length < 2) { if (el) el.style.display = 'none'; return; }
const pts = samples
.filter(s => s.data.cpu_percent != null)
.map(s => ({ t: s.timestamp, v: s.data.cpu_percent }));
if (pts.length < 2) { el.style.display = 'none'; return; }
const W = 600, H = 80, PAD = { top: 6, right: 8, bottom: 18, left: 28 };
const cW = W - PAD.left - PAD.right;
const cH = H - PAD.top - PAD.bottom;
const unitSuffix = opts.unitSuffix || '';
const W = 690, H = 92, PAD = { top: 6, right: 8, bottom: 18, left: 28 };
const tMin = pts[0].t, tMax = pts[pts.length - 1].t;
const tRange = tMax - tMin || 1;
const x = t => PAD.left + ((t - tMin) / tRange) * cW;
// Auto-scale Y axis with 10% padding, clamped to [0, 100]
// Auto-scale Y axis with 10% padding, optionally clamped to opts.yDomain
const vMin = Math.min(...pts.map(p => p.v));
const vMax = Math.max(...pts.map(p => p.v));
const vRange = vMax - vMin || 1;
const vPad = Math.max(vRange * 0.1, 1);
const yLow = Math.max(0, vMin - vPad);
const yHigh = Math.min(100, vMax + vPad);
const domainLow = Array.isArray(opts.yDomain) ? opts.yDomain[0] : 0;
const domainHigh = Array.isArray(opts.yDomain) ? opts.yDomain[1] : Infinity;
const yLow = Math.max(domainLow, vMin - vPad);
const yHigh = Math.min(domainHigh, vMax + vPad);
const yRange = yHigh - yLow || 1;
const y = v => PAD.top + cH - ((v - yLow) / yRange) * cH;
// Build polyline points and filled area path
const linePoints = pts.map(p => `${x(p.t).toFixed(1)},${y(p.v).toFixed(1)}`).join(' ');
const areaPath = `M${x(pts[0].t).toFixed(1)},${(PAD.top + cH).toFixed(1)} ` +
pts.map(p => `L${x(p.t).toFixed(1)},${y(p.v).toFixed(1)}`).join(' ') +
` L${x(pts[pts.length-1].t).toFixed(1)},${(PAD.top + cH).toFixed(1)} Z`;
// Color based on latest absolute CPU %
const latest = pts[pts.length - 1].v;
const strokeColor = latest > 90 ? '#e53935' : latest > 70 ? '#fb8c00' : '#43a047';
const fillColor = latest > 90 ? '#ffcdd2' : latest > 70 ? '#ffe0b2' : '#c8e6c9';
// Compute nice tick step for ~3-5 grid lines
// Compute nice tick step for ~3-5 grid lines, then size the left
// margin to fit the widest label (values/units can run to 3+ digits,
// e.g. RTT samples above 99ms) instead of a fixed guess that clips them.
const rawStep = yRange / 4;
const mag = Math.pow(10, Math.floor(Math.log10(rawStep || 1)));
const niceStep = [1, 2, 5, 10].map(f => f * mag).find(s => yRange / s <= 5) || mag * 10;
const tickStart = Math.ceil(yLow / niceStep) * niceStep;
let gridLines = '';
for (let v = tickStart; v <= yHigh + 0.001; v += niceStep) {
const yy = y(v).toFixed(1);
const label = Number.isInteger(v) ? v : v.toFixed(1);
gridLines += `<line x1="${PAD.left}" y1="${yy}" x2="${PAD.left + cW}" y2="${yy}" stroke="#e0e0e0" stroke-width="1"/>`;
gridLines += `<text x="${(PAD.left - 3).toFixed(1)}" y="${yy}" text-anchor="end" dominant-baseline="middle" font-size="8" fill="#999">${label}</text>`;
const yTicks = [];
for (let v = tickStart; v <= yHigh + 0.001; v += niceStep) yTicks.push(v);
const yTickLabels = yTicks.map(v => (Number.isInteger(v) ? v : v.toFixed(1)) + unitSuffix);
const maxLabelLen = yTickLabels.reduce((m, s) => Math.max(m, s.length), 0);
PAD.left = Math.max(28, Math.ceil(maxLabelLen * 5.5) + 10);
const cW = W - PAD.left - PAD.right;
const cH = H - PAD.top - PAD.bottom;
const x = t => PAD.left + ((t - tMin) / tRange) * cW;
const y = v => PAD.top + cH - ((v - yLow) / yRange) * cH;
// Color based on latest value
const latest = pts[pts.length - 1].v;
const { stroke: strokeColor, fill: fillColor } = opts.colorFor(latest);
// Split into segments wherever the gap between consecutive samples is
// much larger than the typical spacing (e.g. the host was overdue/down
// for a while) — draw each segment separately so missing data reads as
// a visual gap instead of an interpolated line across dead time.
const deltas = [];
for (let i = 1; i < pts.length; i++) deltas.push(pts[i].t - pts[i - 1].t);
deltas.sort((a, b) => a - b);
const medianDelta = deltas[Math.floor(deltas.length / 2)];
const gapThreshold = medianDelta * 2.5;
const segments = [[pts[0]]];
for (let i = 1; i < pts.length; i++) {
if (pts[i].t - pts[i - 1].t > gapThreshold) segments.push([]);
segments[segments.length - 1].push(pts[i]);
}
// X-axis time labels
let linePolylines = '';
let areaPaths = '';
for (const seg of segments) {
if (seg.length < 2) continue;
const segPoints = seg.map(p => `${x(p.t).toFixed(1)},${y(p.v).toFixed(1)}`).join(' ');
linePolylines += `<polyline points="${segPoints}" fill="none" stroke="${strokeColor}" stroke-width="1.5" stroke-linejoin="round"/>`;
const segArea = `M${x(seg[0].t).toFixed(1)},${(PAD.top + cH).toFixed(1)} ` +
seg.map(p => `L${x(p.t).toFixed(1)},${y(p.v).toFixed(1)}`).join(' ') +
` L${x(seg[seg.length-1].t).toFixed(1)},${(PAD.top + cH).toFixed(1)} Z`;
areaPaths += `<path d="${segArea}" fill="${fillColor}" opacity="0.6"/>`;
}
let gridLines = '';
yTicks.forEach((v, i) => {
const yy = y(v).toFixed(1);
gridLines += `<line x1="${PAD.left}" y1="${yy}" x2="${PAD.left + cW}" y2="${yy}" stroke="#e0e0e0" stroke-width="1"/>`;
gridLines += `<text x="${(PAD.left - 3).toFixed(1)}" y="${yy}" text-anchor="end" dominant-baseline="middle" font-size="8" fill="#999">${yTickLabels[i]}</text>`;
});
// X-axis: start/end plus evenly spaced intermediate tickmarks + labels
const fmt = ts => {
const d = new Date(ts * 1000);
return d.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' });
};
const xLabels = `
<text x="${PAD.left}" y="${H - 2}" text-anchor="start" font-size="8" fill="#999">${fmt(pts[0].t)}</text>
<text x="${PAD.left + cW}" y="${H - 2}" text-anchor="end" font-size="8" fill="#999">${fmt(pts[pts.length-1].t)}</text>`;
const xTickCount = 5;
const xAxisY = (PAD.top + cH).toFixed(1);
let xAxisMarks = '';
let xLabels = '';
for (let i = 0; i < xTickCount; i++) {
const t = tMin + (tRange * i) / (xTickCount - 1);
const xx = x(t).toFixed(1);
const anchor = i === 0 ? 'start' : (i === xTickCount - 1 ? 'end' : 'middle');
xAxisMarks += `<line x1="${xx}" y1="${xAxisY}" x2="${xx}" y2="${(PAD.top + cH + 3).toFixed(1)}" stroke="#ccc" stroke-width="1"/>`;
xLabels += `<text x="${xx}" y="${H - 2}" text-anchor="${anchor}" font-size="8" fill="#999">${fmt(t)}</text>`;
}
el.innerHTML = `<svg viewBox="0 0 ${W} ${H}" preserveAspectRatio="none"
style="width:100%;height:${H}px;display:block;">
<defs>
<clipPath id="cpu-clip-${hostname}">
<clipPath id="${opts.clipId}">
<rect x="${PAD.left}" y="${PAD.top}" width="${cW}" height="${cH}"/>
</clipPath>
</defs>
${gridLines}
<line x1="${PAD.left}" y1="${PAD.top}" x2="${PAD.left}" y2="${PAD.top + cH}" stroke="#ccc" stroke-width="1"/>
<line x1="${PAD.left}" y1="${PAD.top + cH}" x2="${PAD.left + cW}" y2="${PAD.top + cH}" stroke="#ccc" stroke-width="1"/>
<g clip-path="url(#cpu-clip-${hostname})">
<path d="${areaPath}" fill="${fillColor}" opacity="0.6"/>
<polyline points="${linePoints}" fill="none" stroke="${strokeColor}" stroke-width="1.5" stroke-linejoin="round"/>
<g clip-path="url(#${opts.clipId})">
${areaPaths}
${linePolylines}
</g>
${xAxisMarks}
${xLabels}
</svg>`;
}
function renderCpuChart(hostname, samples) {
const pts = samples
.filter(s => s.data.cpu_percent != null)
.map(s => ({ t: s.timestamp, v: s.data.cpu_percent }));
renderTimeSeriesChart(`cpu-chart-${hostname}`, pts, {
yDomain: [0, 100],
clipId: `cpu-clip-${hostname}`,
colorFor: (latest) => ({
stroke: latest > 90 ? '#e53935' : latest > 70 ? '#fb8c00' : '#43a047',
fill: latest > 90 ? '#ffcdd2' : latest > 70 ? '#ffe0b2' : '#c8e6c9',
}),
});
}
async function fetchRttHistory(hostname, family) {
const plugin = `rtt_${family.toLowerCase()}`;
const r = await fetch(`/api/0/hosts/${encodeURIComponent(hostname)}/plugins/${plugin}?limit=100`);
if (!r.ok) return [];
const json = await r.json();
return json.samples || [];
}
function renderRttChart(hostname, family, samples, rttThresholds) {
const pts = samples
.filter(s => s.data.rtt != null && s.data.rtt > 0)
.map(s => ({ t: s.timestamp, v: s.data.rtt }));
renderTimeSeriesChart(`rtt-chart-${hostname}-${family}`, pts, {
yDomain: 'auto',
clipId: `rtt-clip-${hostname}-${family}`,
unitSuffix: ' ms',
colorFor: (latest) => {
if (rttThresholds?.critical != null && latest > rttThresholds.critical) {
return { stroke: '#e53935', fill: '#ffcdd2' };
}
if (rttThresholds?.warning != null && latest > rttThresholds.warning) {
return { stroke: '#fb8c00', fill: '#ffe0b2' };
}
return { stroke: '#1976d2', fill: '#bbdefb' };
},
});
}
function renderCpuTable(hostname, d) {
const KEYS = [
['cpu_percent', 'CPU Usage', 'bar'],
@@ -1443,6 +1386,11 @@
document.querySelectorAll('.host-card:not(.collapsed)').forEach(card => {
const hostname = card.dataset.hostname;
fetchHostInfo(hostname).then(data => {
infoCache[hostname] = data;
renderInfoSection(hostname, data);
}).catch(() => {});
card.querySelectorAll('.plugin-accordion:not(.collapsed)').forEach(acc => {
const pname = acc.dataset.plugin;
if (!GLANCE_PLUGINS.includes(pname)) {
+2 -12
View File
@@ -240,7 +240,6 @@
}
.my-ch-name { font-weight: 600; font-size: .9em; color: #222; }
.my-ch-type { padding: 2px 7px; border-radius: 8px; font-size: .72em; font-weight: 600; background: #e8eaf6; color: #3949ab; }
.my-ch-private { padding: 2px 7px; border-radius: 8px; font-size: .72em; font-weight: 600; background: #fce4ec; color: #c62828; }
.my-ch-actions { margin-left: auto; display: flex; gap: 5px; }
.btn-sm-edit { background: #888; color: #fff; border: none; border-radius: 4px; padding: 2px 8px; font-size: .78em; cursor: pointer; }
.btn-sm-edit:hover { background: #666; }
@@ -465,7 +464,7 @@
{% if current_user %}
<div class="section">
<h2>My Channels</h2>
<p style="font-size:.82em;color:#888;margin:0 0 12px">Channels you own. Public channels are available to all users; private channels are visible only to you.</p>
<p style="font-size:.82em;color:#888;margin:0 0 12px">Channels you own are private to you. Global channels are managed by administrators.</p>
<div id="my-channels-list">
{% set my_channels = all_channels | selectattr('owner', 'equalto', current_user.username) | list %}
{% for ch in my_channels %}
@@ -473,7 +472,6 @@
<div class="my-ch-header">
<span class="my-ch-name">{{ ch.name | e }}</span>
<span class="my-ch-type">{{ ch.type | e }}</span>
{% if ch.private %}<span class="my-ch-private">private</span>{% endif %}
<span class="my-ch-actions">
<button class="btn-sm-edit" onclick="openMyChModal('{{ ch.name | e }}')">Edit</button>
<button class="btn-sm-del" onclick="deleteMyChannel('{{ ch.name | e }}')"></button>
@@ -513,11 +511,6 @@
<option value="CRITICAL">CRITICAL only</option>
</select>
</div>
<div class="ch-form-row">
<label style="display:flex;align-items:center;gap:6px;cursor:pointer">
<input type="checkbox" id="my-ch-private"> Private — visible only to you
</label>
</div>
<div id="my-ch-modal-status" class="ch-modal-status"></div>
<div class="ch-modal-footer">
<button class="btn-save" style="background:#888" onclick="closeMyChModal()">Cancel</button>
@@ -744,7 +737,6 @@
document.getElementById('my-ch-type').value = '';
document.getElementById('my-ch-type-fields').innerHTML = '';
document.getElementById('my-ch-min-level').value = 'WARNING';
document.getElementById('my-ch-private').checked = false;
if (name) {
try {
@@ -755,7 +747,6 @@
document.getElementById('my-ch-type').value = ch.type;
onMyChTypeChange();
document.getElementById('my-ch-min-level').value = ch.min_level || 'WARNING';
document.getElementById('my-ch-private').checked = ch.private || false;
(ch.fields || []).forEach(f => {
const inp = document.getElementById('mychf-' + f.key);
if (inp) inp.value = f.value || '';
@@ -774,14 +765,13 @@
const name = document.getElementById('my-ch-name').value.trim();
const type = document.getElementById('my-ch-type').value;
const minLevel = document.getElementById('my-ch-min-level').value;
const isPrivate = document.getElementById('my-ch-private').checked;
const statusEl = document.getElementById('my-ch-modal-status');
statusEl.textContent = '';
if (!name) { statusEl.textContent = 'Name is required.'; statusEl.style.color = '#c62828'; return; }
if (!type) { statusEl.textContent = 'Please select a type.'; statusEl.style.color = '#c62828'; return; }
const body = { name, type, min_level: minLevel, private: isPrivate };
const body = { name, type, min_level: minLevel };
if (_myChSchemas[type]) {
(_myChSchemas[type].fields || []).forEach(sf => {
const inp = document.getElementById('mychf-' + sf.key);
File diff suppressed because it is too large Load Diff
+6
View File
@@ -1251,6 +1251,7 @@ class ThresholdChecker:
title=title,
body=body,
level=lvl,
service=short_path,
),
))
@@ -1534,6 +1535,7 @@ class ThresholdChecker:
title=f"[REMINDER/{alert_state.level.name}] {host_name} {short_path}",
body=body,
level=alert_state.level.name,
service=short_path,
),
))
logger.info("Re-notification sent: %s", message)
@@ -1554,6 +1556,10 @@ class ThresholdChecker:
configured = self.get_thresholds_for_host(hostname)
stale = []
for mp in host.alert_states:
# connectivity.* and rtt are managed by the connection state
# machine, not by threshold config — never purge them.
if mp == "rtt" or mp.startswith("connectivity"):
continue
if self._find_threshold(configured, mp)[0] is not None:
continue
# Also match wildcard pool/partition thresholds (e.g. "zfs_monitor.*.status"
+85 -8
View File
@@ -16,6 +16,11 @@ from . import notify as notify_mod
logger = logging.getLogger(__name__)
eventlog = notify_mod.eventlog
# Plugin data is declared stale after this many collection intervals without a
# fresh sample. >= 2 so a single missed sample never purges live data (the user
# directive: wait at least two full intervals after recovery before going stale).
_STALE_INTERVAL_MULTIPLIER = 3
# SO_TIMESTAMP: kernel attaches a struct timeval to each received datagram.
# Supported on Linux, FreeBSD, and macOS. The constant is not exposed by
# Python's socket module on all platforms
@@ -266,10 +271,15 @@ def restore_connection_timers(hbdclass, ctx):
for afam, conn in list(host.connections.items()):
state = conn.getstate()
if state == hbdclass.Connection.DOWN:
_set_connectivity_alert(host, afam, "CRITICAL")
continue
on_overdue, on_unknown = _make_timer_callbacks(uname, host, ctx)
if state == hbdclass.Connection.UNKNOWN:
_set_connectivity_alert(host, afam, "CRITICAL")
continue
if state == hbdclass.Connection.UP and interval > 0:
elapsed = now - conn.lastbeat
# Give hosts one full (interval + grace) of extra time on startup
@@ -300,11 +310,25 @@ def restore_connection_timers(hbdclass, ctx):
"Restored OVERDUE timer %s/%s: %.0fs remaining",
uname, afam, remaining,
)
# Ensure the connectivity alert is set — it may be missing if
# hbd was shut down before the on_overdue callback had a chance
# to record it.
_set_connectivity_alert(host, afam, "CRITICAL")
restored += 1
logger.info("Restored timers for %d connection(s)", restored)
def _is_rtt_key(plugin_name: str) -> bool:
"""True for synthetic RTT-history keys (rtt_ipv4/rtt_ipv6), not real plugin data."""
return plugin_name.startswith("rtt_")
def _has_real_plugin_data(plugin_data: dict) -> bool:
"""True if plugin_data holds any real (non-RTT) client-collected plugin data."""
return any(not _is_rtt_key(k) for k in plugin_data)
def handle_datagram(msg: dict, addr, transport, ctx: dict):
"""Handle a parsed datagram message.
@@ -371,7 +395,7 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
host.doesack = msg.get("acks", -1)
# send ACK back; ask client to resend plugin info when we have none yet
rmsg = {"time": time.time()}
if not host.plugin_data:
if not _has_real_plugin_data(host.plugin_data):
rmsg["request_update"] = 1
opkt = dicttos("ACK", rmsg)
try:
@@ -385,20 +409,50 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
plugin_name = msg.get("plugin")
if plugin_name:
# Extract plugin fields, dropping protocol metadata fields
# (_interval is the client-declared collection interval, not a metric).
plugin_data = {k: v for k, v in msg.items()
if k not in ("ID", "plugin", "id", "name")}
if k not in ("ID", "plugin", "id", "name", "_interval")}
# Store plugin data with timestamp
host.add_plugin_data(plugin_name, plugin_data, timestamp=now)
# Reset stale timer — 3× the heartbeat interval (min 60 s)
stale_timeout = max(host.interval * 3, 60)
host.reset_plugin_timer(plugin_name, stale_timeout,
_make_plugin_stale_callback(uname, ctx))
# Set the stale timer. Prefer the interval the client declares: it is
# the true collection cadence and is correct from the very first
# post-recovery sample, so we never purge data too early after an
# outage. interval == 0 means a collect-once InfoPlugin that never
# goes stale. Legacy clients omit _interval, so fall back to inferring
# the cadence from the gap between the last two received samples.
history = host.plugin_data.get(plugin_name, [])
declared = msg.get("_interval")
if declared is not None:
if declared > 0:
host.reset_plugin_timer(plugin_name, declared * _STALE_INTERVAL_MULTIPLIER,
_make_plugin_stale_callback(uname, ctx))
else:
host.cancel_plugin_timer(plugin_name)
elif len(history) >= 2:
plugin_interval = max(history[-1][0] - history[-2][0], 1)
host.reset_plugin_timer(plugin_name, plugin_interval * _STALE_INTERVAL_MULTIPLIER,
_make_plugin_stale_callback(uname, ctx))
else:
host.cancel_plugin_timer(plugin_name)
# Remove alert states for metrics present in the previous sample but
# absent now (e.g. a nagios check removed from configuration).
if len(history) >= 2:
prev_keys = set(history[-2][1].keys())
curr_keys = set(plugin_data.keys())
for metric_name in prev_keys - curr_keys:
metric_path = f"{plugin_name}.{metric_name}"
if host.alert_states.pop(metric_path, None) is not None:
eventlog(uname, "INFO", f"stale check removed: {metric_path}")
if (prev_keys - curr_keys) and msg_to_websockets:
msg_to_websockets("host", host.stateinfo())
# If os_info reports an owner and none is configured server-side, apply it
if plugin_name == "os_info":
config_owner = config_mod.get_host_access(cfg, uname).get("owner")
default_owner = config_mod.get_default_owner(cfg)
inferred_owner = plugin_data.get("owner", config_owner or default_owner)
inferred_owner = config_owner or plugin_data.get("owner") or default_owner
host.owner = inferred_owner
logger.info(f"owner for {uname} is {host.owner}")
if DEBUG > 1:
@@ -453,6 +507,7 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
boot = msg.get("boot", 0)
if boot:
# hbc was stared with a -b flag
eventlog(uname, "INFO", "booted")
if host.watched:
asyncio.create_task(notify_mod.send_notification(
@@ -460,11 +515,29 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
notify_mod.Notification(title=f"[INFO] {uname}", body=f"{host.name} booted", level="INFO"),
))
if message:
eventlog(uname, "INFO", "msg: %s" % message, service=service)
eventlog(uname, "INFO", message, service=service)
if conn.getstate() != hbdcls.Connection.UP:
# Transition to UP and log/notify if appropriate
lasts = conn.state
d = conn.newstate(hbdcls.Connection.UP, now)
if boot:
# On reboot, pre-boot plugin data and derived alerts are stale.
# Cancel all plugin timers and wipe plugin state so timers restart
# cleanly from the first two post-boot samples. An ordinary
# reconnect (no boot flag) doesn't invalidate the client's
# already-collected data, so it's left alone — this keeps chart
# history intact across a transient network blip.
for pname in list(host.plugin_timers):
host.cancel_plugin_timer(pname)
for pname in [k for k in host.plugin_data if not _is_rtt_key(k)]:
del host.plugin_data[pname]
stale_plugin_keys = [
k for k in host.alert_states
if k not in ("rtt",) and not k.startswith("connectivity.")
]
for k in stale_plugin_keys:
del host.alert_states[k]
# Clear connectivity alert now that the host is back up
_set_connectivity_alert(host, conn.afam, "OK")
# Don't log/notify RECOVER for a brand-new host seen for the first time —
@@ -504,6 +577,10 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
if interval > 0:
host.interval = interval
# Record RTT history for charting
if rtt is not None:
host.add_plugin_data(f"rtt_{conn.afam.lower()}", {"rtt": rtt}, timestamp=now)
# Timer-based reachability monitoring
# Reset overdue timer on every heartbeat
if interval > 0 and conn.getstate() != hbdcls.Connection.DOWN:
+1 -1
View File
@@ -90,7 +90,7 @@ async def handler(request):
# the client knows to append rather than prepend).
if data.msgs:
try:
for m in reversed(data.msgs):
for m in reversed(data.msgs[-30:]):
host_name = m.get("host") if isinstance(m, dict) else None
if not host_name or _user_can_see_host(user, host_name):
await ws.send_str(json.dumps({"type": "message", "data": m, "history": True}))
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "hbd"
version = "5.3.10"
version = "5.4.3"
description = "Heartbeat monitoring system — client (hbc) and server (hbd)"
readme = "README.md"
requires-python = ">=3.11"
+25 -5
View File
@@ -375,6 +375,10 @@ static const char *jstr(const jval_t *v, const char *def) {
* Config
* ============================================================ */
/* Set from config in cfg_load. Off by default: a CMD packet is an unauthenticated
* UDP datagram, so executing one must be opted into per host. */
static bool g_allow_remote_command = false;
typedef struct {
int hb_port, interval;
char owner[256];
@@ -429,6 +433,9 @@ static void config_load(config_t *cfg, const char *path) {
if ((v = jget(root, "hb_port"))) cfg->hb_port = jint(v, cfg->hb_port);
if ((v = jget(root, "interval"))) cfg->interval = jint(v, cfg->interval);
if ((v = jget(root, "owner"))) snprintf(cfg->owner, sizeof(cfg->owner), "%s", jstr(v, ""));
if ((v = jget(root, "allow_remote_command"))) g_allow_remote_command = jint(v, 0) != 0;
if (g_allow_remote_command)
LOGI("allow_remote_command is true - CMD packets from the server will be executed");
jval_t *plugins = jget(root, "plugins");
@@ -591,7 +598,13 @@ static void conn_recv(conn_t *c) {
LOGD("ACK rtt=%.1fms", c->rtt);
} else if (strcmp(id, "CMD") == 0) {
const char *cmd = kv_get(&msg, "cmd");
if (cmd) {
if (cmd && !g_allow_remote_command) {
LOGI("refused command (allow_remote_command is false): %s", cmd);
kvdict_t rep; kv_clear(&rep);
kv_set(&rep, "service", "command");
kv_set(&rep, "msg", "Refused: allow_remote_command is false");
conn_send(c, "HTB", &rep);
} else if (cmd) {
LOGI("CMD: %s", cmd);
char out[4096] = "";
FILE *p = popen(cmd, "r");
@@ -667,6 +680,7 @@ static void plugin_os_info(conn_t *c, const config_t *cfg) {
if (osver[0]) kv_set(&d, "distro_version_id", osver);
}
#endif
kv_set_int(&d, "_interval", 0); /* InfoPlugin: collect-once, never stale */
conn_send(c, "PLG", &d);
LOGI("sent os_info");
}
@@ -781,6 +795,7 @@ static void plugin_cpu_monitor(conn_t *c, const config_t *cfg) {
}
read_cpu_extras(&d);
kv_set_dbl(&d, "_timestamp", now_ts());
kv_set_int(&d, "_interval", cfg->cpu_interval);
conn_send(c, "PLG", &d);
LOGD("sent cpu_monitor");
}
@@ -796,7 +811,7 @@ static void plugin_cpu_monitor(conn_t *c, const config_t *cfg) {
static void mem_send(conn_t *c,
long long tot, long long used, long long av, long long fr,
long long act, long long ina, long long cac, long long buf,
long long stot, long long sused) {
long long stot, long long sused, int interval) {
kvdict_t d; kv_clear(&d);
kv_set(&d, "plugin", "memory_monitor");
kv_set_ull(&d, "memory_total", (unsigned long long)tot);
@@ -819,6 +834,7 @@ static void mem_send(conn_t *c,
kv_set(&d, "swap_percent", pct);
}
kv_set_dbl(&d, "_timestamp", now_ts());
kv_set_int(&d, "_interval", interval);
conn_send(c, "PLG", &d);
LOGD("sent memory_monitor");
}
@@ -863,7 +879,7 @@ static void plugin_memory_monitor(conn_t *c, const config_t *cfg) {
/* values from /proc/meminfo are in kB */
mem_send(c, tot*1024, used*1024, av*1024, fr*1024,
act*1024, ina*1024, cac*1024, buf*1024,
stot*1024, (stot-sfr)*1024);
stot*1024, (stot-sfr)*1024, cfg->mem_interval);
}
#elif defined(__FreeBSD__) || defined(__DragonFly__)
@@ -889,7 +905,7 @@ static void plugin_memory_monitor(conn_t *c, const config_t *cfg) {
long long cac = (long long)v_cache * ps;
long long av = fr + ina + cac; if (av > tot) av = tot;
long long used = tot - av;
mem_send(c, tot, used, av, fr, act, ina, cac, 0, 0, 0);
mem_send(c, tot, used, av, fr, act, ina, cac, 0, 0, 0, cfg->mem_interval);
}
#elif defined(__NetBSD__)
@@ -910,7 +926,7 @@ static void plugin_memory_monitor(conn_t *c, const config_t *cfg) {
long long used = tot - av;
long long stot = (long long)uvm.swpages * ps;
long long sinuse = (long long)uvm.swpginuse * ps;
mem_send(c, tot, used, av, fr, act, ina, 0, 0, stot, sinuse);
mem_send(c, tot, used, av, fr, act, ina, 0, 0, stot, sinuse, cfg->mem_interval);
}
#endif /* platform memory */
@@ -962,6 +978,7 @@ static void plugin_disk_monitor(conn_t *c, const config_t *cfg) {
char *jval = malloc(MAX_VAL + 1);
if (jval) { snprintf(jval, MAX_VAL, "@%s", json); kv_set(&d, "partitions", jval); free(jval); }
kv_set_dbl(&d, "_timestamp", now_ts());
kv_set_int(&d, "_interval", cfg->disk_interval);
conn_send(c, "PLG", &d);
free(json);
LOGD("sent disk_monitor");
@@ -1067,6 +1084,7 @@ static void plugin_network_monitor(conn_t *c, const config_t *cfg) {
char *jval = malloc(MAX_VAL + 1);
if (jval) { snprintf(jval, MAX_VAL, "@%s", json); kv_set(&d, "interfaces", jval); free(jval); }
kv_set_dbl(&d, "_timestamp", now_ts());
kv_set_int(&d, "_interval", cfg->net_interval);
conn_send(c, "PLG", &d);
free(json);
LOGD("sent network_monitor");
@@ -1125,6 +1143,7 @@ static void plugin_ping_monitor(conn_t *c, const config_t *cfg) {
}
}
kv_set_dbl(&d, "_timestamp", now_ts());
kv_set_int(&d, "_interval", cfg->ping_interval);
conn_send(c, "PLG", &d);
LOGD("sent ping_monitor");
}
@@ -1194,6 +1213,7 @@ static void plugin_nagios_runner(conn_t *c, const config_t *cfg) {
parse_perfdata(output, &d, name);
}
kv_set_dbl(&d, "_timestamp", now_ts());
kv_set_int(&d, "_interval", cfg->nagios_interval);
conn_send(c, "PLG", &d);
LOGD("sent nagios_runner");
}
+17 -3
View File
@@ -41,7 +41,7 @@ from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
# updated by scripts/bumpminor.sh
__version__ = "5.3.10"
__version__ = "5.4.3"
# ---------------------------------------------------------------------------
# Protocol (mirrors hbd/common/proto.py)
@@ -115,9 +115,14 @@ _DEFAULTS: Dict[str, Any] = {
"hb_port": 50003,
"interval": 10,
"owner": None,
"allow_remote_command": False, # Execute shell commands received in CMD packets
"plugins": {},
}
# Set from config in main(). Off by default: a CMD packet is an unauthenticated
# UDP datagram, so executing one must be opted into per host.
_allow_remote_command = False
def _load_config(path: Optional[str] = None) -> Dict[str, Any]:
cfg = dict(_DEFAULTS)
@@ -870,6 +875,10 @@ async def _handle_command(conn: AsyncConnection, msg: Dict[str, Any]):
if not cmd:
return
log = logging.getLogger("hbc.cmd")
if not _allow_remote_command:
log.warning("refused command (allow_remote_command is false): %s", cmd)
await conn.sendto({"service": "command", "msg": "Refused: allow_remote_command is false"})
return
log.info("exec: %s", cmd)
try:
out = subprocess.check_output(
@@ -955,7 +964,7 @@ async def _run_info_plugins(conn: AsyncConnection, plugins: List[Plugin]):
try:
data = await plugin.collect()
if data:
await conn.sendto({"plugin": plugin.name, **data}, "PLG")
await conn.sendto({"plugin": plugin.name, **data, "_interval": plugin.interval}, "PLG")
log.info("sent %s", plugin.name)
except Exception as e:
log.error("%s collect: %s", plugin.name, e)
@@ -968,7 +977,7 @@ async def _run_monitor_group(conn: AsyncConnection, plugins: List[Plugin], inter
try:
data = await plugin.collect()
if data:
await conn.sendto({"plugin": plugin.name, **data}, "PLG")
await conn.sendto({"plugin": plugin.name, **data, "_interval": plugin.interval}, "PLG")
log.debug("sent %s", plugin.name)
except asyncio.CancelledError:
raise
@@ -1180,6 +1189,11 @@ def main(argv=None):
cfg = _load_config(args.configfile)
global _allow_remote_command
_allow_remote_command = bool(cfg.get("allow_remote_command", False))
if _allow_remote_command:
logging.warning("allow_remote_command is true — CMD packets from the server will be executed")
if args.daemon:
_daemonize()
_reconfigure_syslog(level)
File diff suppressed because it is too large Load Diff
+49
View File
@@ -0,0 +1,49 @@
# PyInstaller spec for hbc_windows.exe
# Build with: pyinstaller hbc_windows.spec
#
# Requirements (on Windows):
# pip install pyinstaller
block_cipher = None
a = Analysis(
['hbc_windows.py'],
pathex=[],
binaries=[],
datas=[],
hiddenimports=[],
hookspath=[],
hooksconfig={},
runtime_hooks=[],
excludes=['tkinter', 'unittest', 'email', 'html', 'http', 'urllib', 'xml'],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False,
)
pyz = PYZ(a.pure, a.zlib_archive, cipher=block_cipher)
exe = EXE(
pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
[],
name='hbc_windows',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=False,
upx_exclude=[],
runtime_tmpdir=None,
console=True,
disable_windowed_traceback=False,
argv_emulation=False,
target_arch=None,
codesign_identity=None,
entitlements_file=None,
icon=None,
version=None,
)
+126
View File
@@ -0,0 +1,126 @@
#Requires -RunAsAdministrator
<#
.SYNOPSIS
Install hbc_windows.exe as a Windows Service using NSSM.
.DESCRIPTION
Installs the HeartBeat Client as a Windows Service that starts automatically.
Requires NSSM (Non-Sucking Service Manager) in PATH or alongside this script.
Requires hbc_windows.exe built via: pyinstaller hbc_windows.spec
.PARAMETER Server
HBD server hostname or IP address (required).
.PARAMETER ExePath
Path to hbc_windows.exe. Defaults to the directory containing this script.
.PARAMETER ServiceName
Windows service name. Default: heartbeat-client
.PARAMETER ConfigFile
Path to hbc.json config file. Optional.
.PARAMETER LogFile
Path to log file. Default: C:\ProgramData\heartbeat\hbc.log
.PARAMETER Interval
Heartbeat interval in seconds. Default: 10
.EXAMPLE
.\install_hbc_windows.ps1 -Server hbd.example.com
.\install_hbc_windows.ps1 -Server hbd.example.com -ConfigFile C:\ProgramData\heartbeat\hbc.json
#>
param(
[Parameter(Mandatory = $true)]
[string]$Server,
[string]$ExePath = "",
[string]$ServiceName = "heartbeat-client",
[string]$ConfigFile = "",
[string]$LogFile = "C:\ProgramData\heartbeat\hbc.log",
[int]$Interval = 10
)
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
# Locate hbc_windows.exe
if ($ExePath -eq "") {
$ExePath = Join-Path $PSScriptRoot "hbc_windows.exe"
}
if (-not (Test-Path $ExePath)) {
Write-Error "hbc_windows.exe not found at: $ExePath`nBuild it first with: pyinstaller hbc_windows.spec"
exit 1
}
# Locate NSSM
$nssm = Get-Command nssm -ErrorAction SilentlyContinue
if (-not $nssm) {
$nssmLocal = Join-Path $PSScriptRoot "nssm.exe"
if (Test-Path $nssmLocal) {
$nssm = $nssmLocal
} else {
Write-Error "nssm.exe not found in PATH or alongside this script.`nDownload from https://nssm.cc/download"
exit 1
}
} else {
$nssm = $nssm.Source
}
# Build argument list
$args_list = "--daemon $Server"
if ($ConfigFile -ne "") {
$args_list = "--daemon -c `"$ConfigFile`" $Server"
}
if ($LogFile -ne "") {
$args_list = "$args_list --log-file `"$LogFile`""
}
# Create data directory
$dataDir = "C:\ProgramData\heartbeat"
if (-not (Test-Path $dataDir)) {
New-Item -ItemType Directory -Path $dataDir | Out-Null
Write-Host "Created $dataDir"
}
# Remove existing service if present
$existing = Get-Service -Name $ServiceName -ErrorAction SilentlyContinue
if ($existing) {
Write-Host "Removing existing service '$ServiceName'..."
& $nssm stop $ServiceName 2>$null
& $nssm remove $ServiceName confirm
}
# Install service
Write-Host "Installing service '$ServiceName'..."
& $nssm install $ServiceName $ExePath $args_list
if ($LASTEXITCODE -ne 0) {
Write-Error "nssm install failed (exit $LASTEXITCODE)"
exit 1
}
# Configure service
& $nssm set $ServiceName DisplayName "HeartBeat Client"
& $nssm set $ServiceName Description "Sends heartbeat and plugin metrics to the HBD monitoring server."
& $nssm set $ServiceName Start SERVICE_AUTO_START
& $nssm set $ServiceName AppStdout (Join-Path $dataDir "nssm_stdout.log")
& $nssm set $ServiceName AppStderr (Join-Path $dataDir "nssm_stderr.log")
& $nssm set $ServiceName AppRotateFiles 1
& $nssm set $ServiceName AppRotateBytes 5242880
# Start service
Write-Host "Starting service '$ServiceName'..."
& $nssm start $ServiceName
if ($LASTEXITCODE -ne 0) {
Write-Warning "Service installed but failed to start — check logs in $dataDir"
} else {
Write-Host "Service '$ServiceName' started successfully."
Write-Host "Log file: $LogFile"
Write-Host ""
Write-Host "Useful commands:"
Write-Host " nssm status $ServiceName"
Write-Host " nssm stop $ServiceName"
Write-Host " nssm restart $ServiceName"
Write-Host " nssm remove $ServiceName confirm"
}
+103
View File
@@ -0,0 +1,103 @@
"""Tests for the allow_remote_command gate on CMD packets in hbc."""
import asyncio
import subprocess
import pytest
from hbd.client import main as client_main
from hbd.client.config import CLIENT_DEFAULTS, load_config
class FakeConn:
"""Stand-in for AsyncConnection that records what the client sends back."""
def __init__(self):
self.sent = []
async def sendto(self, msg, msg_id="HTB"):
self.sent.append(msg)
@pytest.fixture
def conn(monkeypatch):
"""A fake connection with remote commands disabled and subprocess booby-trapped."""
monkeypatch.setattr(client_main, "allow_remote_command", False)
def explode(*a, **kw):
raise AssertionError("subprocess must not run when the command is refused")
monkeypatch.setattr(subprocess, "check_output", explode)
return FakeConn()
def test_default_is_false():
assert CLIENT_DEFAULTS["allow_remote_command"] is False
assert load_config("/nonexistent/hbc.yaml")["allow_remote_command"] is False
assert client_main.allow_remote_command is False
def test_refused_when_disabled(conn):
asyncio.run(client_main.handle_command(conn, {"cmd": "id"}))
assert conn.sent == [{
"service": "command",
"msg": "Refused: allow_remote_command is false",
}]
def test_refusal_reported_for_every_attempt(conn):
for _ in range(3):
asyncio.run(client_main.handle_command(conn, {"cmd": "rm -rf /"}))
assert len(conn.sent) == 3
def test_empty_command_is_ignored_either_way(conn, monkeypatch):
asyncio.run(client_main.handle_command(conn, {}))
monkeypatch.setattr(client_main, "allow_remote_command", True)
asyncio.run(client_main.handle_command(conn, {"cmd": ""}))
assert conn.sent == []
def test_executed_when_enabled(conn, monkeypatch):
monkeypatch.setattr(client_main, "allow_remote_command", True)
monkeypatch.setattr(subprocess, "check_output", lambda *a, **kw: b"uid=0(root)")
asyncio.run(client_main.handle_command(conn, {"cmd": "id"}))
assert conn.sent == [{"service": "command", "msg": "OK uid=0(root)"}]
def test_failure_reported_when_enabled(conn, monkeypatch):
monkeypatch.setattr(client_main, "allow_remote_command", True)
def fail(*a, **kw):
raise subprocess.CalledProcessError(1, "false")
monkeypatch.setattr(subprocess, "check_output", fail)
asyncio.run(client_main.handle_command(conn, {"cmd": "false"}))
assert len(conn.sent) == 1
assert conn.sent[0]["msg"].startswith("CalledProcessError ")
def test_config_file_can_opt_in(tmp_path):
cfg_file = tmp_path / "hbc.yaml"
cfg_file.write_text("allow_remote_command: true\n", encoding="utf-8")
assert load_config(str(cfg_file))["allow_remote_command"] is True
@pytest.mark.parametrize("configured, expected", [(True, True), (False, False), (None, False)])
def test_async_main_sets_the_flag_from_config(monkeypatch, configured, expected):
"""async_main must apply the config value before any CMD packet can arrive."""
monkeypatch.setattr(client_main, "allow_remote_command", not expected)
class Bail(Exception):
pass
# gethostname() is the first call after the flag is assigned — bail there so the
# test never opens a socket.
def bail():
raise Bail
monkeypatch.setattr(client_main.socket, "gethostname", bail)
cfg = {} if configured is None else {"allow_remote_command": configured}
with pytest.raises(Bail):
asyncio.run(client_main.async_main(object(), cfg))
assert client_main.allow_remote_command is expected
+247
View File
@@ -0,0 +1,247 @@
"""Tests for ownership rules and scoped config merges (hbd.server.config_access)."""
import pytest
from hbd.server import config_access as ca
# ---------------------------------------------------------------------------
# Visibility helpers
# ---------------------------------------------------------------------------
def test_is_global_when_no_owner():
assert ca.is_global({"type": "pushover"})
assert ca.is_global({"type": "email", "owner": ""})
assert ca.is_global(None) # non-dict is treated as global
assert not ca.is_global({"type": "email", "owner": "alice"})
def test_user_can_use_global_or_own():
assert ca.user_can_use({"type": "pushover"}, "alice")
assert ca.user_can_use({"owner": "alice"}, "alice")
assert not ca.user_can_use({"owner": "bob"}, "alice")
HOSTS = {
"web1": {"owner": "alice", "watch": True},
"web2": {"owner": "bob", "managers": ["alice"]},
"web3": {"owner": "bob", "managers": "carol"}, # string manager form
"web4": {"watch": True}, # unowned
}
def test_user_hosts_owner_and_manager():
assert set(ca.user_hosts(HOSTS, "alice")) == {"web1", "web2"}
assert set(ca.user_hosts(HOSTS, "carol")) == {"web3"}
assert ca.user_hosts(HOSTS, "dave") == {}
assert ca.user_hosts(None, "alice") == {}
CHANNELS = {
"global_ch": {"type": "pushover"},
"alice_ch": {"type": "email", "owner": "alice"},
"bob_ch": {"type": "signal", "owner": "bob"},
}
def test_user_channels_global_plus_own():
assert set(ca.user_channels(CHANNELS, "alice")) == {"global_ch", "alice_ch"}
assert set(ca.user_channels(CHANNELS, "dave")) == {"global_ch"}
TCS = {
"default": {"thresholds": {}},
"alice_tc": {"owner": "alice", "thresholds": {}},
}
def test_user_threshold_configs_global_plus_own():
assert set(ca.user_threshold_configs(TCS, "alice")) == {"default", "alice_tc"}
assert set(ca.user_threshold_configs(TCS, "bob")) == {"default"}
# ---------------------------------------------------------------------------
# merge_hosts_scoped
# ---------------------------------------------------------------------------
MERGE_HOSTS = {
"mine": {"owner": "alice", "watch": True, "notification_channels": ["global_ch"]},
"managed": {"owner": "bob", "managers": ["alice"], "watch": True,
"notification_channels": ["bob_ch"]},
"foreign": {"owner": "bob", "watch": False},
}
MERGE_CHANNELS = {
"global_ch": {"type": "pushover"},
"alice_ch": {"type": "email", "owner": "alice"},
"bob_ch": {"type": "signal", "owner": "bob"},
}
MERGE_TCS = {"alice_tc": {"owner": "alice"}, "bob_tc": {"owner": "bob"}}
def _merge_hosts(payload, existing=None):
return ca.merge_hosts_scoped(
existing if existing is not None else dict(MERGE_HOSTS),
payload, "alice", MERGE_CHANNELS, MERGE_TCS,
)
def _full_payload(**overrides):
"""Payload covering alice's full visible subset, with per-host overrides."""
p = {
"mine": {"owner": "alice", "watch": True, "notification_channels": ["global_ch"]},
"managed": {"watch": True, "notification_channels": ["bob_ch"]},
}
p.update(overrides)
return p
def test_merge_hosts_preserves_foreign_hosts():
result = _merge_hosts(_full_payload())
assert result["foreign"] == {"owner": "bob", "watch": False}
def test_merge_hosts_manager_edits_settings():
result = _merge_hosts(_full_payload(managed={"watch": False, "dyndns": True,
"notification_channels": ["bob_ch"]}))
assert result["managed"]["watch"] is False
assert result["managed"]["dyndns"] is True
# access fields carried over untouched
assert result["managed"]["owner"] == "bob"
assert result["managed"]["managers"] == ["alice"]
def test_merge_hosts_manager_cannot_change_owner():
with pytest.raises(ca.ScopedMergeError, match="managed"):
_merge_hosts(_full_payload(managed={"watch": True, "owner": "alice"}))
def test_merge_hosts_manager_cannot_change_managers():
with pytest.raises(ca.ScopedMergeError, match="managers"):
_merge_hosts(_full_payload(managed={"watch": True, "managers": ["alice", "carol"]}))
def test_merge_hosts_manager_same_access_values_ok():
# Submitting unchanged access fields is not a violation.
result = _merge_hosts(_full_payload(managed={"watch": True, "owner": "bob",
"managers": ["alice"],
"notification_channels": ["bob_ch"]}))
assert result["managed"]["owner"] == "bob"
def test_merge_hosts_manager_cannot_delete():
payload = {"mine": {"owner": "alice", "watch": True}} # 'managed' missing
with pytest.raises(ca.ScopedMergeError, match="managed"):
_merge_hosts(payload)
def test_merge_hosts_owner_can_delete():
payload = _full_payload()
del payload["mine"]
result = _merge_hosts(payload)
assert "mine" not in result
assert "managed" in result and "foreign" in result
def test_merge_hosts_owner_can_transfer_ownership():
result = _merge_hosts(_full_payload(mine={"owner": "bob", "watch": True}))
assert result["mine"]["owner"] == "bob"
def test_merge_hosts_new_host_owner_forced():
result = _merge_hosts(_full_payload(newhost={"watch": True, "owner": "bob"}))
assert result["newhost"]["owner"] == "alice"
def test_merge_hosts_cannot_touch_foreign_host():
with pytest.raises(ca.ScopedMergeError, match="foreign"):
_merge_hosts(_full_payload(foreign={"watch": True}))
def test_merge_hosts_cannot_add_foreign_private_channel():
with pytest.raises(ca.ScopedMergeError, match="bob_ch"):
_merge_hosts(_full_payload(mine={"owner": "alice", "watch": True,
"notification_channels": ["bob_ch"]}))
def test_merge_hosts_keeps_preexisting_foreign_assignment():
# 'managed' already has bob_ch; keeping it is fine.
result = _merge_hosts(_full_payload())
assert result["managed"]["notification_channels"] == ["bob_ch"]
def test_merge_hosts_can_add_global_and_own_channel():
result = _merge_hosts(_full_payload(mine={"owner": "alice", "watch": True,
"notification_channels": ["global_ch", "alice_ch"]}))
assert result["mine"]["notification_channels"] == ["global_ch", "alice_ch"]
def test_merge_hosts_threshold_assignment_validated():
with pytest.raises(ca.ScopedMergeError, match="bob_tc"):
_merge_hosts(_full_payload(mine={"owner": "alice", "watch": True,
"threshold_config": ["bob_tc"]}))
result = _merge_hosts(_full_payload(mine={"owner": "alice", "watch": True,
"threshold_config": ["default", "alice_tc"]}))
assert result["mine"]["threshold_config"] == ["default", "alice_tc"]
def test_merge_hosts_rejects_unknown_fields():
with pytest.raises(ca.ScopedMergeError, match="sneaky"):
_merge_hosts(_full_payload(mine={"owner": "alice", "watch": True, "sneaky": 1}))
def test_merge_hosts_owner_clearing_list_removes_key():
result = _merge_hosts(_full_payload(mine={"owner": "alice", "watch": True}))
assert "notification_channels" not in result["mine"]
# ---------------------------------------------------------------------------
# merge_threshold_configs_scoped
# ---------------------------------------------------------------------------
MERGE_EXISTING_TCS = {
"default": {"thresholds": {"cpu": {"load": {"warning": 2}}}},
"alice_tc": {"owner": "alice", "thresholds": {"cpu": {"load": {"warning": 3}}}},
"bob_tc": {"owner": "bob", "thresholds": {}},
"global_tc": {"thresholds": {}},
}
def _merge_tcs(payload):
return ca.merge_threshold_configs_scoped(dict(MERGE_EXISTING_TCS), payload, "alice")
def test_merge_tcs_new_config_owner_forced():
result = _merge_tcs({"alice_tc": {"thresholds": {}},
"new_tc": {"thresholds": {}, "owner": "bob"}})
assert result["new_tc"]["owner"] == "alice"
def test_merge_tcs_edit_own():
result = _merge_tcs({"alice_tc": {"thresholds": {"mem": {"used": {"warning": 90}}}}})
assert result["alice_tc"]["thresholds"] == {"mem": {"used": {"warning": 90}}}
assert result["alice_tc"]["owner"] == "alice"
def test_merge_tcs_delete_own_when_missing():
result = _merge_tcs({})
assert "alice_tc" not in result
def test_merge_tcs_preserves_global_and_foreign():
result = _merge_tcs({"alice_tc": {"thresholds": {}}})
assert result["default"] == MERGE_EXISTING_TCS["default"]
assert result["bob_tc"] == MERGE_EXISTING_TCS["bob_tc"]
assert result["global_tc"] == MERGE_EXISTING_TCS["global_tc"]
def test_merge_tcs_rejects_default():
with pytest.raises(ca.ScopedMergeError, match="default"):
_merge_tcs({"alice_tc": {"thresholds": {}}, "default": {"thresholds": {}}})
def test_merge_tcs_rejects_global_name_collision():
with pytest.raises(ca.ScopedMergeError, match="global_tc"):
_merge_tcs({"alice_tc": {"thresholds": {}}, "global_tc": {"thresholds": {}}})
def test_merge_tcs_rejects_foreign_owned():
with pytest.raises(ca.ScopedMergeError, match="bob_tc"):
_merge_tcs({"alice_tc": {"thresholds": {}}, "bob_tc": {"thresholds": {}}})
+265
View File
@@ -0,0 +1,265 @@
"""Tests for the dedicated events journal (write path and read path)."""
import asyncio
import json
from hbd.server import journal
def _make_journal(tmp_path, **overrides):
cfg = {"journal_dir": str(tmp_path), "journal_file": "events.journal"}
cfg.update(overrides)
j = journal.MessageJournal(cfg)
assert asyncio.run(j.initialize())
return j
def _read_lines(tmp_path, name="events.journal"):
return (tmp_path / name).read_text(encoding="utf-8").splitlines()
EV1 = {"ts": 1000.0, "host": "h1", "level": "INFO", "service": None, "message": "host up"}
EV2 = {"ts": 2000.0, "host": "h2", "level": "CRITICAL", "service": "cpu", "message": "cpu high"}
def test_log_event_writes_one_json_line(tmp_path):
j = _make_journal(tmp_path)
asyncio.run(j.log_event(EV1))
asyncio.run(j.close())
lines = _read_lines(tmp_path)
assert len(lines) == 1
assert json.loads(lines[0]) == EV1
def test_log_event_appends_in_order(tmp_path):
j = _make_journal(tmp_path)
asyncio.run(j.log_event(EV1))
asyncio.run(j.log_event(EV2))
asyncio.run(j.close())
lines = _read_lines(tmp_path)
assert [json.loads(ln)["ts"] for ln in lines] == [1000.0, 2000.0]
def test_log_event_rotates_at_max_size(tmp_path):
# max_size fits one serialized event line (~77 bytes) but not two, so the
# second write triggers exactly one rotation
j = _make_journal(tmp_path, journal_max_size=120)
asyncio.run(j.log_event(EV1))
asyncio.run(j.log_event(EV2))
asyncio.run(j.close())
backups = list(tmp_path.glob("events.journal.*"))
assert len(backups) == 1
assert json.loads(backups[0].read_text().splitlines()[0]) == EV1
assert json.loads(_read_lines(tmp_path)[0]) == EV2
def test_log_event_noop_when_disabled(tmp_path):
j = journal.MessageJournal(
{"journal_dir": str(tmp_path), "journal_file": "events.journal", "journal_enabled": False}
)
asyncio.run(j.initialize())
asyncio.run(j.log_event(EV1))
assert not (tmp_path / "events.journal").exists()
def test_backfill_seeds_empty_journal(tmp_path):
j = _make_journal(tmp_path)
asyncio.run(j.backfill([EV1, EV2]))
asyncio.run(j.close())
lines = _read_lines(tmp_path)
assert len(lines) == 2
assert json.loads(lines[0]) == EV1
def test_backfill_skipped_when_journal_nonempty(tmp_path):
(tmp_path / "events.journal").write_text(json.dumps(EV1) + "\n")
j = _make_journal(tmp_path) # initialize() picks up the existing size
asyncio.run(j.backfill([EV2]))
asyncio.run(j.close())
assert len(_read_lines(tmp_path)) == 1
def test_get_events_journal_uses_events_config_keys(tmp_path):
journal._events_journal_instance = None
try:
ej = journal.get_events_journal(
{
"journal_dir": str(tmp_path),
"events_journal_file": "ev.jsonl",
"events_journal_max_size": 12345,
"events_journal_max_backups": 3,
}
)
assert ej.journal_file == "ev.jsonl"
assert ej.max_size == 12345
assert ej.max_backups == 3
assert ej.journal_dir == tmp_path
# singleton: second call returns the same instance
assert journal.get_events_journal() is ej
finally:
journal._events_journal_instance = None
def test_get_events_journal_defaults(tmp_path):
journal._events_journal_instance = None
try:
ej = journal.get_events_journal({"journal_dir": str(tmp_path)})
assert ej.journal_file == "events.journal"
assert ej.max_size == 10 * 1024 * 1024
assert ej.max_backups == 10
finally:
journal._events_journal_instance = None
# ---- read path -------------------------------------------------------------
def _write_journal(path, events):
path.write_text("".join(json.dumps(e) + "\n" for e in events), encoding="utf-8")
def _evts(*ts_list):
return [
{"ts": float(t), "host": f"host{i}", "level": "INFO", "service": None, "message": f"msg {t}"}
for i, t in enumerate(ts_list)
]
def test_read_events_newest_first(tmp_path):
_write_journal(tmp_path / "events.journal", _evts(1, 2, 3))
events, more = journal.read_events(tmp_path)
assert [e["ts"] for e in events] == [3.0, 2.0, 1.0]
assert more is False
def test_read_events_limit_and_more(tmp_path):
_write_journal(tmp_path / "events.journal", _evts(1, 2, 3))
events, more = journal.read_events(tmp_path, limit=2)
assert [e["ts"] for e in events] == [3.0, 2.0]
assert more is True
def test_read_events_before_cursor(tmp_path):
_write_journal(tmp_path / "events.journal", _evts(1, 2, 3))
events, _ = journal.read_events(tmp_path, before=3.0)
assert [e["ts"] for e in events] == [2.0, 1.0]
def test_read_events_spans_rotated_files(tmp_path):
# rotated backup holds the oldest events; current file the newest
_write_journal(tmp_path / "events.journal.20260101-000000", _evts(1, 2))
_write_journal(tmp_path / "events.journal.20260201-000000", _evts(3, 4))
_write_journal(tmp_path / "events.journal", _evts(5, 6))
events, more = journal.read_events(tmp_path, limit=10)
assert [e["ts"] for e in events] == [6.0, 5.0, 4.0, 3.0, 2.0, 1.0]
assert more is False
def test_read_events_pagination_across_files(tmp_path):
_write_journal(tmp_path / "events.journal.20260101-000000", _evts(1, 2))
_write_journal(tmp_path / "events.journal", _evts(3, 4))
page1, more1 = journal.read_events(tmp_path, limit=3)
assert [e["ts"] for e in page1] == [4.0, 3.0, 2.0]
assert more1 is True
page2, more2 = journal.read_events(tmp_path, limit=3, before=page1[-1]["ts"])
assert [e["ts"] for e in page2] == [1.0]
assert more2 is False
def test_read_events_host_filter_substring_case_insensitive(tmp_path):
evs = [
{"ts": 1.0, "host": "Wentworth", "level": "INFO", "service": None, "message": "a"},
{"ts": 2.0, "host": "winter", "level": "INFO", "service": None, "message": "b"},
]
_write_journal(tmp_path / "events.journal", evs)
events, _ = journal.read_events(tmp_path, host="went")
assert [e["host"] for e in events] == ["Wentworth"]
def test_read_events_level_filter_exact_case_insensitive(tmp_path):
evs = [
{"ts": 1.0, "host": "h", "level": "CRITICAL", "service": None, "message": "a"},
{"ts": 2.0, "host": "h", "level": "INFO", "service": None, "message": "b"},
]
_write_journal(tmp_path / "events.journal", evs)
events, _ = journal.read_events(tmp_path, level="critical")
assert [e["level"] for e in events] == ["CRITICAL"]
def test_read_events_message_filter(tmp_path):
evs = [
{"ts": 1.0, "host": "h", "level": "INFO", "service": None, "message": "disk almost full"},
{"ts": 2.0, "host": "h", "level": "INFO", "service": None, "message": "all quiet"},
]
_write_journal(tmp_path / "events.journal", evs)
events, _ = journal.read_events(tmp_path, q="Disk")
assert [e["ts"] for e in events] == [1.0]
def test_read_events_skips_malformed_lines(tmp_path):
p = tmp_path / "events.journal"
p.write_text('{"ts": 1.0, "host": "h", "level": "INFO", "message": "ok"}\nnot json\n[1,2]\n')
events, _ = journal.read_events(tmp_path)
assert [e["ts"] for e in events] == [1.0]
def test_read_events_predicate(tmp_path):
_write_journal(tmp_path / "events.journal", _evts(1, 2))
events, _ = journal.read_events(tmp_path, predicate=lambda e: e["host"] == "host0")
assert [e["host"] for e in events] == ["host0"]
def test_read_events_missing_dir(tmp_path):
events, more = journal.read_events(tmp_path / "nope")
assert events == [] and more is False
def test_filter_events_over_in_memory_ring():
ring = _evts(1, 2, 3) # oldest-first, like data.msgs
events, more = journal.filter_events(reversed(ring), limit=2)
assert [e["ts"] for e in events] == [3.0, 2.0]
assert more is True
def test_filter_events_skips_non_dict_entries():
ring = [
{"ts": 1.0, "host": "h", "level": "INFO", "service": None, "message": "a"},
"legacy string entry",
None,
{"ts": 2.0, "host": "h", "level": "INFO", "service": None, "message": "b"},
]
events, more = journal.filter_events(reversed(ring), limit=10)
assert [e["ts"] for e in events] == [2.0, 1.0]
assert more is False
# ---- eventlog wiring --------------------------------------------------------
def test_eventlog_writes_to_events_journal(tmp_path):
from hbd.server import data, notify
journal._events_journal_instance = None
saved_msgs = data.msgs
data.msgs = []
try:
ej = journal.get_events_journal({"journal_dir": str(tmp_path)})
async def scenario():
await ej.initialize()
notify.setup({}, loop=asyncio.get_running_loop())
notify.eventlog("h1", "INFO", "hello world")
await asyncio.sleep(0.05) # let the scheduled journal write run
await ej.close()
asyncio.run(scenario())
lines = _read_lines(tmp_path)
assert len(lines) == 1
ev = json.loads(lines[0])
assert ev["host"] == "h1"
assert ev["level"] == "INFO"
assert ev["message"] == "hello world"
assert isinstance(ev["ts"], float)
finally:
journal._events_journal_instance = None
data.msgs = saved_msgs
notify._loop = None
+217
View File
@@ -0,0 +1,217 @@
"""Tests for flap detection (hbd.server.flap) and its hook in notify.send_notification."""
import asyncio
import json
import pytest
from hbd.server import flap, hbdclass, notify, users as users_mod
@pytest.fixture(autouse=True)
def clean_flap_state():
"""Every test starts with an empty state and a 3-in-10-minutes config."""
flap._state.clear()
flap.setup({"flap_count": 3, "flap_interval": 10})
yield
flap._state.clear()
flap.setup({})
def alerts(n, host="h1", service="cpu", level="CRITICAL"):
return [flap.observe(host, service, level) for _ in range(n)]
def advance(seconds, host="h1", service="cpu"):
"""Backdate a key's recorded timestamps to simulate the clock moving on."""
st = flap._state[(host, service)]
st["events"] = [t - seconds for t in st["events"]]
if st["ok_since"] is not None:
st["ok_since"] -= seconds
# --- tripping ---------------------------------------------------------------
def test_alerts_up_to_flap_count_pass():
assert alerts(3) == [flap.PASS] * 3
assert flap.flapping_services("h1") == []
def test_exceeding_flap_count_trips_then_suppresses():
assert alerts(5) == [flap.PASS, flap.PASS, flap.PASS, flap.TRIP, flap.SUPPRESS]
assert flap.flapping_services("h1") == ["cpu"]
def test_warning_and_critical_both_count():
flap.observe("h1", "cpu", "WARNING")
flap.observe("h1", "cpu", "CRITICAL")
flap.observe("h1", "cpu", "WARNING")
assert flap.observe("h1", "cpu", "CRITICAL") == flap.TRIP
def test_alerts_outside_the_window_do_not_count():
alerts(3)
advance(11 * 60) # older than flap_interval
assert alerts(3) == [flap.PASS] * 3
assert flap.flapping_services("h1") == []
def test_recover_alone_never_trips():
assert [flap.observe("h1", "cpu", "RECOVER") for _ in range(5)] == [flap.PASS] * 5
# --- suppression while flapping --------------------------------------------
def test_recover_and_info_are_suppressed_while_flapping():
alerts(4)
assert flap.observe("h1", "cpu", "RECOVER") == flap.SUPPRESS
assert flap.observe("h1", "cpu", "INFO") == flap.SUPPRESS
def test_info_passes_when_not_flapping():
assert flap.observe("h1", "cpu", "INFO") == flap.PASS
# --- clearing ---------------------------------------------------------------
def test_clears_silently_one_interval_after_recover():
alerts(4)
flap.observe("h1", "cpu", "RECOVER")
advance(10 * 60)
assert flap.flapping_services("h1") == []
assert flap.observe("h1", "cpu", "CRITICAL") == flap.PASS
def test_still_flapping_before_the_interval_elapses():
alerts(4)
flap.observe("h1", "cpu", "RECOVER")
advance(9 * 60)
assert flap.flapping_services("h1") == ["cpu"]
assert flap.observe("h1", "cpu", "CRITICAL") == flap.SUPPRESS
def test_alert_during_the_quiet_window_keeps_it_flapping():
alerts(4)
flap.observe("h1", "cpu", "RECOVER")
advance(9 * 60)
flap.observe("h1", "cpu", "CRITICAL") # resets the quiet window
advance(2 * 60)
assert flap.flapping_services("h1") == ["cpu"]
# --- host removal ------------------------------------------------------------
def test_clear_host_drops_flapping_state_even_without_a_recover():
# A host dropped mid-flap (no RECOVER ever received) must not stay
# flapping forever: nothing will ever set ok_since for it again.
alerts(4)
assert flap.flapping_services("h1") == ["cpu"]
flap.clear_host("h1")
assert flap.flapping_services("h1") == []
assert flap.observe("h1", "cpu", "CRITICAL") == flap.PASS
def test_clear_host_only_affects_the_named_host():
alerts(4, host="h1")
alerts(4, host="h2")
flap.clear_host("h1")
assert flap.flapping_services("h1") == []
assert flap.flapping_services("h2") == ["cpu"]
# --- keying -----------------------------------------------------------------
def test_services_and_hosts_are_tracked_independently():
alerts(4, service="cpu")
assert flap.observe("h1", "disk", "CRITICAL") == flap.PASS
assert flap.observe("h2", "cpu", "CRITICAL") == flap.PASS
assert flap.flapping_services("h1") == ["cpu"]
assert flap.flapping_services("h2") == []
def test_host_level_events_use_the_empty_service_key():
alerts(4, service="")
assert flap.flapping_services("h1") == [""]
# --- disabled ---------------------------------------------------------------
@pytest.mark.parametrize("cfg", [
{"flap_count": 0, "flap_interval": 10},
{"flap_count": 3, "flap_interval": 0},
{},
])
def test_disabled_never_suppresses(cfg):
flap.setup(cfg)
assert alerts(20) == [flap.PASS] * 20
assert flap.flapping_services("h1") == []
# --- integration through the real dispatch path -----------------------------
NOTIFY_CFG = {
"flap_count": 3,
"flap_interval": 10,
"notification_channels": {"ch1": {"type": "pushover", "token": "t", "user": "u"}},
"users": {"alice": {"notification_channels": ["ch1"]}},
}
@pytest.fixture
def delivered(monkeypatch):
"""Wire up a host, a user and a stub channel driver; collect what gets delivered."""
sent = []
monkeypatch.setattr(notify, "_config", dict(notify._config)) # restored on teardown
notify.setup(NOTIFY_CFG)
users_mod.load_users(NOTIFY_CFG)
monkeypatch.setitem(notify._DRIVERS, "pushover", lambda cfg, n: sent.append(n.body) or True)
host = hbdclass.Host("flaphost")
host.watched = True
host.owner = "alice"
yield sent, host
hbdclass.Host.hosts.pop("flaphost", None)
users_mod.load_users({})
def _notify(level, service, body):
n = notify.Notification(title=f"[{level}] flaphost", body=body, level=level, service=service)
asyncio.run(notify.send_notification("flaphost", n))
def test_marker_on_the_tripping_notification_then_nothing(delivered):
sent, host = delivered
for i in range(5):
_notify("CRITICAL", "cpu", f"cpu = 9{i}")
_notify("RECOVER", "cpu", "cpu = 12")
assert sent == ["cpu = 90", "cpu = 91", "cpu = 92", f"cpu = 93 {flap.FLAP_MARKER}"]
assert host.stateinfo()["flapping"] == ["cpu"]
def test_a_flapping_service_does_not_silence_another(delivered):
sent, host = delivered
for i in range(5):
_notify("CRITICAL", "cpu", "cpu = 99")
sent.clear()
_notify("CRITICAL", "disk", "disk = 91")
assert sent == ["disk = 91"]
def test_notifications_resume_after_the_state_clears(delivered):
sent, host = delivered
for i in range(5):
_notify("CRITICAL", "cpu", "cpu = 99")
_notify("RECOVER", "cpu", "cpu = 12")
advance(10 * 60, host="flaphost")
assert host.stateinfo()["flapping"] == []
sent.clear()
_notify("CRITICAL", "cpu", "cpu = 99")
assert sent == ["cpu = 99"]
def test_stateinfo_flapping_survives_json_encoding(delivered):
sent, host = delivered
for i in range(5):
_notify("CRITICAL", "", "IPv4 overdue") # host-level, empty service key
info = host.stateinfo()
assert info["flapping"] == [""]
assert json.loads(json.dumps(info))["flapping"] == [""]
+25
View File
@@ -171,3 +171,28 @@ def test_write_path_preserves_oauth_client_secret(tmp_path):
assert data2["oauth"]["gitea"]["client_secret"] == original_secret, (
f"Expected original secret preserved, got: {data2['oauth']['gitea']['client_secret']!r}"
)
# ---- threshold form payload shape ----
def test_build_threshold_configs_new_shape_with_owner():
form = {
"servers": {
"owner": "alice",
"metrics": {"cpu_monitor.load_15min": {"operator": ">", "warning": 4.0,
"critical": 8.0, "enabled": True}},
},
}
result = http._build_threshold_configs_from_form(form)
assert result["servers"]["owner"] == "alice"
assert result["servers"]["thresholds"]["cpu_monitor"]["load_15min"]["warning"] == 4.0
def test_build_threshold_configs_empty_owner_means_global():
form = {"servers": {"owner": "", "metrics": {"rtt": {"warning": 100.0}}}}
result = http._build_threshold_configs_from_form(form)
assert "owner" not in result["servers"]
def test_build_threshold_configs_ignores_entries_without_metrics():
assert http._build_threshold_configs_from_form({"bad": {"owner": "x"}}) == {}
+39 -1
View File
@@ -1,5 +1,4 @@
"""Tests for _build_host_info helper in http.py."""
import pytest
from unittest.mock import MagicMock
from hbd.server.http import _build_host_info
@@ -172,3 +171,42 @@ def test_build_host_info_covers_empty_when_exact_matches_only():
result = _build_host_info(host, threshold_checker=checker)
t = result["thresholds"][0]
assert t["covers"] == []
class _FakeConnFull:
def __init__(self, afam, addr, state, rtt, statetime, lastbeat, rtts=None):
self.afam = afam
self.addr = addr
self.state = state
self.rtts = rtts if rtts is not None else [rtt]
self.statetime = statetime
self.lastbeat = lastbeat
def test_build_host_info_includes_connections():
host = _FakeHost(connections={
"IPv4": _FakeConnFull("IPv4", "10.0.0.5", "up", 14.2, 1000.0, 2000.0),
"IPv6": _FakeConnFull("IPv6", "fd00::5", "overdue", 0, 1500.0, 1900.0),
})
result = _build_host_info(host)
conns = {c["family"]: c for c in result["connections"]}
assert conns["IPv4"] == {"family": "IPv4", "addr": "10.0.0.5", "state": "up",
"rtt": 14.2, "statetime": 1000.0, "lastbeat": 2000.0}
assert conns["IPv6"]["state"] == "overdue"
def test_build_host_info_connections_empty():
host = _FakeHost()
result = _build_host_info(host)
assert result["connections"] == []
def test_build_host_info_connection_rtt_uses_latest_sample():
"""rtt should reflect the most recent sample, not the oldest retained one."""
host = _FakeHost(connections={
"IPv4": _FakeConnFull("IPv4", "10.0.0.5", "up", None, 1000.0, 2000.0,
rtts=[100.0, 50.0, 14.2]),
})
result = _build_host_info(host)
conns = {c["family"]: c for c in result["connections"]}
assert conns["IPv4"]["rtt"] == 14.2
+23
View File
@@ -0,0 +1,23 @@
"""Tests for the plugin-accordion visibility filter in http.py."""
from hbd.server.http import _visible_plugin_names
def test_visible_plugin_names_excludes_rtt_history_keys():
plugin_data = {
"cpu_monitor": [],
"rtt_ipv4": [],
"rtt_ipv6": [],
"network_monitor": [],
}
result = _visible_plugin_names(plugin_data)
assert sorted(result) == ["cpu_monitor", "network_monitor"]
def test_visible_plugin_names_empty_when_no_plugins():
assert _visible_plugin_names({}) == []
def test_visible_plugin_names_keeps_names_not_prefixed_with_rtt_():
plugin_data = {"nagios_runner": [], "os_info": []}
result = _visible_plugin_names(plugin_data)
assert sorted(result) == ["nagios_runner", "os_info"]
+16 -24
View File
@@ -86,61 +86,53 @@ def test_delete_channel_persisted_after_write(tmp_path):
# ---------------------------------------------------------------------------
# Visibility logic (mirrors http.py _visible_channels_for_user)
# Visibility logic (owner-presence rule, hbd.server.config_access)
# ---------------------------------------------------------------------------
from hbd.server import config_access as ca # noqa: E402
def _visible(config, user):
"""Local copy of the visibility helper for unit testing without the HTTP layer."""
all_channels = config.get("notification_channels") or {}
if user.get("admin"):
return set(all_channels.keys())
username = user["username"]
return {
name for name, cfg in all_channels.items()
if isinstance(cfg, dict) and (not cfg.get("private") or cfg.get("owner") == username)
}
return set(ca.user_channels(all_channels, user["username"]))
CONFIG_VISIBILITY = {
"notification_channels": {
"pub_ch": {"type": "pushover", "token": "t", "user": "u"},
"alice_priv": {"type": "email", "owner": "alice", "private": True,
"alice_priv": {"type": "email", "owner": "alice",
"recipients": ["a@a.com"], "sender": "s@a.com", "smtp_server": "s"},
"bob_priv": {"type": "signal", "owner": "bob", "private": True,
"user": "+1", "recipient": "+2"},
"admin_owned": {"type": "pushover", "token": "t2", "user": "u2", "owner": "adminuser"},
"bob_priv": {"type": "signal", "owner": "bob", "user": "+1", "recipient": "+2"},
"stale_flag": {"type": "pushover", "token": "t2", "user": "u2", "private": True},
}
}
def test_public_channel_visible_to_all():
def test_global_channel_visible_to_all():
for uname in ("alice", "bob", "carol"):
user = {"username": uname, "admin": False}
assert "pub_ch" in _visible(CONFIG_VISIBILITY, user)
assert "pub_ch" in _visible(CONFIG_VISIBILITY, {"username": uname, "admin": False})
def test_private_channel_visible_only_to_owner():
def test_owned_channel_visible_only_to_owner():
alice = {"username": "alice", "admin": False}
bob = {"username": "bob", "admin": False}
carol = {"username": "carol", "admin": False}
bob = {"username": "bob", "admin": False}
assert "alice_priv" in _visible(CONFIG_VISIBILITY, alice)
assert "alice_priv" not in _visible(CONFIG_VISIBILITY, bob)
assert "alice_priv" not in _visible(CONFIG_VISIBILITY, carol)
assert "bob_priv" in _visible(CONFIG_VISIBILITY, bob)
assert "bob_priv" not in _visible(CONFIG_VISIBILITY, alice)
def test_admin_sees_all_channels():
admin = {"username": "adminuser", "admin": True}
visible = _visible(CONFIG_VISIBILITY, admin)
assert visible == {"pub_ch", "alice_priv", "bob_priv", "admin_owned"}
assert _visible(CONFIG_VISIBILITY, admin) == {"pub_ch", "alice_priv", "bob_priv", "stale_flag"}
def test_admin_owned_channel_is_public_by_default():
def test_stale_private_flag_without_owner_is_global():
"""Owner-presence is the single signal; a leftover private flag is ignored."""
alice = {"username": "alice", "admin": False}
assert "admin_owned" in _visible(CONFIG_VISIBILITY, alice)
assert "stale_flag" in _visible(CONFIG_VISIBILITY, alice)
# ---------------------------------------------------------------------------
+102 -10
View File
@@ -24,7 +24,7 @@ def test_sections_have_section_mode():
sections = settings_mod.get_settings_sections(CFG)
for s in sections:
assert "section_mode" in s, f"Section {s['id']} missing section_mode"
assert s["section_mode"] in ("form", "yaml", "channels", "hosts")
assert s["section_mode"] in ("form", "yaml", "channels", "hosts", "thresholds")
def test_sections_have_api_section():
@@ -42,15 +42,13 @@ def test_network_section_has_editable_fields():
assert len(editable) >= 2 # hbd_port, ws_port at minimum
def test_yaml_sections_have_correct_mode():
def test_thresholds_and_dns_section_modes():
sections = settings_mod.get_settings_sections(CFG)
yaml_sections = {s["id"]: s for s in sections if s["section_mode"] == "yaml"}
assert "channels" not in yaml_sections # now uses "channels" mode
assert "hosts" not in yaml_sections # now uses "hosts" mode
assert "thresholds" in yaml_sections
assert "dns" in yaml_sections
assert yaml_sections["thresholds"]["api_section"] == "thresholds"
assert yaml_sections["dns"]["api_section"] == "dns"
by_id = {s["id"]: s for s in sections}
assert by_id["thresholds"]["section_mode"] == "thresholds"
assert by_id["thresholds"]["api_section"] == "thresholds"
assert by_id["dns"]["section_mode"] == "form"
assert by_id["dns"]["api_section"] == "dns"
def test_hosts_section_uses_hosts_mode():
@@ -70,7 +68,7 @@ def test_channels_section_uses_channels_mode():
assert ch["name"] == "pushover_ops"
assert ch["type"] == "pushover"
assert "owner" in ch
assert "private" in ch
assert "editable" in ch
def test_channel_type_schemas_exported():
@@ -112,3 +110,97 @@ def test_users_section_has_user_list():
assert users_sec["users"][0]["username"] == "alice"
# Password hash never exposed
assert "password" not in users_sec["users"][0]
# ---------------------------------------------------------------------------
# Per-user filtering (owners/managers on the settings page)
# ---------------------------------------------------------------------------
from types import SimpleNamespace # noqa: E402
MULTI_CFG = {
**CFG,
"users": {
"alice": {"full_name": "Alice", "admin": True, "password": "x"},
"bob": {"full_name": "Bob", "admin": False, "password": "x"},
},
"notification_channels": {
"global_ch": {"type": "pushover", "token": "t", "user": "u"},
"bob_ch": {"type": "pushover", "token": "t", "user": "u", "owner": "bob"},
"carol_ch": {"type": "pushover", "token": "t", "user": "u", "owner": "carol"},
},
"threshold_configs": {
"bob_tc": {"owner": "bob", "thresholds": {}},
"carol_tc": {"owner": "carol", "thresholds": {}},
},
"hosts": {
"bobhost": {"owner": "bob"},
"managedhost": {"owner": "carol", "managers": ["bob"]},
"carolhost": {"owner": "carol"},
},
}
BOB = SimpleNamespace(username="bob", admin=False)
ADMIN = SimpleNamespace(username="alice", admin=True)
def test_nonadmin_sees_only_three_sections():
sections = settings_mod.get_settings_sections(MULTI_CFG, user=BOB)
assert [s["id"] for s in sections] == ["channels", "hosts", "thresholds"]
def test_nonadmin_sections_hide_admin_fields():
sections = settings_mod.get_settings_sections(MULTI_CFG, user=BOB)
for s in sections:
assert s["fields"] == []
def test_nonadmin_hosts_filtered_with_is_owner():
sections = settings_mod.get_settings_sections(MULTI_CFG, user=BOB)
hosts = next(s for s in sections if s["id"] == "hosts")["hosts"]
by_name = {h["name"]: h for h in hosts}
assert set(by_name) == {"bobhost", "managedhost"}
assert by_name["bobhost"]["is_owner"] is True
assert by_name["managedhost"]["is_owner"] is False
def test_nonadmin_channels_filtered_with_editable():
sections = settings_mod.get_settings_sections(MULTI_CFG, user=BOB)
chans = {c["name"]: c for c in next(s for s in sections if s["id"] == "channels")["channels"]}
assert set(chans) == {"global_ch", "bob_ch"}
assert chans["bob_ch"]["editable"] is True
assert chans["global_ch"]["editable"] is False
def test_admin_sees_everything_with_editable():
sections = settings_mod.get_settings_sections(MULTI_CFG, user=ADMIN)
ids = [s["id"] for s in sections]
assert "network" in ids and "users" in ids
chans = {c["name"]: c for c in next(s for s in sections if s["id"] == "channels")["channels"]}
assert set(chans) == {"global_ch", "bob_ch", "carol_ch"}
assert all(c["editable"] for c in chans.values())
def test_settings_data_pickers_filtered_for_nonadmin():
data = settings_mod.get_settings_data(MULTI_CFG, user=BOB)
assert data["all_channel_names"] == ["bob_ch", "global_ch"]
assert data["all_threshold_configs"] == ["bob_tc"]
def test_no_user_means_admin_view():
sections = settings_mod.get_settings_sections(MULTI_CFG) # auth disabled
assert len(sections) > 3
def test_filtering_unaffected_by_other_users_in_config():
"""The users-section loop must not clobber the requesting username (regression)."""
cfg = dict(MULTI_CFG)
cfg["users"] = {
"alice": {"full_name": "Alice", "admin": True, "password": "x"},
"bob": {"full_name": "Bob", "admin": False, "password": "x"},
"zed": {"full_name": "Zed", "admin": False, "password": "x"}, # bob is not last
}
sections = settings_mod.get_settings_sections(cfg, user=BOB)
hosts = {h["name"] for h in next(s for s in sections if s["id"] == "hosts")["hosts"]}
assert hosts == {"bobhost", "managedhost"}
+153
View File
@@ -0,0 +1,153 @@
"""Tests for RTT history capture in udp.py's handle_datagram."""
import time
from hbd.common.proto import dicttos
from hbd.server import hbdclass
from hbd.server.udp import handle_datagram, parse_message
class _FakeTransport:
def __init__(self):
self.sent = []
def sendto(self, data, addr):
self.sent.append((data, addr))
def _htb(name, rtt=None, interval=0, boot=0):
d = {"name": name, "interval": interval, "id": 0}
if rtt is not None:
d["rtt"] = rtt
if boot:
d["boot"] = boot
return parse_message(dicttos("HTB", d))
def _base_ctx():
return {
"config": {},
"hbdclass": hbdclass,
"msg_to_websockets": None,
"DEBUG": 0,
"verbose": False,
}
def test_handle_datagram_records_rtt_history_for_new_connection():
hbdclass.Host.hosts.pop("rtt-hist-host", None)
handle_datagram(_htb("rtt-hist-host", rtt=42.5), ("127.0.0.1", 50000),
_FakeTransport(), _base_ctx())
host = hbdclass.Host.hosts["rtt-hist-host"]
samples = host.plugin_data.get("rtt_ipv4")
assert samples is not None
assert len(samples) == 1
ts, data = samples[0]
assert data == {"rtt": 42.5}
assert isinstance(ts, float)
def test_handle_datagram_appends_rtt_history_across_heartbeats():
hbdclass.Host.hosts.pop("rtt-hist-host2", None)
transport = _FakeTransport()
ctx = _base_ctx()
handle_datagram(_htb("rtt-hist-host2", rtt=10.0), ("127.0.0.1", 50000), transport, ctx)
handle_datagram(_htb("rtt-hist-host2", rtt=20.0), ("127.0.0.1", 50000), transport, ctx)
host = hbdclass.Host.hosts["rtt-hist-host2"]
samples = host.plugin_data["rtt_ipv4"]
assert [d["rtt"] for _, d in samples] == [10.0, 20.0]
def test_handle_datagram_skips_rtt_history_when_rtt_missing():
hbdclass.Host.hosts.pop("rtt-hist-host3", None)
handle_datagram(_htb("rtt-hist-host3", rtt=None), ("127.0.0.1", 50000),
_FakeTransport(), _base_ctx())
host = hbdclass.Host.hosts["rtt-hist-host3"]
assert "rtt_ipv4" not in host.plugin_data
def test_request_update_fires_on_recovery_even_with_rtt_history():
"""Regression for Finding 1: rtt_* keys must not permanently disable the
request_update gate. A connection recovering from a non-UP state must
still be asked to resend real plugin data, even though rtt_ipv4 already
holds samples from before the drop.
"""
hbdclass.Host.hosts.pop("rtt-hist-host4", None)
transport = _FakeTransport()
ctx = _base_ctx()
# First heartbeat: brand-new host, no plugin data at all yet.
handle_datagram(_htb("rtt-hist-host4", rtt=15.0), ("127.0.0.1", 50000), transport, ctx)
host = hbdclass.Host.hosts["rtt-hist-host4"]
assert host.plugin_data.get("rtt_ipv4") # rtt history now non-empty
# Simulate a recovery: connection was dropped (e.g. OVERDUE->UP after a
# missed heartbeat) and is about to come back UP on the next heartbeat.
conn = host.connections["IPv4"]
conn.state = hbdclass.Connection.DOWN
transport.sent.clear()
handle_datagram(_htb("rtt-hist-host4", rtt=16.0), ("127.0.0.1", 50000), transport, ctx)
ack_data, _ = transport.sent[0]
ack = parse_message(ack_data)
assert ack.get("request_update")
def test_ordinary_recovery_preserves_real_plugin_data_and_rtt_history():
"""An ordinary reconnect (no boot flag) — e.g. OVERDUE->UP after a
transient network blip must NOT wipe already-collected real plugin
data (e.g. cpu_monitor, os_info) or rtt_* history. Only an actual
client reboot invalidates that data (see the boot-flag test below).
"""
hbdclass.Host.hosts.pop("rtt-hist-host5", None)
transport = _FakeTransport()
ctx = _base_ctx()
for rtt in (10.0, 11.0, 12.0):
handle_datagram(_htb("rtt-hist-host5", rtt=rtt), ("127.0.0.1", 50000), transport, ctx)
host = hbdclass.Host.hosts["rtt-hist-host5"]
assert len(host.plugin_data["rtt_ipv4"]) == 3
# Simulate a drop, and pretend the client had previously sent real
# plugin data (collected before the connection went down).
conn = host.connections["IPv4"]
conn.state = hbdclass.Connection.DOWN
host.add_plugin_data("os_info", {"os": "linux"}, timestamp=time.time())
assert "os_info" in host.plugin_data
# Ordinary recovery heartbeat — no boot flag.
handle_datagram(_htb("rtt-hist-host5", rtt=13.0), ("127.0.0.1", 50000), transport, ctx)
assert "os_info" in host.plugin_data
assert len(host.plugin_data["rtt_ipv4"]) == 4
def test_boot_recovery_clears_real_plugin_data_but_preserves_rtt_history():
"""A recovery heartbeat carrying the boot flag (client process actually
restarted) must still wipe stale real plugin data, while rtt_* history
(still a valid measurement, unaffected by a client reboot) survives.
"""
hbdclass.Host.hosts.pop("rtt-hist-host6", None)
transport = _FakeTransport()
ctx = _base_ctx()
for rtt in (10.0, 11.0, 12.0):
handle_datagram(_htb("rtt-hist-host6", rtt=rtt), ("127.0.0.1", 50000), transport, ctx)
host = hbdclass.Host.hosts["rtt-hist-host6"]
assert len(host.plugin_data["rtt_ipv4"]) == 3
conn = host.connections["IPv4"]
conn.state = hbdclass.Connection.DOWN
host.add_plugin_data("os_info", {"os": "linux"}, timestamp=time.time())
assert "os_info" in host.plugin_data
# Recovery heartbeat with boot=1 — client process actually restarted.
handle_datagram(_htb("rtt-hist-host6", rtt=13.0, boot=1), ("127.0.0.1", 50000), transport, ctx)
assert "os_info" not in host.plugin_data
assert len(host.plugin_data["rtt_ipv4"]) == 4