Compare commits
37 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 8da3d550eb | |||
| a76d0fc840 | |||
| 94cbb31c48 | |||
| ae60844a8a | |||
| 49fa310361 | |||
| 28e2180f7b | |||
| ce0590f015 | |||
| f50acca509 | |||
| 72fc82b91f | |||
| 46f8c32c0b | |||
| 691f62aa69 | |||
| cffc9805f9 | |||
| 917d6a401b | |||
| 2bd3a9beb6 | |||
| 5523c60866 | |||
| ab37ac7194 | |||
| f811a19d80 | |||
| 6239825f43 | |||
| b56245bb23 | |||
| 331c4e804d | |||
| 9fd945a481 | |||
| 26df08eeff | |||
| 5819dd6b25 | |||
| 6fb67f8615 | |||
| e70ae6f176 | |||
| a77f6d380c | |||
| 6aae2a1dab | |||
| 85ee0e1040 | |||
| c4f09e9ced | |||
| 64710fd4cd | |||
| 1f5e7465a3 | |||
| b290b21e23 | |||
| 65c4267847 | |||
| 462a445235 | |||
| 368e178f93 | |||
| 6905bf266a | |||
| b6dcce4f35 |
@@ -267,6 +267,41 @@ All plugin metrics can be thresholded:
|
||||
- **Network**: errors_total, dropped packets, connection counts
|
||||
- **Nagios**: exit_code mapping (0=OK, 1=WARNING, 2=CRITICAL)
|
||||
|
||||
### Per-Host Threshold Profiles
|
||||
|
||||
Named threshold configurations let different hosts use different limits. A host's `threshold_config` can be a single name or a **list** — lists are applied left-to-right so profiles compose without duplication:
|
||||
|
||||
```yaml
|
||||
threshold_configs:
|
||||
default:
|
||||
thresholds:
|
||||
cpu_monitor:
|
||||
cpu_percent: {warning: 80, critical: 90}
|
||||
memory_monitor:
|
||||
memory_percent: {warning: 85, critical: 95}
|
||||
|
||||
tight_cpu: # override CPU limits only
|
||||
thresholds:
|
||||
cpu_monitor:
|
||||
cpu_percent: {warning: 60, critical: 75}
|
||||
|
||||
db_disk: # add a database partition check
|
||||
thresholds:
|
||||
disk_monitor:
|
||||
partitions:
|
||||
/var/lib/postgresql:
|
||||
percent: {warning: 75, critical: 88}
|
||||
|
||||
hosts:
|
||||
web-01:
|
||||
threshold_config: default # single profile
|
||||
|
||||
db-01:
|
||||
threshold_config: [tight_cpu, db_disk] # layered: CPU override + extra disk check
|
||||
```
|
||||
|
||||
Each named config's overrides are applied in order on top of the defaults. Metrics not mentioned in a profile are inherited unchanged.
|
||||
|
||||
See [docs/THRESHOLD_ALERTING.md](docs/THRESHOLD_ALERTING.md) for comprehensive documentation including best practices, troubleshooting, and advanced configuration.
|
||||
|
||||
---
|
||||
@@ -441,6 +476,68 @@ plugins:
|
||||
|
||||
All monitoring plugins default to 5-minute (300 second) intervals, but can be customized as needed.
|
||||
|
||||
### hbc_mini — single-file client (no external dependencies)
|
||||
|
||||
`scripts/hbc_mini.py` is a self-contained version of the heartbeat client that requires only Python 3.8+ and no external packages. Copy it to any host and run it directly — no virtualenv, no `pip install`.
|
||||
|
||||
```bash
|
||||
# Basic usage
|
||||
python3 hbc_mini.py your-server.example.com
|
||||
|
||||
# Run as daemon
|
||||
python3 hbc_mini.py -d your-server.example.com
|
||||
|
||||
# Send a boot message
|
||||
python3 hbc_mini.py -b your-server.example.com
|
||||
|
||||
# Send a one-off message
|
||||
python3 hbc_mini.py -m "maintenance starting" your-server.example.com
|
||||
```
|
||||
|
||||
**Config:** `~/.hbc.json` (same keys as `~/.hbc.yaml`, JSON format). Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"hb_port": 50003,
|
||||
"interval": 30,
|
||||
"plugins": {
|
||||
"ping_monitor": {
|
||||
"interval": 60,
|
||||
"hosts": ["8.8.8.8", "192.168.1.1"]
|
||||
},
|
||||
"nagios_runner": {
|
||||
"interval": 300,
|
||||
"commands": [
|
||||
{"name": "check_load", "command": "/usr/lib/nagios/plugins/check_load -w 5,4,3 -c 10,8,6"}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Plugin availability:**
|
||||
|
||||
| Plugin | Platform | Data source |
|
||||
|---|---|---|
|
||||
| `os_info` | all | `platform` stdlib |
|
||||
| `ping_monitor` | all | `ping` subprocess |
|
||||
| `nagios_runner` | all (not Windows) | subprocess |
|
||||
| `cpu_monitor` | Linux | `/proc/stat` |
|
||||
| `memory_monitor` | Linux | `/proc/meminfo` |
|
||||
| `disk_monitor` | Linux, macOS, BSD | `df -P` subprocess |
|
||||
| `network_monitor` | Linux | `/proc/net/dev` |
|
||||
|
||||
**What is not available compared to the full `hbc`:**
|
||||
|
||||
- No YAML config (use JSON instead)
|
||||
- No `filesystem_info` plugin
|
||||
- `cpu_monitor` does not report per-core usage or CPU frequency (no psutil)
|
||||
- Plugins cannot be loaded from external `.py` files — all plugins are compiled in
|
||||
|
||||
Everything else — heartbeat protocol, ACK/CMD/UPD handling, `hb_install.sh`-based self-update, daemonize, syslog — is identical to the full client.
|
||||
|
||||
---
|
||||
|
||||
## 🐞 Debugging in VS Code
|
||||
|
||||
This repository includes a ready-to-use `.vscode/launch.json` with configurations to run or attach the VS Code debugger to `hbd`.
|
||||
|
||||
+183
-47
@@ -814,34 +814,32 @@ Planned features:
|
||||
|
||||
## Multi-Threshold Configuration
|
||||
|
||||
**New in version 2.0**: Support for multiple named threshold configurations with per-host mapping.
|
||||
Support for multiple named threshold configurations with per-host mapping and composable layering.
|
||||
|
||||
### Overview
|
||||
|
||||
The multi-threshold feature allows you to:
|
||||
- Define multiple sets of threshold configurations
|
||||
- Map different hosts to different threshold sets
|
||||
- Define multiple named threshold configurations
|
||||
- Assign one or more configurations to each host
|
||||
- Compose configurations by layering — each named config's overrides are applied in order on top of the defaults
|
||||
- Use different sensitivity levels for different environments
|
||||
- Maintain a default configuration for unmapped hosts
|
||||
|
||||
### Configuration Structure
|
||||
|
||||
Named configurations are defined under `threshold_configs`. Each host selects which ones to use via `threshold_config` in the `hosts` section (a string for a single config, or a list to layer multiple):
|
||||
|
||||
```yaml
|
||||
# Optional: Set the default configuration name (defaults to "default")
|
||||
# Optional: set the default configuration name (defaults to "default")
|
||||
default_threshold_config: "default"
|
||||
|
||||
# Define multiple named threshold configurations
|
||||
threshold_configs:
|
||||
# Configuration name 1
|
||||
default:
|
||||
thresholds:
|
||||
# Standard threshold definitions
|
||||
cpu_monitor:
|
||||
cpu_percent:
|
||||
warning: 80.0
|
||||
critical: 90.0
|
||||
|
||||
# Configuration name 2
|
||||
high_sensitivity:
|
||||
thresholds:
|
||||
cpu_monitor:
|
||||
@@ -849,7 +847,6 @@ threshold_configs:
|
||||
warning: 60.0
|
||||
critical: 75.0
|
||||
|
||||
# Configuration name 3
|
||||
low_sensitivity:
|
||||
thresholds:
|
||||
cpu_monitor:
|
||||
@@ -857,14 +854,77 @@ threshold_configs:
|
||||
warning: 90.0
|
||||
critical: 95.0
|
||||
|
||||
# Map specific hosts to specific configurations
|
||||
host_threshold_mapping:
|
||||
prod-web-01: high_sensitivity
|
||||
prod-web-02: high_sensitivity
|
||||
dev-server-01: low_sensitivity
|
||||
# Unmapped hosts use default_threshold_config
|
||||
hosts:
|
||||
prod-web-01:
|
||||
threshold_config: high_sensitivity # single config
|
||||
|
||||
dev-server-01:
|
||||
threshold_config: low_sensitivity
|
||||
|
||||
# Hosts with no threshold_config use default_threshold_config
|
||||
```
|
||||
|
||||
### Composable Configurations (list form)
|
||||
|
||||
`threshold_config` can be a list. Configs are applied **left to right**: the defaults are the base, then each named config's overrides are layered on top. Later entries in the list win on any metric they define.
|
||||
|
||||
```yaml
|
||||
threshold_configs:
|
||||
default:
|
||||
thresholds:
|
||||
cpu_monitor:
|
||||
cpu_percent: {warning: 80, critical: 90}
|
||||
memory_monitor:
|
||||
memory_percent: {warning: 85, critical: 95}
|
||||
disk_monitor:
|
||||
partitions:
|
||||
/:
|
||||
percent: {warning: 80, critical: 90}
|
||||
|
||||
# Tighter CPU limits for busy servers
|
||||
high_cpu_load:
|
||||
thresholds:
|
||||
cpu_monitor:
|
||||
cpu_percent: {warning: 60, critical: 75}
|
||||
|
||||
# Tighter disk limits for data-heavy servers
|
||||
busy_disk:
|
||||
thresholds:
|
||||
disk_monitor:
|
||||
partitions:
|
||||
/:
|
||||
percent: {warning: 70, critical: 85}
|
||||
|
||||
hosts:
|
||||
# Gets default thresholds only
|
||||
web-01:
|
||||
threshold_config: default
|
||||
|
||||
# Gets tighter CPU limits, default memory and disk
|
||||
build-server:
|
||||
threshold_config: high_cpu_load
|
||||
|
||||
# Layers both: tighter CPU AND tighter disk, default memory
|
||||
db-01:
|
||||
threshold_config: [high_cpu_load, busy_disk]
|
||||
|
||||
# Three layers: busy_disk overrides high_cpu_load if they conflict
|
||||
storage-01:
|
||||
threshold_config: [default, high_cpu_load, busy_disk]
|
||||
```
|
||||
|
||||
**How layering works:**
|
||||
|
||||
Starting from the `default` thresholds:
|
||||
|
||||
| Layer | Applied config | Effect |
|
||||
|-------|---------------|--------|
|
||||
| Base | `default` | all default thresholds |
|
||||
| +1 | `high_cpu_load` | cpu_percent overridden to 60/75 |
|
||||
| +2 | `busy_disk` | disk percent overridden to 70/85; cpu_percent stays at 60/75 |
|
||||
|
||||
Each named config only overrides the metrics it explicitly defines. Metrics not mentioned in a config inherit from the layers beneath.
|
||||
|
||||
### Use Cases
|
||||
|
||||
#### 1. Environment-Based Thresholds
|
||||
@@ -887,11 +947,15 @@ threshold_configs:
|
||||
warning: 90.0 # More relaxed for dev
|
||||
critical: 98.0
|
||||
|
||||
host_threshold_mapping:
|
||||
prod-web-01: production
|
||||
prod-web-02: production
|
||||
dev-web-01: development
|
||||
dev-web-02: development
|
||||
hosts:
|
||||
prod-web-01:
|
||||
threshold_config: production
|
||||
prod-web-02:
|
||||
threshold_config: production
|
||||
dev-web-01:
|
||||
threshold_config: development
|
||||
dev-web-02:
|
||||
threshold_config: development
|
||||
```
|
||||
|
||||
#### 2. Server Role-Based Thresholds
|
||||
@@ -914,7 +978,7 @@ threshold_configs:
|
||||
warning: 70.0
|
||||
critical: 85.0
|
||||
memory_monitor:
|
||||
percent:
|
||||
memory_percent:
|
||||
warning: 90.0 # Databases can use high memory
|
||||
critical: 97.0
|
||||
disk_monitor:
|
||||
@@ -927,17 +991,23 @@ threshold_configs:
|
||||
cache:
|
||||
thresholds:
|
||||
memory_monitor:
|
||||
percent:
|
||||
memory_percent:
|
||||
warning: 95.0 # Redis/Memcached can use very high memory
|
||||
critical: 99.0
|
||||
|
||||
host_threshold_mapping:
|
||||
web-01: webserver
|
||||
web-02: webserver
|
||||
db-01: database
|
||||
db-02: database
|
||||
redis-01: cache
|
||||
memcached-01: cache
|
||||
hosts:
|
||||
web-01:
|
||||
threshold_config: webserver
|
||||
web-02:
|
||||
threshold_config: webserver
|
||||
db-01:
|
||||
threshold_config: database
|
||||
db-02:
|
||||
threshold_config: database
|
||||
redis-01:
|
||||
threshold_config: cache
|
||||
memcached-01:
|
||||
threshold_config: cache
|
||||
```
|
||||
|
||||
#### 3. Sensitivity Levels
|
||||
@@ -952,7 +1022,7 @@ threshold_configs:
|
||||
partitions:
|
||||
/:
|
||||
percent:
|
||||
warning: 70.0 # Very sensitive
|
||||
warning: 70.0
|
||||
critical: 80.0
|
||||
hysteresis: 0.15
|
||||
|
||||
@@ -976,12 +1046,69 @@ threshold_configs:
|
||||
critical: 98.0
|
||||
hysteresis: 0.05
|
||||
|
||||
host_threshold_mapping:
|
||||
payment-gateway: critical
|
||||
auth-server: critical
|
||||
web-01: standard
|
||||
web-02: standard
|
||||
test-server: relaxed
|
||||
hosts:
|
||||
payment-gateway:
|
||||
threshold_config: critical
|
||||
auth-server:
|
||||
threshold_config: critical
|
||||
web-01:
|
||||
threshold_config: standard
|
||||
web-02:
|
||||
threshold_config: standard
|
||||
test-server:
|
||||
threshold_config: relaxed
|
||||
```
|
||||
|
||||
#### 4. Composable Profiles
|
||||
|
||||
Build host-specific thresholds by combining small, focused configs:
|
||||
|
||||
```yaml
|
||||
threshold_configs:
|
||||
# Baseline — everything at default levels
|
||||
default:
|
||||
thresholds:
|
||||
cpu_monitor:
|
||||
cpu_percent: {warning: 80, critical: 90}
|
||||
memory_monitor:
|
||||
memory_percent: {warning: 85, critical: 95}
|
||||
|
||||
# Overlay: tighter CPU only
|
||||
tight_cpu:
|
||||
thresholds:
|
||||
cpu_monitor:
|
||||
cpu_percent: {warning: 60, critical: 75}
|
||||
|
||||
# Overlay: tighter memory only
|
||||
tight_memory:
|
||||
thresholds:
|
||||
memory_monitor:
|
||||
memory_percent: {warning: 70, critical: 85}
|
||||
|
||||
# Overlay: extra disk partition for database servers
|
||||
db_disk:
|
||||
thresholds:
|
||||
disk_monitor:
|
||||
partitions:
|
||||
/var/lib/postgresql:
|
||||
percent: {warning: 75, critical: 88}
|
||||
|
||||
hosts:
|
||||
# Plain web server
|
||||
web-01:
|
||||
threshold_config: default
|
||||
|
||||
# Build server: tight CPU, default memory and disk
|
||||
build-01:
|
||||
threshold_config: tight_cpu
|
||||
|
||||
# Database: tight CPU + tight memory + extra disk partition
|
||||
db-01:
|
||||
threshold_config: [tight_cpu, tight_memory, db_disk]
|
||||
|
||||
# Replica database: tight memory + extra disk, normal CPU
|
||||
db-02:
|
||||
threshold_config: [tight_memory, db_disk]
|
||||
```
|
||||
|
||||
### Backward Compatibility
|
||||
@@ -1012,16 +1139,25 @@ threshold_configs:
|
||||
|
||||
### Configuration Priority
|
||||
|
||||
1. **Host-specific mapping**: If host is in `host_threshold_mapping`, use that config
|
||||
2. **Default config**: Use `default_threshold_config`
|
||||
3. **First alphabetically**: If default not found, use first config alphabetically
|
||||
4. **Legacy fallback**: If `threshold_configs` not present, use `thresholds`
|
||||
1. **Host `threshold_config` (list)**: Layer each named config's overrides left-to-right on top of the defaults
|
||||
2. **Host `threshold_config` (string)**: Use that single named config directly
|
||||
3. **`host_threshold_mapping`** (legacy): Same as above, string only
|
||||
4. **`default_threshold_config`**: Used for hosts with no mapping
|
||||
5. **First alphabetically**: If the default config is not found, use the first config alphabetically
|
||||
6. **Legacy `thresholds` section**: Used when `threshold_configs` is absent entirely
|
||||
|
||||
### Example: Complete Multi-Threshold Setup
|
||||
### Backward Compatibility
|
||||
|
||||
See `hbd/config_multi_threshold_example.yaml` for a complete example with:
|
||||
- 4 named configurations (default, high_sensitivity, low_sensitivity, database)
|
||||
- Host-to-config mappings for production, development, and test systems
|
||||
- Specialized database server thresholds
|
||||
- Custom display messages with plugin data
|
||||
The legacy `host_threshold_mapping` top-level key and the flat `thresholds` section are still fully supported:
|
||||
|
||||
```yaml
|
||||
# Still works — equivalent to hosts: {prod-web-01: {threshold_config: high_sensitivity}}
|
||||
host_threshold_mapping:
|
||||
prod-web-01: high_sensitivity
|
||||
|
||||
# Still works — equivalent to threshold_configs: {default: {thresholds: ...}}
|
||||
thresholds:
|
||||
cpu_monitor:
|
||||
cpu_percent: {warning: 80, critical: 90}
|
||||
```
|
||||
|
||||
|
||||
+1
-1
@@ -14,4 +14,4 @@ Install options:
|
||||
"""
|
||||
|
||||
__all__ = ["__version__"]
|
||||
__version__ = "5.1.5"
|
||||
__version__ = "5.1.16"
|
||||
|
||||
+14
-1
@@ -55,6 +55,7 @@ class AsyncConnection:
|
||||
|
||||
self.transport: Optional[asyncio.DatagramTransport] = None
|
||||
self.protocol: Optional[asyncio.DatagramProtocol] = None
|
||||
self._dead = False
|
||||
|
||||
self.logger = logging.getLogger(f"hbc.conn.{addr}")
|
||||
|
||||
@@ -92,6 +93,9 @@ class AsyncConnection:
|
||||
msg: Message dictionary
|
||||
msg_id: Message ID (HTB, PLG, etc.)
|
||||
"""
|
||||
if self._dead:
|
||||
return
|
||||
|
||||
if not self.transport:
|
||||
await self.open()
|
||||
|
||||
@@ -166,7 +170,9 @@ class HeartbeatProtocol(asyncio.DatagramProtocol):
|
||||
|
||||
def error_received(self, exc):
|
||||
"""Handle protocol errors."""
|
||||
self.logger.error(f"Protocol error: {exc}")
|
||||
self.logger.warning(f"Protocol error on {self.connection.addr}: {exc} — dropping connection")
|
||||
self.connection._dead = True
|
||||
self.connection.close()
|
||||
|
||||
|
||||
async def handle_command(conn: AsyncConnection, msg: dict):
|
||||
@@ -519,6 +525,13 @@ async def async_main(args, config):
|
||||
for sig in (signal.SIGTERM, signal.SIGINT):
|
||||
loop.add_signal_handler(sig, stop)
|
||||
|
||||
def _sighup():
|
||||
global dorestart
|
||||
dorestart = True
|
||||
stop()
|
||||
|
||||
loop.add_signal_handler(signal.SIGHUP, _sighup)
|
||||
|
||||
# Start async tasks
|
||||
# Heartbeat senders (one per connection)
|
||||
for conn in connections:
|
||||
|
||||
@@ -60,6 +60,7 @@ class OSInfoPlugin(InfoPlugin):
|
||||
"python_version": platform.python_version(),
|
||||
"python_implementation": platform.python_implementation(),
|
||||
"hbc_version": hbc_version,
|
||||
"hbc_type": "full",
|
||||
}
|
||||
|
||||
# Add Linux-specific distribution info
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
"""
|
||||
ZFS pool monitoring plugin for Heartbeat.
|
||||
|
||||
Collects per-pool health, capacity, and cumulative I/O statistics via zpool(8).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import shutil
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from hbd.client.plugin import MonitorPlugin
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _int(s: str) -> Optional[int]:
|
||||
try:
|
||||
return int(s.strip().rstrip("KMGTkBkmgt%x"))
|
||||
except (ValueError, AttributeError):
|
||||
return None
|
||||
|
||||
|
||||
def _float(s: str) -> Optional[float]:
|
||||
try:
|
||||
return float(s.strip().rstrip("%x"))
|
||||
except (ValueError, AttributeError):
|
||||
return None
|
||||
|
||||
|
||||
class ZFSMonitorPlugin(MonitorPlugin):
|
||||
"""Monitor ZFS pool health, capacity, and I/O statistics.
|
||||
|
||||
Collects per pool:
|
||||
- health: ONLINE, DEGRADED, FAULTED, etc.
|
||||
- size / alloc / free: total, allocated and free bytes
|
||||
- capacity: percentage used (0-100)
|
||||
- frag: fragmentation percentage
|
||||
- dedup: deduplication ratio
|
||||
- read_ops / write_ops: cumulative I/O operations since last boot/clear
|
||||
- read_bw / write_bw: cumulative bytes transferred since last boot/clear
|
||||
|
||||
Configuration:
|
||||
interval: collection interval in seconds (default: 300)
|
||||
pools: list of pool names to monitor (default: all)
|
||||
"""
|
||||
|
||||
name = "zfs_monitor"
|
||||
description = "ZFS pool health, capacity, and I/O statistics"
|
||||
interval = 300
|
||||
|
||||
def __init__(self, config: Optional[Dict[str, Any]] = None):
|
||||
super().__init__(config)
|
||||
self.interval = self.config.get("interval", 300)
|
||||
self._pools_filter: Optional[List[str]] = self.config.get("pools", None)
|
||||
|
||||
async def initialize(self) -> bool:
|
||||
if not shutil.which("zpool"):
|
||||
self.skip_reason = "zpool not found"
|
||||
return False
|
||||
logger.info("ZFS monitor initialized (interval: %ds)", self.interval)
|
||||
return True
|
||||
|
||||
async def _run(self, *args: str) -> List[str]:
|
||||
"""Run a command and return its stdout lines, or [] on error."""
|
||||
try:
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
*args,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.DEVNULL,
|
||||
)
|
||||
stdout, _ = await asyncio.wait_for(proc.communicate(), timeout=15)
|
||||
return stdout.decode(errors="replace").splitlines()
|
||||
except (FileNotFoundError, asyncio.TimeoutError) as exc:
|
||||
logger.warning("zfs_monitor: %s: %s", args[0], exc)
|
||||
return []
|
||||
|
||||
async def _zpool_list(self) -> Dict[str, Dict]:
|
||||
"""Return per-pool health and capacity from `zpool list`."""
|
||||
lines = await self._run(
|
||||
"zpool", "list", "-H", "-p",
|
||||
"-o", "name,health,size,alloc,free,cap,frag,dedup",
|
||||
)
|
||||
pools: Dict[str, Dict] = {}
|
||||
for line in lines:
|
||||
parts = line.split("\t")
|
||||
if len(parts) < 8:
|
||||
continue
|
||||
name = parts[0].strip()
|
||||
if self._pools_filter and name not in self._pools_filter:
|
||||
continue
|
||||
pools[name] = {
|
||||
"health": parts[1].strip(),
|
||||
"size": _int(parts[2]),
|
||||
"alloc": _int(parts[3]),
|
||||
"free": _int(parts[4]),
|
||||
"capacity": _float(parts[5]),
|
||||
"frag": _float(parts[6]),
|
||||
"dedup": _float(parts[7]),
|
||||
}
|
||||
return pools
|
||||
|
||||
async def _zpool_iostat(self) -> Dict[str, Dict]:
|
||||
"""Return per-pool cumulative I/O counters from `zpool iostat`."""
|
||||
lines = await self._run("zpool", "iostat", "-H", "-p")
|
||||
io: Dict[str, Dict] = {}
|
||||
for line in lines:
|
||||
parts = line.split("\t")
|
||||
if len(parts) < 7:
|
||||
continue
|
||||
name = parts[0].strip()
|
||||
if not name or name.startswith(" "):
|
||||
continue
|
||||
io[name] = {
|
||||
"read_ops": _int(parts[3]),
|
||||
"write_ops": _int(parts[4]),
|
||||
"read_bw": _int(parts[5]),
|
||||
"write_bw": _int(parts[6]),
|
||||
}
|
||||
return io
|
||||
|
||||
async def _collect_metrics(self) -> Dict[str, Any]:
|
||||
pools, io = await asyncio.gather(self._zpool_list(), self._zpool_iostat())
|
||||
for name, stats in io.items():
|
||||
if name in pools:
|
||||
pools[name].update(stats)
|
||||
return {"pools": pools}
|
||||
|
||||
|
||||
plugin = ZFSMonitorPlugin
|
||||
+5
-6
@@ -144,17 +144,16 @@ def cmd_notify(args):
|
||||
url=f"{base_url}/plugins" if base_url else "",
|
||||
)
|
||||
|
||||
# Bypass min_level for explicit test sends; run async channels directly
|
||||
import asyncio
|
||||
from .notify import _send_matrix_async, _send_sms_voipms_async, _DRIVERS
|
||||
ch_type = channel_cfg.get("type", "")
|
||||
print(f"Sending via {args.channel} ({ch_type}): {title} — {args.message}")
|
||||
|
||||
if ch_type in ("matrix", "sms_voipms"):
|
||||
from .notify import _send_matrix_async, _send_sms_voipms_async
|
||||
driver_async = _send_matrix_async if ch_type == "matrix" else _send_sms_voipms_async
|
||||
ok = asyncio.run(driver_async(channel_cfg, notif))
|
||||
if ch_type == "matrix":
|
||||
ok = asyncio.run(_send_matrix_async(channel_cfg, notif))
|
||||
elif ch_type == "sms_voipms":
|
||||
ok = asyncio.run(_send_sms_voipms_async(channel_cfg, notif))
|
||||
else:
|
||||
from .notify import _DRIVERS
|
||||
driver = _DRIVERS.get(ch_type)
|
||||
if driver is None:
|
||||
print(f"Error: unknown channel type '{ch_type}'", file=sys.stderr)
|
||||
|
||||
@@ -225,7 +225,7 @@ def get_watchhosts(config):
|
||||
hosts_config = config.get("hosts", {})
|
||||
if isinstance(hosts_config, dict):
|
||||
for host_name, host_attrs in hosts_config.items():
|
||||
if isinstance(host_attrs, dict) and host_attrs.get("watch", False):
|
||||
if isinstance(host_attrs, dict) and host_attrs.get("watch", True):
|
||||
watchhosts.append(host_name)
|
||||
return watchhosts
|
||||
|
||||
|
||||
@@ -95,7 +95,7 @@ class Connection:
|
||||
if not Null:
|
||||
d["addr"] = self.addr
|
||||
if self.rtts[-1]:
|
||||
d["rtt"] = "%0.1f" % self.rtts[-1]
|
||||
d["rtt"] = "%d" % round(self.rtts[-1])
|
||||
elif self.state == Connection.UNKNOWN:
|
||||
d["rtt"] = ""
|
||||
else:
|
||||
@@ -286,7 +286,7 @@ class Host:
|
||||
Host.hosts[name] = self
|
||||
self.num = num
|
||||
self.dyn = False
|
||||
self.watched = False
|
||||
self.watched = True
|
||||
self.upcount = 0
|
||||
self.interval = 0
|
||||
self.doesack = -1
|
||||
@@ -304,6 +304,7 @@ class Host:
|
||||
|
||||
def statedict(self):
|
||||
d = {}
|
||||
d["raw_name"] = self.name
|
||||
d["name"] = self.name
|
||||
if self.dyn:
|
||||
d["name"] += "*"
|
||||
|
||||
+52
-2
@@ -1,7 +1,11 @@
|
||||
"""HTTP server implementation using aiohttp and jinja2."""
|
||||
|
||||
import asyncio
|
||||
import datetime
|
||||
import json
|
||||
import platform
|
||||
import socket
|
||||
import sys
|
||||
import time
|
||||
import urllib.parse
|
||||
import os
|
||||
@@ -111,6 +115,7 @@ async def start(
|
||||
This function is intended to be awaited inside the main asyncio event loop.
|
||||
"""
|
||||
get_now = get_now or (lambda: time.time())
|
||||
_start_epoch = time.time()
|
||||
|
||||
async def old_index(request):
|
||||
_require_auth_redirect(request)
|
||||
@@ -253,7 +258,9 @@ async def start(
|
||||
extra_scripts=extra_scripts,
|
||||
hbd_version=hbd_version,
|
||||
hosts=[
|
||||
hbdclass.Host.hosts[h].stateinfo() for h in sorted(hbdclass.Host.hosts)
|
||||
hbdclass.Host.hosts[h].stateinfo()
|
||||
for h in sorted(hbdclass.Host.hosts)
|
||||
if _can_operate_host(current_user, hbdclass.Host.hosts[h])
|
||||
],
|
||||
messages=data.msgs[-30:],
|
||||
current_user=current_user.to_dict() if current_user else None,
|
||||
@@ -505,7 +512,7 @@ async def start(
|
||||
hosts_with_plugins = []
|
||||
for hostname in sorted(hbdclass.Host.hosts.keys()):
|
||||
host = hbdclass.Host.hosts[hostname]
|
||||
if not _can_view_host(current_user, host):
|
||||
if not _can_operate_host(current_user, host):
|
||||
continue
|
||||
if host.plugin_data:
|
||||
hosts_with_plugins.append({
|
||||
@@ -806,6 +813,48 @@ async def start(
|
||||
)
|
||||
return web.Response(text=body, content_type="text/html")
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# About page
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
async def about_page(request):
|
||||
"""GET /about — version, runtime, and project information."""
|
||||
current_user, _ = _require_auth_redirect(request)
|
||||
pkg_dir = os.path.dirname(__file__)
|
||||
templates_dir = config.get("templates_dir", os.path.join(pkg_dir, "templates"))
|
||||
env = jinja2.Environment(loader=jinja2.FileSystemLoader(templates_dir))
|
||||
from hbd import __version__ as hbd_version
|
||||
|
||||
uptime_secs = int(time.time() - _start_epoch)
|
||||
days, rem = divmod(uptime_secs, 86400)
|
||||
hours, rem = divmod(rem, 3600)
|
||||
mins, secs = divmod(rem, 60)
|
||||
if days:
|
||||
uptime_str = f"{days}d {hours}h {mins}m"
|
||||
elif hours:
|
||||
uptime_str = f"{hours}h {mins}m {secs}s"
|
||||
else:
|
||||
uptime_str = f"{mins}m {secs}s"
|
||||
|
||||
start_dt = datetime.datetime.fromtimestamp(_start_epoch)
|
||||
start_time_str = start_dt.strftime("%Y-%m-%d %H:%M:%S")
|
||||
|
||||
tmpl = env.get_template("about.html")
|
||||
body = tmpl.render(
|
||||
title="About - Heartbeat",
|
||||
header="About",
|
||||
hbd_version=hbd_version,
|
||||
python_version=f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro} ({platform.python_implementation()})",
|
||||
server_hostname=socket.gethostname(),
|
||||
start_epoch=int(_start_epoch),
|
||||
start_time_str=start_time_str,
|
||||
uptime_str=uptime_str,
|
||||
host_count=len(hbdclass.Host.hosts),
|
||||
current_user=current_user.to_dict() if current_user else None,
|
||||
active_page="about",
|
||||
)
|
||||
return web.Response(text=body, content_type="text/html")
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Settings page (admin only)
|
||||
# -------------------------------------------------------------------------
|
||||
@@ -859,6 +908,7 @@ async def start(
|
||||
web.get("/live", live),
|
||||
web.get("/plugins", plugins_page),
|
||||
web.get("/alerts", alerts_page),
|
||||
web.get("/about", about_page),
|
||||
web.get("/profile", profile_page),
|
||||
web.get("/settings", settings_page),
|
||||
web.get("/static/{path:.*}", static),
|
||||
|
||||
@@ -210,7 +210,6 @@ async def _run_async(config, config_path=None):
|
||||
ctx = dict(
|
||||
config=config,
|
||||
hbdclass=hbdclass,
|
||||
log=eventlog,
|
||||
msg_to_websockets=msg_to_websockets,
|
||||
msg_journal=msg_journal,
|
||||
threshold_checker=threshold_checker,
|
||||
@@ -237,7 +236,6 @@ async def _run_async(config, config_path=None):
|
||||
restore_ctx = dict(
|
||||
config=config,
|
||||
hbdclass=hbdclass,
|
||||
log=eventlog,
|
||||
msg_to_websockets=msg_to_websockets,
|
||||
threshold_checker=threshold_checker,
|
||||
)
|
||||
|
||||
+25
-53
@@ -15,7 +15,6 @@ their own ``notification_channels`` list. When no users are configured the
|
||||
server runs silently (no notifications sent).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import asyncio
|
||||
import logging
|
||||
import smtplib
|
||||
@@ -30,13 +29,10 @@ from . import ws as ws_mod
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
msg_to_websockets = ws_mod.broadcast
|
||||
|
||||
# Module-level state set via setup()
|
||||
_config: dict = {}
|
||||
_loop: Optional[asyncio.AbstractEventLoop] = None
|
||||
|
||||
# Tracks which channels fired a WARNING/CRITICAL per host.
|
||||
# {host_name: set of channel_names} — used to route RECOVER to the same channels.
|
||||
@@ -73,11 +69,9 @@ class Notification:
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def setup(cfg: dict, loop: Optional[asyncio.AbstractEventLoop] = None):
|
||||
"""Initialize notifier from configuration dict and event loop."""
|
||||
global _config, _loop
|
||||
"""Initialize notifier from configuration dict."""
|
||||
global _config
|
||||
_config = dict(cfg)
|
||||
if loop is not None:
|
||||
_loop = loop
|
||||
|
||||
|
||||
def reload_config(cfg: dict):
|
||||
@@ -299,17 +293,6 @@ async def _send_sms_voipms_async(channel_cfg: dict, notif: Notification) -> bool
|
||||
return False
|
||||
|
||||
|
||||
def _send_sms_voipms(channel_cfg: dict, notif: Notification) -> bool:
|
||||
"""Dispatch voip.ms SMS send onto the shared event loop."""
|
||||
if _loop is None:
|
||||
logger.warning("sms_voipms: event loop not available")
|
||||
return False
|
||||
future = asyncio.run_coroutine_threadsafe(_send_sms_voipms_async(channel_cfg, notif), _loop)
|
||||
try:
|
||||
return future.result(timeout=15)
|
||||
except Exception as e:
|
||||
logger.error("sms_voipms send timed out or failed: %s", e)
|
||||
return False
|
||||
|
||||
|
||||
async def _send_matrix_async(channel_cfg: dict, notif: Notification) -> bool:
|
||||
@@ -357,40 +340,23 @@ async def _send_matrix_async(channel_cfg: dict, notif: Notification) -> bool:
|
||||
await client.close()
|
||||
|
||||
|
||||
def _send_matrix(channel_cfg: dict, notif: Notification) -> bool:
|
||||
"""Dispatch matrix send onto the shared event loop."""
|
||||
if _loop is None:
|
||||
logger.warning("matrix: event loop not available")
|
||||
return False
|
||||
future = asyncio.run_coroutine_threadsafe(_send_matrix_async(channel_cfg, notif), _loop)
|
||||
try:
|
||||
return future.result(timeout=15)
|
||||
except Exception as e:
|
||||
logger.error("matrix send timed out or failed: %s", e)
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Channel dispatcher
|
||||
# Channel dispatcher (all async — sync drivers run in a thread executor)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Sync drivers kept for `hbd notify` CLI usage (asyncio.run wraps them there).
|
||||
_DRIVERS = {
|
||||
"pushover": _send_pushover,
|
||||
"email": _send_email,
|
||||
"mattermost": _send_mattermost,
|
||||
"signal": _send_signal,
|
||||
"sms_voipms": _send_sms_voipms,
|
||||
"matrix": _send_matrix,
|
||||
}
|
||||
|
||||
_TIMEOUT = 15 # seconds per channel send
|
||||
|
||||
def _dispatch_to_channel(channel_name: str, channel_cfg: dict, notif: Notification) -> bool:
|
||||
"""Send *notif* to a single named channel, honouring min_level.
|
||||
|
||||
RECOVER always bypasses min_level — a recovery is always relevant if the
|
||||
channel was configured for any alerting (handles the restart-then-recover case
|
||||
where _alerted_channels is empty and we fall through to the normal loop).
|
||||
"""
|
||||
async def _dispatch_to_channel(channel_name: str, channel_cfg: dict, notif: Notification) -> bool:
|
||||
"""Send *notif* to a single named channel, honouring min_level."""
|
||||
level = notif.level.upper()
|
||||
if level != "RECOVER":
|
||||
min_level = channel_cfg.get("min_level", "WARNING").upper()
|
||||
@@ -398,14 +364,24 @@ def _dispatch_to_channel(channel_name: str, channel_cfg: dict, notif: Notificati
|
||||
logger.debug(
|
||||
"channel '%s': skipping level %s (min_level=%s)", channel_name, level, min_level
|
||||
)
|
||||
return True # not an error — filtered intentionally
|
||||
return True # filtered intentionally
|
||||
|
||||
ch_type = channel_cfg.get("type", "")
|
||||
driver = _DRIVERS.get(ch_type)
|
||||
if driver is None:
|
||||
try:
|
||||
if ch_type == "matrix":
|
||||
return await asyncio.wait_for(_send_matrix_async(channel_cfg, notif), timeout=_TIMEOUT)
|
||||
if ch_type == "sms_voipms":
|
||||
return await asyncio.wait_for(_send_sms_voipms_async(channel_cfg, notif), timeout=_TIMEOUT)
|
||||
sync_driver = _DRIVERS.get(ch_type)
|
||||
if sync_driver is None:
|
||||
logger.warning("unknown channel type '%s' for channel '%s'", ch_type, channel_name)
|
||||
return False
|
||||
return driver(channel_cfg, notif)
|
||||
return await asyncio.wait_for(
|
||||
asyncio.to_thread(sync_driver, channel_cfg, notif), timeout=_TIMEOUT
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
logger.error("channel '%s' timed out after %ds", channel_name, _TIMEOUT)
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -419,7 +395,7 @@ def _build_url(host_name: str) -> str:
|
||||
return f"{base_url}/plugins#{host_name}"
|
||||
|
||||
|
||||
def send_notification(host_name: str, notif: Notification) -> dict:
|
||||
async def send_notification(host_name: str, notif: Notification) -> dict:
|
||||
"""Dispatch *notif* to all managers/owner of *host_name*.
|
||||
|
||||
Looks up the host's owner + managers, resolves each user's
|
||||
@@ -469,16 +445,12 @@ def send_notification(host_name: str, notif: Notification) -> dict:
|
||||
if not channel_cfg:
|
||||
continue
|
||||
try:
|
||||
ch_type = channel_cfg.get("type", "")
|
||||
driver = _DRIVERS.get(ch_type)
|
||||
if driver:
|
||||
ok = driver(channel_cfg, notif)
|
||||
ok = await _dispatch_to_channel(channel_name, channel_cfg, notif)
|
||||
results[channel_name] = ok
|
||||
if ok:
|
||||
logger.info("recover sent to channel '%s': %s", channel_name, notif.title)
|
||||
except Exception as e:
|
||||
logger.error("error sending recover to channel '%s': %s", channel_name, e)
|
||||
# Clear the alerted set once recovery is delivered
|
||||
del _alerted_channels[host_name]
|
||||
return results
|
||||
|
||||
@@ -489,14 +461,14 @@ def send_notification(host_name: str, notif: Notification) -> dict:
|
||||
continue
|
||||
for channel_name in user.notification_channels:
|
||||
if channel_name in results:
|
||||
continue # already dispatched to this channel this notification
|
||||
continue
|
||||
channel_cfg = global_channels.get(channel_name)
|
||||
if not channel_cfg:
|
||||
logger.warning("channel '%s' not defined in notification_channels", channel_name)
|
||||
results[channel_name] = False
|
||||
continue
|
||||
try:
|
||||
ok = _dispatch_to_channel(channel_name, channel_cfg, notif)
|
||||
ok = await _dispatch_to_channel(channel_name, channel_cfg, notif)
|
||||
results[channel_name] = ok
|
||||
if ok:
|
||||
logger.info("notification sent to channel '%s': %s", channel_name, notif.title)
|
||||
|
||||
+54
-2
@@ -24,7 +24,7 @@ sensitive bool True when the raw value must never be shown
|
||||
# Credential field names that should always be masked.
|
||||
_SECRET_KEYS = frozenset({
|
||||
"password", "token", "user_key", "api_key", "secret",
|
||||
"smtp_password", "smtp_user",
|
||||
"smtp_password", "smtp_user", "api_password", "access_token",
|
||||
})
|
||||
|
||||
_CHANNEL_TYPE_LABELS = {
|
||||
@@ -181,6 +181,48 @@ def get_settings_sections(config: dict) -> list:
|
||||
"notification_channels": attrs.get("notification_channels", []),
|
||||
})
|
||||
|
||||
# ---- Threshold configurations -----------------------------------------
|
||||
def _parse_metric_row(metric_path, metric_cfg):
|
||||
if not isinstance(metric_cfg, dict):
|
||||
return None
|
||||
return {
|
||||
"metric": metric_path,
|
||||
"operator": metric_cfg.get("operator", ">"),
|
||||
"warning": metric_cfg.get("warning"),
|
||||
"critical": metric_cfg.get("critical"),
|
||||
"hysteresis": metric_cfg.get("hysteresis"),
|
||||
"count": metric_cfg.get("count", 1),
|
||||
"enabled": metric_cfg.get("enabled", True),
|
||||
}
|
||||
|
||||
threshold_config_list = []
|
||||
raw_tconfigs = config.get("threshold_configs") or {}
|
||||
if raw_tconfigs:
|
||||
for cfg_name, cfg_data in sorted(raw_tconfigs.items()):
|
||||
if not isinstance(cfg_data, dict):
|
||||
continue
|
||||
metrics = [
|
||||
r for r in (
|
||||
_parse_metric_row(mp, mc)
|
||||
for mp, mc in (cfg_data.get("thresholds") or {}).items()
|
||||
) if r
|
||||
]
|
||||
threshold_config_list.append({
|
||||
"name": cfg_name,
|
||||
"metrics": sorted(metrics, key=lambda m: m["metric"]),
|
||||
})
|
||||
elif config.get("thresholds"):
|
||||
metrics = [
|
||||
r for r in (
|
||||
_parse_metric_row(mp, mc)
|
||||
for mp, mc in config["thresholds"].items()
|
||||
) if r
|
||||
]
|
||||
threshold_config_list.append({
|
||||
"name": "default",
|
||||
"metrics": sorted(metrics, key=lambda m: m["metric"]),
|
||||
})
|
||||
|
||||
# ---- Hosts summary ----------------------------------------------------
|
||||
hosts_list = []
|
||||
for hname, hcfg in (config.get("hosts") or {}).items():
|
||||
@@ -188,7 +230,7 @@ def get_settings_sections(config: dict) -> list:
|
||||
continue
|
||||
hosts_list.append({
|
||||
"name": hname,
|
||||
"watch": bool(hcfg.get("watch", False)),
|
||||
"watch": bool(hcfg.get("watch", True)),
|
||||
"dyndns": bool(hcfg.get("dyndns", False)),
|
||||
"owner": hcfg.get("owner", ""),
|
||||
"managers": hcfg.get("managers", []),
|
||||
@@ -312,6 +354,16 @@ def get_settings_sections(config: dict) -> list:
|
||||
"hosts": hosts_list,
|
||||
"fields": [],
|
||||
},
|
||||
{
|
||||
"id": "thresholds",
|
||||
"title": "Threshold Configurations",
|
||||
"description": "Named alert threshold sets. Each defines warning/critical levels per metric.",
|
||||
"threshold_configs": threshold_config_list,
|
||||
"fields": [
|
||||
field("default_threshold_config", "Default config", "text",
|
||||
"Threshold config used for hosts with no explicit mapping."),
|
||||
],
|
||||
},
|
||||
{
|
||||
"id": "runtime",
|
||||
"title": "Runtime",
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
{% include 'head.html' %}
|
||||
|
||||
<style>
|
||||
html, body { overflow: visible; }
|
||||
|
||||
.container {
|
||||
max-width: 700px;
|
||||
margin: 0 auto;
|
||||
}
|
||||
|
||||
h1 {
|
||||
color: #333;
|
||||
margin-bottom: 4px;
|
||||
font-size: 1.5em;
|
||||
}
|
||||
|
||||
.subtitle {
|
||||
color: #666;
|
||||
margin-bottom: 24px;
|
||||
font-size: 0.9em;
|
||||
}
|
||||
|
||||
.section {
|
||||
background: #fff;
|
||||
border-radius: 8px;
|
||||
box-shadow: 0 1px 6px rgba(0,0,0,0.1);
|
||||
padding: 20px 24px;
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
|
||||
.section h2 {
|
||||
font-size: 1em;
|
||||
font-weight: 700;
|
||||
color: #333;
|
||||
margin: 0 0 16px;
|
||||
padding-bottom: 10px;
|
||||
border-bottom: 1px solid #eee;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
}
|
||||
|
||||
.info-row {
|
||||
display: flex;
|
||||
align-items: baseline;
|
||||
padding: 8px 0;
|
||||
border-bottom: 1px solid #f5f5f5;
|
||||
font-size: 0.9em;
|
||||
}
|
||||
.info-row:last-child { border-bottom: none; }
|
||||
|
||||
.info-label {
|
||||
width: 160px;
|
||||
flex-shrink: 0;
|
||||
color: #666;
|
||||
font-size: 0.88em;
|
||||
}
|
||||
|
||||
.info-value {
|
||||
color: #222;
|
||||
word-break: break-all;
|
||||
}
|
||||
|
||||
.info-value a {
|
||||
color: #0066cc;
|
||||
text-decoration: none;
|
||||
}
|
||||
.info-value a:hover { text-decoration: underline; }
|
||||
|
||||
.version-badge {
|
||||
display: inline-block;
|
||||
padding: 3px 12px;
|
||||
background: #e8f0fe;
|
||||
color: #1a73e8;
|
||||
border-radius: 12px;
|
||||
font-size: 0.85em;
|
||||
font-weight: 600;
|
||||
font-family: monospace;
|
||||
}
|
||||
|
||||
.hb-logo {
|
||||
font-size: 2.5em;
|
||||
font-weight: 700;
|
||||
color: #0066cc;
|
||||
letter-spacing: -1px;
|
||||
margin-bottom: 6px;
|
||||
}
|
||||
|
||||
.hb-tagline {
|
||||
color: #555;
|
||||
font-size: 0.95em;
|
||||
}
|
||||
|
||||
.logo-section {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 20px;
|
||||
padding: 8px 0 4px;
|
||||
}
|
||||
|
||||
.logo-text { flex: 1; }
|
||||
</style>
|
||||
|
||||
<body>
|
||||
{% include 'nav.html' %}
|
||||
|
||||
<div class="container">
|
||||
<h1>{{ header }}</h1>
|
||||
<p class="subtitle">Heartbeat monitoring system</p>
|
||||
|
||||
<div class="section">
|
||||
<div class="logo-section">
|
||||
<div class="logo-text">
|
||||
<div class="hb-logo">Heartbeat</div>
|
||||
<div class="hb-tagline">Lightweight host monitoring over UDP</div>
|
||||
</div>
|
||||
<span class="version-badge">v{{ hbd_version }}</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<h2>Version</h2>
|
||||
<div class="info-row">
|
||||
<span class="info-label">Server version</span>
|
||||
<span class="info-value">{{ hbd_version }}</span>
|
||||
</div>
|
||||
<div class="info-row">
|
||||
<span class="info-label">Python</span>
|
||||
<span class="info-value">{{ python_version }}</span>
|
||||
</div>
|
||||
<div class="info-row">
|
||||
<span class="info-label">License</span>
|
||||
<span class="info-value">MIT</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<h2>Runtime</h2>
|
||||
<div class="info-row">
|
||||
<span class="info-label">Host</span>
|
||||
<span class="info-value">{{ server_hostname }}</span>
|
||||
</div>
|
||||
<div class="info-row">
|
||||
<span class="info-label">Started</span>
|
||||
<span class="info-value">{{ start_time_str }}</span>
|
||||
</div>
|
||||
<div class="info-row">
|
||||
<span class="info-label">Uptime</span>
|
||||
<span class="info-value" id="uptime-value">{{ uptime_str }}</span>
|
||||
</div>
|
||||
<div class="info-row">
|
||||
<span class="info-label">Hosts monitored</span>
|
||||
<span class="info-value">{{ host_count }}</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<h2>Contact & Source</h2>
|
||||
<div class="info-row">
|
||||
<span class="info-label">Author</span>
|
||||
<span class="info-value">Andreas Wrede</span>
|
||||
</div>
|
||||
<div class="info-row">
|
||||
<span class="info-label">Email</span>
|
||||
<span class="info-value"><a href="mailto:aew@wrede.ca">aew@wrede.ca</a></span>
|
||||
</div>
|
||||
<div class="info-row">
|
||||
<span class="info-label">Repository</span>
|
||||
<span class="info-value"><a href="https://git.wrede.ca/andreas/heartbeat" target="_blank" rel="noopener">git.wrede.ca/andreas/heartbeat</a></span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
<script>
|
||||
(function() {
|
||||
var startEpoch = {{ start_epoch }};
|
||||
var el = document.getElementById('uptime-value');
|
||||
if (!el) return;
|
||||
function fmt(s) {
|
||||
var d = Math.floor(s / 86400);
|
||||
var h = Math.floor((s % 86400) / 3600);
|
||||
var m = Math.floor((s % 3600) / 60);
|
||||
var sec = s % 60;
|
||||
if (d > 0) return d + 'd ' + h + 'h ' + m + 'm';
|
||||
if (h > 0) return h + 'h ' + m + 'm ' + sec + 's';
|
||||
return m + 'm ' + sec + 's';
|
||||
}
|
||||
function tick() {
|
||||
var up = Math.floor(Date.now() / 1000 - startEpoch);
|
||||
el.textContent = fmt(up);
|
||||
}
|
||||
tick();
|
||||
setInterval(tick, 1000);
|
||||
})();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -9,7 +9,7 @@
|
||||
margin: 0 auto;
|
||||
}
|
||||
|
||||
h1 { color: #333; margin-bottom: 10px; font-size: 1.5em; }
|
||||
h1 { color: #333; margin-bottom: 5px; margin-top: 15px; font-size: 1.5em; }
|
||||
|
||||
.subtitle {
|
||||
color: #666;
|
||||
|
||||
@@ -15,6 +15,7 @@
|
||||
body {
|
||||
margin: 0;
|
||||
padding: 10px;
|
||||
padding-top: 60px;
|
||||
background: #f5f5f5;
|
||||
}
|
||||
h1 { font-size: 1.5em; color: #333; margin: 0 0 5px; }
|
||||
@@ -23,11 +24,14 @@
|
||||
|
||||
/* Navigation bar — shared across all pages */
|
||||
.nav {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
right: 0;
|
||||
z-index: 200;
|
||||
background: #fff;
|
||||
padding: 6px 12px;
|
||||
margin-bottom: 10px;
|
||||
box-shadow: 0 2px 4px rgba(0,0,0,.1);
|
||||
border-radius: 4px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
|
||||
@@ -45,6 +45,7 @@
|
||||
h1 {
|
||||
color: #333;
|
||||
margin-bottom: 5px;
|
||||
margin-top: 15px;
|
||||
font-size: 1.5em;
|
||||
}
|
||||
|
||||
@@ -235,6 +236,8 @@
|
||||
color: #ff9800;
|
||||
font-weight: 700;
|
||||
}
|
||||
#ntable a.host-link { color: inherit; text-decoration: none; }
|
||||
#ntable a.host-link:hover { text-decoration: underline; }
|
||||
</style>
|
||||
<script type="text/javascript">
|
||||
var cnt = 0;
|
||||
@@ -244,11 +247,13 @@
|
||||
var HBD_VERSION = "{{ hbd_version }}";
|
||||
|
||||
function hostNameHtml(data) {
|
||||
var rawName = data.raw_name || data.name.replace(/<[^>]+>/g, '').replace('*', '').trim();
|
||||
var nameHtml = data.name;
|
||||
if (!data.hbc_version || data.hbc_version !== HBD_VERSION) {
|
||||
nameHtml += ' 🥀';
|
||||
}
|
||||
return data.dyn ? '<b>' + nameHtml + '</b>' : nameHtml;
|
||||
var display = data.dyn ? '<b>' + nameHtml + '</b>' : nameHtml;
|
||||
return '<a class="host-link" href="/plugins#' + encodeURIComponent(rawName) + '">' + display + '</a>';
|
||||
}
|
||||
|
||||
function setup() {
|
||||
@@ -403,7 +408,7 @@
|
||||
);
|
||||
if (data.connections[i].state == "up") {
|
||||
state = '<span class="state-up">up</span>';
|
||||
latency = Number.parseFloat(data.connections[i].rtts[0]).toFixed(2);
|
||||
latency = String(Math.round(Number.parseFloat(data.connections[i].rtts[0])));
|
||||
} else {
|
||||
if (data.connections[i].state == "unknown") {
|
||||
state = "";
|
||||
@@ -510,7 +515,7 @@
|
||||
<tbody id="ntablebody">
|
||||
{% for host in hosts %}
|
||||
<tr class="{% if host.alert_critical_unacked > 0 or host.alert_critical_acked > 0 %}row-critical{% elif host.alert_warning_unacked > 0 or host.alert_warning_acked > 0 %}row-warning{% endif %}">
|
||||
<td data-name="{{ host.name }}">{{ host.name }}{% if not host.hbc_version or host.hbc_version != hbd_version %} 🥀{% endif %}</td>
|
||||
<td data-name="{{ host.name }}"><a class="host-link" href="/plugins#{{ host.raw_name | urlencode }}">{{ host.name }}{% if not host.hbc_version or host.hbc_version != hbd_version %} 🥀{% endif %}</a></td>
|
||||
<td style="text-align: center; color: #ff9800; font-weight: bold;">
|
||||
{%- set warning_unacked = host.alert_warning_unacked -%}
|
||||
{%- set warning_acked = host.alert_warning_acked -%}
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
{% if current_user and current_user.admin %}
|
||||
<a href="/settings"{% if active_page == "settings" %} class="active"{% endif %}>Settings</a>
|
||||
{% endif %}
|
||||
<a href="/about"{% if active_page == "about" %} class="active"{% endif %}>About</a>
|
||||
</div>
|
||||
<div class="nav-clock" title="Click for full-screen clock">
|
||||
<canvas id="swiss-clock" width="44" height="44"></canvas>
|
||||
|
||||
@@ -16,6 +16,7 @@
|
||||
h1 {
|
||||
color: #333;
|
||||
margin-bottom: 5px;
|
||||
margin-top: 15px;
|
||||
font-size: 1.5em;
|
||||
}
|
||||
|
||||
@@ -382,7 +383,7 @@
|
||||
</div>
|
||||
|
||||
<div class="host-body">
|
||||
{% set plugin_order = ['os_info','cpu_monitor','memory_monitor','disk_monitor','network_monitor','nagios_runner','filesystem_info'] %}
|
||||
{% set plugin_order = ['os_info','cpu_monitor','memory_monitor','disk_monitor','network_monitor','zfs_monitor','nagios_runner','filesystem_info'] %}
|
||||
{% for plugin in plugin_order if plugin in host.plugins %}
|
||||
<div class="plugin-accordion collapsed"
|
||||
data-hostname="{{ host.name }}"
|
||||
@@ -672,6 +673,19 @@
|
||||
text = `${count} filesystem${count !== 1 ? 's' : ''}`;
|
||||
break;
|
||||
}
|
||||
case 'zfs_monitor': {
|
||||
const pools = d.pools || {};
|
||||
const names = Object.keys(pools);
|
||||
if (names.length === 0) { text = 'No pools'; break; }
|
||||
const degraded = names.filter(n => pools[n].health && pools[n].health !== 'ONLINE');
|
||||
text = names.map(n => {
|
||||
const p = pools[n];
|
||||
const cap = p.capacity != null ? ` ${p.capacity.toFixed(0)}%` : '';
|
||||
return `${n}${cap}`;
|
||||
}).join(' · ');
|
||||
if (degraded.length) text += ` ⚠ ${degraded.map(n => pools[n].health).join(',')}`;
|
||||
break;
|
||||
}
|
||||
default:
|
||||
text = 'Loaded';
|
||||
}
|
||||
@@ -693,6 +707,7 @@
|
||||
case 'memory_monitor': html = renderMemoryTable(cached.data); break;
|
||||
case 'disk_monitor': html = renderDiskTables(cached.data); break;
|
||||
case 'network_monitor':html = renderNetworkTables(cached.data); break;
|
||||
case 'zfs_monitor': html = renderZfsTables(cached.data); break;
|
||||
case 'nagios_runner': html = renderNagiosTable(cached.data); break;
|
||||
case 'filesystem_info':html = renderFilesystemTable(cached.data); break;
|
||||
default: html = renderGenericTable(cached.data); break;
|
||||
@@ -1023,6 +1038,66 @@
|
||||
return html;
|
||||
}
|
||||
|
||||
function renderZfsTables(d) {
|
||||
const pools = d.pools || {};
|
||||
const names = Object.keys(pools);
|
||||
if (names.length === 0) return '<div class="no-data">No ZFS pools found</div>';
|
||||
|
||||
const healthCls = h => {
|
||||
if (!h || h === 'ONLINE') return 'pct-ok';
|
||||
if (h === 'DEGRADED') return 'pct-warn';
|
||||
return 'pct-crit';
|
||||
};
|
||||
|
||||
let pt = '<table class="data-table"><thead><tr>'
|
||||
+ '<th>Pool</th><th>Health</th>'
|
||||
+ '<th class="num">Size</th><th class="num">Used</th>'
|
||||
+ '<th class="num">Free</th><th class="num">Cap %</th>'
|
||||
+ '<th class="num">Frag %</th><th class="num">Dedup</th>'
|
||||
+ '</tr></thead><tbody>';
|
||||
for (const name of names) {
|
||||
const p = pools[name];
|
||||
const cap = p.capacity != null ? p.capacity : 0;
|
||||
const capCls = cap > 90 ? 'pct-crit' : cap > 75 ? 'pct-warn' : 'pct-ok';
|
||||
pt += `<tr>
|
||||
<td class="iface-name">${escHtml(name)}</td>
|
||||
<td class="${healthCls(p.health)}">${escHtml(p.health || '—')}</td>
|
||||
<td class="num">${formatBytes(p.size || 0)}</td>
|
||||
<td class="num">${formatBytes(p.alloc || 0)}</td>
|
||||
<td class="num">${formatBytes(p.free || 0)}</td>
|
||||
<td class="num ${capCls}">${cap.toFixed(1)}%</td>
|
||||
<td class="num">${p.frag != null ? p.frag.toFixed(1) + '%' : '—'}</td>
|
||||
<td class="num">${p.dedup != null ? p.dedup.toFixed(2) + 'x' : '—'}</td>
|
||||
</tr>`;
|
||||
}
|
||||
pt += '</tbody></table>';
|
||||
|
||||
const hasIo = names.some(n => pools[n].read_ops != null);
|
||||
if (!hasIo) return pt;
|
||||
|
||||
let iot = '<table class="data-table"><thead><tr>'
|
||||
+ '<th>Pool</th>'
|
||||
+ '<th class="num">Read ops</th><th class="num">Write ops</th>'
|
||||
+ '<th class="num">Read BW</th><th class="num">Write BW</th>'
|
||||
+ '</tr></thead><tbody>';
|
||||
for (const name of names) {
|
||||
const p = pools[name];
|
||||
iot += `<tr>
|
||||
<td class="iface-name">${escHtml(name)}</td>
|
||||
<td class="num">${p.read_ops != null ? p.read_ops.toLocaleString() : '—'}</td>
|
||||
<td class="num">${p.write_ops != null ? p.write_ops.toLocaleString() : '—'}</td>
|
||||
<td class="num">${p.read_bw != null ? formatBytes(p.read_bw) : '—'}</td>
|
||||
<td class="num">${p.write_bw != null ? formatBytes(p.write_bw) : '—'}</td>
|
||||
</tr>`;
|
||||
}
|
||||
iot += '</tbody></table>';
|
||||
|
||||
return `<div class="flex-tables">
|
||||
<div><div class="table-section-label">Pools</div>${pt}</div>
|
||||
<div><div class="table-section-label">I/O (cumulative)</div>${iot}</div>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
function renderGenericTable(d) {
|
||||
let html = '<table class="data-table"><thead><tr><th>Field</th><th>Value</th></tr></thead><tbody>';
|
||||
for (const [k, v] of Object.entries(d)) {
|
||||
@@ -1081,6 +1156,19 @@
|
||||
// ── Init ────────────────────────────────────────────────────────────────
|
||||
|
||||
document.addEventListener('DOMContentLoaded', () => {
|
||||
// If a host fragment is in the URL, expand and scroll to that host;
|
||||
// otherwise expand the first host as before.
|
||||
const hash = window.location.hash;
|
||||
if (hash) {
|
||||
const hostname = decodeURIComponent(hash.slice(1));
|
||||
const card = document.querySelector(`.host-card[data-hostname="${hostname}"]`);
|
||||
if (card) {
|
||||
card.classList.remove('collapsed');
|
||||
fetchHostGlance(hostname);
|
||||
setTimeout(() => card.scrollIntoView({ behavior: 'smooth', block: 'start' }), 150);
|
||||
return;
|
||||
}
|
||||
}
|
||||
const first = document.querySelector('.host-card');
|
||||
if (first) {
|
||||
first.classList.remove('collapsed');
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
max-width: 960px;
|
||||
}
|
||||
|
||||
h1 { color: #333; margin-bottom: 4px; font-size: 1.5em; }
|
||||
h1 { color: #333; margin-bottom: 5px; margin-top: 15px; font-size: 1.5em; }
|
||||
.subtitle { color: #666; margin-bottom: 24px; font-size: 0.9em; }
|
||||
|
||||
/* ---- Sidebar + content layout ---- */
|
||||
@@ -23,7 +23,7 @@
|
||||
width: 180px;
|
||||
flex-shrink: 0;
|
||||
position: sticky;
|
||||
top: 20px;
|
||||
top: 60px;
|
||||
}
|
||||
|
||||
.sidebar-nav a {
|
||||
@@ -254,6 +254,17 @@
|
||||
.host-bool { text-align: center; }
|
||||
.dot-yes { color: #2e7d32; font-size: 1.1em; }
|
||||
.dot-no { color: #ddd; font-size: 1.1em; }
|
||||
|
||||
/* ---- Threshold configurations ---- */
|
||||
.thresh-config { margin: 12px 20px 20px; }
|
||||
.thresh-config-name {
|
||||
font-weight: 600; font-size: 0.9em; color: #1a237e;
|
||||
margin-bottom: 6px;
|
||||
}
|
||||
.mini-table .warn { color: #e65100; font-weight: 600; }
|
||||
.mini-table .crit { color: #b71c1c; font-weight: 600; }
|
||||
.mini-table .dim { color: #aaa; }
|
||||
.mini-table .metric-path { font-family: monospace; font-size: 0.88em; }
|
||||
</style>
|
||||
|
||||
<body>
|
||||
@@ -394,6 +405,49 @@
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
|
||||
{# ---- Threshold configurations section ---- #}
|
||||
{% if section.id == "thresholds" %}
|
||||
{% if section.threshold_configs %}
|
||||
{% for tc in section.threshold_configs %}
|
||||
<div class="thresh-config">
|
||||
<div class="thresh-config-name">{{ tc.name }}</div>
|
||||
{% if tc.metrics %}
|
||||
<div style="overflow-x: auto;">
|
||||
<table class="mini-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Metric</th>
|
||||
<th>Op</th>
|
||||
<th>Warning</th>
|
||||
<th>Critical</th>
|
||||
<th>Hysteresis</th>
|
||||
<th>Count</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% for m in tc.metrics %}
|
||||
<tr {% if not m.enabled %} style="opacity:0.45"{% endif %}>
|
||||
<td class="metric-path">{{ m.metric }}</td>
|
||||
<td>{{ m.operator or '>' }}</td>
|
||||
<td class="warn">{{ m.warning if m.warning is not none else '—' }}</td>
|
||||
<td class="crit">{{ m.critical if m.critical is not none else '—' }}</td>
|
||||
<td class="dim">{{ '%.0f%%' % (m.hysteresis * 100) if m.hysteresis else '—' }}</td>
|
||||
<td class="dim">{{ m.count }}</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
{% else %}
|
||||
<span class="val-empty">No thresholds defined.</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endfor %}
|
||||
{% else %}
|
||||
<div class="field-row"><span class="val-empty">No threshold configurations defined.</span></div>
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
|
||||
{# ---- Hosts section ---- #}
|
||||
{% if section.id == "hosts" %}
|
||||
{% if section.hosts %}
|
||||
|
||||
+114
-48
@@ -9,10 +9,11 @@ This module provides a flexible threshold checking system that:
|
||||
- Supports multiple comparison operators
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import time
|
||||
from enum import Enum
|
||||
from typing import Dict, Any, Optional, Tuple, Callable
|
||||
from typing import Dict, List, Any, Optional, Tuple, Callable
|
||||
from . import notify as notify_mod
|
||||
from .config import THRESHOLD_DEFAULTS
|
||||
|
||||
@@ -328,14 +329,17 @@ class ThresholdChecker:
|
||||
renotify_interval: Seconds between repeat notifications (default: 1 hour)
|
||||
journal: Optional MessageJournal instance for logging threshold events
|
||||
"""
|
||||
# Named threshold configurations: {config_name: {metric_path: ThresholdConfig}}
|
||||
# Named threshold configurations (pre-merged: defaults + overrides): {config_name: {metric_path: ThresholdConfig}}
|
||||
self.threshold_configs = {}
|
||||
|
||||
# Raw overrides only for each named config (no defaults baked in): {config_name: {metric_path: ThresholdConfig}}
|
||||
self.threshold_raw_configs: Dict[str, Dict[str, ThresholdConfig]] = {}
|
||||
|
||||
# Single threshold set for backward compatibility: {metric_path: ThresholdConfig}
|
||||
self.thresholds = {}
|
||||
|
||||
# Host to config name mapping: {host_name: config_name}
|
||||
self.host_config_mapping = {}
|
||||
# Host to ordered list of config names: {host_name: [config_name, ...]}
|
||||
self.host_config_mapping: Dict[str, List[str]] = {}
|
||||
|
||||
# Default config name to use when no mapping exists
|
||||
self.default_config = "default"
|
||||
@@ -372,6 +376,7 @@ class ThresholdChecker:
|
||||
|
||||
# Clear old configuration
|
||||
self.threshold_configs.clear()
|
||||
self.threshold_raw_configs.clear()
|
||||
self.thresholds.clear()
|
||||
self.host_config_mapping.clear()
|
||||
self.grace_seconds = float(config.get("grace", 2))
|
||||
@@ -424,9 +429,10 @@ class ThresholdChecker:
|
||||
self._parse_plugin_thresholds(plugin_name, plugin_thresholds, target_dict=effective_defaults)
|
||||
|
||||
self.threshold_configs["default"] = dict(effective_defaults)
|
||||
self.threshold_raw_configs["default"] = {}
|
||||
logger.info("Registered 'default' threshold config with %d metrics", len(effective_defaults))
|
||||
|
||||
# Parse each named configuration, seeding it with effective_defaults first
|
||||
# Parse each named configuration
|
||||
for config_name, config_data in threshold_configs.items():
|
||||
if config_name == "default":
|
||||
continue # already handled above
|
||||
@@ -440,33 +446,41 @@ class ThresholdChecker:
|
||||
continue
|
||||
|
||||
logger.info("Parsing threshold configuration: %s", config_name)
|
||||
self.threshold_configs[config_name] = dict(effective_defaults)
|
||||
|
||||
# Raw overrides only (used for multi-config layering)
|
||||
raw_overrides: Dict[str, ThresholdConfig] = {}
|
||||
thresholds_config = config_data["thresholds"]
|
||||
for plugin_name, plugin_thresholds in thresholds_config.items():
|
||||
if not isinstance(plugin_thresholds, dict):
|
||||
continue
|
||||
if isinstance(plugin_thresholds, dict):
|
||||
self._parse_plugin_thresholds(plugin_name, plugin_thresholds, target_dict=raw_overrides)
|
||||
self.threshold_raw_configs[config_name] = raw_overrides
|
||||
|
||||
self._parse_plugin_thresholds(
|
||||
plugin_name,
|
||||
plugin_thresholds,
|
||||
target_dict=self.threshold_configs[config_name]
|
||||
)
|
||||
# Pre-merged version (defaults + overrides) for single-config fast path
|
||||
self.threshold_configs[config_name] = dict(effective_defaults)
|
||||
self.threshold_configs[config_name].update(raw_overrides)
|
||||
|
||||
# Parse host to config mapping from two possible sources
|
||||
# 1. New format: hosts section with threshold_config attribute
|
||||
# Parse host → config list mapping from two possible sources
|
||||
|
||||
def _normalise(value) -> List[str]:
|
||||
"""Accept a string or list; always return a list."""
|
||||
if isinstance(value, list):
|
||||
return [str(v) for v in value]
|
||||
return [str(value)]
|
||||
|
||||
# 1. hosts section with threshold_config attribute (string or list)
|
||||
if "hosts" in config:
|
||||
hosts_config = config["hosts"]
|
||||
if isinstance(hosts_config, dict):
|
||||
for host_name, host_attrs in hosts_config.items():
|
||||
if isinstance(host_attrs, dict) and "threshold_config" in host_attrs:
|
||||
self.host_config_mapping[host_name] = host_attrs["threshold_config"]
|
||||
self.host_config_mapping[host_name] = _normalise(host_attrs["threshold_config"])
|
||||
|
||||
# 2. Legacy format: host_threshold_mapping section (for backward compatibility)
|
||||
# 2. Legacy host_threshold_mapping section (string values only)
|
||||
if "host_threshold_mapping" in config:
|
||||
legacy_mapping = config.get("host_threshold_mapping", {})
|
||||
if isinstance(legacy_mapping, dict):
|
||||
self.host_config_mapping.update(legacy_mapping)
|
||||
for host_name, value in legacy_mapping.items():
|
||||
self.host_config_mapping[host_name] = _normalise(value)
|
||||
|
||||
# Set default config (first one alphabetically or explicitly set)
|
||||
self.default_config = config.get("default_threshold_config", "default")
|
||||
@@ -664,7 +678,10 @@ class ThresholdChecker:
|
||||
)
|
||||
|
||||
def get_thresholds_for_host(self, host_name: str) -> Dict[str, ThresholdConfig]:
|
||||
"""Get the appropriate threshold configuration for a host.
|
||||
"""Get the effective threshold configuration for a host.
|
||||
|
||||
When threshold_config is a list, configs are applied left-to-right on top
|
||||
of the default thresholds so earlier entries can be overridden by later ones.
|
||||
|
||||
Args:
|
||||
host_name: Name of the host
|
||||
@@ -676,23 +693,40 @@ class ThresholdChecker:
|
||||
if self.thresholds and not self.threshold_configs:
|
||||
return self.thresholds
|
||||
|
||||
# Multi-config mode: look up host-specific configuration
|
||||
if self.threshold_configs:
|
||||
config_name = self.host_config_mapping.get(host_name, self.default_config)
|
||||
if not self.threshold_configs:
|
||||
return {}
|
||||
|
||||
if config_name in self.threshold_configs:
|
||||
return self.threshold_configs[config_name]
|
||||
else:
|
||||
config_names = self.host_config_mapping.get(host_name)
|
||||
|
||||
# No host-specific mapping → return pre-merged default
|
||||
if not config_names:
|
||||
return self.threshold_configs.get(self.default_config, {})
|
||||
|
||||
# Single config → fast path using pre-merged copy
|
||||
if len(config_names) == 1:
|
||||
name = config_names[0]
|
||||
if name in self.threshold_configs:
|
||||
return self.threshold_configs[name]
|
||||
logger.warning(
|
||||
"Threshold config '%s' not found for host '%s', using default '%s'",
|
||||
config_name,
|
||||
host_name,
|
||||
self.default_config
|
||||
name, host_name, self.default_config,
|
||||
)
|
||||
return self.threshold_configs.get(self.default_config, {})
|
||||
|
||||
# No thresholds configured
|
||||
return {}
|
||||
# Multiple configs → start from defaults, layer raw overrides in order
|
||||
result = dict(self.threshold_configs.get(self.default_config, {}))
|
||||
for name in config_names:
|
||||
if name == self.default_config:
|
||||
continue # defaults already the base
|
||||
raw = self.threshold_raw_configs.get(name)
|
||||
if raw is None:
|
||||
logger.warning(
|
||||
"Threshold config '%s' not found for host '%s', skipping",
|
||||
name, host_name,
|
||||
)
|
||||
else:
|
||||
result.update(raw)
|
||||
return result
|
||||
|
||||
def check_value(
|
||||
self,
|
||||
@@ -769,6 +803,29 @@ class ThresholdChecker:
|
||||
self._check_pending_or_renotify(host_name, alert_state, metric_path, value, threshold, None)
|
||||
|
||||
return None
|
||||
def _find_threshold(
|
||||
self, thresholds: Dict[str, "ThresholdConfig"], metric_path: str
|
||||
) -> Optional["ThresholdConfig"]:
|
||||
"""Return the threshold for *metric_path*, falling back to suffix matches.
|
||||
|
||||
Allows generic thresholds like ``ping_monitor.rtt_avg`` to match
|
||||
fully-qualified paths like ``ping_monitor.8_8_8_8_rtt_avg``.
|
||||
The exact match is always tried first; then successive leading
|
||||
underscore-delimited segments are stripped from the field name until
|
||||
a match is found or no segments remain.
|
||||
"""
|
||||
if metric_path in thresholds:
|
||||
return thresholds[metric_path]
|
||||
plugin, sep, field = metric_path.partition(".")
|
||||
if not sep:
|
||||
return None
|
||||
parts = field.split("_")
|
||||
for i in range(1, len(parts)):
|
||||
candidate = plugin + "." + "_".join(parts[i:])
|
||||
if candidate in thresholds:
|
||||
return thresholds[candidate]
|
||||
return None
|
||||
|
||||
def check_plugin_data(
|
||||
self,
|
||||
host_name: str,
|
||||
@@ -797,11 +854,10 @@ class ThresholdChecker:
|
||||
for metric_name, value in data.items():
|
||||
metric_path = f"{plugin_name}.{metric_name}"
|
||||
|
||||
if metric_path not in thresholds:
|
||||
threshold = self._find_threshold(thresholds, metric_path)
|
||||
if threshold is None:
|
||||
continue
|
||||
|
||||
threshold = thresholds[metric_path]
|
||||
|
||||
# Get or create alert state
|
||||
if metric_path not in alert_states:
|
||||
alert_states[metric_path] = AlertState(metric_path)
|
||||
@@ -987,23 +1043,23 @@ class ThresholdChecker:
|
||||
value: Any,
|
||||
):
|
||||
"""Send notification and log to journal/eventlog."""
|
||||
try:
|
||||
notify_mod.send_notification(
|
||||
from . import hbdclass
|
||||
host = hbdclass.Host.hosts.get(host_name)
|
||||
if host is not None and not host.watched:
|
||||
eventlog(host_name, lvl, message, service="threshold")
|
||||
return
|
||||
asyncio.get_event_loop().create_task(notify_mod.send_notification(
|
||||
host_name,
|
||||
notify_mod.Notification(
|
||||
title=f"[{lvl}] {host_name}",
|
||||
body=message,
|
||||
level=lvl,
|
||||
),
|
||||
)
|
||||
logger.info("Notification sent: %s", message)
|
||||
except Exception as e:
|
||||
logger.error("Failed to send notification: %s", e)
|
||||
))
|
||||
|
||||
# Log to journal
|
||||
if self.journal is not None:
|
||||
try:
|
||||
import asyncio
|
||||
loop = asyncio.get_event_loop()
|
||||
loop.create_task(self.journal.log_threshold_event(
|
||||
host_name=host_name,
|
||||
@@ -1080,7 +1136,9 @@ class ThresholdChecker:
|
||||
) -> None:
|
||||
"""Handle a state-change transition with grace-period logic.
|
||||
|
||||
Transitioning INTO alert: defers the notification for grace_seconds.
|
||||
Transitioning INTO alert (worsening): defers the notification for grace_seconds.
|
||||
De-escalation within alert states (e.g. CRITICAL→WARNING): no new notification;
|
||||
the metric is still alerting so no RECOVER was sent.
|
||||
Transitioning TO OK:
|
||||
- Still in grace window (pending_since set): suppresses both the alert
|
||||
and the recovery — the spike never warranted a page.
|
||||
@@ -1100,12 +1158,20 @@ class ThresholdChecker:
|
||||
alert_state.pending_since = None
|
||||
else:
|
||||
self._send_notification(host_name, lvl, message, metric_path, old_level, new_level, value)
|
||||
else:
|
||||
elif new_level.value > old_level.value:
|
||||
# Worsening (OK→WARNING, OK→CRITICAL, WARNING→CRITICAL): schedule notification.
|
||||
alert_state.pending_since = time.time()
|
||||
logger.debug(
|
||||
"Alert deferred (%.0fs grace): %s on %s = %s",
|
||||
self.grace_seconds, metric_path, host_name, value,
|
||||
)
|
||||
else:
|
||||
# De-escalation within alert states (e.g. CRITICAL→WARNING): metric is still
|
||||
# alerting but did not recover, so no new notification.
|
||||
logger.debug(
|
||||
"De-escalation %s→%s for %s on %s, no notification",
|
||||
old_level.name, new_level.name, metric_path, host_name,
|
||||
)
|
||||
|
||||
def _check_pending_or_renotify(
|
||||
self,
|
||||
@@ -1195,20 +1261,20 @@ class ThresholdChecker:
|
||||
else:
|
||||
message = f"REMINDER ({alert_state.level.name}): {host_name} - {metric_path} = {value} (ongoing for {int(now - alert_state.since)}s)"
|
||||
|
||||
try:
|
||||
notify_mod.send_notification(
|
||||
from . import hbdclass
|
||||
host = hbdclass.Host.hosts.get(host_name)
|
||||
if host is None or host.watched:
|
||||
asyncio.get_event_loop().create_task(notify_mod.send_notification(
|
||||
host_name,
|
||||
notify_mod.Notification(
|
||||
title=f"[REMINDER/{alert_state.level.name}] {host_name}",
|
||||
body=message,
|
||||
level=alert_state.level.name,
|
||||
),
|
||||
)
|
||||
))
|
||||
logger.info("Re-notification sent: %s", message)
|
||||
alert_state.last_notification = now
|
||||
alert_state.notification_count += 1
|
||||
logger.info("Re-notification sent: %s", message)
|
||||
except Exception as e:
|
||||
logger.error("Failed to send re-notification: %s", e)
|
||||
|
||||
def get_active_alerts(self, alert_states: Dict[str, AlertState]) -> list:
|
||||
"""
|
||||
|
||||
+21
-15
@@ -211,10 +211,11 @@ def _make_timer_callbacks(uname, host, ctx):
|
||||
connection.newstate(connection.__class__.OVERDUE, now, cfg.get("grace", 2))
|
||||
msg = f"{connection.afam} overdue"
|
||||
eventlog(uname, "CRITICAL", msg)
|
||||
notify_mod.send_notification(
|
||||
if host.watched:
|
||||
asyncio.create_task(notify_mod.send_notification(
|
||||
uname,
|
||||
notify_mod.Notification(title=f"[CRITICAL] {uname}", body=msg, level="CRITICAL"),
|
||||
)
|
||||
))
|
||||
# Track in alert_states so the Alerts Dashboard shows this
|
||||
_set_connectivity_alert(host, connection.afam, "CRITICAL")
|
||||
if threshold_checker:
|
||||
@@ -315,7 +316,6 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
|
||||
|
||||
cfg = ctx.get("config", {})
|
||||
hbdcls = ctx.get("hbdclass")
|
||||
log = ctx.get("log")
|
||||
msg_to_websockets = ctx.get("msg_to_websockets")
|
||||
DEBUG = ctx.get("DEBUG", 0)
|
||||
verbose = ctx.get("verbose", False)
|
||||
@@ -408,10 +408,11 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
|
||||
|
||||
if res:
|
||||
eventlog(uname, "WARNING", res)
|
||||
notify_mod.send_notification(
|
||||
if host.watched:
|
||||
asyncio.create_task(notify_mod.send_notification(
|
||||
uname,
|
||||
notify_mod.Notification(title=f"[WARNING] {uname}", body=res, level="WARNING"),
|
||||
)
|
||||
))
|
||||
|
||||
interval = int(msg.get("interval", 0) or 0)
|
||||
shutdown = msg.get("shutdown", 0)
|
||||
@@ -421,10 +422,11 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
|
||||
|
||||
if boot:
|
||||
eventlog(uname, "INFO", "booted")
|
||||
notify_mod.send_notification(
|
||||
if host.watched:
|
||||
asyncio.create_task(notify_mod.send_notification(
|
||||
uname,
|
||||
notify_mod.Notification(title=f"[INFO] {uname}", body=f"{host.name} booted", level="INFO"),
|
||||
)
|
||||
))
|
||||
if message:
|
||||
eventlog(uname, "INFO", "msg: %s" % message, service=service)
|
||||
|
||||
@@ -438,13 +440,18 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
|
||||
if not newh:
|
||||
if d == 0 or lasts == "unknown":
|
||||
m = "%s is up" % (conn.afam)
|
||||
elif d < 4:
|
||||
# Transient blip (likely client restart) — skip log and notification
|
||||
m = None
|
||||
else:
|
||||
m = "%s back after being %s for %s" % (conn.afam, lasts, dur(d))
|
||||
if m:
|
||||
eventlog(uname, "RECOVER", m)
|
||||
notify_mod.send_notification(
|
||||
if host.watched:
|
||||
asyncio.create_task(notify_mod.send_notification(
|
||||
uname,
|
||||
notify_mod.Notification(title=f"[RECOVER] {uname}", body=m, level="RECOVER"),
|
||||
)
|
||||
))
|
||||
|
||||
if boot or newh:
|
||||
host.upcount = host.doesack
|
||||
@@ -454,10 +461,11 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
|
||||
if shutdown:
|
||||
m = "%s shutdown" % conn.afam
|
||||
eventlog(uname, "INFO", m)
|
||||
notify_mod.send_notification(
|
||||
if host.watched:
|
||||
asyncio.create_task(notify_mod.send_notification(
|
||||
uname,
|
||||
notify_mod.Notification(title=f"[INFO] {uname}", body=m, level="INFO"),
|
||||
)
|
||||
))
|
||||
conn.newstate(hbdcls.Connection.DOWN, now)
|
||||
_set_connectivity_alert(host, conn.afam, "CRITICAL")
|
||||
|
||||
@@ -491,12 +499,10 @@ def handle_datagram(msg: dict, addr, transport, ctx: dict):
|
||||
op, rmsg = host.cmds[0]
|
||||
if op == "CMD":
|
||||
del host.cmds[0]
|
||||
if log:
|
||||
log(uname, "command sent")
|
||||
eventlog(uname, "INFO", "command sent")
|
||||
elif op == "UPD":
|
||||
del host.cmds[0]
|
||||
if log:
|
||||
log(uname, "update initiated")
|
||||
eventlog(uname, "INFO", "update initiated")
|
||||
opkt = dicttos(op, rmsg)
|
||||
try:
|
||||
transport.sendto(opkt, addr)
|
||||
|
||||
+52
-9
@@ -13,7 +13,8 @@ from . import data
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_connections: set = set()
|
||||
# Map of WebSocket → User object (or None when auth is disabled)
|
||||
_connections: dict = {}
|
||||
_loop: Optional[asyncio.AbstractEventLoop] = None
|
||||
_get_hosts: Optional[Callable[[], Iterable]] = None
|
||||
_verbose: bool = False
|
||||
@@ -34,22 +35,52 @@ def setup(
|
||||
_verbose = verbose
|
||||
|
||||
|
||||
def _user_can_see_host(user, host_name: str) -> bool:
|
||||
"""Return True if *user* may see updates for *host_name* (manager or higher)."""
|
||||
from . import hbdclass, users as users_mod
|
||||
if user is None or not users_mod.users_enabled():
|
||||
return True
|
||||
if user.admin:
|
||||
return True
|
||||
host = hbdclass.Host.hosts.get(host_name)
|
||||
if host is None:
|
||||
return False
|
||||
return host.is_manager(user.username)
|
||||
|
||||
|
||||
def _get_token(request) -> str:
|
||||
"""Extract session token from request (mirrors logic in http.py)."""
|
||||
auth = request.headers.get("Authorization", "")
|
||||
if auth.startswith("Bearer "):
|
||||
return auth[7:].strip()
|
||||
token = request.headers.get("X-Auth-Token", "")
|
||||
if token:
|
||||
return token
|
||||
return request.cookies.get("hbd_session", "")
|
||||
|
||||
|
||||
async def handler(request):
|
||||
"""aiohttp WebSocket upgrade handler — register as GET /ws."""
|
||||
from aiohttp import web
|
||||
from . import users as users_mod
|
||||
|
||||
ws = web.WebSocketResponse()
|
||||
await ws.prepare(request)
|
||||
|
||||
_connections.add(ws)
|
||||
token = _get_token(request)
|
||||
user = users_mod.get_session_user(token) if token else None
|
||||
|
||||
_connections[ws] = user
|
||||
remote = request.remote
|
||||
logger.info("WebSocket connected from %s", remote)
|
||||
|
||||
try:
|
||||
# Send current host state to the new client
|
||||
# Send current host state, filtered to hosts this user may see
|
||||
if _get_hosts:
|
||||
try:
|
||||
for h in list(_get_hosts()):
|
||||
host_name = h.get("raw_name") or h.get("name", "")
|
||||
if _user_can_see_host(user, host_name):
|
||||
await ws.send_str(json.dumps({"type": "host", "data": h}))
|
||||
except Exception as e:
|
||||
logger.error("Error sending initial hosts: %s", e)
|
||||
@@ -74,7 +105,7 @@ async def handler(request):
|
||||
except Exception as e:
|
||||
logger.exception("WebSocket handler error from %s: %s", remote, e)
|
||||
finally:
|
||||
_connections.discard(ws)
|
||||
_connections.pop(ws, None)
|
||||
logger.info("WebSocket disconnected from %s", remote)
|
||||
|
||||
return ws
|
||||
@@ -83,25 +114,37 @@ async def handler(request):
|
||||
def broadcast(typ: str, payload) -> bool:
|
||||
"""Thread-safe broadcast to all connected WebSocket clients.
|
||||
|
||||
For host and plugin updates, only sends to clients whose user has
|
||||
manager-or-higher access to that host. Other message types are
|
||||
broadcast to all clients.
|
||||
|
||||
Can be called from any thread; schedules sends on the event loop.
|
||||
Returns False if the loop is not running yet.
|
||||
"""
|
||||
if not _loop:
|
||||
return False
|
||||
|
||||
# Determine the host name for access-filtered message types
|
||||
host_name: Optional[str] = None
|
||||
if typ in ("host", "plugin"):
|
||||
host_name = payload.get("raw_name") or payload.get("host") or payload.get("name")
|
||||
|
||||
jmsg = json.dumps({"type": typ, "data": payload})
|
||||
|
||||
async def _send_all():
|
||||
dead = set()
|
||||
for ws in list(_connections):
|
||||
for ws, user in list(_connections.items()):
|
||||
try:
|
||||
if not ws.closed:
|
||||
await ws.send_str(jmsg)
|
||||
else:
|
||||
if ws.closed:
|
||||
dead.add(ws)
|
||||
continue
|
||||
if host_name is not None and not _user_can_see_host(user, host_name):
|
||||
continue
|
||||
await ws.send_str(jmsg)
|
||||
except Exception:
|
||||
dead.add(ws)
|
||||
for ws in dead:
|
||||
_connections.discard(ws)
|
||||
_connections.pop(ws, None)
|
||||
|
||||
asyncio.run_coroutine_threadsafe(_send_all(), _loop)
|
||||
return True
|
||||
|
||||
+7
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "hbd"
|
||||
version = "5.1.5"
|
||||
version = "5.1.16"
|
||||
description = "Heartbeat monitoring system — client (hbc) and server (hbd)"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11"
|
||||
@@ -34,6 +34,9 @@ server = [
|
||||
"matrix-nio>=0.24",
|
||||
]
|
||||
|
||||
# Minimal client — hbc_mini only, no external dependencies
|
||||
mini = []
|
||||
|
||||
# Install both client and server
|
||||
all = [
|
||||
"hbd[client,server]",
|
||||
@@ -54,6 +57,9 @@ dev = [
|
||||
hbd = "hbd.server.cli:main"
|
||||
hbc = "hbd.client.main:main"
|
||||
|
||||
[tool.setuptools]
|
||||
script-files = ["scripts/hb_install.sh", "scripts/hbc_mini.py"]
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["."]
|
||||
include = ["hbd*"]
|
||||
|
||||
@@ -4,12 +4,14 @@ set -e
|
||||
uv version --bump patch
|
||||
VER=$(uv version --short)
|
||||
sed -i".bak" "s/__version__ = \"[0-9.]*\"\(.*\)$/__version__ = \"$VER\"\1/" hbd/__init__.py
|
||||
sed -i".bak" "s/__version__ = \"[0-9.]*\"\(.*\)$/__version__ = \"$VER\"\1/" scripts/hbc_mini.py
|
||||
|
||||
# commit pyproject.toml
|
||||
git commit -m "version $VER" pyproject.toml hbd/__init__.py
|
||||
git commit -m "version $VER" pyproject.toml hbd/__init__.py scripts/hbc_mini.py
|
||||
git push
|
||||
# tag version
|
||||
git tag -a v$VER -m "Version $VER"
|
||||
git push --tags
|
||||
|
||||
rm hbd/__init__.py.bak
|
||||
rm scripts/hbc_mini.py.bak
|
||||
|
||||
+45
-27
@@ -12,11 +12,14 @@
|
||||
set -e
|
||||
what=$1
|
||||
on_ha=0
|
||||
where=""
|
||||
venv=""
|
||||
[ "$2" = "HA" ] && on_ha=1
|
||||
[ -z "$what" ] && what="client"
|
||||
|
||||
if [ -d /homeassistant ]; then
|
||||
echo "cannot install in HA, running \"docker exec homeassistant $0 $@\""
|
||||
docker exec homeassistant $0 $@
|
||||
if [ -d /homeassistant ]; then # if running from HA command line
|
||||
echo "HA, running \"docker exec homeassistant /config/bin/hb_install.sh $@\""
|
||||
docker exec homeassistant /config/bin/hb_install.sh $@ HA
|
||||
rc=$?
|
||||
if [ $rc -ne 0 ]; then
|
||||
echo "Failed to install heartbeat in HA, please check the logs for more details"
|
||||
@@ -24,11 +27,12 @@ if [ -d /homeassistant ]; then
|
||||
fi
|
||||
exit 0
|
||||
fi
|
||||
if [ -d /config ]; then
|
||||
echo "Installing on HA"
|
||||
|
||||
if [ $on_ha -eq 1 ] || [ -r /.dockerenv ] && [ -d /config/bin ]; then
|
||||
# Installing under docker on Home Assistant OS, using /config/bin for executables and /config/venvs for virtual environments
|
||||
echo "Home Assistant OS detected, installing under docker"
|
||||
where="/config/bin"
|
||||
venv="/config/venvs"
|
||||
on_ha=1
|
||||
else
|
||||
if [ ! -d $HOME/.local/bin ] && [ ! -d $HOME/bin ]; then
|
||||
echo "No suitable bin directory found in PATH, please add either $HOME/.local/bin or $HOME/bin to your PATH"
|
||||
@@ -43,24 +47,32 @@ else
|
||||
echo "No suitable bin directory found in PATH, please add either $HOME/.local/bin or $HOME/bin to your PATH"
|
||||
exit 1
|
||||
fi
|
||||
if [ "$what" = "mini" ]; then
|
||||
venv=""
|
||||
else
|
||||
venv="$HOME/venvs"
|
||||
fi
|
||||
fi
|
||||
echo "Installing $what to $where"
|
||||
if [ ! -z "$venv" ]; then
|
||||
echo "Using virtual environment at $venv/hbd"
|
||||
fi
|
||||
|
||||
echo "Installing heartbeat $what"
|
||||
|
||||
if [ ! -d $venv/hbd ]; then
|
||||
set +e
|
||||
python3 -m pip --version > /dev/null 2>&1
|
||||
rc=$?
|
||||
set -e
|
||||
if [ $rc -ne 0 ]; then
|
||||
# truenas does not have pip installed by default, so we need to fetch get-pip.py and install pip
|
||||
if [ "$venv" != "" ] && [ ! -d $venv/hbd ]; then
|
||||
arg=""
|
||||
have_pip=$(python3 -c "import pip" 2>/dev/null &> /dev/null && echo "Installed" || echo "Not Installed")
|
||||
if [ "$have_pip" = "Not Installed" ]; then
|
||||
# some systems do not have pip installed by default, so we need to fetch get-pip.py and install pip
|
||||
echo "pip is not installed, fetching get-pip.py and installing pip"
|
||||
arg="--without-pip"
|
||||
fi
|
||||
mkdir -p $venv
|
||||
have_venv=$(python3 -c "import venv" &> /dev/null && echo "Installed" || echo "Not Installed")
|
||||
have_venv=$(python3 -c "import venv" 2>/dev/null &> /dev/null && echo "Installed" || echo "Not Installed")
|
||||
if [ "$have_venv" = "Not Installed" ]; then
|
||||
if [ "$have_pip" = "Not Installed" ]; then
|
||||
echo "python has no venv, and no pip to install virtualenv, cannot continue"
|
||||
exit 1
|
||||
fi
|
||||
echo "python venv module not found, installing virtualenv"
|
||||
python3 -m pip install --user virtualenv
|
||||
python3 -m virtualenv $venv/hbd --system-site-packages $arg
|
||||
@@ -74,24 +86,30 @@ if [ ! -d $venv/hbd ]; then
|
||||
deactivate
|
||||
fi
|
||||
|
||||
if [ ! -z "$venv" ]; then
|
||||
. $venv/hbd/bin/activate
|
||||
fi
|
||||
if [ "$what" = "mini" ]; then
|
||||
curl -s -o $where/hbc_mini https://git.wrede.ca/andreas/heartbeat/raw/branch/master/scripts/hbc_mini.py
|
||||
chmod +x $where/hbc_mini
|
||||
else
|
||||
python3 -mpip install --upgrade --index-url https://git.wrede.ca/api/packages/andreas/pypi/simple/ --extra-index-url https://pypi.org/simple hbd[$what]
|
||||
fi
|
||||
|
||||
if [ ! -z "$venv" ]; then
|
||||
echo "linking executables to $where"
|
||||
if [ "$what" = "server" ]; then
|
||||
rm -f $where/hbd
|
||||
ln -sf $(which hbd) $where/hbd
|
||||
echo "hbd installed, you can run it with \"$where/hbd\" or \"hbd\" if $where is in your PATH"
|
||||
else
|
||||
elif [ "$what" = "client" ]; then
|
||||
rm -f $where/hbc
|
||||
ln -sf $(which hbc) $where/hbc
|
||||
fi
|
||||
rm -f $where/hb_install.sh
|
||||
cp "$0" $where/hb_install.sh
|
||||
chmod +x $where/hb_install.sh
|
||||
if [ $on_ha -eq 1 ]; then
|
||||
echo "restarting hbc "
|
||||
job=$(grep run_hbc configuration.yaml | sed 's/run_hbc://')
|
||||
$job
|
||||
else
|
||||
echo "hbc installed, you can run it with \"$where/hbc\" or \"hbc\" if $where is in your PATH"
|
||||
fi
|
||||
ln -sf $(which hb_install.sh) $where/hb_install.sh
|
||||
fi
|
||||
echo "Installation complete. To upgrade, run the following:"
|
||||
echo " $where/hb_install.sh $what"
|
||||
echo "To install on another machine, run the following obtain the install script and run it:"
|
||||
echo "from https://git.wrede.ca/andreas/heartbeat/raw/branch/master/scripts/hb_install.sh"
|
||||
echo "and then run sh hb_install.sh [mini|client]"
|
||||
Executable
+1154
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user