# Heartbeat

Heartbeat is a lightweight host monitoring system built around a simple idea: each machine you want to monitor runs a small client (`hbc`) that sends a UDP "heartbeat" packet to a central server (`hbd`) on a regular interval. If a heartbeat stops arriving, you get notified. Alongside reachability, clients can ship system metrics — CPU, memory, disk, network — and the server will alert you when any of those cross a threshold.

## How it works

```
  [ monitored host ]              [ your server ]
  ┌─────────────┐  UDP 50003     ┌────────────────────────┐
  │     hbc     │ ────────────>  │  hbd                   │
  │             │                │  host state tracking   │
  │  plugins:   │  <──────────── │  threshold alerting    │
  │  cpu, mem,  │  ACK / CMD     │  notifications         │
  │  disk, ...  │                │  web dashboard + API   │
  └─────────────┘                └────────────────────────┘
```

- **hbd** — the server daemon. Tracks which hosts are alive, evaluates metric thresholds, fires notifications, serves the web dashboard and REST API.
- **hbc** — the client. Sends heartbeats and plugin data over UDP. Runs on any Linux/BSD/macOS host.
- **hbc_mini** — a zero-dependency single-file alternative (`hbc_mini.py` or `hbc_mini.c`) for hosts where you can't install Python packages.

Notifications can go to Pushover, email, Mattermost, Matrix, Signal, or VoIP.ms SMS. The dashboard shows host connectivity, RTT graphs, active alerts, and per-host plugin metrics in real time via WebSocket.

---

## Getting started

This tutorial sets up a server on one machine and a client on a second machine. You'll end up with a working dashboard and your first host being monitored.

### 1. Install the server

On the machine that will run `hbd`:

```bash
git clone https://git.wrede.ca/andreas/heartbeat.git
cd heartbeat
python3 -m venv .venv
source .venv/bin/activate
pip install .
```

Verify the install:

```bash
hbd --help
```

### 2. Create a server config

Create `~/.hb.yaml`:

```yaml
hb_port: 50003       # UDP port — clients send heartbeats here
hbd_port: 50004      # HTTP port — web dashboard and API
ws_port: 50005       # WebSocket port — live dashboard updates

interval: 20         # Expected heartbeat interval (seconds)
grace: 2             # Seconds of slack before a host is considered overdue

pickfile: ~/.hb.pick
pidfile:  ~/.hb.pid
logfile:  ~/.hb.log
```

That's enough to get started. No hosts, no users, no notifications needed yet — the server will accept any client that connects.

### 3. Start the server

```bash
hbd serve -c ~/.hb.yaml -f -v
```

`-f` keeps it in the foreground so you can watch the log. You should see:

```
Heartbeat daemon starting on UDP :50003, HTTP :50004, WS :50005
```

Open `http://your-server:50004/live` in a browser. The dashboard is empty for now.

### 4. Install the client on a host to monitor

On the machine you want to monitor (must be able to reach the server on UDP 50003):

```bash
pip install hbd      # or: copy scripts/hbc_mini.py if you can't install packages
```

#### Quick start — no config file

```bash
hbc your-server.example.com
```

Within a few seconds the server log will show the host checking in, and it will appear on the dashboard.

#### With a config file

Create `~/.hbc.yaml` on the client host:

```yaml
hb_port: 50003
interval: 10         # Send a heartbeat every 10 seconds

plugins:
  cpu_monitor:
    interval: 60
  memory_monitor:
    interval: 60
  disk_monitor:
    interval: 60
```

Then start the client:

```bash
hbc -c ~/.hbc.yaml your-server.example.com
```

Send a boot message at startup so the server logs when the host came up:

```bash
hbc -b -c ~/.hbc.yaml your-server.example.com
```

Run as a daemon (logs go to syslog):

```bash
hbc -d -b -c ~/.hbc.yaml your-server.example.com
```

### 5. View the dashboard

Open `http://your-server:50004/live`. You'll see the monitored host, its last heartbeat time, and RTT. Click the host name to see plugin metrics.

Navigate to `/plugins/<hostname>` for CPU, memory, and disk graphs.

### 6. Add a notification channel (optional)

Edit `~/.hb.yaml` on the server:

```yaml
notification_channels:
  pushover_ops:
    type: pushover
    token: YOUR_APP_TOKEN
    user: YOUR_USER_KEY

users:
  alice:
    password: pbkdf2:sha256:...    # generate: hbd passwd alice
    admin: true
    notification_channels: [pushover_ops]

default_owner: alice
```

Generate the password hash:

```bash
hbd passwd alice
```

Paste the output into the config, then reload:

```bash
hbd reload
```

Test the channel:

```bash
hbd notify
```

### 7. Set a threshold alert (optional)

Add to `~/.hb.yaml`:

```yaml
thresholds:
  cpu_monitor:
    cpu_percent:
      warning: 80.0
      critical: 90.0
  disk_monitor:
    partitions:
      /:
        percent:
          warning: 80.0
          critical: 90.0
```

Reload: `hbd reload`. The server will now alert when a monitored host crosses these values.

---

## What's next

| Topic | Where to look |
|---|---|
| Full server config reference | [README — Server](https://git.wrede.ca/andreas/heartbeat/src/branch/master/README.md#server-hbd) |
| Client options and all plugins | [README — Client](https://git.wrede.ca/andreas/heartbeat/src/branch/master/README.md#client-hbc) |
| Threshold alerting details | [THRESHOLD_ALERTING.md](https://git.wrede.ca/andreas/heartbeat/src/branch/master/docs/THRESHOLD_ALERTING.md) |
| Notification channels | [NOTIFICATIONS.md](https://git.wrede.ca/andreas/heartbeat/src/branch/master/docs/NOTIFICATIONS.md) |
| User accounts and roles | [USERS.md](https://git.wrede.ca/andreas/heartbeat/src/branch/master/docs/USERS.md) |
| Writing a custom plugin | [PLUGIN_DEVELOPMENT.md](https://git.wrede.ca/andreas/heartbeat/src/branch/master/docs/PLUGIN_DEVELOPMENT.md) |
| Nagios check integration | [NAGIOS_INTEGRATION.md](https://git.wrede.ca/andreas/heartbeat/src/branch/master/docs/NAGIOS_INTEGRATION.md) |
| REST API | [HTTP_API.md](https://git.wrede.ca/andreas/heartbeat/src/branch/master/docs/HTTP_API.md) |
| Zero-dependency client | [README — hbc_mini](https://git.wrede.ca/andreas/heartbeat/src/branch/master/README.md#hbc_mini--zero-dependency-client) |