mirror of
https://github.com/ipnet-mesh/meshcore-hub.git
synced 2026-08-06 08:52:50 +02:00
docs: document Route Health Monitoring for v0.16 release
- Add docs/routes.md feature/operational doc (evaluation semantics, visibility, observer scoping, background evaluator, prerequisites) - Add v0.16.0 'Route Health Monitoring' section to docs/upgrading.md (above the existing Observer Ingestion Filters subsection) - docs/configuration.md: cross-ref header + FEATURE_ROUTES (Feature Flags) and ROUTE_EVALUATOR_INTERVAL_SECONDS (Collector) rows - README.md: Features bullet + docs tree entry + Documentation list entry - docs/seeding.md: cross-link to routes.md from the Routes section - docker-compose.yml: wire FEATURE_ROUTES (web) and ROUTE_EVALUATOR_INTERVAL_SECONDS (collector) as passthroughs
This commit is contained in:
Louis King
@@ -70,6 +70,7 @@ flowchart LR
|
||||
|
||||
- **Event Persistence**: Store messages, advertisements, telemetry, and trace data
|
||||
- **Raw Packet Inspection**: Capture, browse, and search raw wire packets; a deduplicated packet view shows every observer reception and routing path, with clickable path-hash badges that look up the matching nodes
|
||||
- **Route Health Monitoring**: Define multi-hop mesh routes and track each link's health over time with traffic-light status, per-day history, and recent-match attribution
|
||||
- **REST API**: Query historical data with filtering and pagination
|
||||
- **Node Tagging**: Add custom metadata to nodes for organization
|
||||
- **Web Dashboard**: Visualize network status, node locations, and message history
|
||||
@@ -334,6 +335,7 @@ meshcore-hub/
|
||||
│ ├── letsmesh.md # LetsMesh packet decoding details
|
||||
│ ├── maintenance.md # Backup and restore procedures
|
||||
│ ├── observer.md # Remote observers and PACKETCAPTURE_* reference
|
||||
│ ├── routes.md # Route health monitoring
|
||||
│ ├── seeding.md # Seed data format and import guide
|
||||
│ ├── upgrading.md # Upgrade guide for breaking changes
|
||||
│ └── webhooks.md # Webhook configuration reference
|
||||
@@ -347,6 +349,7 @@ meshcore-hub/
|
||||
- [docs/configuration.md](docs/configuration.md) - Single source of truth for environment variables
|
||||
- [docs/deployment.md](docs/deployment.md) - Production setup, reverse proxy, multi-instance, API scaling, Redis
|
||||
- [docs/observer.md](docs/observer.md) - Remote packet-capture observers and `PACKETCAPTURE_*` reference
|
||||
- [docs/routes.md](docs/routes.md) - Route health monitoring and link status
|
||||
- [docs/maintenance.md](docs/maintenance.md) - Backup and restore procedures
|
||||
- [docs/database.md](docs/database.md) - Database backends (SQLite/PostgreSQL) and migration
|
||||
- [docs/upgrading.md](docs/upgrading.md) - Upgrade guide for breaking changes
|
||||
|
||||
@@ -256,6 +256,8 @@ services:
|
||||
- SPAM_WEIGHT_PATH=${SPAM_WEIGHT_PATH:-0.75}
|
||||
- SPAM_WEIGHT_NAME=${SPAM_WEIGHT_NAME:-0.25}
|
||||
- SPAM_RESCORE_INTERVAL_SECONDS=${SPAM_RESCORE_INTERVAL_SECONDS:-120}
|
||||
# Route health monitoring (background evaluator cadence; 0 disables).
|
||||
- ROUTE_EVALUATOR_INTERVAL_SECONDS=${ROUTE_EVALUATOR_INTERVAL_SECONDS:-60}
|
||||
command: ["collector"]
|
||||
healthcheck:
|
||||
test: ["CMD", "meshcore-hub", "health", "collector"]
|
||||
@@ -419,6 +421,8 @@ services:
|
||||
# Spam detection toggle is on by default; this also drives the backend
|
||||
# scoring/hide switch on the collector and api.
|
||||
- FEATURE_SPAM_DETECTION=${FEATURE_SPAM_DETECTION:-true}
|
||||
# Routes page (route health monitoring) is on by default.
|
||||
- FEATURE_ROUTES=${FEATURE_ROUTES:-true}
|
||||
command: ["web"]
|
||||
healthcheck:
|
||||
test:
|
||||
|
||||
@@ -4,7 +4,7 @@ This document is the **single source of truth** for MeshCore Hub environment var
|
||||
|
||||
Variables are grouped by feature. Each section below links to the feature's dedicated document (where one exists) for setup, architecture, and operational details. The companion documents no longer duplicate these tables — they link back here.
|
||||
|
||||
> **Cross-references:** [deployment.md](deployment.md) (production setup, scaling, Redis operational notes) · [database.md](database.md) (backend setup, migration runbook) · [observer.md](observer.md) (packet-capture observer vars, which live there because they configure an external image) · [auth.md](auth.md) (OIDC architecture, IdP guides) · [webhooks.md](webhooks.md) (payload format, URL routing) · [letsmesh.md](letsmesh.md) (packet decoding) · [content.md](content.md) (custom pages, media, logos) · [i18n.md](i18n.md) (translations) · [seeding.md](seeding.md) (seed YAML) · [maintenance.md](maintenance.md) (backup/restore)
|
||||
> **Cross-references:** [deployment.md](deployment.md) (production setup, scaling, Redis operational notes) · [database.md](database.md) (backend setup, migration runbook) · [observer.md](observer.md) (packet-capture observer vars, which live there because they configure an external image) · [routes.md](routes.md) (route health monitoring feature overview) · [auth.md](auth.md) (OIDC architecture, IdP guides) · [webhooks.md](webhooks.md) (payload format, URL routing) · [letsmesh.md](letsmesh.md) (packet decoding) · [content.md](content.md) (custom pages, media, logos) · [i18n.md](i18n.md) (translations) · [seeding.md](seeding.md) (seed YAML) · [maintenance.md](maintenance.md) (backup/restore)
|
||||
|
||||
---
|
||||
|
||||
@@ -71,6 +71,7 @@ The collector subscribes to MQTT events and persists them to the database. For p
|
||||
| Variable | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `CHANNEL_REFRESH_INTERVAL_SECONDS` | `300` | Seconds between channel-key refresh from the database (minimum `10`) |
|
||||
| `ROUTE_EVALUATOR_INTERVAL_SECONDS` | `60` | Seconds between route health evaluations. `0` disables the background evaluator (route cards then stay `unknown`). See [routes.md](routes.md) |
|
||||
|
||||
### Observer Ingestion Filters
|
||||
|
||||
@@ -233,6 +234,7 @@ Control which pages are visible in the web dashboard. Disabled features are full
|
||||
| `FEATURE_RADIO_CONFIG` | `true` | Show radio config panel on home page |
|
||||
| `FEATURE_PACKETS` | `true` | Enable the `/packets` raw-packet browser. In Compose this also drives `RAW_PACKET_CAPTURE_ENABLED` on the collector |
|
||||
| `FEATURE_SPAM_DETECTION` | `true` | Show the "show potential spam" toggle on `/messages`. In Compose this also drives `SPAM_DETECTION_ENABLED` on the collector + api — see [Spam Detection](#spam-detection) |
|
||||
| `FEATURE_ROUTES` | `true` | Enable the `/routes` page (route health monitoring). See [routes.md](routes.md) |
|
||||
|
||||
**Dependencies:** Dashboard auto-disables when all of Nodes/Advertisements/Messages are disabled. Map auto-disables when Nodes is disabled. Members auto-disables when OIDC is disabled (set via `OIDC_ENABLED`).
|
||||
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
# Route Health Monitoring
|
||||
|
||||
The **Routes** page (`/routes`) lets operators define monitored multi-hop mesh routes and track each one's health over time. A background evaluator on the collector matches captured packet paths against each route's configured node sequence within a rolling time window and rolls the result up into a traffic-light status card, with a per-day history strip and a list of recent matching transmissions.
|
||||
|
||||
For the environment-variable reference, see [configuration.md → Feature Flags](configuration.md#feature-flags) (`FEATURE_ROUTES`) and [→ Collector](configuration.md#collector) (`ROUTE_EVALUATOR_INTERVAL_SECONDS`). To define routes from YAML, see [seeding.md → Routes](seeding.md#routes).
|
||||
|
||||
> **Prerequisite:** route matching reads from the `packet_path_hops` table, which is populated by **Raw Packet Capture**. Keep `FEATURE_PACKETS=true` (the default) so packet paths continue to be captured; with capture off, route cards stay at `unknown` once the existing window of hops ages out.
|
||||
|
||||
## How health is evaluated
|
||||
|
||||
A route is an ordered list of two or more nodes (the configured path). For each captured packet reception, the evaluator walks the reception's path-hash sequence and checks whether the route's nodes appear **in order, as a subsequence** (intermediate hops are allowed). When `reversible` is set (the default), the reverse-ordered path is also accepted, so a packet travelling `B → ... → A` counts toward an `A → B` route.
|
||||
|
||||
Each route carries these knobs:
|
||||
|
||||
| Field | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `match_width` | `1` | Path-hash prefix width in bytes (1/2/3). Higher widths disambiguate nodes that share a short public-key prefix. |
|
||||
| `window_hours` | `24` | Rolling lookback window for the live status card. |
|
||||
| `packet_count_threshold` | `3` | Distinct matching packets at/above which the route is `healthy`. |
|
||||
| `clear_threshold` | _(2× threshold)_ | Comfort bar for the `clear`/`marginal` split. Omit/null to use twice the threshold. |
|
||||
| `max_hop_span` | _(unlimited)_ | Caps the position gap between the first and last matched node, to reject matches that wander too far. |
|
||||
| `reversible` | `true` | Also match the path in reverse direction. |
|
||||
| `enabled` | `true` | When `false`, the route is skipped by the evaluator and reports `unknown`/`no_coverage`. |
|
||||
|
||||
The result is reported on two axes:
|
||||
|
||||
- **State** — `healthy` (≥ threshold distinct matches in the window), `unhealthy` (some packets were observed but too few matched the configured path), or `no_coverage` (no in-scope packets at all in the window).
|
||||
- **Quality** — `clear` (≥ effective clear bar), `marginal` (healthy but below the comfort bar), `failing` (unhealthy), or `unknown` (no coverage / route disabled).
|
||||
|
||||
The state/quality split lets the dashboard render a single traffic-light band while keeping the underlying reason visible: a `marginal` route is technically up but losing margin, and a `failing` route has traffic in the window but the configured path isn't completing.
|
||||
|
||||
### Observer scoping
|
||||
|
||||
By default every observer's receptions contribute to every route. A route may instead scope itself to an explicit **observer allow-list** (`route_observers`); when set, only receptions from those observer nodes are matched. Use this to ignore noisy or off-path observers that would otherwise drown out the signal. Observer nodes that don't yet exist in the database are skipped with a warning rather than failing the seed.
|
||||
|
||||
### Background evaluator
|
||||
|
||||
The collector runs a background thread that re-evaluates every enabled route on a fixed cadence and upserts the result into `route_results` (one row per route, holding the latest `state`, `quality`, `matched_count`, and the threshold snapshots used). The cadence is controlled by `ROUTE_EVALUATOR_INTERVAL_SECONDS` (default `60`); set it to `0` to disable the evaluator, in which case route cards remain at `unknown` until it is re-enabled. The per-route history strip and the `/api/v1/routes/{id}/history` endpoint evaluate on demand over the raw `packet_path_hops` table and are not dependent on the background evaluator.
|
||||
|
||||
## Visibility
|
||||
|
||||
Routes carry the same role-based visibility levels as channels — `community`, `member`, `operator`, `admin`. A user only sees routes whose visibility is at or below their role's maximum level. Seeded routes default to `community` (visible to everyone); set a higher level to restrict a route to operators/admins only. Visibility is enforced on both the list and detail endpoints, so a hidden route's existence is not leaked.
|
||||
|
||||
## Defining routes
|
||||
|
||||
Routes are keyed by their `from`/`to` endpoint labels and upserted by that pair. There are two ways to create them:
|
||||
|
||||
- **Seed YAML** — add a `routes.yaml` to your `SEED_HOME` and run the seed process. See [seeding.md → Routes](seeding.md#routes) for the format and rules (path nodes must already exist in the database; the `(from, to)` pair must be unique).
|
||||
- **API** — `POST /api/v1/routes` (admin only) creates a route, with a `/preview` endpoint that dry-runs matching against an unsaved configuration so you can tune thresholds before committing. See `SCHEMAS.md` for the request/response shapes.
|
||||
|
||||
The `/routes` page renders the live status card, the per-day history strip, recent matching transmissions (with observer attribution), and — for admins — inline edit/delete controls.
|
||||
@@ -111,6 +111,8 @@ meshcore-hub collector channel remove --name MyChannel
|
||||
|
||||
## Routes
|
||||
|
||||
> See [routes.md](routes.md) for the route health monitoring feature overview and evaluation semantics.
|
||||
|
||||
Routes define multi-hop mesh paths to monitor for health. Each route is keyed by its `from`/`to` endpoint labels and is upserted (created or updated) by that label pair.
|
||||
|
||||
### Routes YAML Format
|
||||
|
||||
@@ -4,6 +4,27 @@ This guide covers upgrading from a previous MeshCore Hub release to the current
|
||||
|
||||
## v0.16.0
|
||||
|
||||
### Route Health Monitoring
|
||||
|
||||
A new **Routes** page lets operators define monitored multi-hop mesh routes (an ordered list of two or more nodes) and track each one's health. A background evaluator on the collector matches captured packet paths against each route's configured node sequence within a configurable time window and rolls the result up into a traffic-light **quality** band (`clear` / `marginal` / `failing` / `unknown`) plus a **state** (`healthy` / `unhealthy` / `no_coverage`). Routes carry the same role-based visibility levels as channels (`community` / `member` / `operator` / `admin`), can be scoped to specific observers, and are matched in both directions by default (`reversible`). Define them via `routes.yaml` in `SEED_HOME` (see [seeding.md → Routes](seeding.md#routes)) or the `/api/v1/routes` API; see [routes.md](routes.md) for the feature overview.
|
||||
|
||||
**On by default** and non-breaking — the page renders and the evaluator runs with no configuration. Hide the page with `FEATURE_ROUTES=false`; stop the evaluator with `ROUTE_EVALUATOR_INTERVAL_SECONDS=0`.
|
||||
|
||||
**Database migration required:**
|
||||
|
||||
```
|
||||
meshcore-hub db upgrade
|
||||
```
|
||||
|
||||
This creates five tables — `routes`, `route_nodes`, `route_observers`, `route_results`, `packet_path_hops` — and backfills `packet_path_hops` from existing `raw_packets.decoded`. Route health therefore relies on **Raw Packet Capture** being enabled (`FEATURE_PACKETS=true`, the default) so packet paths continue to be captured. The migration runs automatically on Docker startup; the schema change is additive and safe on both SQLite and Postgres.
|
||||
|
||||
**New optional environment variables (all safe to omit):**
|
||||
|
||||
| Variable | Default | Description |
|
||||
| ----------------------------------- | ------- | ---------------------------------------------------------------------------------------------------- |
|
||||
| `FEATURE_ROUTES` | `true` | Show the `/routes` page and nav entry. On by default. |
|
||||
| `ROUTE_EVALUATOR_INTERVAL_SECONDS` | `60` | Collector background evaluator cadence in seconds. `0` disables the evaluator (cards stay `unknown`). |
|
||||
|
||||
### Observer Ingestion Filters (allow/deny remote observers)
|
||||
|
||||
Remote observers contribute to the Hub by publishing decoded packets to your MQTT broker, and anyone with broker access can do so. You can now restrict which observers are ingested by their public key with two new **optional** collector variables:
|
||||
|
||||
Reference in New Issue
Block a user