Add complete route health monitoring feature that tracks whether packets
traverse expected multi-hop paths through the mesh network.
Models & migration:
- 5 new models: PacketPathHop, Route, RouteNode, RouteObserver, RouteResult
- Alembic migration with keyset-paginated backfill of existing packets
Collector:
- store_raw_packet refactored to persist path hops via bulk insert
- Matching engine (collector/routes.py) with subsequence matching, quality
bands (clear/marginal/failing/no_coverage), and collision detection
- Background route evaluator (60s loop) wired into subscriber lifespan
- Route seed loader in CLI (resolves by public_key, matching YAML format)
API:
- 6 CRUD endpoints + preview endpoint under /api/v1/routes
- Schemas accept node_public_keys (64-char hex) instead of internal UUIDs
- 5 Prometheus gauges for route health metrics
- Packet groups endpoint reads from hop table
Web UI:
- Full SPA routes page with summary strip, grouped cards, expandable detail
- Routes nav entry in both desktop (spa.html) and mobile (app.js) navbars
- Home page nav card with feature gate
- API proxy access mapping for v1/routes endpoints
- Visual node-search path builder with autocomplete dropdown, ordered chips
with reorder/remove controls, and paste-64-char-key support
- Observer picker with same search UX (unordered chips)
- i18n strings in en.json and nl.json
Config:
- feature_routes flag (default: true)
- route_evaluator_interval_seconds (default: 60)
- routes_file seed path
- example/seed/routes.yaml
Persist the path-hash byte width (1/2/3) as a nullable Integer column on
RawPacket, computed at ingest by the collector and backfilled for historical
rows via a self-contained Python migration. Replace the per-request Python
decode loop in the grouped-list route with a SQL MAX() aggregate + HAVING
filter, and add a discrete <select> filter (Any/1B/2B/3B) to the /packets
SPA page.
- Model: add path_hash_bytes column to RawPacket (nullable Integer)
- Migration: batch_alter_table add_column + keyset-paginated Python backfill
reading decoded via Core select() on sa.JSON column (portable across
SQLite and Postgres)
- Collector: compute path_hash_bytes at ingest via two-tier path-hash
extraction (decoded.path -> payload.decoded.pathHashes)
- API: add func.max() aggregate to group query, HAVING filter on
?path_hash_bytes=1|2|3 param; delete Phase 3 decode loop and dead helper
- Frontend: add path-width select filter wired through query/apiParams/
pagination/headerParams; add i18n keys (en/nl)
- Tests: 1186 passed, 22 skipped; collector + API + model coverage
Adverts/Messages rows now link directly to the deduplicated packet-detail
page (/packets/hash/:hash). Each path hop renders as a clickable badge
opening a popover that looks up nodes by public-key prefix via the new
pubkey_prefix query param on GET /api/v1/nodes (case-insensitive
startswith). Adds a derived path_hash_bytes field on GroupedPacketRead.
Defaults changed: FEATURE_PACKETS now defaults to true and
RAW_PACKET_RETENTION_DAYS to 7 (independent of DATA_RETENTION_DAYS).
Fixes a mypy arg-type error by explicitly annotating the packet_hash
list as list[str].
The packet-group detail observer table only ever showed the hop count
because _extract_path_hashes looked at decoded.payload.decoded.pathHashes,
but normal packets carry the routing path at the top level as decoded.path.
Read decoded.path first (falling back to the old pathHashes location).
Frontend: render each hop as its own badge joined by arrows, wrapping on
narrow screens, with centre-truncation for paths over 16 hops. Badges carry
a path-hash-badge class + data-path-hash hook for a future node lookup.
Also expose v1/packet-groups as an open endpoint in the web proxy mapping
(it is not a prefix of v1/packets) and rebase the packet_hash index
migration onto the current head.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Fix black formatting on packet_groups.py and raw_packets.py
- Fix mypy error: annotate sort_exprs dict as dict[str, Any] so
asc()/desc() receive a typed expression, not object
- Add test_packet_groups.py with 33 tests covering list/detail
endpoints, filters, redaction, observer hydration, path_hashes
extraction, and the _extract_path_hashes helper (98% line coverage)
https://claude.ai/code/session_01NH2rZzuHzasJj12SZeRhbJ
Louis King